Monday, July 16, 2007

JavaScript Web Spider - Powered by Yahoo

pdp has released a proof-of-concept web spider written completely in JavaScript. It is a pure client-side tool, requiring no server support other than the Yahoo Site Explorer service it leverages. The spider is very efficient - it can index the files and directory structure of a web site within a few queries, making it very fast and efficient. The only limitation is that only can fetch pages already indexed by Yahoo.

As pdp points out in his writeup, it would be simple to modify this code to identify vulnerabilities in spidered sites and exploit them in real-time. We will almost certainly see XSS/AJAX worms leveraging this sort of technique to target and compromise other web applications, and they will be very difficult to block.