Searching for web pages
My feature in today’s Guardian newspaper (If distributed processing can search for aliens, why not web pages?) is about the Majestic-12 distributed search engine which has notched up some pretty impressive statistics. Inspired by the dominance of the big three search engines (Google, Yahoo and MSN), the UK-based project hopes to put search back into the hands of the internet community. If you’d like to test out Majestic-12’s fast search of the new one billion page index, go here. Curious how it all works? See a more detailed explanation of the technology.
That was a good timely article for me Michael.
I can’t give away details at the moment, but tackling comapnies such as Google purely on size will be difficult. But Search as a concept on the web is only 10% complete (e.g. delicious is an interesting example of a “parallel distributed indexer and search engine” that uses people and is about quality).
regards,
steven