| Rank |
Spider |
Description |
Hits |
|
| 1 |
bright.net caching robot |
|
178 |
26.61% |
64 |
21.19% |
| 2 |
Googlebot |
Google's crawler |
134 |
20.03% |
62 |
20.53% |
| 3 |
Scooter |
Scooter is AltaVista's prime index agent |
85 |
12.71% |
29 |
09.60% |
| 4 |
Inktomi Slurp |
Indexing documents for the HotBot search engine (www.hotbot.com), collecting Web statistics |
42 |
06.28% |
22 |
07.28% |
| 5 |
WiseNut Robot |
WiseNut search engine robot |
32 |
04.78% |
24 |
07.95% |
| 6 |
Xenu's Link Sleuth |
|
25 |
03.74% |
10 |
03.31% |
| 7 |
Search4free Robot |
|
25 |
03.74% |
16 |
05.30% |
| 8 |
FAST-WebCrawler |
FAST |
14 |
02.09% |
7 |
02.32% |
| 9 |
Grub-client |
Distributed web crawler. People who choose to download and run the client will assist in building the Web's largest, most accurate database of URLs. |
14 |
02.09% |
11 |
03.64% |
| 10 |
Perl LWP |
|
13 |
01.94% |
13 |
04.30% |
| 11 |
Girafabot |
Spider used by www.girafa.com |
12 |
01.79% |
1 |
00.33% |
| 12 |
AskJeeves Robot |
AskJeeves search engine robot |
10 |
01.49% |
2 |
00.66% |
| 13 |
Unknown robot |
Unknown robot |
10 |
01.49% |
3 |
00.99% |
| 14 |
Openfind data gatherer |
|
8 |
01.20% |
2 |
00.66% |
| 15 |
GAIS Lab Robot |
Gaisbot is the agent software of GAIS which crawls web sites all over the world, in order to build a search engine |
8 |
01.20% |
2 |
00.66% |
| 16 |
Search.ch Robot |
Search.ch search engine robot |
7 |
01.05% |
2 |
00.66% |
| 17 |
CNET_Snoop Link Checker |
Software archive robot |
6 |
00.90% |
3 |
00.99% |
| 18 |
arks |
The Arks robot is used to build the database for the dpsindia/lawvistas.com search service |
6 |
00.90% |
3 |
00.99% |
| 19 |
PHP Pad File Parser |
Software archive robot |
5 |
00.75% |
2 |
00.66% |
| 20 |
SurveyBot |
Whois Source ctawler |
4 |
00.60% |
1 |
00.33% |
| 21 |
Overture Crawler |
Overture web crawler, now resolving to FAST Search & Transfer |
4 |
00.60% |
4 |
01.32% |
| 22 |
IBM Almaden Crawler |
Crawler from IBM, usage unknown |
3 |
00.45% |
1 |
00.33% |
| 23 |
WiseGuys Robot |
Web robot used to build search engines maintained by WiseGuys |
3 |
00.45% |
2 |
00.66% |
| 24 |
Walhello appie |
The appie-spider is used to collect and index web pages for the Walhello search engine |
2 |
00.30% |
1 |
00.33% |
| 25 |
larbin |
|
2 |
00.30% |
1 |
00.33% |
| 26 |
ia_archiver |
Alexa robot |
2 |
00.30% |
1 |
00.33% |
| 27 |
Fluid Dynamics Search Engine robot |
Crawls remote sites as part of a shareware search engine program |
2 |
00.30% |
1 |
00.33% |
| 28 |
MSIE Crawler |
Updating subscription content |
2 |
00.30% |
2 |
00.66% |
| 29 |
Pompos |
The goal of Pompos is to collect as many documents as possible for the dir.com search engine |
2 |
00.30% |
2 |
00.66% |
| 30 |
InternetSeer |
SiteCheck is InternetSeer's Web mapping and monitoring system |
2 |
00.30% |
2 |
00.66% |
| 31 |
Unitix PAD Spider |
Software archive robot |
2 |
00.30% |
2 |
00.66% |
| 32 |
Voila Robot |
Web spider used by the Voila search engine |
2 |
00.30% |
1 |
00.33% |
| 33 |
Robozilla |
Robozilla visits all the links within the Open Directory periodically, marking the ones that return errors for review |
1 |
00.15% |
1 |
00.33% |
| 34 |
Microsoft URL Control |
None |
1 |
00.15% |
1 |
00.33% |
| 35 |
Ingrid |
Indexing documents for the Ilse search engine (http://ilse.nl) |
1 |
00.15% |
1 |
00.33% |
| |
Total |
|
669 |
302 |
