Crawler
My News Crawler is an 'aggregation' script.
More Info 
Developer: Detlef Horchler
License: GNU General Public License (GPL)
Operating System: unix, Win nt, xp
Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.
Developer: Heritrix
License: GNU General Public License (GPL)
Operating System: Independent
Free, full featured text search engine program for web sites, completely in Java.
Developer: Java Search Engine
License: Freeware
Operating System: Linux, Unix, Windows
This script sends an automatic email notification whenever your site is visited.
Developer: http://www.vascript.co.n...
License: Freeware
Operating System: Unix
Ever wish you could just open up a program and have it generate an index of all links on your site.
Developer: Hawk Roberts
License: Freeware
Operating System: All
Open Source Web Search Engine.
Developer: shen139
License: GNU General Public License (GPL)
Operating System: Win, UNIX
OpenSearchServer is built on Lucene and written in Java and it can be integrated into almost any kind of application without the need to produce Java code.
Developer: Jaeksoft SarL
License: GNU General Public License version 3.0 (GPLv3)
Operating System: Macintosh, Linux, Windows
Fluid Dynamics Search Engine is multi-platform compatible.
Developer: http://www.xav.com/scrip...
License: Freeware
Operating System: Unix/Linux/Windows
This tool is made up of three seperate tools.
Developer: B.D. Brown
License: Freeware
Operating System: Linux
Meta-tags are used frequently by most search engines.
Developer: Keyur Parmar
License: Freeware
Operating System: Linux
The script now is able to crawl/spider your website, create your sitemaps, ping Google, Yahoo, MSN, Ask.
Developer: Waleed GadElKareem
License: Freeware
Operating System: ALL