GWebCrawler & Google Sitemap Creator
is a free source code web indexing engine running under the
MS Windows environment.
This project, is written in VB.NET. Program is currently using
only one thread to browse and index the site, but in my tests,
adding new threads didn't make big difference or make it any
faster.
Program has very high execution and
running speed and it is also very small in size and very
importantly it's released under the GPL Licence. As of now, this
project is not very mature yet, however, it’s an one-man-project
and that is why there’re still some bugs and missing features.
If you have any question or suggestion
about this project, please drop me a mail (trytobreak @
gmail.com), I’ll love to hear from you. As well, if you have any
feature request, you can always do the demand, but there’s no
warranty that I’ll implement it.
I hope you enjoy GWebCrawler & Google Sitemap Creator as
much as I enjoyed coding it.
Here are the features
of GWebCrawler & Google Sitemap Creator:
- Include only if URL contains field - available
- Exclude URLs containing field -
available
- URL variables to parse out -
only one variable is supported as of now
- Save / Load Form's settings
- Save / Load Queue file
- Create Google Sitemap XML file
- See the Queue list while
processing
- Thread Priority can be set
while script is running
- Show details of processing as
how many unique urls were parse out, which one is being
processed...
- Shows current version in real
time
Requirements:
WINDOWS 98 / ME / 2000/ 2003 /
NT/ XP
At least 64mb of memory
This program is written in Microsoft .NET. Meaning, that
Microsoft Framework 1.1 or above is required to run it, so
requirements on hard drive space are as follows:
- 30mb of free hard drive space, if Framework not installed
- less than 1mb of free hard drive space if Framework
installed already
Download:
Download
GWebCrawler 1.7 beta:
Click Here -
72kb
(No installation
required. Just run webcrawler.exe)
If you'd like to help me out with this project, please email
me (trytobreak @ gmail.com) and I will email you the source
code.