LinkkURI is a Perl script that can be used to find non-working links from a web site. You may download, use, distribute and modify the program under the terms of the MIT Licence.
Here are some examples on the use of LinkkURI:
LinkkURI gave the following report on the early version of this
web page (it was then on different server):
The numbers at the beginning of lines are HTTP response codes. Only the RedHat archive was unavailable at the time of the test. The form of the report is purposefully simple to make it easy to manipulate with the standard unix text tools, like grep, sed or perl.
LinkkURI utilizes powerful Perl modules. They take care of all technical details, including URL parsing and redirections.
WARNING: Be careful with the website crawl (-c and -V options). You don't want to crawl too much, BOFH might get nervous... Try to ensure that your crawler stays on one site (it should, but people are creative in these kinds of things...). Don't try to crawl (i.e. download) somebody else's site. To be on the safe side, the number of GET-requests used to get all links from a page is limited to 100. You can increase the limit by editing the source code, if necessary. The number of URLs to be checked (HEAD-requests) is unlimited.
This program tries to be environmentally friendly, though: pages are fetched only once. If only header is needed, then only header is requested. Before downloading a page for links linkkuri checks the header for text/html content type. The program obeys the Robot Exclusion Protocol by default. The web host is contacted at most once per minute, be patient. Set your email address to the beginning of the source code, as required by the protocol.
To use LinkkURI you need to have Perl installed. In addition you need the following packages:
You can find the above mentioned packages from CPAN.
If you use Linux with RPM system you can
download the packages from
ftp.redhat.com (mirror at
ftp.funet.fi). Install them with command
rpm -ivh package-name.rpm.
Download version 0.01.02 and save it as linkkuri. You must give it execute-privileges: chmod a+x linkkuri Check that the first line of the file points to your perl interpreter (it is /usr/bin/perl by default). Read the licence before downloading the file. Note that the program is at very early stage of development (meaning: it is just a quick hack with a stupid name).
URL: http://www.iki.fi/kaip/linkkuri.htmlCopyright © 1999 Kai Puolamäki (Kai.Puolamaki@iki.fi)