Simple tools for using in conjunction with the LDSpider
-
stripURL.sh - Downloads a list of .au domains and sorts them into a text file
-
testForAU.sh - Tests domains to see if they are hosted in Australia, removes domains from file if not
-
crawl.sh - Crawls a URL seeking RDF links and stores them in a text file to be used as seeds for the LDSpider