-----------------------------------------------------------------------
# THEKLA - Web spider for web application pentesting
# (c) 2009- Volker Tanger <volker.tanger@wyae.de>
#
# http://www.wyae.de/software/thekla/
# All rights reserved. Distributable under "Modified BSD" license 
-----------------------------------------------------------------------

Changes:

1x.y.x	MAJOR TODO: rewrite pattern matching to generic (type, matchstring)
	IMPORTANT todo: feature: identify repeat content via SHA1(content)
	todo: feature: Auto-Cookie (auch: je Domain)
	todo: bugfix: print times > 24h - 1d
	todo: feature: match InnerHtml
	todo: feature: find known JS toolkits (top 37 ajax toolkits)
	todo: feature: find webstate/viewstate? PUT-Method (REST)
	todo: feature: what other identifiers for reloading
	todo: scanner (similar to Burp):
		- Passwort mit Autocomplete
		- match PHPINFO:  allow_url_fopen allow_url_include
		- find SWFs 

12.8.29	bugfix: added <base href=...> processing for JS and forms, too
	bugfix: corrected PHPSESSID regex

12.8.28	feature: added <base href=...> processing
	bugfix: made phpsess global variable

12.3.28	feature: manual Cookie support

11.11.25 feature "cut after hash" for URLs
	feature: decoding "&amp;" in GET parameter to "&" 
	feature: inhibit multiple-URL-encoding (i.e. the dreaded %252525...)
	feature: added sorted "logurls" file listing all visited URLS
	
11.2.9	feature: added excluded file types
	feature: saving HTTP errors into errors file, too
	feature: writes unsorted files during read, re-writes sorted
		 list after successful completion
	feature: print visited URLs sorted into file
	feature: find/visit frame/iframe targets
	feature: added PHPSESSID keeping for URL-based session-ID
	bugfix: adjusted a number of RegEx-matches	

11.1.7	bugfix: print times > 24h
	feature: reads HTML 30x redirects
	feature: finds AJAX requests (requests only, not targets)
	feature: reads JS files   <script ... src="URL"...>
	feature: list external JS loads

11.1.6	bugfix: added except-handling for strange download bugs
	feature: changed log file format / output
	feature: find and log error messages, lorem ipsum, admin
	         interfaces, email addresses, ...

10.3.1	feature: added time run and remaining time

10.1.26	library change: from urllib to urllib2 (urllib is deprecated)
		unfortunately lost proxy support (until next update)
	feature: shortened (value-free) param-urls list
	bugfix: first forms were listed without actions - fixed

9.6.11	bugfix: muliple URLs only worked for links, but not in getForms()

9.6.4	feature: added muliple allowed and forbidden URLs

9.5.25	feature: added proxy support

9.5.19	first published version


-----------------------------------------------------------------------