CeWL - Custom Word List Generator

CeWL - Custom Word List Generator

CeWL (pronounced "cool") is a ruby app which spiders a given URL to a specified depth, optionally following external links, and returns a list of words which can then be used for password crackers such as John the Ripper.

It also has an associated command line app, FAB (Files Already Bagged) which uses the same metadata extraction techniques to create author/creator lists from already downloaded.


CeWL needs the RubyGems package to be installed along with the following gems:
  • nokogiri
  • mime-types
  • mini_exiftool
  • rubyzip
  • spider

These can be installed by running
bundle install
from the CeWL directory. The mini_exiftool gem also requires the exiftool application to be installed.


cewl [OPTION] ... URL
	--help, -h: show help
	--keep, -k: keep the downloaded file
	--depth x, -d x: depth to spider to, default 2
	--min_word_length, -m: minimum word length, default 3
	--offsite, -o: let the spider visit other sites
	--write, -w file: write the output to the file
	--ua, -u user-agent: user agent to send
	--no-words, -n: don't output the wordlist
	--meta, -a include meta data
	--meta_file file: output file for meta data
	--email, -e include email addresses
	--email_file file: output file for email addresses
	--meta-temp-dir directory: the temporary directory used by exiftool when 
                                   parsing files, default /tmp
	--count, -c: show the count for each word found

		--auth_type: digest or basic
		--auth_user: authentication username
		--auth_pass: authentication password

	Proxy Support
		--proxy_host: proxy host
		--proxy_port: proxy port, default 8080
		--proxy_username: username for proxy, if required
		--proxy_password: password for proxy, if required

		--header, -H: in format name:value - can pass multiple

	--verbose, -v: verbose

	URL: The site to spider.

No comments

Powered by Blogger.