Wordlist is a Ruby library and CLI for reading, combining, mutating, and building wordlists, efficiently.
- Supports reading
.txtwordlists, and.gz,.bz2,.xz,.zip, and.7zcompressed wordlists. - Supports building wordlists from arbitrary text. Also supports
.gz,.bz2,.xz,.zip, and.7zcompression. - Provides an advanced lexer for parsing text into words.
- Can parse/skip digits, special characters, whole numbers, acronyms.
- Can normalize case, apostrophes, and acronyms.
- Supports wordlist operations for combining multiple wordlists together.
- Supports wordlist modify or mutating the words in the wordlist on the fly.
- Also provides a
wordlistcommand. - Fast-ish
Open a wordlist for reading:
wordlist = Wordlist.open("passwords.txt")Open a compressed wordlist for reading:
wordlist = Wordlist.open("rockyou.txt.gz")Enumerate through a wordlist:
wordlist.each do |word|
puts word
endCreate an in-memory list of literal words:
words = Wordlist::Words["foo", "bar", "baz"]Concat two wordlists together:
(wordlist1 + wordlist2).each do |word|
puts word
endUnion two wordlists together:
(wordlist1 | wordlist2).each do |word|
puts word
endSubtract one wordlist from the other:
(wordlist1 - wordlist2).each do |word|
puts word
endCombine every word from wordlist1 with the words from wordlist2:
(wordlist1 * wordlist2).each do |word|
puts word
endCombine the wordlist with itself multiple times:
(wordslist ** 3).each do |word|
puts word
endFilter out duplicates from multiple wordlists:
(wordlist1 + wordlist2 + wordlist3).uniq.each do |word|
puts word
endConvert every word in a wordlist to lowercase:
wordlist.downcase.each do |word|
puts word
endConvert every word in a wordlist to UPPERCASE:
wordlist.upcase.each do |word|
puts word
endCapitalize every word in a wordlist:
wordlist.capitalize.each do |word|
puts word
endRun String#tr on every word in a wordlist:
wordlist.tr('_','-').each do |word|
puts word
endRun String#sub on every word in a wordlist:
wordlist.sub("fish","phish").each do |word|
puts word
endRun String#gsub on every word in a wordlist:
wordlist.gsub(/\d+/,"").each do |word|
puts word
endPerforms every possible mutation of each word in a wordlist:
wordlist.mutate(/[oae]/, {'o' => '0', 'a' => '@', 'e' => '3'}).each do |word|
puts word
end
# dog
# d0g
# firefox
# fir3fox
# firef0x
# fir3f0x
# ...Enumerates over every possible case variation of every word in a wordlist:
wordlist.mutate_case.each do |word|
puts word
end
# cat
# Cat
# cAt
# caT
# CAt
# CaT
# cAT
# CAT
# ...Wordlist::Builder.open('path/to/file.txt.gz') do |builder|
# ...
endAdd individual words:
builder.add(word)Adding an Array of words:
builder.append(words)Parsing text:
builder.parse(text)Parsing a file's content:
builder.parse_file(path)- ruby >= 3.0.0
zcat/gzip(for reading/writing.gzwordlists)bzcat/bzip2(for reading/writing.bz2wordlists)xzcat/xz(for reading/writing.xzwordlists)unzip/zip(for reading/writing.zipwordlists)7za(for reading/writing.7zwordlists)
$ gem install wordlistgem.add_dependency 'wordlist', '~> 1.0'gem 'wordlist', '~> 1.0'usage: wordlist { [options] WORDLIST ... | --build WORDLIST [FILE ...] }
Wordlist Reading Options:
-f {txt|gzip|bz2|xz|zip|7zip}, Sets the desired wordlist format
--format
--exec COMMAND Runs the command with each word from the wordlist.
The string "{}" will be replaced with each word.
Wordlist Operations:
-U, --union WORDLIST Unions the wordlist with the other WORDLIST
-I, --intersect WORDLIST Intersects the wordlist with the other WORDLIST
-S, --subtract WORDLIST Subtracts the words from the WORDLIST
-p, --product WORDLIST Combines every word with the other words from WORDLIST
-P, --power NUM Combines every word with the other words from WORDLIST
-u, --unique Filters out duplicate words
Wordlist Modifiers:
-C, --capitalize Capitalize each word
--uppercase, --upcase Converts each word to UPPERCASE
--lowercase, --downcase Converts each word to lowercase
-t, --tr CHARS:REPLACE Translates the characters of each word
-s, --sub PATTERN:SUB Replaces PATTERN with SUB in each word
-g, --gsub PATTERN:SUB Replaces all PATTERNs with SUB in each word
-m, --mutate PATTERN:SUB Performs every possible substitution on each word
-M, --mutate-case Switches the case of each letter in each word
Wordlist Building Options:
-b, --build WORDLIST Builds a wordlist
-a, --[no-]append Appends to the new wordlist instead of overwriting it
-L, --lang LANG The language to expect
--stop-words WORDS... Ignores the stop words
--ignore-words WORDS... Ignore the words
--[no-]digits Allow digits in the middle of words
--special-chars CHARS Allows the given special characters inside of words
--[no-]numbers Parses whole numbers in addition to words
--[no-]acronyms Parses acronyms in addition to words
--[no-]normalize-case Converts all words to lowercase
--[no-]normalize-apostrophes Removes "'s" from words
--[no-]normalize-acronyms Removes the dots from acronyms
General Options:
-V, --version Print the version
-h, --help Print the help output
Examples:
wordlist rockyou.txt.gz
wordlist passwords_short.txt passwords_long.txt
wordlist sport_teams.txt -p beers.txt -p digits.txt
cat *.txt | wordlist --build custom.txt
Reading a wordlist:
$ wordlist rockyou.txt.gzReading multiple wordlists:
$ wordlist sport_teams.txt beers.txtCombining every word from one wordlist with another:
$ wordlist sport_teams.txt -p beers.txt -p all_four_digits.txt
coors0000
coors0001
coors0002
coors0003
...Combining every word from one wordlist with itself, N times:
$ wordlist words.txt -P 3Mutating every word in a wordlist:
$ wordlist passwords.txt -m o:0 -m e:3 -m a:@
dog
d0g
firefox
fir3fox
firef0x
fir3f0x
...Executing a command on each word in the wordlist:
$ wordlist directories.txt --exec "curl -X POST -F 'user=joe&password={}' -o /dev/null -w '%{http_code} {}' https://$TARGET/login"Building a wordlist from a directory of .txt files:
$ wordlist --build wordlist.txt dir/*.txtBuilding a wordlist from STDIN:
$ cat *.txt | wordlist --build wordlist.txt user system total real
Wordlist::Builder#parse_text (size=5.4M) 1.943605 0.003809 1.947414 ( 1.955960)
Wordlist::File#each (N=1000) 0.000544 0.000000 0.000544 ( 0.000559)
Wordlist::File#concat (N=1000) 0.001143 0.000000 0.001143 ( 0.001153)
Wordlist::File#subtract (N=1000) 0.001360 0.000000 0.001360 ( 0.001375)
Wordlist::File#product (N=1000) 0.536518 0.005959 0.542477 ( 0.545536)
Wordlist::File#power (N=1000) 0.000015 0.000001 0.000016 ( 0.000014)
Wordlist::File#intersect (N=1000) 0.001389 0.000000 0.001389 ( 0.001407)
Wordlist::File#union (N=1000) 0.001310 0.000000 0.001310 ( 0.001317)
Wordlist::File#uniq (N=1000) 0.000941 0.000000 0.000941 ( 0.000948)
Wordlist::File#tr (N=1000) 0.000725 0.000000 0.000725 ( 0.000736)
Wordlist::File#sub (N=1000) 0.000863 0.000000 0.000863 ( 0.000870)
Wordlist::File#gsub (N=1000) 0.001240 0.000000 0.001240 ( 0.001249)
Wordlist::File#capittalize (N=1000) 0.000821 0.000000 0.000821 ( 0.000828)
Wordlist::File#upcase (N=1000) 0.000760 0.000000 0.000760 ( 0.000769)
Wordlist::File#downcase (N=1000) 0.000544 0.000001 0.000545 ( 0.000545)
Wordlist::File#mutate (N=1000) 0.004656 0.000000 0.004656 ( 0.004692)
Wordlist::File#mutate_case (N=1000) 24.178521 0.000000 24.178521 ( 24.294962)
Copyright (c) 2009-2023 Hal Brodigan
See {file:LICENSE.txt} for details.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent