Didier Stevens

sortcanon.py is a tool to sort text files according to some canonicalization function. For example, sorting domains or ipv4 addresses.

This is actually an old tool, that I still had to publish. I just updated it to Python 3.

This is the man page:

Usage: sortcanon.py [options] [files]
Sort with canonicalization function

Arguments:
@file: process each file listed in the text file specified
wildcards are supported

Valid Canonicalization function names:
 domain: lambda x: '.'.join(x.split('.')[::-1])
 ipv4: lambda x: [int(n) for n in x.split('.')]
 length: lambda x: len(x)

Source code put in the public domain by Didier Stevens, no Copyright
Use at your own risk
https://DidierStevens.com

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -m, --man             Print manual
  -c CANONICALIZE, --canonicalize=CANONICALIZE
                        Canonicalization function
  -r, --reverse         Reverse sort
  -u, --unique          Make unique list
  -o OUTPUT, --output=OUTPUT
                        Output file

Manual:

sortcanon is a tool to sort the content of text files according to some
canonicalization function.
The tool takes input from stdin or one or more text files provided as argument.
All lines from the different input files are put together and sorted.

If no option is used to select a particular type of sorting, then normal
alphabetical sorting is applied.

Use option -o to write the output to the given file, in stead of stdout.

Use option -r to reverse the sort order.

Use option -u to produce a list of unique lines: remove all doubles before
sorting.

Option -c can be used to select a particular type of sorting.
For the moment, 2 options are provided:

domain: interpret the content of the text files as domain names, and sort them
first by TLD, then domain, then subdomain, and so on ...

length: sort the lines by line length. The longest lines will be printed out
last.

ipv4: sort IPv4 addresses.

You can also provide your own Python lambda function to canonicalize each line
for sorting.
Remark that this involves the use of the Python eval function: do only use this
with trusted input.

dns-query-async.py is a tool to perform DNS queries in parallel.

This is the man page:

Usage: dns-query-async.py [options] command file
Program to perform asynchronous DNS queries

accepted commands: gethost,getaddr

Source code put in the public domain by Didier Stevens, no Copyright
Use at your own risk
https://DidierStevens.com

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -m, --man             Print manual
  -o OUTPUT, --output=OUTPUT
                        Output to file (# supported)
  -s NAMESERVERS, --nameservers=NAMESERVERS
                        List of nameservers (,-separated)
  -n NUMBER, --number=NUMBER
                        Number of simultaneous requests (default 10000)
  -t TRANSFORM, --transform=TRANSFORM
                        Transform input (%%)

Manual:

This tool performs asynchronous DNS queries. By default, it will perform 10000
queries simultaneously.

The first argument is a command. There are 2 commands for the moment: gethost
and getaddr
The second argument is a filename: a text file containing the items to resolve.

Use command getaddr to lookup the IP address of the hostnames provided in the
input file.
Example:
 dns-query-async.py getaddr names.txt
Result:
 didierstevens.com,1,96.126.103.196
 didierstevenslabs.com,1,96.126.103.196
 Duration: 0.20s

Use command gethost to lookup the hostnames of the IP addresses provided in the
input file.
Example:
 dns-query-async.py gethost ips.txt

Use option -s to provide the name servers to use (comma separated list).

Use option -n to change the number of asyncio workers (10000 default).

Use option -t to transform the input list and perform lookups.
For example, take list of subdomains/hostnames https://github.com/m0nad/DNS-
Discovery/blob/master/wordlist.wl
Issue the following command:
 dns-query-async.py -t %%.example.com getaddr wordlist.wl
Result:
 0.example.com,0,Domain name not found
 009b.example.com,0,Domain name not found
 01.example.com,0,Domain name not found
 02.example.com,0,Domain name not found
 03.example.com,0,Domain name not found
 1.example.com,0,Domain name not found
 10.example.com,0,Domain name not found
 101a.example.com,0,Domain name not found

The %% in %%.example.com is replaced by each hostname/subdomain in wordlist.wl
and then resolved.

Use option -o to write the output to a file.

Here is a new tool I’m releasing as beta: pngdump.py.

It’s a tool to analyze PNG files. Unlike jpegdump, you can not yet select items for further analysis.

A couple of my tools can produce JSON output, using my own format (myjson).

This output can then be piped into another tool, like strings.py or file-magic.py.

I’m now releasing a tool that can be put into a command pipe to filter the JSON data: myjson-filter.py

For example, here I use myjson-filter.py to remove all items that are XML files (based on the content: starting with <?xml) before strings are extracted with strings.py:

More info in this ISC diary entry I wrote: “Method For String Extraction Filtering“.