I have now merged these 2 tools, and added support for other protocols transported by IPv4 and IPv6. And I will no longer maintain simple_tcp_stats.py and simple_udp_stats.py.
This new tool simple_ip_stats.py is a Python program that reads pcap files and produces simple statistics for each IP connection per protocol.
For the moment, it calculates the entropy of the data (without packet reassembling) of each connection (both directions) and reports this in a CSV file:
XORSelection is a 010 Editor script I wrote some time ago, and it is included in the 010 Editor script repository. You provided it with an XOR key (ASCII or HEX), and then it will XOR-encode the file open in 010 Editor (or a selection of that file).
This new version brings options.
The input box of XORSelection can still be used to provide an ASCII key (e.g. Secret) or an HEX key (e.g. 0x536563726574), and the program will behave like before.
But if you provide no input and click OK, a second input box will be displayed, to input options (prior versions display an error message).
If you type h, you’ll get a simple help dialog, and then the program will terminate:
The options you can enter are r, l and/or s.
Use option r (reverse key) if the key has to be reversed before being used. Example: Secret -> terceS.
I introduced this option because I regularly need to use little-endian 4-byte XOR keys.
Use option l (literal key) if you need to use an ASCII key that starts with 0x (otherwise this key will be parsed as an HEX key).
Option s (shift) can be used to shift the key.
Here is an example to explain the shift option: assume the key is Secret, and that you want to decode a selection of an encrypted file. That selection will most likely not align with the key (e.g. the first byte of the selection was not encoded with the first byte of the key): lets assume you need to start decoding from the 3rd byte of the key: then you need a shift to the right of 2 positions, e.g. option s2. Remark that you can also shift to the left, then you use a negative integer, for example s-1.
A colleague had to create a list of email addresses from a list of names (given name + surname). Some of the names had letters with accents: these accents had to be removed to keep the basic letter, in order to form a list of email addresses. For example, “andré” had to be converted to “andre”.
I found the Python module Unicode, and told my colleague he could use that module together with my python-per-line.py to generate his list. It turned out I had to make a change to my python-per-line.py tool first, so that it would handle Unicode input properly.
It works as follows. Take this Unicode text file:
Using unidecode method unidecode with python-per-line.py is done like this:
Remark that “é” has been converted to “e”.
Here is a list of names:
And here is the command to convert this list to email addresses:
Remark that personal names might be more complex than the simple case of “given name + surname”, and that the Python expression might have to be adapted accordingly.
This new version of python-per-line.py, a utility to execute a Python expression for every line in its input text files(s), adds option –encoding to handle encodings like Unicode (Python 3.7 required).
This new version of hex-to-bin.py, a tool to convert hexadecimal data to binary data, has a new option to ignore al characters/bytes that are not hexadecimal digits: -H –hexonly.
This option can be used to parse obfuscated, hexadecimal dumps of PE files, for example:
And there are also options if you want to take only lowercase hexadecimal digits into account (–loweronly) or uppercase hexadecimal digits (–upperonly).
With version 0.0.16 (we are now at version 0.0.18), I updated my zipdump.py tool to handle (deliberately) malformed ZIP files. My zipdump tool uses Python’s ZIP module to analyze ZIP files.
Now, zipdump has a an option (-f) to scan arbitrary binary files for ZIP records.
I will show here how this feature can be used, by analyzing a sample Xavier Mertens wrote a diary entry about. This sample is a Word document with macros, an OOXML (Office Open XML format) file (.docm). It is malformed, because 1) there’s an extra byte at the beginning and 2) there’s a byte missing at the end.
When you use my zipdump tool to look at the file, you get an error:
Using option -f l (list), we can find all PKZIP records inside arbitrary, binary files:
When using option -f with value l, a listing will be created of all PKZIP records found in the file, plus extra data. Some of these entries in this report will have an index, that can be used to select the entry.
In this example, 2 entries can be selected:
p: extra bytes at the beginning of the file (prefix)
1: an end-of-central-directory record (PK0506 end)
Using option -f p, we can select the prefix (extra data at the beginning of the file) for further analysis:
And from this hex/ascii dump, we learn that there is one extra byte at the beginning of the ZIP file, and that it is a newline characters (0x0A).
Using option -f 1, we can select the EOCD record to analyze the ZIP file:
As this generates an error, we need to take a closer look at the EOCD record by adding option -i (info):
With this info, we understand that the missing byte makes that the comment length field is one byte short, and this causes the error seen in previous image.
ZIP files can contain comments (for the ZIP container, and also for individual files): these are stored at the end of the PKZIP records, preceded by a 2-byte long, little-endian integer. This integer is the length of the comment. If there is no comment, this integer is zero (0x00).
Hence, the byte we are missing here is a NULL (0x00) byte. We can append a NULL byte to the sample, and then we should be able to analyze the ZIP file. In stead of modifying the sample, I use my tool cut-bytes.py to add a single NULL byte to the file (suffix option: -s #h#00) and then pipe this into zipdump:
File 5 (vbaProject.bin) contains the VBA macros, and can be piped into oledump.py:
Didier Stevens 