A colleague had to create a list of email addresses from a list of names (given name + surname). Some of the names had letters with accents: these accents had to be removed to keep the basic letter, in order to form a list of email addresses. For example, “andré” had to be converted to “andre”.
I found the Python module Unicode, and told my colleague he could use that module together with my python-per-line.py to generate his list. It turned out I had to make a change to my python-per-line.py tool first, so that it would handle Unicode input properly.
It works as follows. Take this Unicode text file:
Using unidecode method unidecode with python-per-line.py is done like this:
Remark that “é” has been converted to “e”.
Here is a list of names:
And here is the command to convert this list to email addresses:
Remark that personal names might be more complex than the simple case of “given name + surname”, and that the Python expression might have to be adapted accordingly.
This new version of python-per-line.py, a utility to execute a Python expression for every line in its input text files(s), adds option –encoding to handle encodings like Unicode (Python 3.7 required).
This new version of hex-to-bin.py, a tool to convert hexadecimal data to binary data, has a new option to ignore al characters/bytes that are not hexadecimal digits: -H –hexonly.
This option can be used to parse obfuscated, hexadecimal dumps of PE files, for example:
And there are also options if you want to take only lowercase hexadecimal digits into account (–loweronly) or uppercase hexadecimal digits (–upperonly).
With version 0.0.16 (we are now at version 0.0.18), I updated my zipdump.py tool to handle (deliberately) malformed ZIP files. My zipdump tool uses Python’s ZIP module to analyze ZIP files.
Now, zipdump has a an option (-f) to scan arbitrary binary files for ZIP records.
I will show here how this feature can be used, by analyzing a sample Xavier Mertens wrote a diary entry about. This sample is a Word document with macros, an OOXML (Office Open XML format) file (.docm). It is malformed, because 1) there’s an extra byte at the beginning and 2) there’s a byte missing at the end.
When you use my zipdump tool to look at the file, you get an error:
Using option -f l (list), we can find all PKZIP records inside arbitrary, binary files:
When using option -f with value l, a listing will be created of all PKZIP records found in the file, plus extra data. Some of these entries in this report will have an index, that can be used to select the entry.
In this example, 2 entries can be selected:
p: extra bytes at the beginning of the file (prefix)
1: an end-of-central-directory record (PK0506 end)
Using option -f p, we can select the prefix (extra data at the beginning of the file) for further analysis:
And from this hex/ascii dump, we learn that there is one extra byte at the beginning of the ZIP file, and that it is a newline characters (0x0A).
Using option -f 1, we can select the EOCD record to analyze the ZIP file:
As this generates an error, we need to take a closer look at the EOCD record by adding option -i (info):
With this info, we understand that the missing byte makes that the comment length field is one byte short, and this causes the error seen in previous image.
ZIP files can contain comments (for the ZIP container, and also for individual files): these are stored at the end of the PKZIP records, preceded by a 2-byte long, little-endian integer. This integer is the length of the comment. If there is no comment, this integer is zero (0x00).
Hence, the byte we are missing here is a NULL (0x00) byte. We can append a NULL byte to the sample, and then we should be able to analyze the ZIP file. In stead of modifying the sample, I use my tool cut-bytes.py to add a single NULL byte to the file (suffix option: -s #h#00) and then pipe this into zipdump:
File 5 (vbaProject.bin) contains the VBA macros, and can be piped into oledump.py:
A small change in this new version of XORSearch: option -n now also takes a negative value (output characters left of keyword) or an explicit positive value (output characters right of keyword).
Didier Stevens 