Didier Stevens

MyJSON Tools

I have several tools that can produce and consume data in a JSON format I informally call MyJSON.

Let’s illustrate with a simple example. I created a ZIP file with 2 small files inside it: a text file and a png file.

Analyzing this with zipdump.py gives the following output:

zipdump.py is one of my tools that can also produce JSON output that contains the files contained inside the analyzed ZIP file. This is done via option –jsonoutput:

Let’s pretty print this with jq:

It’s easy to see now, that the dictionary contains an entry “items”, which is a list. That list contains dictionaries, one for each item (in this example, an item is a file contained inside the ZIP file). Each dictionary contains an “id” entry (here, the number generated by zipdump), a “name” entry (here, the name of the contained file) and a “content” entry. That content entry contains the BASE64 encoded content of the item (here, the content of the contained file).

This JSON output can be piped into other tools that can accept this JSON format. file-magic.py is one of these tools. By default, it operates on files, but by using option –jsoninput, it will consume JSON data from stdin:

The result is the identification of each contained file inside the ZIP file, based on its content.

file-magic.py can also produce JSON output:

Notice that there is an extra field now for each item: “magic”. This field contains the type detected by file-magic.py.

This JSON output can be filtered with a tool like myjson-filter.py (this is a tool that consumes JSON data from stdin by default):

In this case, the JSON output is the same as the JSON input, as myjson-filter.py wasn’t given any options to filter the JSON data. Let’s do that now: let’s filter for PNG files with option -t:

As can be seen, the JSON output now only contains one item: the png image.

If desired, myjson-filter.py can also write items to disk. For example, let’s write all PNG files to disk with name equal to the sha256 hash of their content and extension .png:

This little example illustrates how my tools that support MyJSON data can work together.

The tools than can produce MyJSON output (option –jsonoutput) to stdout are:

The tools than can accept MyJSON input (option –jsoninput) from stdin are:

The tools than only accept MyJSON input from stdin are:

And if you want to write your own program that can process MyJSON data, my Python program template for binary files process-binary-files.py also supports this format.

Blog at WordPress.com.