Didier Stevens

Saturday 31 December 2022

Combining zipdump, file-magic And myjson-filter

Filed under: maldoc,Malware — Didier Stevens @ 9:38

In this blog post, I show how you can combine my tools zipdump.py, file-magic.py and myjson-filter.py to select and analyze files of a particular type.

I start with a daily batch of malware files published by Malware Bazaar.

I let it produce JSON output using option –jsonoutput, that can be consumed by some of my tools, like file-magic.py, my tool to identify files based on the content using the libmagic library.

In the output above, we can see that most files are PE files (Windows executables).

For this example, I’m interested in Office files (ole files). I can filter the output of file-magic.py for that with option -r. Libmagic identifies this type of file as “Composite Document File …”, thus I filter for Composite:

This gives me a list of malicious Office documents. I want to extract URLs from them, but I don’t want to extract all of these files from the ZIP container to disk, and do the URL extraction file per file.

I want to do this with a one-liner. 🙂

What I’m going to do, is use file-magic’s option –jsonoutput, so that it augments the json output of zipdump with the file type, and then I use my tool myjson-filter.py to filter that json output for files that are only of a type that contains the word Composite. With this command:

This produces JSON output that contains the content of each file of type Composite, found inside the ZIP container.

This output can be consumed by my tool strings.py, to extract all the strings.

Side note: if you want to know first which files were selected for processing, use option -l:

Let’s pipe the filtered JSON output into strings.py, with options to produce a list of unique strings (-u) that contain the word http (-s http), like this:

I use my tool re-search.py to extract a list of unique URLs:

I filter out common URLs found in Office documents:

And finally, I sort the URLs by domain name using my tool sortcanon.py:

The adobe URLs are not malicious, but the other ones could be.

This one-liner allows me to quickly process daily malware batches, looking for easy IOCs (cleartext URLs in Office documents) without writing any malicious file to disk.

zipdump.py --jsonoutput 2020-10-24.zip | file-magic.py --jsoninput --jsonoutput | myjson-filter.py -t Composite | strings.py --jsoninput -u -s http | re-search.py -u -n url -F officeurls | sortcanon.py -c domain

Remark that by using an option to search for strings with the word http (-s http), I reduce the output of strings to be processed by re-search.py, so that the search is faster. But that limits you (mostly) to URLs with protocol http or https.

Leave out this option if you want to search for all possible protocols, or try -s “://”.

Thursday 29 December 2022

Update: zipdump.py Version 0.0.24

Filed under: My Software,Update — Didier Stevens @ 0:00

A small update to option -W of zipdump.py.

Next to value vir, you can now also specify values hash and hashvir.

hash: write each file with name equal to the SHA256 of the content of the file.

hashvir: write each file with name equal to the SHA256 of the content of the file plus extension .vir.

zipdump_v0_0_24.zip (http)
MD5: 33E7B7602263CB2C23D59C7EDEC8666C
SHA256: 1BEF40A9B567DAE84563FEA1B4DE8E0BD7F5926F7FCFF6D7086D2643133FBACE

Wednesday 28 December 2022

Powerstrip With Neon Lamp Switch

Filed under: Elec,Hardware — Didier Stevens @ 0:00

There are powerstrips with a switch that lights up when the switch is turned on. Like this one:

These switches (certainly older models) often use a neon lamp as light source.

I measured the electric energy consumption of a powerstrip with switch on and neon lamp burning (without anything plugged into the powerstrip’s outlets).

It consumed 7,8582 Wh over 24 hours, thus it drew on average 0,327 W.

That’s about 6 times more than the standby power of my Apple USB charger A2347 (0,0530 W).

FYI: although the switch is turned on in the above picture, you don’t see the neon lamp burning.

That’s because of the AC power here in Belgium is 230V and 50Hz.

50Hz means that the current is 0 A 100 times per second, and thus the neon lamp does not light up around these 0 A current values.

So the picture above was taken at a moment that the lamp wasn’t lighting up because the current was (almost) 0 A.

I will go into more details in an upcoming blog post.

I did not conduct tests with powerstrips that use LEDs in stead of neon lamps yet, because all the powerstrips with LEDs I have, also have a builtin USB charger, and that draws power too.

Tuesday 27 December 2022

Combining dns-pydivert And dnsresolver

Filed under: Malware,My Software,Networking — Didier Stevens @ 0:00

I use my tools dns-pydivert and dnsresolver.py for dynamic analysis of software (malware and benign software).

On the virtual machine where I’m doing dynamic analysis, I disable IPv6 support.

I install dnslib and run dnsresolver.py with a command like this, for example:

dnsresolver.py "type=resolve,label=example.com,answer=. 1 IN A" "type=forwarder,server="

The first command is a resolve command: DNS A queries for example.com will be resolved to IPv4 address with TTL 1 minute.

The second command is a forwarder command: all DNS requests not handled by other commands, are forwarded to Make sure that the IPv4 address of the DNS server you forward requests to, is different from the VM’s default DNS server, otherwise this forwarding will be redirected by dns-pydivert too.

I don’t use this second resolver command if the VM is isolated from the Internet, I only use it when I want to allow some interaction with the Internet.

Then I install pydivert and run dns-pydivert.py as administrator.

You can’t run dns-pydivert.py properly without administrative permissions:

When dns-pydivert.py and dnsresolver.py are running, DNS traffic is altered according to our settings.

For example (picture above), when I issue a “ping google.com” command inside the VM, dns-pydivert sees this first DNS packet and configures itself with the addresses in this packet: is the IPv4 address of the Windows VM and is the IPv4 address of this Windows VM’s DNS server.

It alters this first request to be redirected to the VM itself ( ->

Then dnsresolver receives this packet, and forwards it to DNS server It receives a reply from DNS server, and forwards it to the Windows VM (

Then dns-pydivert sees this reply, and changes its source from to, so that it appears to come from the Windows VM’s default DNS server.

When I do the same (picture above) for example.com (ping example.com), the query is redirected to dnsresolver, which resolves this to with a TTL of 1 minute (per resolve commands configuration).

Thus the ping command pings the localhost, instead of example.com’s web server.

And when I kill dns-pydivert (picture above) and issue a “ping example.com” again after waiting for 1 minute, the query is no longer redirected and example.com’s web server is pinged this time.

I used ping here to illustrate the process, but often it’s HTTP(S) traffic that I want to redirect, and then I also use my simple-listener.py tool to emulate simple web servers.

Remark that this will only redirect DNS traffic (per the configuration). This does not redirect traffic “directed” at IPv4 addresses (as opposed to hostnames).

This can be done too with pydivert, and I will probably release a tool for that too.

Monday 26 December 2022

New Tool: dns-pydivert.py

Filed under: Announcement,My Software — Didier Stevens @ 0:00

dns-pydivert is a tool that uses WinDivert, a “user-mode packet capture-and-divert package for Windows” to divert IPv4 DNS packets to and from the machine it is running on.

This tool requires admin rights.

When started, it listens for IPv4 UDP packets with source and/or destination port equal to 53.
When this tools processes its first UDP packet with destination port 53, it considers the source address of this packet as the DNS client’s IPv4 address (e.g., the Windows machine this tool is running on) and the destination address to be the IPv4 address of the DNS server used by the client.
From then on, all IPv4 UDP packets with source or destination port 53 (including that first packet) are altered by the tool.
All IPv4 UDP packets with destination port 53, have their destination address changed to the IPv4 address of the client.
All IPv4 UDP packets with source port 53, have their source address changed to the IPv4 address of the DNS server.

This tool can be used to redirect all DNS IPv4 traffic to the machine itself, where a tool like dnsresolver.py can handle the DNS requests.


  • This tool does not handle IPv6.
  • This tool does not check if the UDP packets to and/or from port 53 are actual DNS packets.
  • This tool ignores DNS traffic over TCP.
  • This tool does not handle queries to multiple DNS servers (different IPv4 addresses) correctly.
dns-pydivert_V0_0_1.zip (http)
MD5: BEAB8F9D180E15B27EB86CBEF7429216
SHA256: 7CB4BA7A4ABC0788AB8CE3F2DD1006DF86AD5D80943A4716FC3E62F1FA2100F6

Sunday 25 December 2022

Update: dnsresolver.py Version 0.0.2

Filed under: My Software,Update — Didier Stevens @ 0:00

This update to dnsresolver.py, my custom DNS server, adds a command to forward DNS request.

With this forward command, all requests that are not handled by other commands, are forwarded to the provided DNS server.

dnsresolver_V0_0_2.zip (http)
MD5: D96EA9517E106C4C9E3668AB6799B150
SHA256: 611C1540FE7FA2016E38689A153681428BBF3EAFC927A62342310A93022B3EC4

Saturday 24 December 2022

Update: myjson-filter.py Version 0.0.3

Filed under: My Software,Update — Didier Stevens @ 0:00

This update of myjson-filter.py adds an option (-t) to filter on the magic field added by file-magic.py.

To be explained in an upcoming blog post.

myjson-filter_V0_0_3.zip (http)
MD5: AB8AF505B120D02AD1A9846A72A340B5
SHA256: AB73314ACCD65EC765D6DDA629AF273FF882D293F11F6A2EA8FC633B019E5836

Friday 23 December 2022

Update: file-magic.py Version 0.0.5

Filed under: My Software,Update — Didier Stevens @ 0:00

This update of file-magic.py brings option –jsonoutput to augment json input data with a magic field.

To be explained in an upcoming blog post after myjson-filter.py update is released.

file-magic_V0_0_5.zip (http)
MD5: 5B4CB4EE75E1CAC7705E33CCE4809E10
SHA256: 876F9AC31E1EC395EB93922AA2A7EFA027534F7343500648FE0A036021C7F1B9

Thursday 22 December 2022

Update: nsrl.py Version 0.0.4

Filed under: My Software,Update — Didier Stevens @ 0:00

This is a bug fix version for my nsrl.py script, a tool to check hashes with the NSRL list.

nsrl_V0_0_4.zip (http)
MD5: 6F72B03493C73E88CB3771C860BC76D0
SHA256: D68039B8654C1D52CD1C12670C7E885E462B72BF23892E86BE86E6381C95B669

Wednesday 21 December 2022

Update: InteractiveSieve Version

Filed under: My Software,Update — Didier Stevens @ 0:00

New features:

  • Loading files from command line arguments
  • Column index to right click-menu
  • “Hide (if equal to prev and next)” to right click-menu
  • “Values separator…” to right click-menu
  • “Hide duplicates” to right click-menu
  • Added column filtering when loading files

InteractiveSieve_V_0_9_2_0.zip (http)
MD5: 74A4019A36199C5057207184341FB639
SHA256: D9481C99F44FCEA0729F526B70E307881E2128FB1EB23DF135790EDB4392CB4A
Next Page »

Blog at WordPress.com.