Didier Stevens

Sunday 12 July 2020

Quickpost: curl

Filed under: Networking,Quickpost — Didier Stevens @ 0:00

Since I learned that Windows 10 has curl pre-installed now, I notice I use it more often.

Here are some quick notes, mainly for myself:


Using Tor:

curl –socks5-hostname http://didierstevens.com

Option socks5-hostname uses SOCKS5 protocol and does name resolution of the hostname via the SOCKS5 protocol (and not local DNS)


Removing the User-Agent header:

curl –header “User-Agent:” http://didierstevens.com

Option –header (-H) can also be used to remove a header: provide the header name with colon, provide no header value.


Using a custom User-Agent header (-A –user-agent):

curl –user-agent “Mozilla/5.0 DidierStevens” http://didierstevens.com


Saving received data:

curl –dump-header 01.headers –output 01.bin.vir –trace 01.trace –trace-time http://didierstevens.com

Option —dump-header (-D) saves the headers, option –output (-o) saves the body, –trace creates a trace file and –trace-time adds timestamps to the trace file.


Option to ignore certificate errors: -k –insecure


Putting it all together:

curl –socks5-hostname –user-agent “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.116 Safari/537.36” –insecure –dump-header 01.headers –output 01.data –trace 01.trace –trace-time https://didierstevens.com


Quickpost info

Saturday 30 May 2020

New Tool: simple_ip_stats.py

Filed under: My Software,Networking — Didier Stevens @ 9:50

Some time ago, I created a tool to calculate the entropy of TCP data for a colleague. And a bit later, he asked me for a tool for UDP.

I have now merged these 2 tools, and added support for other protocols transported by IPv4 and IPv6. And I will no longer maintain simple_tcp_stats.py and simple_udp_stats.py.

This new tool simple_ip_stats.py is a Python program that reads pcap files and produces simple statistics for each IP connection per protocol.

For the moment, it calculates the entropy of the data (without packet reassembling) of each connection (both directions) and reports this in a CSV file:

TCP;;’GET ‘;364;5.42858024035
TCP;;’GET ‘;426;5.46464090792

simple_ip_stats_V0_0_1.zip (https)
MD5: 0482F3667E4EE6444350D9B0A146F764
SHA256: 480DCF2C82030EF996A6C1C3FEFCAAB77C000EC72DECA91329298C9BCC578BAD

Monday 18 May 2020

Quickpost: curl And SSPI Proxy Authentication

Filed under: Networking,Quickpost — Didier Stevens @ 0:00

curl with SSPI feature supports integrated authentication to a proxy: you don’t need to provide credentials.

The command is the following:

curl –proxy proxyname:8080 –proxy-ntlm -U : https://www.didierstevens.com/index.html

This curl command uses a proxy (–proxy) and authenticates to the proxy (–proxy-ntlm) without providing explicit credentials (-U :).

curl will use an SSPI to perform integrated authentication to the proxy. This is explained on curl’s man page:

If you use a Windows SSPI-enabled curl binary and do either Negotiate or NTLM authentication then you can tell curl to select the user name and password from your environment by specifying a single colon with this option: “-U :”.

curl’s SSPI feature can also be used to authenticate to an internal IIS server.

Windows’ built-in curl version supports SSPI. You can use the version option to check if your version of curl supports SSPI:


Quickpost info

Monday 23 March 2020

Quickpost: User-Agent: Microsoft Office Excel 2014

Filed under: Networking,Quickpost — Didier Stevens @ 14:25

To start: there is no version 2014 of Microsoft Office.

That’s why I was intrigued when I saw User Agent String “Microsoft Office Excel 2014” appearing in Wireshark when I did some tests with Excel’s data importing features.

With Excel 2019, when I get data from a CSV file and provide an URL (in stead of a local filename) like this:

Excel will issue several OPTIONS and HEAD requests, with different User Agent Strings:

And finally, a GET request to download the file:

Xavier Mertens has mentioned User Agent String “Microsoft Office Excel 2014” in another context: “Microsoft Apps Diverted from Their Main Use“.

Quickpost info

Tuesday 28 January 2020

etl2pcapng: Support For Process IDs

Filed under: Forensics,Networking — Didier Stevens @ 0:00

You can start a packet capture on a vanilla Windows machine with command “netsh trace start capture=yes” (and end it with “netsh trace stop”).

This packet capture file, with extension .etl, can not be opened with Wireshark. Until recently, I used Microsoft’s Message Analyzer, but this tool is no longer supported and installation files have been removed from Microsoft’s site.

In comes etl2pcapng, a new open-source utility from Microsoft that converts an .etl file to .pcapng format:

Utility that converts an .etl file containing a Windows network packet capture into .pcapng format“.

I contributed to version 1.3.0 of etl2pcapng, by adding a comment containing the Process ID to each packet. etl files contain metadata (like the PID of the process associated with the network traffic) that got lost when translating to pcapng format. As the pcapng format has no option to store the PID for each packet, but it supports packet comments, I stored the PID inside packet comments:

Notice this warning by Microsoft:

The output pcapng file will have a comment on each packet indicating the PID of the current process when the packet was logged. WARNING: this is frequently not the same as the actual PID of the process which caused the packet to be sent or to which the packet was delivered, since the packet capture provider often runs in a DPC (which runs in an arbitrary process). The user should keep this in mind when using the PID information.

Monday 18 November 2019

Update: tcp-honeypot.py Version 0.0.7

Filed under: My Software,Networking,Update — Didier Stevens @ 0:00

This new version of tcp-honeypot.py, a simple TCP honeypot and listener, brings TCP_ECHO and option -f as new features.

TCP_ECHO can be used to send back any incoming data (echo). Like this:

dListeners = {4444: {THP_LOOP: 10,THP_ECHO: None,},}

TCP_ECHO also takes a function, which’s goal is to transform the incoming data and return it. Here is an example with a lambda function that converts all lowercase letters to uppercase:

dListeners = {4444: {THP_LOOP: 10,THP_ECHO: lambda x: x.upper(),},}

If persistence is required across function calls, a custom class can also be provide. This class has to implement a method with name Process (input: incoming data, output: transformed data). Consult the man page (option -m) for more details.

And option -f (format) can be used to change the output format of data.
Possible values are: repr, x, X, a, A, b, B
The default value (repr) output’s data on a single line using Python’s repr function.
a is an ASCII/HEX dump over several lines, A is an ASCII/HEX dump too, but with duplicate lines removed.
x is an HEX dump over several lines, X is an HEX dump without whitespace.
b is a BASE64 dump over several lines, B is a BASE64 without whitespace.



Sunday 20 October 2019

New Tool: simple_tcp_stats.py

Filed under: My Software,Networking — Didier Stevens @ 10:25

My new tool simple_tcp_stats.py is a Python program that reads pcap files and produces simple statistics for each TCP connection.

For the moment, it calculates the entropy of the data (without packet reassembling) of each TCP connection (both directions) and reports this with a CSV file:

ConnectionID;head;Size;Entropy;’GET ‘;364;5.42858024035;’GET ‘;426;5.46464090792;’HTTP’;3308;6.06151478505;’HTTP’;493;6.73520107812


simple_tcp_stats_V0_0_1.zip (https)
MD5: 606DB4208BBC5908D9F32A68DDF90AC6
SHA256: 68B275C58736AE450D23BEA82CC1592936E541E00726D8ED95F5CA8ACB02B7CE

Wednesday 2 October 2019

Shark Jack Capture File

Filed under: Hardware,Networking — Didier Stevens @ 0:00

I have a new toy: a “Shark Jack“. It’s a small device sold by Hak5 that performs a nmap scan (-sP) when plugged into a network port (that’s the default “payload”).

In this blog post, I’m sharing the network capture of a scan performed in this “test environment”:

The device (small black box, almost square) between the Shark Jack (SJ) and the router is my “Packet Squirrel”: a simple network capture device.

A couple of observations:

  1. The SJ was tested with its original firmware (1.0.0)
  2. The SJ will randomize its MAC address
  3. The SJ performs 2 full DHCP handshakes prior to the nmap scan
  4. The SJ listens on port 53 (tcp and udp) using dnsmasq (observed while scanning)

Example of different MAC addresses after before and after reboot:

root@shark:~# ifconfig
eth0 Link encap:Ethernet HWaddr 2E:AF:43:F2:3E:22
inet addr: Bcast: Mask:


root@shark:~# ifconfig
eth0 Link encap:Ethernet HWaddr 86:72:96:71:C3:3C
inet addr: Bcast: Mask:


And it can get quite hot while charging, as can be observed in this thermal image:

shark_jack_capture.zip (https)
MD5: 9E5C1187D64A6EC7284C06464E791F01
SHA256: 5153F5C7B559BEC1539B0395F97C5852064D7ED9309B837F11A9381EA6ED4C88

Wednesday 7 August 2019

Downloading Executables Over DNS: Capture Files

Filed under: maldoc,Networking — Didier Stevens @ 0:00

In my BruCON training “Malicious Documents For Red Teams” (October 2019), we will cover downloading of files over DNS. I Tweeted about downloading Mimikatz via DNS-over-HTTPS with an Excel sheet.

I’m not releasing the Python code to serve files via DNS, nor the VBA code to download files over DNS/DoH: this is reserved for the attendees of my training.

But here I am sharing capture files of the downloads via DNS, so that you can understand how traffic looks like, and how to detect it.

Capture files inside the ZIP container (password is infected):

  1. 1-dns-txt.pcap: downloading of files via DNS TXT records, EICAR file (binary, hexadecimal and BASE64 encoded) and Mimikatz.exe (BASE64 encoded)
  2. 2-DoH-txt.pcap: downloading of Mimikatz.exe via DNS TXT records via dns.google.com (Google’s DNS over HTTPS)
  3. 3-DoH-txt-domain-fronting.pcap: same as 2, but with domain fronting (www.google.com)
  4. 4-DoH-txt.pcapng: same as 2, but in a PCAPNG file with decryption keys
  5. 5-DoH-txt.pcapng: same as 4, but with shorter DNS TXT records (to help with decryption)

DNS_TXT_captures.zip (https)
MD5: 5DB5091B9B641E9B8DA0E29CE9870981
SHA256: 49858B8BBA851B86EAB2DB6C5F329C5B587A3B1C7EB1A1E6028BCFBCDF445ECC

Tuesday 30 July 2019

Quickpost: tcp-honeypot.py & Browser Tests

Filed under: My Software,Networking,Quickpost — Didier Stevens @ 0:00

tcp-honeypot.py is a Python program that allows you to define listeners using dictionaries: they listen on a given TCP port and process connections according to their configuration.

It started as a simple TCP honeypot, but now I use it too if I need a small network server.

For my quickpost “Quickpost: Browsers & Content-Disposition“, I needed a simple web server that would serve a page that I could fully control (headers & body).

I did this with tcp-honeypot. Dictionary dListeners (used by tcp-honeypot) defines the listeners: the keys are the TCP port numbers to listen on, and the values are dictionaries with configuration entries.

As I wanted to serve 3 different pages, I resorted to listen on 3 different ports (8080, 8081, 8082), each would serve a different page. Each dictionary for these listeners contains one entry with key THP_REPLY. Because each listener is very simple it listens for a connection and reads incoming data, discards it, and then sends its reply (regardless of input).

Here is the code to do this (file content-disposition-test.py):

#!/usr/bin/env python

__description__ = 'TCP honeypot configuration for Content-Disposition tests'
__author__ = 'Didier Stevens'
__version__ = '0.0.1'
__date__ = '2019/04/03'

Source code put in public domain by Didier Stevens, no Copyright
Use at your own risk

  2019/04/03: start


dListeners = {
    8080:    {THP_REPLY: TW_CRLF(['HTTP/1.1 200 OK', 'Content-Disposition: inline', '', 'Line 1', 'Line 2', 'Line 3'])},
    8081:    {THP_REPLY: TW_CRLF(['HTTP/1.1 200 OK', 'Content-Disposition: attachment', '', 'Line 1', 'Line 2', 'Line 3'])},
    8082:    {THP_REPLY: TW_CRLF(['HTTP/1.1 200 OK', 'Content-Disposition: attachment; filename="test.js"', '', 'Line 1', 'Line 2', 'Line 3'])},

THP_REPLY configures a listener to read incoming data when a TCP connection is established, then send a reply (the value of dictionary entry THP_ENTRY) and then close the connection. This value is a string: the HTTP message (start-line, headers and body) to be send to the browser. In stead of defining one long string with start-line, headers and body, separated with carriage return & newline (CR NL), I use convenience function TW_CRNL. When you call convenience function TW_CRNL (Terminate With CR NL) with a list of strings, it terminates each string with CR NL (\r\n) and concatenates all strings into one string, that is returned by the function.

To start the server with this configuration, I just have to run tcp-honeypot.py with content-disposition-test.py as argument.

There are other methods to do this, for example using a single port. I’ll describe these methods in an upcoming blog post.

Quickpost info

Next Page »

Blog at WordPress.com.