Didier Stevens

Thursday 4 June 2015

Regular Expressions With Comments

Filed under: My Software — Didier Stevens @ 20:01

Many flavors of regular expressions support comments now. You can make your regular expression a bit more readable by adding comments. Like in programming languages, where a comment does not change the behavior of the program, a regular expression comment does not change the behavior of the regular expression.

A regular expression comment is written like this: (?#comment) where comment can be any text, as long it is not ).

Here is an example of a regular expression for a simple email address: [A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,6}

And here is the same regular expression with a comment (bold): [A-Z0-9._%+-]+@(?#domain)[A-Z0-9.-]+\.[A-Z]{2,6}

Why am I posting this? Because I’m using this in my new Snort rules I’m blogging about soon.

Monday 18 May 2015

Howto: Install Wireshark Dissectors

Filed under: My Software,Wireshark — Didier Stevens @ 0:00

I teach a Wireshark class at Brucon 2015.

If you want to use my Wireshark dissectors like TCP Flag dissector, but don’t know how to install a Wireshark dissector, then watch this video howto:

Monday 11 May 2015

Detecting Network Traffic from Metasploit’s Meterpreter Reverse HTTP Module

Filed under: Networking — Didier Stevens @ 5:52

I teach a Wireshark class at Brucon 2015.

I took a closer look at Metasploit’s Meterpreter network traffic when reverse http mode is used.

The Meterpreter client will make regular HTTP requests to the Metasploit server to check if it has commands ready to be executed. This is how a request looks like:



The client sends an HTTP POST request with a 4-byte payload: RECV. The URI has the following pattern: 4 or 5 alphanumeric characters, an underscore and 16 alphanumeric characters. The 16 alphanumeric characters are chosen at random, and the 4 or 5 alphanumeric characters are some kind of checksum.

I checked Meteasploit’s source code: these characteristics of Meterpreter’s Reverse HTTP protocol are hardcoded.

What is not hardcoded, but parametrized with a variable, is the User Agent String. By default, it is “Mozilla/4.0 (compatible; MSIE 6.1; Windows NT)”, but it is an option that can be changed.

I’ve tested the detection of Metasploit Meterpreter traffic with this User Agent String in several environments, and never encountered a false positive. You might think that “Mozilla/4.0 (compatible; MSIE 6.1; Windows NT)” is quite common as a User Agent String, but it is not. “MSIE 6.1” is pretty rare (according to Wikipedia, there is no Internet Explorer version 6.1), and “Windows NT” without version number is also rare. Combined, I’ve never seen this User Agent String except for Metasploit Meterpreter traffic. The only User Agent String seen in-the-wild that comes close to this one is “Mozilla/4.0 (compatible; MSIE 6.1; Windows XP)”. But I suggest that you check your environment for this Metasploit User Agent String if you want to be sure no false positives will be generated in your environment.

But like I said, the User Agent String is an option, and can be easily changed by the Metasploit operator. That’s why I also developed a method to detect Metasploit Meterpreter Reverse HTTP traffic looking for its hardcoded characteristics: a POST request with RECV payload to a recognizable URI.

Here is the Snort rule:

# Snort rules by Didier Stevens (http://DidierStevens.com)
# 2015/05/01 - 2015/05/10
# Thanks to Nathan Fowler for helping out with performance optimization
# I start numbering my rules at SID 1618000
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit Meterpreter"; flow:to_server,established; content:"RECV"; http_client_body; depth:4; fast_pattern; isdataat:!0,relative; urilen:23<>24,norm; content:"POST"; pcre:"/^\/[a-z0-9]{4,5}_[a-z0-9]{16}\/$/Ui"; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/05/11/detecting-network-traffic-from-metasploits-meterpreter-reverse-http-module/; sid:1618008; rev:1;)

This Snort rule looks for traffic from your internal network to the outside. You need to change the rule if you want to detect internal-only traffic.

Here is an example of an alert:

[**] [1:1618008:1] Metasploit Meterpreter [**]
[Classification: A Network Trojan was detected] [Priority: 1] 
05/11-22:26:31.236007 ->
TCP TTL:64 TOS:0x0 ID:21177 IpLen:20 DgmLen:212 DF
***A**** Seq: 0x1B677291  Ack: 0x861008DD  Win: 0x7680  TcpLen: 20
[Xref => http://blog.didierstevens.com/2015/05/11/detecting-network-traffic-from-metasploits-meterpreter-reverse-http-module/]

Based on the Metasploit User Agent Strings I published a couple of months ago, I made these Snort rules:

# Snort rules by Didier Stevens (http://DidierStevens.com)
# 2015/04/30
# I start numbering my rules at SID 1618000
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/4.0 (compatible\; MSIE 6.0\; Windows NT 5.1)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618000; rev:1;)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/4.0 (compatible\; MSIE 6.1\; Windows NT)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618001; rev:1;)
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/4.0 (compatible\; MSIE 7.0\; Windows NT 6.0)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618002; rev:1;)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/4.0 (compatible\; MSIE 7.0\; Windows NT 6.0\; Trident/4.0\; SIMBAR={7DB0F6DE-8DE7-4841-9084-28FA914B0F2E}\; SLCC1\; .N|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618003; rev:1;)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/4.0 (compatible\; Metasploit RSPEC)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618004; rev:1;)
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/5.0 (Windows\; U\; Windows NT 5.1\; en-US) AppleWebKit/525.13 (KHTML, like Gecko) Chrome/ Safari/525.13|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618005; rev:1;)
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/5.0 (compatible\; Googlebot/2.1\; +http://www.google.com/bot.html)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618006; rev:1;)
#alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Metasploit User Agent String"; flow:to_server,established; content:"User-Agent|3a| Mozilla/5.0 (compatible\; MSIE 10.0\; Windows NT 6.1\; Trident/6.0)|0d 0a|"; http_header; classtype:trojan-activity; reference:url,blog.didierstevens.com/2015/03/16/quickpost-metasploit-user-agent-strings/; sid:1618007; rev:1;)

Remark that I commented-out the Snort rules that I expect to generate too many false positives. But it’s best that you check what User Agent Strings are common in your environment, before you deploy and uncomment these rules.

Update: these rules are designed for an environment where egress traffic has to go to an HTTP port. If your environment allows all destination ports (like Metasploit’s default 4444 port), then replace $HTTP_PORTS by any in the rules you deploy. Thanks @securitygen for the remark.

snort-rules-V0_0_1.zip (https)
MD5: 526AAC1CE1E8576633498223DFA07E3D
SHA256: 7694E4E884E12068BC2A32714D3B0C48060B12C80E4093AFB6B1563E2EDA5E8D

Wednesday 6 May 2015

Update: NAFT Version 0.0.9

Filed under: Forensics,My Software,Networking,Update — Didier Stevens @ 13:55

This update to NAFT adds support for YARA. YARA rules can be used to search through the heap, like this:

naft-icd.py -y IOS_canary.yara –decoders decoder_xor1 heap r870-core

Address      Bytes     Prev     Next Ref     PrevF    NextF Alloc PC  what
83AB9498 0000004100 83AB9444 83ABA4CC 001  -------- -------- 80B5CC7C  8253709C
 YARA rule: IOS_canary

Rule IOS_canary.yara searches for a canary value inside the blocks.

rule IOS_canary
        $canary = {FD 01 10 DF}

NAFT_V0_0_9.zip (https)
MD5: FEBBDB892D631275A95A0FEA59F8519F
SHA256: 95F42F109623F2BA6D8A9FFB013CBB0B5E995F02E5EB35F8E83A62B8CA8B86D0

Wednesday 29 April 2015

pdf-parser: A Method To Manipulate PDFs Part 2

Filed under: My Software,PDF — Didier Stevens @ 0:00

I provide 2 days of Hacking PDF training at HITB Amsterdam. This is one of the methods I teach.

Maarten Van Horenbeeck posted a diary entry (July 2008) explaining how scripts and data are stored in PDF documents (using streams), and demonstrated a Perl script to decompress streams. A couple of months before, I had started developing my pdf-parser tool, and Maarten’s diary entry motivated me to continue adding features to pdf-parser.

Extracting and decompressing a stream (for example containing a JavaScript script) is easy with pdf-parser. You select the object that contains the stream (example object 5: -o 5) and you “filter” the content of the stream (-f ). The command is:

pdf-parser.py –o 5 –f sample.pdf

In PDF jargon, streams are compressed using filters. You have all kinds of filters, for example ZLIB DEFLATE, but also lossy compressions like JPEG. pdf-parser supports a couple of filters, but not all, because the implementation of some of them (mostly the lossy ones) differs between vendors and PDF applications.


A recent article published by Virus Bulletin on JavaScript stored inside a lossy stream gave me the opportunity to implement a method I had worked out manually.

The problem: you need to decompress a stream and you have no decompression algorithm.

The solution: you use the PDF application to decompress the stream.

The method: you create a new PDF document with the stream as embedded file, and then save the embedded file using the PDF application.

The detailed method: when you need to decompress a stream for which you have no decompressor (or no decompressor identical to the target application), you create a new PDF document into which you include the object with the stream as an embedded file. PDF documents support embedded files. For example, if you have a PDF document explaining a financial method, you can include a spreadsheet in the PDF document as an embedded file. The embedded file is stored as an object with a stream, and the compression can be any method supported by the PDF application. Crafting this PDF document with embedded file manually requires many manipulations and calculations, and is thus a very good candidate for automation.

Figure: this PDF embeds a file called vbanner2.jpg

With pdf-parser, you can use this method as follows:

  1. Create a Python program that generates the PDF document with embedded file. Use pdf-parser like this (in this example, the data stream you want to decompress is in object 5 of PDF file sample.pdf): pdf-parser.py –generateembedded 5 sample.pdf > embedded.py
  2. Execute the Python program to create the PDF file: embedded.py embedded.pdf
  3. Open the created PDF file embedded.pdf with the target application (Adobe Reader for the Virus Bulletin example), and save the embedded file to disk
  4. The saved file contains the decompressed stream

You can find my PDF tools here.

Remark: the generated Python program requires my module mPDF.py, which can also be found on my PDF tools page.

Remark 2: don’t use this method when the stream contains an exploit for the decompressor.

Monday 27 April 2015

Update: virustotal-search Version 0.1.2 Daily Quota Handling and CVEs

Filed under: My Software,Update — Didier Stevens @ 0:00

This new version op virustotal-search adds a bunch of options to manage the local database, and 2 features I want to highlight here:

1) If you exceed your daily quota, virustotal-search will now do a clean stop. You can use option -w (waitquota) to instruct virustotal-search to wait until your daily quota is reset, and then continue. The quota reset is tested by doing a query every hour.

2) A new column was added to the CSV output: CVEs. virustotal-search will extract CVE numbers from AV detection signatures and report them in column CVEs.

And I also worked together with VirusTotal so that you get a proper error message when you submit an invalid search request (for example MD5 hash prefixed with $).

virustotal-search_V0_1_2.zip (https)
MD5: 62C8031738E6E20FEC38337010496DF6
SHA256: 317AF862A62CF78FC58604EDB77AA3C00EC1543D2337EC634749C25CC5E4908C

Thursday 23 April 2015

MS15-034: PoC Excel Video

Filed under: Hacking,Vulnerabilities — Didier Stevens @ 19:31

Since I like to hack with Excel, I made a PoC for MS15-034 in VBA/Excel.

PS: If you want to see my videos as soon as they are published, subscribe to my video blog videos.DidierStevens.com or YouTube Channel.

Here’s the video:

Friday 17 April 2015

MS15-034 Detection: Some Observations

Filed under: Networking,Vulnerabilities — Didier Stevens @ 9:15

Several detection rules (SNORT, F5, …) are being published these days to detect exploitation of vulnerability MS15-034.

If you are making or modifying such detection rules, I want to share some observations with you.

MS15-034 can be exploited with a GET request with a specially crafted Range header.

Here is the example we’ll use: Range: bytes=2-18446744073709551615

Referring to RFC 2616 section 14.35.1, you can see that this is not the only way to specify a range. Here is the BNF:

ranges-specifier = byte-ranges-specifier
byte-ranges-specifier = bytes-unit “=” byte-range-set
byte-range-set  = 1#( byte-range-spec | suffix-byte-range-spec )
byte-range-spec = first-byte-pos “-” [last-byte-pos]
first-byte-pos  = 1*DIGIT
last-byte-pos   = 1*DIGIT

suffix-byte-range-spec = “-” suffix-length
suffix-length = 1*DIGIT

First of all, whitespace is allowed. So Range: bytes = 2 – 18446744073709551615 is valid (and also caused a BSOD on my test machine).

Second, numbers can have leading zeroes. So Range: bytes=2-018446744073709551615 is valid (and also caused a BSOD on my test machine).

Third, multiple ranges are allowed. So Range: bytes=2-3,4-18446744073709551615 is valid (this did not cause a BSOD on my test machine).

If you are using rules that don’t detect these cases properly, then attackers can easily evade detection. One space character could be all it takes to evade detection. If the rule looks for string “-18446744073709551615”, then using string “- 18446744073709551615” in the attack (extra space character added) will evade detection.

Thursday 16 April 2015

pdf-parser: A Method To Manipulate PDFs Part 1

Filed under: My Software,PDF,Update — Didier Stevens @ 0:00

I provide 2 days of Hacking PDF training at HITB Amsterdam. This is one of the methods I teach.

Sometimes when I analyze PDF documents (benign or malicious), I want to reduce the PDF to its essential objects. But when one removes objects in a PDF, indexes need to be updated and references updated/removed. To automate this process as much as possible, I updated my pdf-parser program to generate a Python program that in turn, generates the original PDF.

Thus when I want to make changes to the PDF (like removing objects), I generate its corresponding Python program, and then I edit this Python program.

I do this simply with option -g.


Then you can edit the Python program, and when you run it, it will generate a new PDF file.

You can also use option -g together with option -f to filter the streams before they are inserted in the Python program. This gives you the decompressed streams in the Python program, opening them up to editing.

In this example, without option -f the Python statement for the stream object is:

oPDF.stream(5, 0, 'x\x9cs\nQ\xd0w3T02Q\x08IS040P0\x07\xe2\x90\x14\x05\r\x8f\xd4\x9c\x9c|\x85\xf0\xfc\xa2\x9c\x14M\x85\x90,\x05\xd7\x10\x00\xdfn\x0b!', '<<\r\n /Length %d\r\n /Filter /FlateDecode\r\n>>')

And with option -f, it becomes:

oPDF.stream2(5, 0, 'BT /F1 24 Tf 100 700 Td (Hello World) Tj ET', '', 'f')

The generated Python program relies on my mPDF library found in my PDF make tools.

pdf-parser_V0_6_2.zip (https)
MD5: D6717F1CA6B9DA2392E63F0DABF590DD
SHA256: 4DC0136062E9A5B6D84C74696005531609BD0299887B70DDFFAA19115BF2E746

Wednesday 15 April 2015

PDF Password Cracking With John The Ripper

Filed under: Encryption,PDF — Didier Stevens @ 0:00

I have a video showing how to use oclHashcat to crack PDF passwords, but I was also asked how to do this with John The Ripper on Windows.

It’s not difficult.

Download the latest jumbo edition john-the-ripper-v1.8.0-jumbo-1-win-32.7z from the custom builds page.

Decompress this version.

Download the previous jumbo edition John the Ripper 1.7.9-jumbo-5 (Windows binaries, ZIP, 3845 KB).

Extract file cyggcc_s-1.dll from the previous jumbo edition, and copy it to folder John-the-Ripper-v1.8.0-jumbo-1-Win-32\run.

Generate the hash for the password protected PDF file (I’m using my ex020.pdf exercise file) and store it in a file (pdf2john.py is a Python program, so you need to have Python installed):

John-the-Ripper-v1.8.0-jumbo-1-Win-32\run\pdf2john.py ex020.pdf > ex020.hash

Start John The Ripper:

John-the-Ripper-v1.8.0-jumbo-1-Win-32\run\john.exe ex020.hash

Loaded 1 password hash (PDF [MD5 SHA2 RC4/AES 32/32])
Will run 8 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
secret           (ex020.pdf)
1g 0:00:00:00 DONE 2/3 (2015-03-29 22:39) 10.20g/s 125071p/s 125071c/s 125071C/s
Use the "--show" option to display all of the cracked passwords reliably
Session completed

By starting John The Ripper without any options, it will first run in single crack mode and then in wordlist mode until it finds the password (secret).

But you can also provide your own wordlists (with option –wordlist) and use rules (option –rules) or work in incremental mode (–incremental).

« Previous PageNext Page »

The Rubric Theme. Blog at WordPress.com.


Get every new post delivered to your Inbox.

Join 322 other followers