Didier Stevens

Wednesday 13 July 2016

Practice ntds.dit File Part 2: Extracting Hashes

Filed under: Encryption — Didier Stevens @ 0:00

There are several how-tos on the Internet explaining you how to extract hashes from the Active Directory database file. I used this how-to for Kali Linux: https://blog.joelj.org/windows-password-audit-with-kali-linux/

The tools libesedb and ntdsxtract are used in this how-to.

I encountered an error when making libesedb:

libcfile_support.c:742:2: error: #error Missing file remove function

Make sure to read the comment from May 6, 2016 for this how-to: it offers a solution for this error. Edit libcfile/libcfile_support.c and add this line at the top:

#define HAVE_UNLINK 1

First we get the Active Directory database file I published and unzip it:

20160710-210725

Next we use libesedb (command esedbexport) to export the tables from ntds.dit:

20160710-210856

This may take some time, depending on the size of the database.

20160710-210938

The exported tables are in folder ntds.dit.export:

20160710-211024

Then we use ntdsxtract (command dsusers.py) to export the hashes (LM and NTLM) from the exported tables. First we export the hashes in a format suitable for John the Ripper. We store the files in folder dump. This command also takes the SYSTEM registry hive (file system) to extract the system key to decrypt the hashes.

20160710-211607

We let the command create the folder dump:

20160710-211642

Next the tool detects 2 schemas in the exported tables (objects 5 and 1480). First we try schema object 5:

20160710-211702

This fails:

20160710-211723

So we start again with schema object 1480, but first we need to remove the dump folder:

20160710-211757

20160710-211816

20160710-211831

Now you can find the extracted hashes (lm.john.out and nt.john.out) in folder dump:

20160710-211852

20160710-212259

Next we repeat the same command but export hashes in a format suitable for hashcat:

20160710-211921

20160710-211932

Now you can find the extracted hashes (lm.ocl.out and nt.ocl.out) in folder dump:

20160710-211954

20160710-212049

If you want these hash files to crack the passwords without having to run through this how-to, you can download them here:

ntds-hashes.zip (https)
MD5: B0A84D756C211A97087BA307F0CE5739
SHA256: 009520798DD34831C47ADAC47D6DEB3C153FC44BD9D400A0BB813EBA46728D86

Tuesday 12 July 2016

Practice ntds.dit File Part 1

Filed under: Encryption — Didier Stevens @ 0:00

I’m publishing a sample Active Directory database file (ntds.dit) together with the corresponding SYSTEM registry hive so that you can practise hash extraction and password cracking.

This ntds.dit and system file come from a virtual machine I installed just for this purpose: Windows Server 2003 Standard Edition with SP1 (English). The reason I selected an old Windows version, is that 2003 still supports LM hashes by default.

I changed the password policy to allow very weak passwords:

20160710-125218

I added 40 users: 20 users with passwords taken from the rockyou database leak and 20 users with random passwords (varying in length from 1 to 20 characters). Some of the passwords I randomly selected from rockyou are longer than 14 characters: when a password is longer than 14 characters, Windows does not store a LM hash for that password.

You can find many how-tos on the Internet showing you how to extract the LM and NTLM hashes from the Active Directory database file. I too will posts examples of hash extraction and password cracking.

Happy cracking!

ntds.zip (https)
MD5: F20E477D9784E009777F286ABF718FA3
SHA256: F5EBBF57B3C646FC339ECEEE03063BEDE9E0E7FC8254B0E57A77CC4036134B04

Monday 11 July 2016

hashcat 3.00 “fatal error: ‘inc_vendor.cl’ file not found”

Filed under: Encryption — Didier Stevens @ 0:00

When I tested hashcat 3.00 I got an error: “fatal error: ‘inc_vendor.cl’ file not found”. The fix for such errors is to update your (GPU) drivers. Unfortunately, I could not update at the moment but I still wanted to play with this new toy:-)

Here is the output with the errors (the warnings are another indication that a driver update is necessary):

1

The error is a build error. hashcat was not able to compile the code for a brute-force LM hash attack for my GPU (m03000_a3.2ad8800f.kernel). Older versions of hashcat included compiled code for different attacks, hashes and devices. But since version 2.0 (when hashcat became open source) hashcat compiles this code when necessary: Just In Time (JIT). You can see this in the kernels folder. In older versions (< 2.0) of hashcat, this folder is populated with code. In the newer versions, it is empty.

The build log indicates why this error occurs: file inc_vendor.cl was not found (this file is present in the OpenCL folder) when compiling m03000_a3.cl.

My workaround is to temporarily make the OpenCL folder the working directory, and then run hashcat so that it can compile the kernels:

2

Now you can find the compiled kernels in the kernels folder:

4

The mask I specify with this hashcat command is also much smaller: ?1 This way, hashcat terminates quickly and I can move back to the directory I want and use the full mask ?1?1?1?1?1?1?1:

3

This time there is no build error, because the necessary kernels are already build and taken from the cache (folder kernels).

Remember, this is a workaround, not a fix. The fix is to update the drivers. I updated the drivers later (not without issues), deleted the content of the kernels folder, and started again. This time, without build errors.

FYI: I generated the LM hashes for this example with this website: https://www.tobtu.com/lmntlm.php

5

Wednesday 6 July 2016

YouTube Video Promo

Filed under: Announcement,Didier Stevens Labs — Didier Stevens @ 8:37

I produced 32 technical videos in 2015. You can find them on YouTube and my video blog (sometimes I also post beta versions of my new tools along with the video on my video blog).

I decided to run a promo for my Didier Stevens Labs videos: If you buy one of my products, you get to download the original MP4 files I uploaded to my free YouTube channel. This offer is also valid for existing clients.

YouTube Video Promo

Monday 13 June 2016

Update:oledump.py Version 0.0.24

Filed under: My Software,Update — Didier Stevens @ 0:00

oledump.py has the –calc option to calculate the MD5 hashes of each stream (if you need another hash algorithm, use option –extra).

This time I needed the hashes of the decompressed macro streams, and not of the raw streams. So I updated oledump.py to support using options –calc and -v together (and also option –extra and -v). When you use option –calc (or –extra) with option -v, raw macro streams (indicator m or M) will be decompressed and the hash of the decompressed macro will be calculated.

I needed this option to compare two samples that were different, but probably very similar.

Here I can see that the hashes of the macro streams are identical, hence that although I have 2 different samples, the VBA code is identical.

20160608-215121

oledump_V0_0_24.zip (https)
MD5: F1BFD24FBC72966D54C365B57E662700
SHA256: 4C175874EFDF7DB3264038BFACFD44F1B9060E834189FF3CBAA6C8EBD9D7F680

Saturday 11 June 2016

Overview of Content Published In May

Filed under: Announcement — Didier Stevens @ 0:00

Here is an overview of content I published in May:

Blog posts:

YouTube videos:

Videoblog posts:

SANS ISC Diary entries:

Tuesday 7 June 2016

Recovering A Ransomed PDF

Filed under: PDF — Didier Stevens @ 0:00

I was contacted to help with a PDF file encrypted by ransomware. Just like another case I helped with, the file was not completely encrypted. The file had parts with low entropy, as byte-stats.py shows:

20160606-220414

Searching for endobj, I noticed the file contained PDF objects:

20160606-221145

So I stripped the beginning of the file that was encrypted:

20160606-221331

This file can be parsed by pdf-parser. Now I’m going to try to rebuild this PDF. First I check if it contains an object referencing all pages:

20160606-221658

As you can see, it doesn’t. So I will add the missing objects:

20160606-222040

Object 2 (the missing /Pages object) needs to reference all pages still present in the document (/Kids list). I make a list of all /Page objects with the following command:

20160606-222418

And then I update object 2 /Pages with the 87 /Page objects I found (dictionary entries /Kids and /Count):

20160606-222617

When I open this PDF with a PDF reader, I get 87 pages. All of them are blank, except the last one:

20160606-222854

The pages are blank because of missing fonts definitions:

20160606-223238

I add some generic font definitions:

20160606-223431

This gives me the following PDF:

20160606-223618

AS you can see, not all text is readable, that’s because I did not select the right font. Some trial and error with different fonts would allow me to further recover the document.

This method can also help you with corrupt PDF documents. Of course, this is not a complete recovery. We miss the first pages that were encrypted.

Wednesday 1 June 2016

Major Update For zipdump.py

Filed under: My Software,Update — Didier Stevens @ 0:00

I released a first, simple version of zipdump.py, a tool to analyze ZIP files and their content. But I’ve made major changes to the tool (like support for YARA) that I release today.

zipdump can also be used to pipe a sample into my other analysis tools like oledump.py.

zipdump_v0_0_3.zip (https)
MD5: 100E4B1E1E9F542EB244C9A0766C35FF
SHA256: A5219D7C88FF78A8D7C93B9EEF19D085F9FA92944CAE492F293164213329988F

Here is the man page:

 

Usage: zipdump.py [options] [zipfile]
ZIP dump utility

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -m, --man             Print manual
  -s SELECT, --select=SELECT
                        select index nr or name
  -S SEPARATOR, --separator=SEPARATOR
                        Separator character (default )
  -o OUTPUT, --output=OUTPUT
                        Output to file
  -d, --dump            perform dump of first file or selected file
  -D, --dumpall         perform dump of all files or selected file
  -x, --hexdump         perform hex dump of first file or selected file
  -X, --hexdumpall      perform hex dump of all files or selected file
  -a, --asciidump       perform ascii dump of first file or selected file
  -A, --asciidumpall    perform ascii dump of all files or selected file
  -e, --extended        report extended information
  -p PASSWORD, --password=PASSWORD
                        The ZIP password to be used (default infected)
  -y YARA, --yara=YARA  YARA rule file (or directory or @file) to check files
                        (YARA search doesn't work with -s option)
  --yarastrings         Print YARA strings
  -C DECODERS, --decoders=DECODERS
                        decoders to load (separate decoders with a comma , ;
                        @file supported)
  --decoderoptions=DECODEROPTIONS
                        options for the decoder
  -v, --verbose         verbose output with decoder errors
  -c CUT, --cut=CUT     cut data
  -r, --regular         if the ZIP file contains a single ZIP file, handle it
                        like a regular (non-ZIP) file
  -z, --zipfilename     include the filename of the ZIP file in the output
  -E EXTRA, --extra=EXTRA
                        add extra info (environment variable: ZIPDUMP_EXTRA)

Manual:

zipdump is a tool to analyze ZIP files.
The ZIP file can be provided as an argument, via stdin (piping) and it may
also be contained in a (password protected) ZIP file.

When providing zipdump with a file to analyze, it will report on the content
of the ZIP file, like in this example:
C:\Demo>zipdump.py example.zip
Index Filename     Encrypted Timestamp
    1 Dialog42.exe         0 2012-02-25 12:08:26
    2 readme.txt           0 2015-11-24 19:40:12

The first column, Index, is an index that zipdump assigns to each file inside
the ZIP file. You can use it with option -s (select) to select a file for
further analysis.
Filename is the filename of the contained file.
Encrypted is a flag indicating if the file is encrypted (1) or not (0).
And the last column (Timestamp) is the timestamp of the file inside the
archive.

Option -s takes the index number or the filename to select a file.

By default, the separator used to delimit columns is the space character. When
the default separator is used, padding is added to lign up the columns.
Another separator character can be selected with option -S. No padding is used
when the separator is provided (even if it is the space character).
C:\Demo>zipdump.py -S ; example.zip
Index;Filename;Encrypted;Timestamp;
1;Dialog42.exe;0;2012-02-25 12:08:26;
2;readme.txt;0;2015-11-24 19:40:12;

When a file is selected, the properties of this file are displayed:
C:\Demo>zipdump.py -s 1 example.zip
Index Filename     Encrypted Timestamp
    1 Dialog42.exe         0 2012-02-25 12:08:26

The content of the selected file can also be dumped.
Use option -x to perform an hexdump:
C:\Demo>zipdump.py -s 1 -x example.zip
4D 5A 50 00 02 00 00 00 04 00 0F 00 FF FF 00 00
B8 00 00 00 00 00 00 00 40 00 1A 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...

Use option -a to perform an ascii/hexdump:
C:\Demo>zipdump.py -s 1 -a example.zip
00000000: 4D 5A 50 00 02 00 00 00 04 00 0F 00 FF FF 00 00  MZP.............
00000010: B8 00 00 00 00 00 00 00 40 00 1A 00 00 00 00 00  +.......@.......
00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
...

Use option -d to perform a raw dump:
C:\Demo>zipdump.py -s 2 -d example.zip
test

A raw dump is useful to pipe the output into another command:
C:\Demo>zipdump.py -s 1 -d example.zip | pecheck.py
PE check for '':
Entropy: 6.425034 (Min=0.0, Max=8.0)
MD5     hash: 9b7f8260724e2cb643ad0729ec995b40
...

When options -x, -a or -d are used without selecting a file (option -s), the
first file in the ZIP file is selected and dumped.
When options -X, -A or -D are used without selecting a file (option -s), all
files in the ZIP file are selected and dumped.

The output produced by zipdump.py can de written to a file with option -o.

If the ZIP file is password protected, zipdump.py will try with password
'infected'. Option -p can be used to provide a different password to open the
ZIP file.

If the ZIP file contains a single ZIP file, the contained ZIP file will be
considered to be the ZIP file to analyze. To prevent this, use option -r.
Option -r handles the contained ZIP file as a regular file.

Option -z can be used to include the name of the zipfile in the report:
C:\Demo>zipdump.py -z -S ; example.zip
Index;Zipfilename;Filename;Encrypted;Timestamp;
1;example.zip;Dialog42.exe;0;2012-02-25 12:08:26;
2;example.zip;readme.txt;0;2015-11-24 19:40:12;

This can be useful when reports of many ZIP files are merged together.

Option -e extends the amount of information reported:
C:\Demo>zipdump.py -e example.zip
Index Filename     Encrypted Timestamp           MD5
Filesize Entropy       Magic HEX Magic ASCII Null bytes Control bytes
Whitespace bytes Printable bytes High bytes
    1 Dialog42.exe         0 2012-02-25 12:08:26
9b7f8260724e2cb643ad0729ec995b40    58120 6.42503434625 4d5a5000  MZP.
13014          6403             1678           19366      17659
    2 readme.txt           0 2015-11-24 19:40:12
098f6bcd4621d373cade4e832627b4f6        4 1.5            74657374 test
0             0                0               4          0

Columns MD5, Filesize and Entropy should be self-explanatory.
The Magic columns (HEX and ASCII) report the first 4 bytes of the file.
The remaining columns provide more statistical data about the contained file.
They count the number of bytes of a particular type found inside the contained
file. The byte types are: null bytes, control bytes, whitespace, printable
bytes and high bytes.

If you need other data than displayed by option -e, use option -E (extra).
This option takes a parameter describing the extra data that needs to be
calculated and displayed for each file. The following variables are defined:
  %INDEX%: the index of the file
  %ZIPFILENAME%: the filename of the ZIP container
  %FILENAME%: the filename of the contained file
  %ENCRYPTED%: encrypted indicator
  %TIMESTAMP%: timestamp
  %LENGTH%': the length of the file
  %MD5%: calculates MD5 hash
  %SHA1%: calculates SHA1 hash
  %SHA256%: calculates SHA256 hash
  %ENTROPY%: calculates entropy
  %HEADHEX%: display first 20 bytes of the file as hexadecimal
  %HEADASCII%: display first 20 bytes of the file as ASCII
  %TAILHEX%: display last 20 bytes of the file as hexadecimal
  %TAILASCII%: display last 20 bytes of the file as ASCII
  %HISTOGRAM%: calculates a histogram
                 this is the prevalence of each byte value (0x00 through 0xFF)
                 at least 3 numbers are displayed separated by a comma:
                 number of values with a prevalence > 0
                 minimum values with a prevalence > 0
                 maximum values with a prevalence > 0
                 each value with a prevalence > 0
  %BYTESTATS%: calculates byte statistics
                 byte statistics are 5 numbers separated by a comma:
                 number of NULL bytes
                 number of control bytes
                 number of whitespace bytes
                 number of printable bytes
                 number of high bytes

Example adding the SHA256 hash to the report:
C:\Demo>zipdump.py -E "%SHA256%" example.zip
Index Filename     Encrypted Timestamp
    1 Dialog42.exe         0 2012-02-25 12:08:26
0a391054e50a4808553466263c9c3b63e895be02c957dbb957da3ba96670cf34
    2 readme.txt           0 2015-11-24 19:40:12
9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

The parameter for -E may contain other text than the variables, which will be
printed. Escape characters \n and \t are supported.
Example displaying the MD5 and SHA256 hash per file, separated by a -
character:
C:\Demo>zipdump.py -E "%MD5%-%SHA256%" example.zip
Index Filename     Encrypted Timestamp
    1 Dialog42.exe         0 2012-02-25 12:08:26 9b7f8260724e2cb643ad0729ec995
b40-0a391054e50a4808553466263c9c3b63e895be02c957dbb957da3ba96670cf34
    2 readme.txt           0 2015-11-24 19:40:12 098f6bcd4621d373cade4e832627b
4f6-9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

If the extra parameter starts with !, then it replaces the complete output
line (in stead of being appended to the output line).
Example:
C:\Demo>zipdump.py -E "!%FILENAME%;%SHA256%" example.zip
Dialog42.exe;0a391054e50a4808553466263c9c3b63e895be02c957dbb957da3ba96670cf34
readme.txt;9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08

To include extra data with each use of zipdump, define environment variable
ZIPDUMP_EXTRA with the parameter that should be passed to -E. When environment
variable ZIPDUMP_EXTRA is defined, option -E can be ommited. When option -E is
used together with environment variable ZIPDUMP_EXTRA, the parameter of option
-E is used and the environment variable is ignored.

zipdump supports YARA rules. Installation of the YARA Python module is not
mandatory if you don't use YARA rules.
You provide the YARA rules with option -y. You can provide one file with YARA
rules, an at-file (@file containing the filenames of the YARA files) or a
directory. In case of a directory, all files inside the directory are read as
YARA files.
All files inside the ZIP file are scanned with the provided YARA rules, you
can not use option -s to select an individual file.

Example:
C:\Demo>zipdump.py -y contains_pe_file.yara example.zip
Index Filename     Decoder YARA namespace        YARA rule
    1 Dialog42.exe         contains_pe_file.yara Contains_PE_File

In this example, you use YARA rule contains_pe_file.yara to find PE files
(executables) inside ZIP files. The rule triggered for file 1, because it
contains an EXE file.

If you want more information about what was detected by the YARA rule, use
option --yarastrings like in this example:
C:\Demo>zipdump.py -y contains_pe_file.yara --yarastrings example.zip
Index Filename     Decoder YARA namespace        YARA rule
    1 Dialog42.exe         contains_pe_file.yara Contains_PE_File 000000 $a
4d5a 'MZ'

YARA rule contains_pe_file detects PE files by finding string MZ followed by
string PE at the correct offset (AddressOfNewExeHeader).
The rule looks like this:
rule Contains_PE_File
{
    meta:
        author = "Didier Stevens (https://DidierStevens.com)"
        description = "Detect a PE file inside a byte sequence"
        method = "Find string MZ followed by string PE at the correct offset
(AddressOfNewExeHeader)"
    strings:
        $a = "MZ"
    condition:
        for any i in (1..#a): (uint32(@a[i] + uint32(@a[i] + 0x3C)) ==
0x00004550)
}

To deal with encoded files, zipdump supports decoders. A decoder is a type of
plugin, that will bruteforce a type of encoding on each file. For example,
decoder_xor1 will encode each file via XOR and a key of 1 byte. So
effectively, 256 different encodings of the file will be scanned by the YARA
rules. 256 encodings because: XOR key 0x00, XOR key 0x01, XOR key 0x02, ...,
XOR key 0xFF
Here is an example:
C:\Demo>zipdump.py -y contains_pe_file.yara -C decoder_xor1 example.zip
Index Filename            Decoder             YARA namespace        YARA rule
    1 Dialog42.exe                            contains_pe_file.yara
Contains_PE_File
    3 Dialog42.exe.XORx14 XOR 1 byte key 0x14 contains_pe_file.yara
Contains_PE_File

The YARA rule triggers on file 3. It contains a PE file encoded via XORing
each byte with key 0x14.

You can specify more than one decoder separated by a comma ,.
C:\Demo>zipdump.py -y contains_pe_file.yara -C
decoder_xor1,decoder_rol1,decoder_add1 example.zip

Some decoders take options, to be provided with option --decoderoptions.

Use option -v to have verbose error messages when debugging your decoders.

Option -c (--cut) allows for the partial selection of a file. Use this option
to "cut out" part of the file.
The --cut option takes an argument to specify which section of bytes to select
from the file. This argument is composed of 2 terms separated by a colon (:),
like this:
termA:termB
termA and termB can be:
- nothing (an empty string)
- a positive decimal number; example: 10
- an hexadecimal number (to be preceded by 0x); example: 0x10
- a case sensitive string to search for (surrounded by square brackets and
single quotes); example: ['MZ']
- an hexadecimal string to search for (surrounded by square brackets);
example: [d0cf11e0]
If termA is nothing, then the cut section of bytes starts with the byte at
position 0.
If termA is a number, then the cut section of bytes starts with the byte at
the position given by the number (first byte has index 0).
If termA is a string to search for, then the cut section of bytes starts with
the byte at the position where the string is first found. If the string is not
found, the cut is empty (0 bytes).
If termB is nothing, then the cut section of bytes ends with the last byte.
If termB is a number, then the cut section of bytes ends with the byte at the
position given by the number (first byte has index 0).
When termB is a number, it can have suffix letter l. This indicates that the
number is a length (number of bytes), and not a position.
termB can also be a negative number (decimal or hexademical): in that case the
position is counted from the end of the file. For example, :-5 selects the
complete file except the last 5 bytes.
If termB is a string to search for, then the cut section of bytes ends with
the last byte at the position where the string is first found. If the string
is not found, the cut is empty (0 bytes).
No checks are made to assure that the position specified by termA is lower
than the position specified by termB. This is left up to the user.
Search string expressions (ASCII and hexadecimal) can be followed by an
instance (a number equal to 1 or greater) to indicate which instance needs to
be taken. For example, ['ABC']2 will search for the second instance of string
'ABC'. If this instance is not found, then nothing is selected.
Search string expressions (ASCII and hexadecimal) can be followed by an offset
(+ or - a number) to add (or substract) an offset to the found instance. For
example, ['ABC']+3 will search for the first instance of string 'ABC' and then
select the bytes after ABC (+ 3).
Finally, search string expressions (ASCII and hexadecimal) can be followed by
an instance and an offset.
Examples:
This argument can be used to dump the first 256 bytes of a PE file located
inside the file: ['MZ']:0x100l
This argument can be used to dump the OLE file located inside the file:
[d0cf11e0]:
When this option is not used, the complete file is selected.

Sunday 29 May 2016

Update: pecheck.py Version 0.5.1

Filed under: My Software,Update — Didier Stevens @ 10:12

This version offers more info about the overlay:

20160529-115403

pecheck-v0_5_1.zip (https)
MD5: F045A67AC1ECCF129030DFCE316383A9
SHA256: 9F6EFD34455D530BD3A867FEDD40C1E9538E8B7299E538AAC73D936EDF9904EF

Saturday 21 May 2016

Update: pecheck.py Version 0.5.0

Filed under: My Software,Update — Didier Stevens @ 20:46

This version of pecheck adds support for YARA rules and overlays.

20160521-223253

pecheck-v0_5_0.zip (https)
MD5: B873F8B5F6D408E4026010F010EA5FC4
SHA256: 7FCE12A8B10BEFF0C991B652CEDE376C187E74F23C603BF1A9250C9E7756AB48

« Previous PageNext Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 450 other followers