Didier Stevens

Monday 1 January 2018

New Tool: What Is New?

Filed under: My Software — Didier Stevens @ 0:00

Isn’t the beginning of a new year a good moment to release a new tool called what-is-new.py? 🙂

It’s actually an old tool, I started this in 2012, because it’s something I have to do often: I have a recurring list, and I need to know what items on that list are new (in a nutshell, that’s the problem I tried to solve).

For example, every week I produce a list of User Agent Strings for the requests to my web servers. With a single what-is-new.py command, I can see what User Agent Strings have never hit my servers before.

what-is-new.py takes one argument and a bunch of files. The argument is the name of the database (a Python pickle file). The input can be a single file, several files or stdin. Every line in these files that was not seen before (i.e. not in the database) will be listed by what-is-new.py

In the following example, I run 2 files through what-is-new (files 1.txt and 2.txt) with database demo. The database doesn’t exist yet, hence all lines of the first file are considered new. But with the second invocation, you can see that only line 4 is new.

The tool has several options, I invite you to take a look at the help (-h) and manual (-m).


what-is-new_V0_0_1.zip (https)
MD5: 02067A60EA2EBEE29E98CAF31CEDDF37
SHA256: A4499A230D1925C164531A68C0E8F4FE016882A44D6EDBFF9F4D7BFFA29D14A4


  1. […] New Tool: What Is New? […]

    Pingback by Overview of Content Published In January | Didier Stevens — Thursday 1 February 2018 @ 0:00

  2. […] Tools: jq and what-is-new.py. […]

    Pingback by Retrieving and Processing JSON Data (BTC Example) – Didier Stevens Videos — Sunday 15 July 2018 @ 12:06

RSS feed for comments on this post. TrackBack URI

Leave a Reply (comments are moderated)

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.