Didier Stevens

Sometimes I read the following programmer’s question:

When I launch my program from the command line, I get a new prompt immediately. What API should I call to let my program display a new prompt only when it is done?

This is often related to scripting: calling this kind of program from a BAT file is a problem, because the BAT file will continue executing while the program is still running.

In fact, this behavior is not defined through coding in the program itself, but by a setting in the header of the program. For WIN32 applications, the value of the SUBSYSTEM parameter is 2 (IMAGE_SUBSYSTEM_WINDOWS_GUI) for a GUI application and 3 (IMAGE_SUBSYSTEM_WINDOWS_CUI) for a console application. A programmer can change this setting by selecting the appropriate option for his compiler. And if one can’t recompile the program, just use a binary editor or a PE file editor. It’s important to understand that console applications are not limited to console interaction, and GUI applications are not limited to GUI interaction. A console application can create windows and a GUI application can create consoles.

It is the shell (cmd.exe) that decides if it waits for the end of the launched program or not, based on the value of the SUBSYSTEM parameter. Take a look at the source code of cmd.exe for ReactOS (this is Open Source):

After your program is successfully started (CreateProcess), the shell will check if the new process is a console or GUI application (IsConsoleProcess). If it’s a console application, the shell will wait for the program to terminate (WaitForSingleObject), and then it will set the ErrorLevel to the return code. But if it’s a GUI application, it will not wait and it will set the ErrorLevel immediately to 0. That is why you immediately get a new command prompt when you launch a GUI application from the shell: it’s the shell itself that decides not to wait.

So if a programmer wants cmd.exe to wait for the end of its program, he has to declare it as a console application, even if it uses a GUI. There is a drawback however, because when this program is launched from explorer.exe (and not cmd.exe), a console will be automatically created by Windows. The only trick I found to avoid this is to hide the console programmaticaly (but it will still appear briefly when your program is stared).

You can try the following experiment to better understand the SUBSYSTEM parameter without resorting to programming:

• take a copy of notepad.exe
• change the SUBSYSTEM parameter of this copy from 2 to 3
• launch the copy from Windows Explorer
• launch the copy from cmd.exe
• launch the copy from a BAT file

Quickpost info

Ever started cmd.exe to see this: “The command prompt has been disabled by your administrator”?

It means that a GPO has been set to disable cmd.exe. This is not the same as the Software Restriction Policies. There is a special policy for cmd (and another one for regedit).

When started, cmd.exe checks for the existence of a certain key in the registry and decides to continue execution based on the value of this key. This key is DisableCMD and sits in Software\Policies\Microsoft\Windows\System. For regedit.exe, the key is DisableRegistryTools in Software\Microsoft\Windows\CurrentVersion\Policies\System.

There are many hacks to bypass this technique, depending on what kind of control you have as a user. When you have only control over the content of the programs you execute, use this trick: edit a copy of cmd.exe with a binary editor, search for DisableCMD and change it to something else, like DisableAMD. This copy of cmd.exe will now look for a key that doesn’t exist, and thus continue execution. For regedit, I renamed the key to DisableRegistryFools.

Mark Russinovich has another, elegant hack for this: he starts the program and injects a DLL that intercepts calls to the registry API and filters the return values. Limited users can inject DLLs into their own processes. But since Microsoft bought Sysinternals, his tool (GPdisable) is not available anymore.

Quickpost info

After reading my zero byte padding post, someone asked me how McAfee intercepted scripts.

The Microsoft VB script and JS script engines are COM objects. Looking at the CLSID registry data for these COM objects, you’ll find this info (Windows XP SP2):

VB Script Language
HKEY_CLASSES_ROOT\CLSID\{B54F3741-5B07-11cf-A4B0-00AA004A55E8}
InprocServer32 -> C:\WINDOWS\system32\VBScript.dll

JScript Language
HKEY_CLASSES_ROOT\CLSID\{f414c260-6ac0-11cf-b6d1-00aa00bbbb58}
InprocServer32 -> C:\WINDOWS\system32\JScript.dll

When McAfee VirusScan with ScriptScan is installed, the InprocServer32 reference for both COM objects is modified: C:\Program Files\Network Associates\VirusScan\scriptproxy.dll

This is how VirusScan intercepts script execution. They install a “proxy” (scriptproxy.dll) that will scan the scripts before they are passed on to the appropriate scripting engine (VBScript.dll or JScript.dll).

One important implication of this mechanism, is that ScriptScan will only protect script execution when the scripts are executed with the MS COM objects, like IE does. But Firefox doesn’t work with COM, it has its own JS engine (SpiderMonkey), so ScriptScan does not scan scripts executed by Firefox.

There are documented cases where scriptscan causes problems on servers, the proposed solution is to remove the proxy: regsvr32 /u scriptproxy.dll

I wonder if there is malware out there using this trick? And one can also write his own proxy DLL to intercept scripts.

Of course, McAfee VirusScan is not the only AV providing protection against malicious scripts, most modern AV provide this. For example, Kaspersky’s Anti-Virus uses the same technique, but their proxy DLL is scrchpg.dll.