When you enable NOP sled monitoring, HeapLocker will create a new thread to periodically check (every second) newly committed virtual pages that are readable and writable. When a NOP sled is detected inside these pages with a length equal to or longer than NOPSledLengthMin, HeapLocker will suspend all threads (except this monitoring thread used by HeapLocker) and warn the user that a NOP-sled was detected.
For HeapLocker, a NOP sled is a sequence of single-byte instructions; these may be different or the same instructions. For a list of all single-byte instructions recognized by HeapLocker, take a look array abNOPSledDetection in the source code.
With a classic heap spray, the NOP sled will be detected long before the vulnerability is exploited. But with more sophisticated techniques, it is possible that the NOP sled is detected too late, i.e. that the shellcode already executed. Or it’s also possible that HeapLocker is too early, i.e. that it scans the new page before the NOP sled was written to it. But to prevent this, I wait 1 second between the detection of a new page and the NOP sled scan of that page.
I’ve had some false positives with this detection, that’s why you can configure HeapLocker to ask the user for confirmation.