Cracking +RCG's Example 4 (Self32.exe)
(short keys are not long enough)

by Zer0+

Keeper of the lists

(26 December 1997, very slightly edited by fravia+)

---

Courtesy of fravia's page of reverse engineering

---

Well, I'm happy to publish this essay by Zer0+, which should actually be classified among the 'advanced' cracking serie... it's not stuff for newbyes at all. But since the whole 'our protection' project is quite 'tough', I assume that people reading this knows already our trade enough. If not, study more!

Sender: zer0degree(at)hotmail(point)com

Cracking +RCG's Example 4 (Self32.exe)

First of all I would like to thank +RCG for having written 
this exerc-ice (sic! :) it was a nice christmas present for
me to work on.
I have cracked it at christmas' eve in about 5 hours,
which might suggest you that the protection scheme he used
was weak. Deadly wrong! This type of protection is very
strong and I could tackle it only because +RCG used a
short key (I am sure he wanted it to be crackable) and
a few month ago I was also playing with the idea to write
a protection like this... and at that time I spent a lot of
time to figure out the possible attacks against this
type of protections.

Ok, lets see our target Self32.exe (8192 bytes) is a
crippleware (the open and save functions disabled) plus a
nagscreen, but no registration dialog. It comes with a
94 bytes long icname.dat file.
First, I fired up Filemon and run Self32 to see what it
is doing with icname.dat. Well, it did not touch it (yet).
Instead it worked a lot on windows winsock and url dlls
(+RCG might explain me what it was doing with them :),
what I did not take seriously, specially because it run
without problems when I nuked these windows files.
I noticed however that the program looks for a file
called USER.DAT. I made a small 20 byte file for him and
run it again. Well, it read 16 bytes from it then went on
to look for another file called CODE.DAT. Ok, lets make
it for him, too. It reads another 16 bytes from this file
and surprisingly, the initial nagscreen is gone. Lets see
the missing functions, Open, BIG CRASH!

BINGO!! This was the moment I realised that the protection
is exactly what I was thinking some time ago. The code of
the missing functions are encrypted and the two 16 bytes
read from the files are used to decrypt it, obviously
producing garbage code in this case, which leads to a
program crash when the program tries to execute it.

Ok, its time to use Windasm. In the disassembled list the
first thing which caught my eye was a long list of NOPs.
AHA, the decrypted code comes here!
I pinpointed the places where the program opens and reads
the USER.DAT and CODE.DAT files. Wrote them down, and
decided to trace the decryption process with Softice.
 
After setting BPX breakpoints at 4013D8 and 401442
where the program starts to work on USER.DAT and CODE.DAT
respectively, I run the program. It breaks at 4013D8,
opens the USER.DAT file, reads 16 bytes , closes it,
and sums up the first 10 bytes in AX and finally stores
the result at [0040215C]. (instruction 40143C) 

After the program does exactly the same thing with the
CODE.DAT file and finally the result ANDed with the stored
value at [0040215C].

:0040148F mov esi, 00402008         ;pointer to the read string
:00401494 xor eax, eax
:00401496 mov ecx, 00000009
:0040149B add ax, word ptr [esi]    ;summing up a word
:0040149E inc esi                   ;shifting one byte
:0040149F loop 0040149B
:004014A1 and eax, 0000FFFF       
:004014A6 and dword ptr [0040215C], eax  ;stored here!
:004014AC jmp 004014B6
:004014AE nop
:004014AF nop
:004014B0 nop

So, finally we have an interesting value at 40215C, let's
see what it is used for (bpmw 40215C r). If we choose the
Open function we break at 4013CB. Here our magic value is
used to XOR a set of bytes which are starting with E6,57
71,43...values. AH, this is the encoded instructions of
the missing function. Indeed, if we trace on further the
XORed bytes are copied by a write file function to the
place of the NOPs starting at 4014D7 . (+RCG might tell us
how exactly this is done). Then the execution is passed
to 4014D7 and soon big crash is comming :). The picture
is exactly the same if we try the Save function, so the
same encrypted code containes both functions.
After a short search we discover that the encrypted code
is stored in the icname.dat file (lenght, code bytes).
So, to summerise we have a 59 byte long encryted machine
code plus a two byte long key to XOR it with to get a
meaningful function.

Some time ago, when I was thinking about how to crack this
type of protections my main conclusion was that the shorter
the key the more sensitive the protection to the known
plain text attack. Here we have a two byte long key so
it worth to check out the encoded text, I might pick up
some pattern by just looking at it. (If not then I can
write a small program to try every possible keys.)

I was looking for two bytes which are repeated several
times. I have quickly found that 59,62 occures two times
in the first few bytes, so lets work on this. What are
these bytes can be after decoding? I supposed that the
function refers to some addresses in the program all
looking like 0004xxxx, which are xxxx4000 in the code,
so lets suppose 59,62 is encoded 40,00. We need to xor
59 by 19 to get 40 and 62 by 62 to get 0. So the bytes
of our key will be 19 and 62. At this point I looked
again at the encoded bytes and realized that 62 and 19
occures frequently among them, even right after each
which means 0000 after decoding, which is an other
common two bytes in meaningful 32 bit machine code.
At this point I was almost completely sure I have 
found the key. (I was lucky to get it right at the
first try!) So in Softice I patched the program to
use 6219 for decryption and checked the decoded
function at 4014D7.

PUSH DWORD PTR [00402168]
CALL KERNEL!CloseHandle
PUSH 30
PUSH 004021EF
PUSH 004014FE
PUSH DWORD PTR [00402148]
CALL USER!MessageBoxA
JMP  004011D5

Well, it looks good for me :) and sure enough I got the
congratulation message after running it.

The crack: First, I thought I will write my name in the
USER.DAT file and calculate a regcode for the CODE.DAT
and the two will give the correct 6219 key. However, I
realized that the values from the two file are ANDed
together and there is no correct regcode which turns
the value obtained from my name to 6219 (mistake??).
At this point I decided that I keep my name in the
USER.DAT file (at least 16 bytes long), supply a
bogus 16 byte string in the CODE.DAT file (this way
the nag screen is eliminated) and patch the program
at 4014A6 MOV WORD PTR [0040215C], 6219 to always
use the correct key. (Conveniently, we have three
NOPs there to allow enough space for the patch :)

Final thoughts for shareware programmers:

- The only reason I could crack this program was
the fact that +RCG used (I am sure intentionally)
a two byte long key. If the key is,  let's say,
16 byte long the recogniseable patterns of the
encoded functions disappear, and the long key
makes the brute force trying impossible, too.
A long key IMHO makes the protection practically
equivalent to really leaving out the functions,
but with the ability to unlock them with a regkey.
Very appealing to shareware programmers, I guess.

- +RCG filled the place of the missing function
with NOPs and let the program run through them.
Another approach can be that the place is filled
with junk code and a jump at the begining used to
avoid it. When the real code is filled in, the
jump is removed. Having big amount of junk code
there at the first place really confuses disassemblers
and makes the code more problematic to follow.

- Some thoughts on how to implement a protection
like this. (Warning, I have never tested them in
practice :) I don't know whether +RCG used assembler
or not to code this protection ( I suspect he used
extensively), but it might be coded mainly in high
level language. The main problem to find the begining
and the end of the code we want to replace. This can
be done by including marker assembler instructions
in the code. (in c and c++ by using asm directive)
This instructions should do nothing and be easily
distinguishable from compiler generated code.
For example:

void protectedfunction (int x) {

	int y;

	asm {			//begining marker instructions
		INC EAX,
		INC EAX,
		DEC EAX,
		DEC EAX,
		...etc
	}

	y = x+2;		//real code
	...etc

	asm {			//end marking instructions
		DEC ECX,
		DEC ECX,
		NOP,
		INC ECX,
		INC ECX,
	}
	
	return;
}

If a function is written like above, it's easy to
pinpoint the beginning and end of the code we want to
remove and replace some of the begining marking
instructions with a calculated jump to the return.
All the stripping out of the code and putting it later
back can be done with a small utility which works
on the exe file. (BTW, +RCG patched in the correct
function on the fly, a remarkable achivement, but
I guess difficult to manage only in c or pascal).

Well, I think it's enough for now. 
Some time ago the topic - how to twist even more a
protection like this - was on the +HCU maillists (that 
I'm proud to manage :), so curious people could check 
the maillists out for more suggestions. 


Greetings to +RCG and all other +HCUkers

Zer0+

Back to 'Our protections'

---

homepage links anonymity +ORC students' essays academy database
tools cocktails antismut CGI-scripts search_forms mail_fravia
Is reverse engineering legal?