For the second pwnable challenge, we have to generate a hash collision. Read more about a hash collision here. Put simply, it’s when a hash function maps two keys to the same generated hash value:

Assume a set of n objects map to a range R. If n > |R|, the probability of a hash collision is 1, meaning they are guaranteed to occur. See pigeonhole principle.

The code provided, col.c, looks like this:

#include <stdio.h>
#include <string.h>
unsigned long hashcode = 0x21DD09EC;
unsigned long check_password(const char* p){
        int* ip = (int*)p;
        int i;
        int res=0;
        for(i=0; i<5; i++){
                res += ip[i];
        }
        return res;
}

int main(int argc, char* argv[]){
        if(argc<2){
                printf("usage : %s [passcode]\n", argv[0]);
                return 0;
        }
        if(strlen(argv[1]) != 20){
                printf("passcode length should be 20 bytes\n");
                return 0;
        }

        if(hashcode == check_password( argv[1] )){
                system("/bin/cat flag");
                return 0;
        }
        else
                printf("wrong passcode.\n");
        return 0;
}

First thing to notice here is that we have to pass a string of exactly 20 characters to proceed to the next step. After that, the code executes the check_password() function call. If the value returned upon executing the function call matches with hashcode, we’ll capture the flag.

Notice that hashcode is set to the following hex string:

unsigned long hashcode = 0x21DD09EC;

Which in decimal translates to:

>>> int("0x21DD09EC", 16)
568134124

So we need to provide our input in such a way that upon check_password()’s execution we get the value 568134124.

We should now focus on the function check_password():

unsigned long check_password(const char* p){
        int* ip = (int*)p;
        int i;
        int res=0;
        for(i=0; i<5; i++){
                res += ip[i];
        }
        return res;
}

Notice that we pass the pointer to our input value, which is then converted to an array of pointers in ip. We then declare a variable res, which iterates 5 times to generate the final res value which gets matched against the hashcode value. The math works out - our input is expected to be exactly 20 bytes long, and since each int value is worth 4 bytes, the loop iterates 5 times.
Since the decimal value of hashcode isn’t exactly divisible by 5, we have the following equation:

4x + y = 568134124

A simple solution to above equation would yield x as 113626824, and y as 113626828. In hex:

>>> hex(113626824)
'0x6c5cec8'
>>> hex(113626828)
'0x6c5cecc'

Notice that the system is little endian:

col@pwnable:~$ lscpu | grep Endian
Byte Order:            Little Endian

Which means we’ll have to input our string in little endian order:

col@pwnable:~$ python -c 'print "\xc8\xce\xc5\xc8"*4 + "\xcc\xce\xc5\x06"'
�������������������

And if we pass this as input to the program:

col@pwnable:~$ ./col `python -c 'print "\xc8\xce\xc5\x06"*4 + "\xcc\xce\xc5\x06"'`
daddy! I just managed to create a hash collision :)