http://forum.xda-developers.com/showthread.php?t=2050393

Well... I have attached a debugger to native code, set breakpoints, analyzed registers, memory, etc. It wasn't that easy though. It took me several days to start debugging and get first key, but I got second one in about 1 hour.

Actually I don't really need that key, I can't even play Angry Birds Rio on my old G1, but it was challenging and I love challenges ;) Plus I have learnt a LOT about gdb, assembler, ARM architecture, etc.

So I want to thank you, Goddchen, for giving me an opportunity to learn & play :)

Ok, let's move on...

First, I have disassembled libangrybirds.so using IDA Pro 5.5 . I was able to examine code and attach IDA to gdbserver on a device, but unfortunately it wasn't working properly. IDA was thinking that libangrybirds.so is a main binary of a process it attached to, but it should look into loaded shared libs instead. Weird, but I didn't find a way to attach it properly. And this is pity, because IDA is a great tool and it would make debugging a pleasure, but I had to use gdb instead.

Second, Android has problems with debugging multi-threaded native code. MT support was added in NDK r5 and because of some bug it's not possible on a system older than Gingerbread.

Third, you could attach gdb manually, but ndk-gdb script does great work for you. You will have to do some tricks to use it with 3rd party app though.

Fourth, it seems libangrybirds.so is a Java code compiled to native or something like that. There are objects like FileInputStream, ByteOutputStream, etc., but there are also some API differencies. We'll see String and Array<uchar> objects, but it's usually easy to find a pointer to simple uchar[].

Steps to start native code debugging:

1. Upgrade to Gingerbread (Yeah, I had to do that. Hacking requires you to sacrifice yourself a bit ;) ). Or you could use an emulator.
2. Install NDK >= r5 .
3. Decode Angry Birds Rio using apktool. You could just unzip it, but decoded app is much more similiar to original sources, so it's more compatible with NDK. For example ndk-gdb reads AndroidManifest.xml to get package name. Of course you could fake simple AndroidManifest.xml and other files if you want.
4. Rename lib dir to libs.
5. Fake jni/Android.mk file. I have copied one from hello-jni sample and didn't even bother to modify module name: http://pastebin.com/HMBXt5cm .
6. Copy libs/armeabi*/libangrybirds.so to obj/local/armeabi*/ . Normally this is done by ndk-build command.
7. Fake libs/armeabi*/gdb.setup file. It should be something like: http://pastebin.com/BYm13RKz , but second line isn't that important.
8. Angry Birds Rio apk contains old gdbserver and you need one from NDK r5. Grab ${NDK_ROOT}/toolchains/arm-linux-androideabi-4.4.3/prebuilt/gdbserver and push it to /data/data/com.rovio.angrybirdsrio/lib .
9. Ufff... you could now try to run: ndk-gdb --verbose --launch=com.rovio.ka3d.App .
10. After few seconds you should see "(gdb)" prompt and game should be paused on the device.
11. Run 'info shared' and check if libangrybirds.so is loaded. If not then something is wrong.

Ok, let's find a key for levels lua files:

1. Set a breakpoint for GameLua::loadLevel() - find this method in IDA Pro and copy its EXPORT name:
   
   Code:

   (gdb) br _ZN7GameLua9loadLevelEN4lang6StringE
   Breakpoint 1 at 0x80468e4c

2. Resume game and open some level. You should hit a breakpoint:
   
   Code:

   (gdb) c
   Continuing.
   [New Thread 5857]
   [Switching to Thread 5857]Breakpoint 1, 0x80468e4c in GameLua::loadLevel () from /home/brutall/t-angrybirds/com.rovio.angrybirdsrio-1/obj/local/armeabi/libangrybirds.so

3. Look into IDA and note there are 2 lang::String objects passed as first arguments to method, so pointers are in R1 and R2 registers. We need to examine these objects and find pointers to raw char[]. Fortunately lang::String is very simple wrapper around char[], so pointer is first (and only one, I think) member of String:
   
   Code:

   (gdb) x/4x $r1
   0x4395e66c:	0x00a405f0	0x00153b28	0x804ec778	0x00000000
   (gdb) x/s 0x00a405f0
   0xa405f0:	 "levels/warehouse/Level190"

   Yey, finally we see something :)
4. Let's move to lang::AESUtil::decrypt() method. It's named _ZN4lang7AESUtil7decryptERKNS_5ArrayIhEES4_RS2_, so:
   
   Code:

   (gdb) advance _ZN4lang7AESUtil7decryptERKNS_5ArrayIhEES4_RS2_
   0x80539894 in lang::AESUtil::decrypt () from /home/brutall/t-angrybirds/com.rovio.angrybirdsrio-1/obj/local/armeabi/libangrybirds.so

5. As you can see decrypt() gets 3 Array<uchar> objects and 2 of them are const. It's quite easy to guess they're: key, encrypted data and container for decrypted data. Let's check this:
   
   Code:

   (gdb) x/4x $r1
   0x1592b0:	0x00159528	0x00000020	0x00000020	0x7b206e65

   0x00000020 = 32 - yes, length of AES key :) First 4 bytes of an Array object is a pointer to raw char[] and second 4 bytes contain length of an array. Now we could read contents of an Array:
   
   Code:

   (gdb) x/s 0x00159528
   0x159528:	 "USCaPQpA4TSNVxMI1v9SK9UC0yZuAnb2a"

   :) As you can see there are 33 chars instead of 32. This is because Array stores its length, so char[] isn't null-terminated. Ignore last "a" char.
6. We could also look into second const Array to be sure that encoded string is exactly the same as contents of lua file:
   
   Code:

   (gdb) x/4x $r2
   0x4395d6f4:	0x009ca248	0x000004a0	0x000004a0	0x00000378
   (gdb) x/4x 0x009ca248
   0x9ca248:	0x3347b5dc	0x26048446	0x1a0c1231	0x35d3f99c

   First 16 bytes are the same, length of data is also ok.

As you can see there is AES::BlockMode passed to AES: ecrypt(). It would be quite hard to interpret it without headers, so I was trying various block modes and I found that CBC with empty initial vector decodes to string starting with '7z'. For me that meant: mission successfull :)

Ok, highscores.lua and settings.lua files now. Technique is very similar, but there are some differences:

• Different keys.
• They aren't loaded using GameLua::loadLevel(), but GameLua::loadPersistentFile(). You could find this very easily, searching for "highscores.lua" in IDA.
• If you examine GameLua::loadPersistentFile() method you will see it doesn't load files using FileInputStream, but io::AppDataInputStream, so we have to be sure, what exactly is being decrypted.
• Annoying thing is that gdb can't catch highscores/settings loading, because they're loaded too soon - before gdb attach itself.

Maybe there is a better solution to last problem, but I've decided to add some Thread.sleep() call just after System.loadLibrary(), so gdb will attach before highscores.lua loading.

1. Open smali/com/rovio/ka3d/App.smali, and add 2 lines of code just after loadLibrary() call in onCreate() method:
   
   Code:

   invoke-static {v0}, Ljava/lang/System;->loadLibrary(Ljava/lang/String;)Vconst-wide/16 v0, 5000invoke-static {v0, v1}, Ljava/lang/Thread;->sleep(J)V

2. Run ndk-gdb --verbose --launch=com.rovio.ka3d.App .
3. Set a breakpoint for GameLua::loadPersistentFile() method and check which file is being loaded:
   
   Code:

   (gdb) br _ZN7GameLua18loadPersistentFileERKN4lang6StringE
   Breakpoint 1 at 0x80457030
   (gdb) c
   Continuing.
   [New Thread 6735]
   [Switching to Thread 6735]Breakpoint 1, 0x80457030 in GameLua::loadPersistentFile () from /home/brutall/t-angrybirds/com.rovio.angrybirdsrio-1/obj/local/armeabi/libangrybirds.so
   (gdb) x/s $r2
   0x4395e3b8:	 "highscores.lua"

   I'm not sure why it's R2, not R1 and why there is no lang::String, but char[] directly. I think this isn't a pointer to String, but String itself, passed to method in registers, so its char[] is in R2.
4. Now advance to lang::AESUtil::decrypt() method and read key as usual:
   
   Code:

   (gdb) advance _ZN4lang7AESUtil7decryptERKNS_5ArrayIhEES4_RS2_
   0x80539894 in lang::AESUtil::decrypt () from /home/brutall/t-angrybirds/com.rovio.angrybirdsrio-1/obj/local/armeabi/libangrybirds.so
   (gdb) x/4x $r1
   0x159294:	0x00159620	0x00000020	0x00000020	0x00159518
   (gdb) x/s 0x00159620
   0x159620:	 "44iUY5aTrlaYoet9lapRlaK1Ehlec5i0"

5. Because of that AppDataInputStream object, we need to check if encrypted data is the same as file contents. Pull highscores.lua file from a device and run:
   
   Code:

   (gdb) x/4x $r2
   0x4395ddc4:	0x0015bc00	0x00000040	0x00000040	0x00000001
   (gdb) x/16x 0x0015bc00
   0x15bc00:	0x2271b777	0xe6f19f4c	0x2489a316	0xfae1aee2
   0x15bc10:	0x82e0ef38	0xe84fc25d	0xb196adac	0xbf030439
   0x15bc20:	0xb6b9bade	0x3046af12	0xe8eeeb0d	0x20e8037c
   0x15bc30:	0x1a405edf	0xc218f7f6	0xc29209e2	0x9ad03e8c

   Yeah, this is my highscores.lua file.
6. Same for settings.lua file to check if it's encrypted with the same key. It is.
7. After decrypting these files we'll see some weird chars at the end of decoded data. Few seconds on the Wikipedia and we'll know this is just PKCS7 padding scheme.

Now we have got everything we want :)

Ahh, not exactly everything... I would be really happy to know, how to properly attach IDA for debugging - it would be much easier, even if gdb interface is also very good.