I finally gave up on this project since I will continue developing it on an arm platform (based on the EMX module, will post a video soon).
I got it to the point where you could browse files from the device with the application on the android phone and selecting a file started playing it.
Also the ethernet part was working (I could listen radio stations) but you can only test it by changing code on the device (no implementation on the Android side).
So, here is the complete source code for the xmega and for the android device. The android project can be imported directly in Eclipse.
Android Application Source Code
XMEGA Source Code