I've developed a FTP like multi-threaded server application as a target for this challenge of the month. It has been coded in c and compiled by VC++ 2008. This is a three step challenge:
Step 1- Find the correct "passphrase" format to logon to the server and get the "Access Granted" message. (You may use a debugger like Ollydbg to do Live RE for this step).
Step 2- Do vulnerability research on the server software. There is at least one exploitable bug but there could be more bugs or error conditions. Try to spot a memory corruption bug and write a denial of service exploit for it.
Step3- Convert your DoS exploit to a code execution exploit to get a connect-back shell.
If you have questions on the challenge, post them here (or to behrang AT sensepost.com)
[you should be able to run the server on just about anything - bug will be exploitable even under XP-SP*]