Hold your finger on the app until it shakes and a red x appears. Click that x and it will delete.

Btw, I still have my original iPod