I replaced the battery on my old iPod mini, though I can't specifically recall where I got it. (The cost was only about 20 dollars. I'm pretty sure I just googled "iPod" battery.)

The actual replacement was a little tricky, even with the tool provided. I kinda scratched the case up trying to pry the two white plastic caps off. Once you get the case off, the rest is easy. As long as you expect/don't mind a few cosmetic scratches, it's well worth it.