I just beat the game on Chieftan as Japan (I thought it was Easy, not Beginner) and now want a real challenge because I STEAMROLLED the other Civs (though actually got a science victory). How big of a jump is Chieftan to Prince? This is essentially my first Civ game (I played II when I was too young to really know what I was doing), so I want a challenge, but I don't want to be overwhelmed. Can any Civ vets give me some pointers?

It's a bit of a jump, but if you feel you understand the basics like how to increase food production without destroying happiness or how to keep your gold from going in the red, then you won't have trouble. 
 
Warlord is barely a step up from Cheftain so you probably won't be satisfied. I'm not the highest level player and I find Prince provides a good challenge. Every level after that starts to handicap you and gives the AI advantages.
 
On Prince I win most games, but there's enough opposition that I'm not perpetually in first place.  I end up neck-in-neck with at least half of the Civs, which makes the game way more engaging than just building your empire and treating other Civs like speedbumps in the way. 

considering that the AI is ridiculously moronic you shouldn't have a problem beating it on the higher levels.  they really made this game a lot easier this time around.

Prince you will have to manage happiness better and you won't be able to just build all buildings and still be in the green.

Biggest thing you'll notice is the barbarians will be more persistent than they were on chieftain.  The AI will suck just as hard but build and tech faster.

Ramp up the difficulty. Do not play as Japan. Go for a non-conquest victory. And you got your challenge.

If you have any strategy at all you'll still find prince ridiculously easy.