I'm on my way to Japan in May. We're all nerds here, where should I go? What should I check out? What dumb food should I eat/drink?
Have you a rough idea of where in Japan you're going to? Where you go depends what you're interested in.
Sightseeing & taking photos & olde Japanised atmos = Kyoto/Nara, kansai region, rural areas & Kyushu further west, maybe Hakone or whatever for some hot springs.
Shopping and geekery = Tokyo and Akihabara specifically.
I'm a sumo nut so I always recommend it to people who happen to be holidaying in Japan while a tournament is on. Just so happens that the Tokyo tournament runs from May 12th-26th. Tickets go on sale from April 6th on the official site and it's usually most worth checking out on either the final or penultimate day, since that's when the championship usually gets decided.
I've found the Lonely Planet guides to be pretty awesome and exhaustive. Buy one, flick through it, fold some page corners, bish bash bosh. I tend to combine that with the reviews of accommodation on Tripadvisor and work from there.
go to one of the sega world arcades in akihabara and play virtua fighter! check out kotobukiya!
by a japanese 3ds and instantly regret it when you bring it home and never play on it because everything is in japanese!
visit the ghibli museum!
eat 'crunky balls nude' and drink 'the creamy' coffee in a can!
Eat kare raisu, buy coffee and tea (hot and cold) from vending machines, go to FamilyMart at 3am to get onigiri.
I'd recommend visiting a shrine in the midst of all that tech-badassery. Meiji shrine is really impressive if you're in Tokyo.
What everybody else said (especially that bit about a Japanese 3DS)
If you go to the Ghibli museum, I believe all the online purchasing info is in Japanese. I want to say J-List has a service to buy them for you (but I can't access them from work).
Most Japanese people read English way better than they speak it. If you want to communicate something, writing is way better. This is really useful for taxis.
If you aren't going on business, you can get a railpass that will let you rain all over for free. It's massively worth it if you can make use of it.
Don't be afraid of eating at any place because of language. Nearly everywhere has plastic/wax models of the menu in the front window. Assuming you aren't Asian, just point at what you want for the hostess by the door. That food will show up for you. Then the bill. No tipping, it'll only confuse things.
Don't bring in any porn, the laws are pretty strict.
If you're travelling around a lot, get a rail pass. Very worth it, and you can use it on the circle line within Tokyo (which hits most tourist spots).
Don't be embarrassed if you can't read a menu, just point at what you want and smile. If you know a tiny bit of Japanese, give it a go. They'll appreciate you making the effort.
If you have a girlfriend, get her made up as a geisha at Maiko in Kyoto. Expensive, but worth it.
If you've done any reading on this you don't need me to tell you to go to Akihabara, take a day and go in every store.
This website tells you how to get tickets to the Ghibli Museum, but you need to do it a fair bit in advance.
If you're tall, people will look at you. Embrace it.
Kyoto is the greatest city in the world.
shi shi remooooon