Hi @dw
You lucky devil.
I can help with this one, being a semi regular visitor... you need to head to Shinjuku station in Tokyo and the surrounding area, particularly if you're on a vinyl hunt.
Shinjuku also good place to stay for a first trip to Japan if you don't mind the hustle and bustle. Not sure of your budget and requirements but have stayed before at the un-flashy but central Shinjuku Prince Hotel.
That means you're also central for all the record shops, like...
Disk Union (the main store) - 6 floors of musical goodness is your first port of call. DISK UNION HERE
See also this blog which mentions the other 20-odd record stores in Shinjuku
VINYL COLLECTOR
Apart from Disk Union which is fairly easy to find as it's in a side street in the main shopping area right by the station, the problem you will have as a westerner is finding the smaller specialist stores. As you probably know, you often have to look up or down to find them as they are not at street level, and asking directions with even GSCE-level Japanese (my level) often results in confused looks.
Good luck - and can I send you a wants list? 
Shinjuku aside, Shibuya in Tokyo (with the famous X-shaped pedestrian crossing from Lost in Translation) also good for records. Used to be a six storey Tower Records there, but also smaller stores in side streets.
Near Shimbashi Station is also good place to stay as again you're right near a major train station and can just jump straight on the metro to all the above places. Lots of hotels nearby.