- Platten Pedro is way out in the 'old west' and it is oozing the atmosphere of an old bookstore (which I enjoy a lot). It has a decent soundtrack section. After I had gone through three shelves high up in the back room (where I didn't find much of interest), I was deemed worthy of the more interesting stuff, which is right next to the owner's counter (on his left, when entering the store). All the records in there appear to be at least twenty years old.
- Hard Wax is amazing for techno, but if that is your thing you will know that anyway.
- Bis aufs Messer - Antoni is spot on, these guys are awesome. They come from a punk/hardcore background, which is also reflected in a lot of the prices. Soundtracks aren't really their focus, I think. Their stock is pretty well documented online.
- Rumpsti Pumsti is great for experimental/Avantgarde/Sound art etc. Again, their website represents their stock pretty neatly.
- Close to Bis aufs Messer is hhv, who generally have great prices and certainly the biggest selection. Hardly anything is actually displayed in the store, though, which means you'll have to order online in advance and choose the 'pick up in store' option. The store itself is, unless you're interested in hiphop, not actually that interesting.
- Lastly, there is a seller called "recordsale-de" on discogs, who has some 380,000 items (in fact, they boast to be "#1 seller on discogs"). In some cases, the prices are excellent, in others, they make little sense. When ordering through them, you can pick up your records somewhere in Mitte, iirc at a cinema box office. I have never done that because I became aware of the scheme too late on my last visit, as it takes some time (a couple of days, if memory serves, depending on the weekdays) for them to pull your order and have it ready for pick-up at that place.
EDIT: I have just seen that pickup is no longer possible. Sorry if I have caused confusion.
Note that hhv and that discogs seller (and probably Bis aufs Messer as well) have competitive shipping rates, even internationally. If you have scored on the flea markets that Antoni has mentioned it might be more convenient to have your purchases from these stores sent to you. 
