Skully'z at 907 Bourbon Street is a very small store that carries a good selection of indie & punk. They do stock OSTs, Votary, Death Waltz etc. There are a few crates of used records. The guy was really friendly. It is in the "rainbow" part of Bourbon St. If you keep walking all the way up Bourbon St. to Esplanade and take a left, you'll find a restaurant called Port of Call. They make a really, really mean burger served with a loaded baked potato on the side. It's worth a visit.
Peaches Records at 408 North Peters Street is a much larger store, but it's a bit of a tourist trap. They do stock some great records, but I found their prices to be appalling... also the owner was fake sugary sweet to us while she was simultaneously being a giant bitch to her employees in front of us, so that kind of harshed the vibe. I did pick up a couple of good used blues records and I think a copy of Maurice Jarre's The Bride.
These are really the only two shops I am aware of that are walkable from the French Quarter. There may be more now, NOLA is a hip town that is fairly vibrant at the moment. Let me know if you find any other good digging spots, since I travel to NOLA for work 2-4 times a year.
Other places to eat, Jacques Imo's is great 
http://jacques-imos.com/ (though not walkable from the French Quarter), Irene's Cuisine fancy Italian-Creole is one of the best, and recently I was very impressed by a place called Andy's Bistro in Metairie, they served a soft shell crab on special that just blew the doors off of every other soft shell crab dish I've ever had. 
http://www.andybistro.com/index.php
Enjoy the trip, I love New Orleans.