What are some fun, "off the beaten path" things to do in Atlanta?

My wife and I are looking at being in Atlanta from 11/14 (F) - 11/19 (W), and are looking at things to do in terms of alternative theater, comedy shows/improv, blues/jazz/creole clubs. We found a few great locations this way when we went to our second city of Chicago last year, so I hope the ATL natives can help some lost tourists enjoy your city even moreso than the CHI crowd helped us to. Tall order though...thanks in advance all!