For historical accuracy, there are no "self-defense martial arts". There are martial arts, and there are fighting styles, and there are martial arts that have degenerated into mere exercise at various dojos. But martial arts are the arts of war, and war necessarily comprises both offense and defense (how can a person honestly learn how to defend them self if they don't understand how they can be earnestly attacked? You have to learn how to effectively attack people, and then learn how to effectively protect yourself, and then learn how to bypass the defenses and then come up with more defenses, and the cycle continues ad infinitum).
Any martial art that has a history of having been used in actual warfare would be a good style to learn. So the biggest limiting factor is actually the dojo instructor. There are a lot of bad instructors out there, either because they don't know what they are doing, or because they do know what they are doing but they sadly don't know how to properly teach it. If you can't find a good instructor who knows how to teach a proper martial art, then I would err towards ... well, actually I'd suggest you stay with the MMA gym because
you said you wanted to learn how to defend yourself.
I hate "MMA" because I don't view it as real martial arts, but I do acknowledge it teaches people how to beat other people up, and how to take some hits, and how to dodge some hits, and how to fight on the ground, which are all important aspects of fighting for your defense and the defense of others. One thing that it lacks, and which honestly most other modern martial arts lack, and that you'll have to make up for one your own or elsewhere, is fighting against multiple opponents. This is an important aspect of street fighting, even if you're only trying to defend, yet so few places (at least around me) seem to even touch the subject.
On the other hand, if you're really jonesin' to learn some traditional martial arts, then I would suggest books on Karate, Tae Kwon Doe, and Tai Chi Chuan (library memberships are cheap!). The first two are simple enough, and the third one is about as traditional as it comes. You won't learn anything even moderately advanced in any books, but you'll be able to learn the basics enough to be able to figure out which ones appeal to you and which ones you might want to check out further. I suggest these three for the basic foundations of many martial arts, something that MMA seems to tend to skip over.
And finally, if there are different types of dojos in your area, most good places will let random people come in and watch a session or two. Just walk in, and stand around by the door until someone comes to talk to you, and let them know what you're looking for. They will likely try to sell you on their dojo, so listen patiently and ask
every question you have but don't buy in until you've had a chance to check out a few, and even then don't buy into any of them if none of them seem good enough
Or, maybe, buy into the cheapest for a couple of months just for the extra dojo experience.