Manually tweaking the AO seems to be the best solution. I've brightened everything up and I think I've reached a decent middle ground. I'll definitely be tweaking this a lot more (and redoing some parts that didn't bake correctly) but I'm happier with the results.
Yes, that looks great!