You don't think this would be one issue people have with Karate? I mean, if you just want to improve health, fitness, flexibility, coordination, balance, and self confidence, all you need is a gym with a yoga class. Just because you started doing it for those reasons, doesn't mean that Karate is designed for that. It was designed as a combat system. Those other things are extra benefit that you can get without going to a Karate dojo. But if you can only get those things, and you can't learn to fight well, you must wonder, why am I here?
You stated that the ultimate goal of 'the way' of martial arts is to kick butt. Perhaps for some martial arts but not necessarily for others. I joined Shito Ryu Shukokai because I am approaching middle age, I was slightly overweight and wanted to improve my health and fitness, flexibility, coordination, balance and self confidence. I needed stress release from the day to day grind. For me, karate was very effective in assisting me to reach these goals.