In a serious situation, yes I'll wear my life jacket every time. If i'm on a wild white water rafting trip, I'm going to wear a life jacket even if I may have some swimming ability.
Think of martial arts training as safety bubble that protects you from attackers. You train your forms/poomse/katas to develop muscle strength, endurance, and technique that will better prepare you for the reality of the street.
You don't try to be "cool" and have a fight club with kids hitting each other. Where is the controlled contact? Where is the adult supervision? Why are they wearing gloves when they wont' have them in the street?