Announcement

Collapse
No announcement yet.

Suggestions for constructing a valid Survey on fighting

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    Suggestions for constructing a valid Survey on fighting

    OK, my recent article deconstructing the "90% of all fights go to the ground" myth got me thinking. Because none of the existing data sources pass research muster, it'd be really, really nice to have one that does. The problem is, I'm not going to shell out that thousands of dollars (hundreds of thousands?) to have a Research Group run a national poll.

    So, failing that, I'm soliciting ideas for actually getting the necessary data in such a way that actually will pass muster.

    Suggestions?

    Peace favor your sword,
    Kirk

    #2
    Survey of trial transcripts, maybe?

    Comment


      #3
      Maybe. I admit that I hadn't thought of that. Unfortunately, it misses everything that doesn't go to trial or that the trial itself doesn't detail. :(

      Peace favor your sword,
      Kirk

      Comment


        #4
        The reason for the question will affect the wording of the specific question and thus may affect your research methods (or narrow down your options). Personally I would do what a lot of scientists do and narrow the question at first to something easily measurable and specific. After that question is answered then look to broaden it or look at another easily measurable area before attempting to generalize.

        For example if the purpose of looking at this myth is to help train police in more effective self defense techniques then one might look analyze all police video recordings related to officer assault or response to violent situations. Then you could make a conclusion along the lines of "based on the data in X county between the time of Y and Z, fights involving police went to the ground N percentage of times".....

        Comment


          #5
          I work in academia and so far one of the biggest revelations I've had in my career is that if it's not clear how you should measure something, chances are you haven't properly narrowed down what it is you want to measure.

          Get a nice specific question/hypothesis first. It's probably going to end up sounding like a very unambitious question, but it's much better to have an unambitious question you can answer than a really thought-provoking one that there is no simple answer for.

          Also, outside of a few fields, the information you get from polls is usually bullshit. Avoid polls.

          Comment


            #6
            Originally posted by CrackFox View Post
            I work in academia and so far one of the biggest revelations I've had in my career is that if it's not clear how you should measure something, chances are you haven't properly narrowed down what it is you want to measure.

            Get a nice specific question/hypothesis first. It's probably going to end up sounding like a very unambitious question, but it's much better to have an unambitious question you can answer than a really thought-provoking one that there is no simple answer for.

            Also, outside of a few fields, the information you get from polls is usually bullshit. Avoid polls.
            Agreed 100%.

            Comment


              #7
              Actually just off the top of my head. If you could get a random sample of cctv footage of street fights, you wouldn't actually need a huge number of videos to start making estimates of how often fights go to the ground. You'd need to define properly what "going to the ground" means and you'll have to make sure that sample of fights really is random.

              Comment


                #8
                I wouldn't do random street fights yet because of the high variability in potentially confounding factors...some have weapons...some are under the influence....some aren't. I'd think you'd need some method of control for a first question otherwise when you break down the types of fights (who was involved, types of illegal substances involved, weapons, etc) you'll find probably either high variability or lack of information.

                Also where would you get these CCTV tapes? You tube? Probably not a good idea because only certain videos are put there.

                The ultimate goal of something like this is to answer the original question of what type of martial art should a law abiding citizen or LEO study. The fights on video may encompass neither of those types. Kirk already has an article from a police standpoint, why not take that a step further first before going into more random areas? With police videos you can find out ultimately if drugs/alcohol were involved and what weapons if any were involved and you have one less variable of always having police involvement and could make a conclusion regarding encounters law enforcement might face.
                Last edited by BryanW; 1/09/2012 11:16am, . Reason: fixed spelling

                Comment


                  #9
                  That's a good point. Defining the specific question is key.

                  Initially, the thought was "what percentage of fights 'go to the ground'" but I realized instantly that this was to vague. What is a fight? How is "go the the ground" defined? &tc. Of course, the point of finding out what percentage of "fights" "go to the ground" is to make an informed decision about how much priority ground-fighting should have in your personal training for Self Defense.

                  As these thoughts were rattling around in my brain I also realized that I need some way to collect a statistically significant random sample of the population (in this case, the general population as a whole). All of the data collection methods that I can think of or has been suggested to date significantly filter the sample to a point where I doubt it would be representative of the population. I've had suggestions of CCTV vids, Court Documents, Arrest Records, and web surveys targeted to Martial Arts forums as well as a few other suggestions thrown in for good measure. All suffer from the same problem of not representing the general population. LEO Arrest Records only capture "fights" to which LEO's roll and make an arrest. Court Records only capture information which got to a Trial and in which the testimony is introduced to Court. Web surveys suffer from very specific targeting and Martial Artists represent a vanishingly small percentage of the population as a whole.

                  How do I normalize for all of these factors, never mind getting a valid, significant, random sample?

                  The best I can come up with is a phone survey, but even that has issues.

                  This is why I'm asking for suggestions. :(

                  Peace favor your sword,
                  Kirk

                  Comment


                    #10
                    Provided you have suitable randomization, a sample doesn't have to be as big as most people would think in order to be statistically significant.

                    Comment


                      #11
                      Honestly I WOULD narrow it down to the specific populations and specific aspects. Even if you're are "only getting X type" the data will likely be more valid. Its better to have an accurate conclusion applying to a select population than a contested, unreliable generalizable conclusion (IE: "90% of all fights go to the ground"). Yes the result is less generalizable but may be more useful to someone reading the article for a specific purpose. Also if you continue to get similar results across the board you could always attempt a meta-analysis to support a general statement.

                      The unfortunate piece of this is there aren't any good studies because its not easy to measure. People getting into a street fight don't actively look for a place they can be recorded first.

                      I think any poll in this matter is going to basically be a poll of opinion which will be a terrible data source full of personal bias. Don't waste your time with that. Look for something you can objectively look at and mark down whether or not it went to the ground.

                      You also have to define if "going to the ground" means the fight ends there, is there for 5 seconds...20 seconds.....a percentage of the time of the fight......etc.

                      Comment


                        #12
                        I agree with Crackfox the only way to realistically get enough data is viewing cctv footage.

                        The only way to do that cheaply and without a lot of issues around applying for footage that may be evidence etc...

                        Is to use youtube, which has already been done by someone. I think you mentioned this in your article in the other thread.

                        If you introduced some reasonable criteria for admission for the videos and then gathered a useful sample size you'd probably get some decent results.
                        sigpic

                        Comment


                          #13
                          Originally posted by judoka_uk View Post
                          I agree with Crackfox the only way to realistically get enough data is viewing cctv footage.

                          The only way to do that cheaply and without a lot of issues around applying for footage that may be evidence etc...

                          Is to use youtube, which has already been done by someone. I think you mentioned this in your article in the other thread.

                          If you introduced some reasonable criteria for admission for the videos and then gathered a useful sample size you'd probably get some decent results.
                          I would not use youtube. You need a method that deals with the boring as well as exciting...you need an "all comers" take on things. CCTV is fine if you can get it. If you talk to some sort of management level for the state police, explain what you're doing and spin it that it might help streamline police martial arts training, you might be able to get an IRB to officially approve the study and get access to all the recordings done from a police cruiser during police interaction provided all personally identifying information is removed.

                          Comment


                            #14
                            Originally posted by CrackFox View Post
                            Provided you have suitable randomization, a sample doesn't have to be as big as most people would think in order to be statistically significant.
                            The number my Profs taught was "100" as the lower bounds of sample size.

                            I admit that I haven't kept my finger on the pulse of Statistical methods since those days, low these many years ago but the courses on Stats that I took at U. has been really helpful in smelling out BS when politicians want to push a particular agenda and start spouting numbers.

                            Is 100 still the lower bounds?

                            Peace favor your sword,
                            Kirk

                            Comment


                              #15
                              Originally posted by BryanW View Post
                              I would not use youtube. You need a method that deals with the boring as well as exciting...you need an "all comers" take on things. CCTV is fine if you can get it. If you talk to some sort of management level for the state police, explain what you're doing and spin it that it might help streamline police martial arts training, you might be able to get an IRB to officially approve the study and get access to all the recordings done from a police cruiser during police interaction provided all personally identifying information is removed.
                              Interesting idea. Thanks.

                              Peace favor your sword,
                              Kirk

                              Comment

                              Collapse

                              Edit this module to specify a template to display.

                              Working...
                              X