I'm stumped

Home > Statistics Every Writer Should Know > The Stats Board > Discusssion

I'm stumped
Message posted by Laurie Meisler on May 15, 2000 at 12:00 AM (ET)

This is definitely statistics 101 but I'm stumped about interpreting the data of a survey I conducted. I asked voters to select the individual whom they think does the best work in a specific area and then to rank the individual they selected as "strong", "average" or weak" in a specific characteristic. Well, it turns out that the person who ranks first (meaning he got the most mentions) got 12 strongs, 2 averages and no weaks in this characteristic while the person who ranks fifth got 6 strongs and no averages or weaks. It looks like the fifth place guy ranks highest in this characteristic but isn't he benefiting from getting fewer votes? Is there a calculation I could do that takes number of responses into account? Help!

READERS RESPOND:
(In chronological order. Most recent at the bottom.)

Re: I'm stumped
Message posted by Jim on May 16, 2000 at 12:00 AM (ET)

Did you include an option in your scale that said "Don't know" or "Not aware of candidate"? It sounds like 12 respondents had an opinion about your first candidate and only six responded in relation to candidate 5. However all six thought this person was "strong". What you may not know is do those who did not respond in relation to this candiate have an opinion about him at all. Or do they simply know nothing about him? If they did would they also regard him as "strong"?

This really boils down to interpreting non-responses and how they might affect the actionability of your research. It sounds like a questionnaire design problem and the only remedy may be to recontact those not responding in relation to candidate 5 and trying to find out more.

Re: I'm stumped
Message posted by Phil on May 20, 2000 at 12:00 AM (ET)

It is not exactly clear what the instructions were. That would be important. But consider using "total scores" for each person, not just the averages. That way both the number of mentions and the ratings are taken into account.

Then rank by total score.

(We used to audit quality by frequency and severity. We gave one point for being a defect and then 1, 5, 10, or 20 points depending on the severity of the defect. We then calculated a total score for each unit audited. Demerit systems like this generate useful data over time for various purposes.)

Your $5 contribution helps cover part the $500 annual cost of keeping this site online.