adjusting frequency counts in a chi-square
Message posted by Peter (via 131.193.141.96) on June 26, 2001 at 4:57 PM (ET)

Hello, I’m tackling a stats problem and wanted to get some consultation. I am comparing the frequency of subjects who positively endorse a personality item as describing him or her. I would like to determine if more men or women endorse an item with more frequency. For example, I would like to if more men or women will positively endorse “aggressive” as an adjective that describes him or her. So the data is in this format:

Subject Gender Adjective: Aggressive Adjective: Accommodating
1 Male Yes No
2 Female No Yes
3 Male Yes No
4 Female Yes No
5 Female No Yes

I ran a chi-square analysis to determine if the expected frequencies for each cell would be equal to the observed frequencies of endorsements for each cell for each adjective. So the null hypothesis would be that if we observe 50% of men and 50% of women positively endorse an adjective, the analysis would be not significant and there would be no difference between the frequency of endorsements between men and women for a particular item.

However, the problem we have is that the sample size is imbalanced. In this sample, there are 26% men and 74% women in the sample. Some of the chi-square analyses are significant because the analyses are stating that are more women who endorse an item than men. But part of what makes the test significant is that there are simply women than men and hence a greater chance for the women to have significantly greater frequency counts than men for a particular item. SPSS has an option to insert your own values into the expected frequency counts for each cell. So for example: our total sample size is 258 with 68 males (26%) and 190 females (74%). For a particular adjective “extroverted”, a total of 114 subjects positively endorsed that item, meaning the subjects felt the item described them. Of the 114 subjects, 18 were men and 95 were women. Thus the percentage of men who endorsed the item was 26% and the percentage of women who endorsed the item was 50%. Our question is: did a greater number or proportion of women or men endorse the item? Based on the percentages alone, it looks like more women endorsed the item than men. But is it a significant difference? Furthermore, the chi-square analysis that was conducted on this item was based on the assumption that there would be an equal number of respondents for each cell. In this case, the analysis assumed there would be 56.5 subjects in each cell. But this assumes that there were an equal proportion of men and women in the sample, which is clearly not the case. What I did was calculate an expected value for each cell based on the proportion of men and women in the sample. So of the 114 subjects that endorsed the item, we would expect that 84 women, or 74% of 114, would endorse the item. We would expect that 29 men, or 26% of 114, would endorse the item. I entered the values of 29 and 84 into the expected frequency counts for each respective cell for the Chi –Square analysis. I did get a significant effect based on the adjusted frequency counts. Thus, I am claiming that a significant difference does exist regarding the proportion of men versus women who frequently endorse that item, based on the relative proportions of men and women in the sample.

My question is, do you detect any missteps in my logic in conducting the analysis? Furthermore, even though I adjust the chi-square analysis to account for the disproportionate number of males and females in the sample, this still means that I cannot guarantee for certain that should I continue to gather more data and balance the proportion of males and females, that I might get different results. So my adjusted chi-square results should be interpreted with caution. Your thoughts?
i can be reached at pjiman1@hotmail.com

READERS RESPOND:
(In chronological order. Most recent at the bottom.)

Re: adjusting frequency counts in a chi-square
Message posted by JG (via 128.8.23.162) on June 26, 2001 at 6:50 PM (ET)