Item bias/Conditional ANOVA

Home > Statistics Every Writer Should Know > The Stats Board > Discusssion

Item bias/Conditional ANOVA
Message posted by Rei on September 21, 2000 at 12:00 AM (ET)

In conducting item bias (whether an instrument item is biased towards a group of respondents), conditional ANOVA can be used.
For example, an insturment of 40 items is administered on two samples of respondents in different cultures.
The total score of each respondent is calculalted, followed by recoding each total score into a score level (each score level with at least N=50).
Thus to identify item bias, we can conduct two types of ANOVA.
1) If the effect of culture is significant, it means that uniform bias is present.
2) If the effect of culture & score level is significant, it means that non-uniform bias is present.
(it is probably not suitable here to explain what is uniform bias and waht is non-uniform bias)
Now, my question is, what p-value should be used in examining each test for each instrument item?
If we use the conventional 0.05 level for each test, is there a possiblity of inflating type I error?
But can we treat each test on each item an independent test, thus we simply use p=0.05 as our criterion?
Furthermore, we are actually conducting two tests on each of 40 instrument items, so if we are going to control the inflation of type I error, the p value set will be very very low?
Anyone who has any idea, please kindly enlighten.
Thank you very much.
Rei

READERS RESPOND:
(In chronological order. Most recent at the bottom.)

Re: Item bias/Conditional ANOVA
Message posted by jg on September 23, 2000 at 12:00 AM (ET)

Have you considered co-variance analysis ?

Re: Item bias/Conditional ANOVA
Message posted by jg on September 23, 2000 at 12:00 AM (ET)

It seems to me that you have a general model in which there there is a block effect - typical agricultural ANOVA - that is some treatments are applied by the experimenter while other treatments are applied by nature in terms of which plot the experimental unit uses, etc.

Your $5 contribution helps cover part the $500 annual cost of keeping this site online.