Job Discussion Forums

[wulfrun]

what's the difference between these two? i've heard different explanations...

[johnslat]

All praise to Google:

"Construct validity is the extent to which your test/scale adequately assesses the theoretical concept that you say it does. Say you made a new test of intelligence for example, you would need to be able to claim that it does distinguish between people at different levels of ability. Usually, this would be done by looking at its relationship with other established tests in the area, so you would compare results on your new intelligence on tests to performance on an IQ test for example (and you would hope that people who score high on your test get a high IQ, and low have a low IQ, so you can claim they're measuring the same thing). This is referred to as convergent validity.
Sometimes you might also compare it to performance on tests which have nothing to do with your theoretical concept (e.g. you wouldn't expect your tests results to be correlated with something that has nothing to do with intelligence), which is called divergent validity; so it shows that you are testing what you say you are, and not something else.

Content validity refers to whether the items on your test actually test what you're looking at, and that the test is representative of it. For example, it would be invalid to include a spelling question on a test of mathematical ability (as it isn't testing what you're looking at), and it would also be invalid to test mathematical ability by just using addition questions (as there is a lot more to mathematical ability than addition, e.g. subtraction, multiplication etc.)"

http://answers.yahoo.com/question/index?qid=20090312005335AASzsoX

Construct validity
Construct validity refers to the extent to which operationalizations of a construct (e.g. practical tests developed from a theory) do actually measure what the theory says they do. For example, to what extent is an IQ questionnaire actually measuring "intelligence"?

Construct validity evidence involves the empirical and theoretical support for the interpretation of the construct. Such lines of evidence include statistical analyses of the internal structure of the test including the relationships between responses to different test items. They also include relationships between the test and measures of other constructs. As currently understood, construct validity is not distinct from the support for the substantive theory of the construct that the test is designed to measure. As such, experiments designed to reveal aspects of the causal role of the construct also contribute to construct validity evidence.

Content validity
Content is a non-statistical type of validity that involves �the systematic examination of the test content to determine whether it covers a representative sample of the behavior domain to be measured� (Anastasi & Urbina, 1997 p. 114). For example, does an IQ questionnaire have items covering all areas of intelligence discussed in the scientific literature?

Content validity evidence involves the degree to which the content of the test matches a content domain associated with the construct. For example, a test of the ability to add two numbers should include a range of combinations of digits. A test with only one-digit numbers, or only even numbers, would not have good coverage of the content domain. Content related evidence typically involves subject matter experts (SME's) evaluating test items against the test specifications.

A test has content validity built into it by careful selection of which items to include (Anastasi & Urbina, 1997). Items are chosen so that they comply with the test specification which is drawn up through a thorough examination of the subject domain. Foxcraft et al. (2004, p. 49) note that by using a panel of experts to review the test specifications and the selection of items the content validity of a test can be improved. The experts will be able to review the items and comment on whether the items cover a representative sample of the behaviour domain."

http://en.wikipedia.org/wiki/Validity_(statistics)#Construct_validity

Regards,
John

[wulfrun]

thanks john for taking the time to reply. i did do the Google search first and came up with these links too. the problem is that they're contradictory, or they're not really clear in what the implications would be in a language testing context. what i'm looking for ideally is two hypothetical examples of ESL tests, one that would have high content validity and low construct validity, and one that would have low content validity and low.

(DELTA assignment...)

[Sashadroogie]

Thanks for jogging my memory on this one. Face validity is the only one I can also recall, as it is easy. I.e. does this look like a profesh test.

As for the other two, the way I remember it is that content validity is the degree to which the test really tests material on the course. E.g. high content validity would be where the learners have been studying present perfect, and present perfect is in the test. But a low construct validity would be if the method of testing did not match skills that the learners have. E.g., if the learners were never asked to write about their past experiences (using present perfect) then the test would have low construct validity, despite being on content. I think. Sorry for the vagueness. Just goes to show that once DELTA is passed, it's past : )

[coledavis]

This brings me back to my masters in occupational psychology.

Content: I agree with the last commentator: it contains relevant materials, and the example given is fine.

Construct: The test has a clear relationship with theories, concepts or measures which are considered to underlie a suitable test. My example here would be that results on the test correlate well with other language tests (maybe relating to present perfect, or to use of language at intermediate level, or to ones which are highly regarded in some other relevant way). A construct is essentially a way of looking at something, a concept that summarises various relevant ideas; construct validity means a strong relationship with such an (accepted) construct.

Face: The user (or his mother) thinks it does the trick. E.g. A Russian parent may be delighted that a test is devoted to grammar. That is high face validity. However, the test may have low construct validity because it does not correlate at all well with tests that show application of linguistic concepts and low content validity in that it may not relate to relevant lexis (for example).

Howzat?