<b> Forum for the discussion of assessment and testing of ESL/EFL students </b>

Moderators: Dimitris, maneki neko2, Lorikeet, Enrico Palazzo, superpeach, cecil2, Mr. Kalgukshi2

Post Reply
Posts: 2
Joined: Sat Oct 04, 2014 3:02 am


Post by oneota » Wed Oct 08, 2014 12:26 am

With the growing demand from major Korean firms for OPIc scores, more and more university students in Korea will be taking that test.

1. Strange questions
My sophomores recently took it. They came back shell-shocked. Their answers on their extended background surveys were totally ignored. Or so they said. For example, a male student who probably doesn't even know what a kitchen looks like (okay, I exaggerate a little) was asked to explain how to bake something. Forget the fact that an average Korean household does not have an oven to bake in: my student said he did not give any answer indicating he knew anything about cooking. Another student was asked about violins in an orchestra. She said she doesn't have a clue about classical music. Many students were asked about the people they met at a hagweon - this of students who had never been to hagweons.

2. Evaluations
The evaluations came back very quickly: in about one week. How-ever, the results seemed strange and not consistent with how I would score the students based on the OPIc/ACTFL can-do statements, my knowledge of the individual students, and my training as a Cambridge evaluator of spoken English (OE). while it's possible I might be inflating their abilities because I know them, I can't believe that I am so far off that a student who speaks fluently and confidently on even technical issues was correctly rated as IM2 (just above students who have excessive pauses and don't go beyond single simple sentences, rated at IM1). What's more, and more telling, is the order of the students. One of my better, but definitely not best, student got the highest score. She got about where I would place her or just below (though she told me she was "speechless" at her score), but above the best students.

3. Basis for judging
While ACTFL gives criteria for scoring, it seems (partly based on what they give and partly by trying to figure out why my students did so poorly) that beyond the IM1 level, the ability to make paragraphs and to think on your feet and make up things or re-direct a conversation as well as the ability to formulate questions are necessary. While TOEIC speaking places a premium on having few pauses or fillers, I can't say whether this is an issue with OPIc. One good thing about OPIc is that, unlike TOEIC speaking, it gives the students time enough to speak at some length.

4. Feedback to students
My students told me that, in contrast to TOEIC results, OPIc scores came with no explanation. All they got was a naked score (NH, IM1, IM2, or IM3). This is all that a potential employer might need, but the students hardly knew where to begin with no specifics on their ability and failings (though obviously these had to have been determined, in order for the scores to be calculated).

5. Response
I have written to both ACTFL and Credu (the Korean administers of the test) about my concern about the appropriateness of the individual questions and gotten no response (other than one returned e-mail addressed to a specific contact person). Considering that the test is expensive (understandable since it is evaluated by trained people rather than run through a computer program) and that students' careers can depend in a significant measure on the results, I find it disappointing that the companies responsible seem monolithically unresponsive.

6. Correlation
Samsung/Credu has put out a fine-grained correlation between TOEIC scores and OPIc scores. I can only guess as to the basis of this correlation. It seems that they have looked at the can-do statements of each major category and then simply divided them perfectly evenly.

7. Breakdown
I have read that it is because of the large number of Koreans taking OPIc that Intermediate Mid (IM) was broken down into three sub-sub-levels. How- ever, I can find no mention of these sub-sub-levels on the material from ACTFL and Credu that gives can-do statements as well as narrative descriptions for each level and sub-level. If any-one can come up with can-do statements or even narrative descriptions differentiating these sub-sub-levels, I would be quite grateful.

Any comments on OPIc, either specific responses or otherwise, would be appreciated.[/u]

Post Reply