Student placement testing

<b> Forum for the discussion of assessment and testing of ESL/EFL students </b>

Moderators: Dimitris, maneki neko2, Lorikeet, Enrico Palazzo, superpeach, cecil2, Mr. Kalgukshi2

Post Reply
Molly
Posts: 8
Joined: Thu Feb 06, 2003 1:44 am

Student placement testing

Post by Molly » Thu Feb 27, 2003 1:35 am

We have a problem at the school here in China. They allow the students and or students' parents to choose which classes the students take and at what level they start. The resulting dichotomy in the classroom is predictable. We use primarily the New Concept English textbooks one and two. I am currently trying to develop a proposal to put forth to the school on the necessity of some sort of placement testing, but I want to provide an answer at the same time, not just say "there's a problem". I have never done anything with placement testing before, so I am open to any and all input from those out here who have been there and done it.

stephen
Posts: 97
Joined: Tue Feb 25, 2003 9:06 am

Post by stephen » Thu Feb 27, 2003 2:00 am

Dear Molly

It would help if you mentioned what level(s) and age groups you are teaching.

Establishing a proper system of placing students is problematic at the best of times, but you are in Asia which makes it much worse as "face" will enter the equation. This is going to make it much more difficult to get the students (if they are adults), or their parents (if they are children) to accept the level the student is placed at. On the adults front, I know from experience as I am struggling against unrealistic learner expectations on a daily basis. You defintley need to work out a strategy to deal with "face" if a placement system, no matter how effective, is to succeed.

Regards
Stephen

Molly
Posts: 8
Joined: Thu Feb 06, 2003 1:44 am

Post by Molly » Fri Feb 28, 2003 12:29 am

Stephen,
Thanks for the input. The age group where there is a problem is upper primary and lower junior middle. Fortunately, there doesn't seem to be a problem with the youngest students or adults. I hadn't considered face being an issue, but I should have known. Thanks for the heads-up.
Regards,
Molly

Roger
Posts: 274
Joined: Thu Jan 16, 2003 1:58 am

Post by Roger » Fri Feb 28, 2003 2:43 am

Dear Molly,

I don't want to sound sarcastic but the truth of the matter is: you have too much good faith in the education system of your host country!
Chinese assessments are farcical. Let me tell you this quite without any beating around the bush - I have been seeing it for eight years now!

I don't know what subject you are teaching. I suppose it is ORAL English. If so, then this means your expertise will hardly be appreciated. You are a white face to give the school the sheen of a Potemkin village. Sorry, but all the exams in public schools I have seen were given by Chinese teachers! Do you trust your local colleagues' English proficiency?

Another point is that schools have a custom of passing the same percentage of students every year. They would lose credibility if a disproportionate number of students fail the final exam in any given year! That is why a foreign teacher's opinion on the performance of Chinese students hardly matters! You might rock the boat!

On a more hopeful note:
A private business has hired me to train their staff. The students chose the English level themselves - advanced, intermediate, pre-intermediate, according to the number of years they had studied at school (from 5 to anywhere above ten years!).

I insisted that I give all of them an exam.
My exam consisted of a written part and an oral part.
The written part consisted of mutliple-choice answers and fill-in-the blank statements. In addition, I asked them to write a CURRICULUM VITAE or a RESUMEE.
Knowing that their knowledge of these terms might be shaky I brought three dictionaries with me. They were allowed to use dictionaries throughout the writing exam.
The result was that
- The fill-in-blank part and the multiple-choice answers were done uniformly well, with few people making any mistakes at all.
- The writing of prose, i.e. the resumee, degenerated into an exercise of introducing themselves, which many chose to do in a rather verbose way, not meeting any criteria of a resumee (formalism, conciseness). The English consequently was of varying quality. Luckily, some pre-intermediate students could upgrade to intermediate, and some even to advanced learners. However, I have to teach them how to meet requirements such as writing to the point, following formal guidelines and so on. Some even handed in papers without a name!!!

- The oral was quite, quite disappointing.
Well, don't get me wrong! It was done fairly well by all in terms of answering simple questions such as:
- Where in the text can you find information on topic X, topic Y and topic Z.
BUt in terms of understanding the gist of the information, I must say it was frustrating. Imagine people see a picture of a beetle, a larvae and an egg - and when they are talking about the text that goes with the visuals they cannot pronounce the word "beetle" (bittle, biggle, betel!). Worse, when I asked an unpredictable question containing the word 'beetle', hardly anyone recognised the word 'beetle', with some staring at me as if I did not exist at all!
The worst, however, was that most claimed they had never seen a 'beetle' (or 'betel') - something which is unbelievable as China seems to me to be swarming with *beep* and beetles.
I wonder if they shut nasty things out of their minds. Or if their imagination just is so poor...

I wish you good luck anyway - if you can persuade them that you know English better than they (which will have difficulties going down with them!).

Molly
Posts: 8
Joined: Thu Feb 06, 2003 1:44 am

Post by Molly » Sat Mar 01, 2003 10:19 pm

Hi Roger, I am actually not Molly, I am Brian her husband and just use the same account... anyways, I have in just six short months come to the same conclusion about the public schools, it is hopeless. I was talking about setting up an assessment for the private school we are employed by, they loan us out to public schools but we still have many classes that are at our school.
As far as the question of lack of imagination or some sort of mental block to bad things, my opinion- both.
Brian

stephen
Posts: 97
Joined: Tue Feb 25, 2003 9:06 am

Post by stephen » Thu Mar 06, 2003 11:53 am

Dear Molly and Brian

I think Roger has made some good points. My experience with Chinese learners is based in Taiwan not on the Mainland, but the idea that a native speaker could assess English better than a Chinese person seems to be highly offensive to many of the Chinese, no matter how badly the Chinese person speaks English.

However, that said, if you are looking at establishing some tests, the first thing to do would be to decide what you want each student to be able to do for a certain level. Then you should base your tests around this. It might be useful to have a look at some of the international tests to see what standards they set and what criteria they use. Stuff like the Cambridge Young Learners exams, KET & PET might give you some ideas. Also, I think they might publish their marking criteria, which might give you some ideas for placement. Otherwise establish some standards by consensus amoungst those that will be doing the placement testing. You will obviously need to review these after a period of time.

It will obviously help if the placement test appears as formal as possible, not from an educational but from a face perspective. I would concentrate on productive not receptive skills, because these will be weaker, and because it is unlikely that students will have learned any real reading skills (ie. they will have learnt to translate not to read.)

Hope this helps
Stephen

Roger
Posts: 274
Joined: Thu Jan 16, 2003 1:58 am

Post by Roger » Sun Mar 09, 2003 1:32 am

Dear Brian and Stephen,

sometimes I succeed in getting things my way - for some time! I recently accepted a new part-time job, but made it conditional on students an entrance exam for my classes.
My employer is a PRIVATE company, foreign-owned and engaged in industrial and trading activities, so the GM wanted to see improved communication skills in his senior staff.

I was subtly warned during the job interview that I might run into some resistance on the part of students that had joined the company when it still was a state-owned business. These warnings proved very, very prophetic!

However, here is how I assessed their English:
- A test of their comprehension of a text: Multiple choice answers as
well as ticking a box R or W (two tests);
- writing of a resume. I put two dictionaries on the front desk for
students to use (and I encouraged them to use their own dic/
tionaries as I don't want them to merely memorise things);
- an oral test. In this test, they had to answer a question pertaining
to the test paper they had answered in writing. The text was about
the life cycle of deathwatch beetles and contained graphic visuals.

You might be interested to know that all students passed the first part
of the exam (answering multiple choice questions and ticking b). As for the RESUME, that is another matter!
There were some chaps in the lower two levels that simply filled the page reserved for the resume with English vocables. I have no explanation of why they chose to do this - words of every category, not listed alphabetically nor according to grammar functions, just a hodgepodge of words! If I had seen this in a primary school class, I would have said to muyself the task was over the students' ability, but these were university-trained adults!
Still, there were a majority of texts of prose. Out of some 40 texts, 3 were genuine resumes. The remainder were self-descriptions, that is highly fluent writings about themselves that frequently ended with the somewhat optimistical note that they were hoping to join my class!

Unfortunately, by writing such prose - so different from the laconic style of a formal CV! - I encountered so many grammar trouble spots that I had to use red ink heavily!

On a positive note, I could upgrade a couple of students frintermediate level to advanced level, but on the whole, I had to redefine some of the objectives of the courses:
To improve their OVERALL ENGLISH, by focusing on the effects of poor grammar versus accurate English!

Oh, and the oral English test proved another sore popint: I asked each and every one of them whether they had ever seen a "beetle" in their life! Believe it or not - nobody had seen any beetle before, with many not understanding my question in the first place ("Biggle? Betel? Bittle? I don't understand..."). The text had an accurate drawing of the beetle which looked remarkably like a typical *beep* that I never fail to find in Chinese homes!

Roger
Posts: 274
Joined: Thu Jan 16, 2003 1:58 am

corrigendum

Post by Roger » Sun Mar 09, 2003 1:36 am

My contribution would have needed some proofreading...
Just one correction:
The last line, which mentions "*beep*: In my mind, I was typing "*beep*"

Don't know how this 'beep' got there!

stephen
Posts: 97
Joined: Tue Feb 25, 2003 9:06 am

Post by stephen » Sun Mar 09, 2003 6:44 pm

Roger

Nice to hear that you managed to get a Chinese employer to do things properly rather than paying lip service to sensible student placement. I am constantly battling to get students to take the levels they place into (as opposed to the most advanced which they are not ready for in any context apart from there own minds.)

Regarding your test, I'm always a little cautious about linking oral testing to written material. There is a risk that you are testing reading comprehension as much as oral fluency. Stuff about death watch bettles sounds quite advanced, so I would guess this would place those with weaker reading skills at a disadvantage, even if they spoke with a better level of fluency & accuracy. However, this only comes into play if you are trying to assess both things seperately. This does rather open the can of worms of what to get them to talk about, topics requiring general knowledge would be out (unless these students are radically different from those that have shaped our previous conversations on Dave's forums.) But maybe some questions around topics, possibly things like describing their position and responsibilities, where they went to university, the town/city they live in, or entertainment, could be developed. Of course, it depends on their level, and the higher the level the greater required complexity and hence the more difficult to find things to talk about. But the basic thing is trying to avoid questions that cannot be answered with the ever popular minimalist approach (ie. Yes/No) or that they cannot easily be forced to expand upon.

To give a simple example.
"Do you like .........?"
"Yes"
"Why?"

Also you can often keep going with the on word question approach to push them.

For a very weak student.
"Do you like Shanghi?"
"Yes"
"Why?"
"It is very convienience."
"How is it convienient?"
"It is have good transport."
[Please note learner errors were deliberately inserted.]

For a stronger student.
"Do you like Shanghi?"
"Yes, it is very easy to go to any part of the city. It has many things to do."
"What kind of things?"
"um..you..can..um..go to the movies, or eat lots of delicious food..........."

With the bettle thing...
"Have you ever seen a bettle before?"
No
Possible follow up..
"What would you do if you saw one?" (The only question that sprang to mind)
The way I see it the problems with things like this are, firstly, grammatical complexity, secondly, the fact that "no" is the easiest answer (possibly I should have put this first), thirdly, the requirement of abstract thought, and possibly fourthly, face in that if they admit to seeing one and you ask where they will have to admit that it was in their/someone else's house, and god forbid anyone lose face.

Anyway, let me know what you think. (I hope I'm not trying teach my grandmother to suck eggs.)

As reqards the majority writing prose and not a proper CV. Two things spring to mind. One, do you think they were prepared and memorised answers? I'm sure you know the student thinking at least as well, probably better, than me. It's always a problem! Two, at least you know they can't do it. This means that you know a topic you can start on, assuming their English is good enough. My experience is that even when the grammar is good enough, structure is a big problem (which I'm sure you know already, so I don't know why I'm even mentioning it; too much insomnia and no contact hours tomorrow perhaps.) But if you are planning on teaching correspondence then you might consider a lexical approach (language to execute specific functions, eg. making a point, making a weak complaint, making a strong complaint) and then have them insert them into a structure given to them. (I can recommend a very good book for this which could be tackled by intermediate students and would help with style alot, but I'm not sure what the situations with getting materials is on the Mainland; if your interested drop me a PM and I'll dig up the details.)

Good luck
Stephen

Roger
Posts: 274
Joined: Thu Jan 16, 2003 1:58 am

Post by Roger » Mon Mar 10, 2003 4:47 am

Hello Stephen,

thanks for your informative feedback on my testing. I have learnt quite a few things from this experience, first of all that I need to learn how to do a test.
Normally I can use printed versions and browse through some guide that offers random samples of questions to ask. Normally, I use a NEW CONCEPT ENGLISH test from one of the textbooks. However, this being China decisions are routinely made ad hoc, and the decision to do these tests during regularly-scheduled lessons was sprung on me. I had asked them to schedule tests much earlier, with more time.

In the end, it was not too bad. The students' inability to deliver on the CV/resume requirement is not to be seen too seriously. What it does show is that they have not learnt the exact form of a resume. To me this came as a surprise simply because they were adults and must have applied for jobs before. I felt a resume was best suited to their abilityand interest and came closest to one of the objectives of their English lessons - to learn how to write a report!

As for their inability to understand the word 'beetle', that still rankles me! Maybe, as you said, I should have selected a different topic. However, I did not expect them to understand every word in the text. What a 'beetle' is they could have gleaned not only from the description but from the visuals as well. It is very difficult to decide what went wrong: Perhaps, as I half-suspect, the topic was not "convenient" enough! It's like asking a Chinese person whether they could imagine a divorce in their family! 100% of Chinese would say no, yet this is by no means statistically true! And *beep* are ubiquitous here, so much so that, perhaps, people prefer to ignore them!

I had an alternative question ready for anyone who either spoke too sloor for those who finished early. It was "which place in China is your favourite place?"
I rephrased the question in simpler English when needed, but few had any favourite place apart from their home place!

marukosu
Posts: 7
Joined: Thu Aug 14, 2003 10:26 am
Location: Okinawa, Japan

Post by marukosu » Thu Aug 14, 2003 12:24 pm

Roger wrote:In the end, it was not too bad. The students' inability to deliver on the CV/resume requirement is not to be seen too seriously. What it does show is that they have not learnt the exact form of a resume. To me this came as a surprise simply because they were adults and must have applied for jobs before.
Well, I'm not sure how much help this will be a few months after the fact, but here goes:

Resume-writing is a very culturally-specific task. Despite not knowing much about China, I can pretty much guarantee you that 99.9% of the population has no idea what a western-style resume looks like, what it's meant to be structured like, or what purpose it serves.

As for that strange list of non-related vocabulary terms? You wrote:
There were some chaps in the lower two levels that simply filled the page reserved for the resume with English vocables. I have no explanation of why they chose to do this - words of every category, not listed alphabetically nor according to grammar functions, just a hodgepodge of words!
Remember that you armed them with dictionaries... Look up "resume" in the dictionary--it means "summary." They were summarizing the English they knew for you, by listing all the vocab they could muster... :D

Post Reply