Korean Job Discussion Forums

[Unreal]

I got to school this morning and found a note on my desk. The note was from an anonymous student who said that the students don't think I grade their presentations fairly. I work at a specialized language high school so students are extra sensitive about their English marks. Last time I got a lot of complaints that the grades were too low. Now I'm getting the opposite. I have certain criteria to grade by (given to me by my supervisor) but the criteria are very vague, (for example, "Clarity"), so consistency is not easy to maintain. I get students trying to compare their presentation to other students but after marking 120 different presentations in a week I really can't remember if Jun-soo's pronunciation was not as good as Min-young's.

Does anyone else mark presentations and how do you do it accurately so that students are satisfied? Or is this an impossibility? (Actually only one of my classes has been a problem but they are consistently a problem and I don't want to give them any room for argument.)

[OiGirl]

I assume you have some kind of rubric with your criteria. You could find/develop a sample piece at each scale point and use it to show your students how you grade and help yourself to grade more consistently.

[Unreal]

I have the following categories:

Clarity
Attitude and Posture for the Audience
Gesture and Eye Contact
Argument Structure
Degree of Material Interest and Response from the Audience

This is all I have...it's been translated from Korean by one of the Korean English teachers. Each one has a scale from 2-4, meaning the worst a student can do is 2 (10 total) and the best is 4 (20 total).
The first three are not too bad...but the last two I have a really hard time grading.

[Hanson]

[Unreal]

Thanks for the advice and comments. I do tell the students about the evaluation criteria beforehand and tell them later what they can improve. I think I will give each student a written evaluation with the grade of each section so that they know exactly where they made their mistakes. This will be a lot of extra work, but should eliminate some of these problems.

After thinking about this I realize part of it is the fact that their English speaking ability is only scored through "Clarity". Someone who can barely speak English but puts on a good show can easily score better than a near-native speaker with a slightly boring presentation. I think I will talk to my supervisor about changing the criteria so that English ability scores higher than performance ability...I mean after all it is an English class, not a performance class.

[Woland]

[Hotpants]

Ah, grades, grades...

I do agree with OIgirl about setting in stone some samples under each grade.

Alternatively, can you not show them their grades on a less regular basis? Perhaps collect them all and hand them out at the end of each month. That way, students might be motivated to increase their level of performance each time and you don't have to face such regular onslaught of disatisfied students. Giving too many grades can wear off like any thing else. How would it be if we awarded grades for absolutely everything a student said: "Hi, how are you?" "I'm fine, thank you." "Okay, that's a 'B' because your voice is too throaty."

Also, as your students are apparently capable enough in English to give presentations, and they seem to know what grades are 'best', then how about giving them the role of assessing each other? I've taken this tack because grading can get dull, and it can also be subjective according to the mood I'm in on that day. Now, grading is much more fun. Students will listen more carefully to others if they need to assess what they hear. Students also seem to have a much greater recollection in distinguishing between other students' performances. You either provide all students with assessment forms; select a panel of 5 students each time to be on the evaluation team; or you can ask students to call out what grade they think you should write down. They can also utilize discussion skills as to why they are awarding such grades, and they can be more brutally frank with each other - I tend to be too generous in order to avoid confrontational students. My boss frowned on me letting students grade each other, but I thought what the heck, the boss has no idea of how effective it works out for everyone's English language skills.

[PRagic]

This is what I use for speaking exams. See what you think:

Fluency (Flow and clarity of conversation, not speed)
Accuracy (Grammar, expressions, and idiomatic speech if applicable)
Presentation (Posture, attitude, eye contact...call it as you see it)
Pronunciation (Depending on whether or not you cover this in class)
OPTIONAL (Preparation - depending on whether or not they are preparing their conversations in advance without your input).

5 points max for each category. Has worked well for me.

[HapKi]

In my "Speech and Presentation" class, I pass out an Assessment Score Sheet to 3 random students of the audience, which counts as 50%, while mine counts as the other 50%. We try to get through 4 different styles of presentation through out the semester, including self-introduction, a process "How to.." presentation, informative, and a persuasive selling of a product, a la the Home Shopping Network.

For the current informative presentation, I'm using 10 criteria judged on a 1-5 point system, including:
well-prepared, original, natural voice, use of charts and graphs, eye contact and no reading, length, structure that follows the outline, and so on.

[Woland]

[PRagic]

Good point. See the post above by Woland. Seems like a pretty good set of point schemes. Meant to add that.

[Woland]

[Kyrei]

I have always viewed my scale is fairly simple, especially in comparison to my colleagues who tend to have pages and pages of things to grade. Although it remains highly subjective, it quite often reflects numerically where I feel students belong in the long run. Granted that since I am the only rater, that makes sense, but still, I have had but few problems with the results from students and have alwasys been able to justify the grade sufficiently. I am, however, not in the position that the OP was in terms of high competitiveness among fluent speakers. My students overall tend not be fluent speakers. Having fluent speakers (for example, students who have lived/grown up abroad) in a class that generally are not is a problematic issue that I deal with case by case, although generally, and justly, they get higher grades. Sometimes the world just isn't fair. For those students, effort and attitude factor highly into the values of the following scores.

Here is my scheme. Each individual presenter is give a score out of 15:
Manner rated from 0 - 5:

This category covers their overall presentation manner in terms of speaking VS reading a paper (noting that is it not the essay they had turned in before the presentation which is open on my desk as I watch), making eye contact, general confidence in their presentation. The time that they present also factors into this as a large penalty if they are over or under time. The numeric scale matches a conceptual A (5) thru F (0) scale.

Quality rated from 0 - 4:

This category covers the quality and depth of information presented, its coherence, and the quality of the actual PPT or posters used (or other visual aids). This follows an Excellent (4), Above Average (4), Average (3), Below Average (2), Poor (1) to Highly Unsatisfactory (0).

Fluency rated from 0 - 3:

This category covers basic fluency in speech: do they stutter, inflect naturally, stumble, lose their place and have to start over, etc. and follows a simple scale of Very Good (3), Good (2), Bad (1), Terrible (0).

Clarity rated from 0 - 3:

This category covers pronunciation, enunciation, spelling in the PPT, a basic overview of their grammar usage, and idiosyncracies like giggling, laughing, covering their mouths with hands, etc. Basically anything that gets in the way of my clearly understanding what they are trying to impart. The numbers follow the same semantics as for Fluency.

The total points available for each presenter is 15.

There is a 5 point "Group Mark" which also has five categories, each rated 0 - 5 and then averaged out: Informative, Interesting, Visual Aids, Effort Shown, Length. Since these are averaged out they follow a simple A (5) to F (0) scale and I make a simple qualitative judgement.

Informative is basically how much information they impart in their given time. As I have done this assignment for several years, I have certain preset expectations for what they are going to tell me.
Interesting is, taken with a grain of salt, the audience reaction, as well as my own (eg. are they putting me and others to sleep...).
Visual Aids and Effort Shown are catch-all categories for how well laid out the overall PPT is and how much work they have done in collecting graphics, organising who speaks when and says what, and my observations of them during prep time.
Length is just as it sounds: did their presentation fall within the accepted parameters of time allotted? They start out with 5 and every 10 sec beyond or below drops it another point.

I mark notes in an open comments area for each student about the number of times it struck me that they were "reading" or "speaking", if they were "mumbling", "clear", "hard to understand" or "confident", to name but a few, and I add Xs and Checks beside comments to strengthen or negate them as the student progresses through the presentation. This allows me to look back at the marks and adjust them based on the comments I have made if such seems necessary. For example, if this is the third terrible presentation in a row, I know that I am getting a little cranky so I can look back at the comments made (with modifers like Xs and Checks, etc.) and make sure that those comments seem to match numerically with comparable comments and modifiers other students in other classes or groups.

If a comment is underlined, then it is a strong support, i.e. "reading" underlined twice means reading all the way through the presentation and not looking up much at all (which would be a 1 or 2 on Manner if they at least read a decently prepared paper that was NOT the essay they had turned in the week before). If they simply read the same essay and/or faced the screen (and read that) they get 0.

The Mid-term presentation is graded on the same scale of 20 points for the presentation (15 individual and 5 group), with 10 points for each individual essay (but that is a topic for another post) as the final. When they ask for the results I will tell them, which is often enough to make the Final presentation ten times better. Since the grades are curved anyway, their intrinsic values are not truly representative - their individual values will depend, in the long run, in the overall total and where said student fits on the class ranking before they hit the number cruncher. For curving I use a very handy "square root system" that I can explain if anyone is interested. It is all handled on an Excel spreadsheet that I made up which assigns letter values, calculates how many As and Bs I need, etc. It is a pretty handy sheet. Again, I can send it to people if anyone is intersted. PM and we'll talk.

In the past, I have tried to do peer evaluation but in a class that by nature is competitive in terms of grades available, it was pointless. They all slammed each other! In one class I remember, no one scored over 10 out of 15, which factors out to no one getting over a D+ for their presentation! I still use it as a listening tool but do not take the grades into consideration (although I don't tell them that!).

[Unreal]

Wow, you guys have some great ideas. I really liked Wolan's first post and the idea of making it perfectly clear what each score means. After talking to my supervisor I came up with a new evaluation sheet. I wanted to focus more on English speaking and ideas than performance. Here's my new evaluation sheet, which I think covers everything I'm looking for:

Pronunciation (4 points)
Excellent: All words are spoken well and are easy to understand
Good: Some words not correct but can be understood by context.
Fair: Some words are difficult/impossible to understand.
Inadequate: Whole sentences are difficult to understand.

Fluency (3)
Good: Student speaks clearly and fluently without hesitation.
Fair: Student�s sentences are a little choppy, a bit unclear or too fast.
Inadequate: Student takes long pauses to think or read.

Vocabulary Level and Variety/Wrong usage (3)
Good: Proper level of vocabulary is used to communicate.
Fair: Some words are inappropriate or generally simple.
Inadequate: Too simple vocabulary or misused used words hinder meaning.

Grammar/Sentence Structure (2) (tense, number, articles, appropriate sentence length)
Good: Proper grammar is maintained throughout presentation.
Fair: Some grammatical errors but meaning is still fairly clear.
Inadequate: Grammatical errors hinder meaning of some sentences.

Contents (2) (Arguments are focused and clear and there is good evidence for them)
Good: Clear arguments with good examples
Fair: Argument is not quite clear. Some examples are not adequate.
Inadequate: Argument is hard to understand and lacking good examples.

Appropriate, Challenging and Interesting Topic (2) (Is it thought provoking and relevant?)
Good: The topic is fresh and creative or addresses the topic in a critical way
Fair: Addresses an assigned issue but lacks creativity or critical thinking.
Inadequate: Topic is not one of the ones assigned or fails to hold interest.

Attitude, Gestures, Eye Contact (2)
Good: Maintains a smile and good eye contact. Uses appropriate gestures and movement.
Fair: Keeps a smile and eye contact occasionally. Gestures are few and movement is minimal or inappropriate.
Inadequate: Reading, very little eye contact. Very few gestures/movement.

Audience Response (2)
Good: The audience is very interested in the topic and concentrates on the speaker. Audience wants to be involved in the presentation.
Fair: The audience is paying attention but shows no strong response.
Inadequate: The audience appears to be uninterested.

[vdowd]

One way that I have kept the whole class engaged (28 students) when doing a debate/talk show/presentation is to assign the rest of the group roles as reporters or artist (courtroom, like).

The students can choose reporters or artistic reporter - that way the students who are less able in spelling, writing have an equal chance to contribute. I post the best of their "news reports" on the board as well as the assessment of the group. It does also help me to be less subjective in marking - sometimes the reports have points about the presentation that I did not notice.

Sometimes, I also say that every student listening will have to ask a question of 1 of the panel. Keeps them thinking.

However, I like the idea of also having a student panel involved in the assessment and will use this next time, too.