Job Discussion Forums

5h09un · Joined: 01 Jul 2010 Posts: 140

is the writing portion of this exam graded by a human being, a computer or both?

Facing a Robo-Grader? Just Keep Obfuscating Mellifluously

http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html?src=me&ref=general&pagewanted=all

Glenski · Joined: 15 Jan 2003 Posts: 12844 Location: Hokkaido, JAPAN

Considering the rubric, I'd say no robot/computer could come close to scoring it.
http://www.ets.org/Media/Tests/TOEFL/pdf/Writing_Rubrics.pdf

Wikipedia confirms:
http://en.wikipedia.org/wiki/TOEFL#Internet-based_Test_2
The Writing section measures a test taker's ability to write in an academic setting and consists of two tasks: one integrated task and one independent task. In the integrated task, test-takers read a passage on an academic topic and then listen to a speaker discuss the same topic. The test-taker will then write a summary about the important points in the listening passage and explain how these relate to the key points of the reading passage. In the independent task, the test-taker must write an essay that states, explains, and supports their opinion on an issue, supporting their opinions or choices, rather than simply listing personal preferences or choices. Responses are sent to the ETS OSN and evaluated by four raters.

Sashadroogie · Posted: Tue Apr 24, 2012 12:50 pm Post subject:

I've long suspected that robots write the bilge that I have to correct, in other exams. And if you can hit even close to grading 30 scripts an hour, you will be virtually transformed into a robo-reader yourself.

TOEFL has long led the field in automation. I mean, who can forget the automaton voices that are used in the tests?

WHAT DOES THE MAN IMPLY?

spiral78 · Joined: 05 Apr 2004 Posts: 11534 Location: On a Short Leash

It's actual people, or at least reasonable facsimilies thereof. Cool

gator07 · Joined: 26 Sep 2011 Posts: 45

It sounds like a combination is used but the human raters' responses are used for feedback/score deferrals:
http://www.ets.org/toefl/ibt/scores/

naturegirl321 · Joined: 04 May 2003 Posts: 9041 Location: home sweet home

I was just reading this article about robots vs humans and grading.

http://www.npr.org/2012/04/24/151305392/robot-eyes-as-good-as-humans-when-grading-essays

Sashadroogie · Posted: Wed Apr 25, 2012 5:10 am Post subject:

Robo readers not so good at comprehension, coherence etc? Not to worry. The candidates aren't either...

Glenski · Joined: 15 Jan 2003 Posts: 12844 Location: Hokkaido, JAPAN

hippocampus · Joined: 27 Feb 2012 Posts: 126 Location: Bikini Bottom

Wow! Robo-readers! You know what would be cool? If robots could be policeman; they'd be called 'Robo-cops and... oh, never mind.