Job Discussion Forums

[shikushiku-boy]

"It would be impossible to study all these people directly,
so in such situations, the research concentrates on a small
proportion of the overall group - a sample of the total. One
can usually be fairly confident that results deriving from a survey
of a population sample can be generalized to the whole of that population."
Giddens


Deriving solid data from small random samples is a widely used and
proven research technique (and has been for about 50 years).

You'll have to do a bit better than attacking the sample size.


Both from "English Writing Errors of Japanese Students as
Reported by University Professors" by John Izzo. University of Aizu

[canuck]

http://www.hagi.ac.jp/gakubu/izzoweb/ProfRptErr.htm

They based it on 5 Japanese students.

[Gordon]

What is wrong with attacking a sample size? You cannot generalize an entire nation with 5 people! Only an idiot would do that.

I don't disagree that the Japanese are not good language learners, but I have drawn this conclusion from about 1500 Japanese language learners I have had over the past 4 years, not from 5 individuals.

[Chris21]

That research can certainly be criticized for it's tiny sample size. Any quantative researcher worth her salt would tell you that a MINIMUM sample size should be at least 30, and preferrably much higher than that. A sample based on 5 cases isn't even worth reading. There is too great a chance that an anomaly (i.e. outlier, skew, bias, non-random sample) could completely undermine the data.

The study is also 15 years old. SLA research has come a long way since then.

[moot point]

All right, follks. You have attacekd the OP by claiming the stats don't validate themselves.

I want to wave my finger at all the folks who've recently completed an MA in Applied Linguistics/TESOL/etc. in that this is exactly what you have probably done yourself. I know I have. It is termed "action research" and is encouraged amongst anyone in the education sector.

Sure the numbers don't validate themselves statistically, but this is a trend in education to have teachers evaluate their classes upon reflection. I think it is a positive step to support a bottom-up approach. I have also publised several articles on my own personal teaching experiences and those of others based on small-time action research.

The problem, however, as I see it, is that there is no dominant body to collect all the data/results found by individual ESL/EFL teachers. If we could form a body of some sorts to amass the individual results/feedback of "action research" with the ultimate goal of making guidelines then I think our industry would truly benefit. Until that happens we'll still be considered transients....

[moot point]

...and BTW, it is common knowledge that the Japanese students have the lowest level of English amongst Asians. To question that is silly. Look at any internationally recognised test results (IELTS, TOEIC, TOEFL) to find the results yourself.

[angrysoba]

[moot point]

angrysoba, I love your tag name.

I agree with you, where you are referring to accessibility. North Koreans obviously don't have the chance to take a TOEIC exam as much as Japanese employees are forced to take it. In one country it is an honour, and in another it is a headache and/or obligation.

But by and by, if you've travelled around the region and met folks on a day to day basis I think you could come up with a (very staticitally-challenged) feeling that the Japanese are very inept at speaking English.

The reasons for this run deep and is one of a long discussion on society, wealth, pride, L1 demands, etc.

[angrysoba]

[moot point]

[gaijinalways]

[Chris21]

I think Angry Soba's point about low TOEFL scores in Japan is a good one. Japan scores so low on standardized tests like TOEFL becuase virtually everyone in this country takes the TOEFL at one time or another. While in North Korea, I'd be surprised if anyone other than the elite took the test. I recently saw data that showed the number of TOEFL-takers in Japan was almost 300,000, while in NK it was less than 1000. The number of TOEFL test takers in Japan vastly outnumbers even China and India. Given these numbers, I think Japanese students score very well compared to other Asian countires (the Japanese average score was just under 500).

Also, many of the countries that score higher than Japan are former British colonies (i.e. India, Malaysia, Singapore, HK). Clearly, history gives these countries an advantage.

One last point to consider is that in Japan, the cost of taking the TOEIC test is very affordable, while in third world Asian countries it can be equivalent to a month's salary. Test takers in these countries are going to make sure that they are well prepared before taking any test, or risk throwing away a great deal of money. Whereas in Japan, poor preparation and losing the test fee isn't as big a concern.

I would bet that if you did a true random sample of all Asian countries, Japanese students would be at (or near) the top.

[canuck]

[Glenski]

Reminds me of what Robert Day wrote in How to Write and Publish a Scientific Paper.

One anecdote:

33.3 % of the mice in our study suffered no ill effects.
33.3 % of the mice in our study suffered negative effects.
The third mouse escaped and ran away.

[shikushiku-boy]

Perhaps, Chris21 would like to explain what "true" randomness is.

Is Chris21 saying that the USC Japanese freshmen were selected
by 'false' randomness? If Chris21 is saying that, how does Chris21
know that the randomness with which they were selected was false?

Some people are arguing that you cannot draw conclusions about
the English ability of the entire Japanese population based on the
written English skills of 5 USC Japanese freshmen. Perhaps, these
people can quote the section of the OP which says you can
draw conclusions about the English ability of the entire Japanese population based on the written English skills of 5 USC Japanese freshmen.

I would have thought it is more realistic to compare Japanese TOFEL
scores with those of South Korea, rather than North Korea.

I doubt there is another country in the world that devotes the time,
money, and effort to learning English that Japan does.

Q: Are the results achieved commensurate with the time, money, and effort expended?