Grading Software fooled by BABEL

BABEL, the Basic Automatic B.S. Essay Language generator, is software created by Les Perelman and others [chronicle.com] at MIT. Perelman was a "Director of Undergraduate Writing" at MIT. He has used BABEL to generate nonsense essays and feed them to automated essay grading software. The BABEL output gets high marks with sentences like "Privateness has not been and undoubtedly never will be lauded, precarious, and decent." I think Perelman has a point here. Clever students will no doubt learn to game any such grading system to their benefit. Teachers must question what, if any, benefit an automated essay grading system has.

Back in 2005, Perelman discovered an excellent predictor of score [nytimes.com] on an SAT essay test. It was the length of the essay. No other variable he examined correlated nearly as well with the score. The top scoring essays had many factual errors, too. No matter, according to SAT. The writing quality depends not on the correctness of any facts, according to SAT. Perelman's advice for scoring well is to practice writing fast and make up facts. Perhaps the high scorers can land a job with Fox News?

A paper by Perelman in the Journal of Writing Assessment critiques automatic scoring of essays.

Hybrid Pedagogy

Earlier today I stumbled onto hybrid pedagogy, a website for the eponymous open access journal, while reading a post from Rebecca Schuman, a blogger at Slate who covers education. The director, Jesse Stommel, is an assistant professor at UW-Madison. Give their site a whirl if you're into hybrid learning and pedagogy.

Text Set for NSA Mass Surveillance

Text Sets are a tool teachers can use to scaffold instruction for struggling learners. They generally focus on one topic or theme and are a very useful tool for improving comprehension and literacy. Text sets popped into my head when I found this article about the NSA and its mass surveillance. The author does not use the term "text set," but that is what he's proposing to provide context for the journalistic coverage of the NSA following the revelations by Edward Snowden last summer.

Anyone familiar with the Snowden/NSA story has no doubt read an article in which a journalist has compared the NSA to Big Brother in George Orwell's novel "1984." It's a metaphor with legs (is that a meta-metaphor?). Mr. Berlatsky points out that 1984 is not a book that paints the most relevant picture of the current government-sanctioned surveillance of the citizenry. He argues that other works do. He even goes so far as to say that 1984 can "enslave thought." I think he may be referring to the phenomenon of journalists who repeat what other journalists have already tweeted or written, even after a statement has been found to be unsupported by the facts. On The Media is all over this case.

On to the text set, culled from the article. A caveat: this text set might not be appropriate for scaffolding.

• Phillip K. Dick, Do Androids Dream of Electric Sheep?
• Franz Kafka, The Trial
• Arun Kundnani, The Muslims are Coming
• Maureen F. McHugh, China Mountain Zhang

to which I would add these nonfiction works:

• Patrick Radden Keefe, Chatter
• Bruce Schneier, Secrets & Lies
• James Bamford, The Puzzle Palace