« A data communications starter kit for university re-opening | Main | Answers on excess deaths, test accuracy and quick testing »

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

Roel Peters

Hi there, it's brave to be the devil's advocate in this post. But I'm not sure the attack on the model is misguided. I don't think many argue that the rate of A-levels should be around 25%. The problem are the parameters that go into the model. The final paragraphs of this post elaborate on this briefly: existing inequities are propagated through the model. The feeling of no/less impact on the predictions must be tough to cope with.

Kaiser

RP: Of course, in the U.S. also, any measure of school achievement is highly correlated with household income and housing prices. Asking a model to resolve this deep-seated problem is asking too much. If society is willing to agree on targets, such as downgrading "good schools" so their students get fewer As than before, then a model could be built to operationalize that. But now we will get cries of reverse discrimination. I'm thinking the biggest mistake in this A-Levels exercise is to have teachers grade on the same scale as the actual test. Will elaborate on this in a new post.

Thomas Dietterich

Isn't the main issue that the grade should depend only on the student and not on the other factors? When the student sits the exam, the score they earn is entirely determined by them. But this was replaced by a model of "guilt by association", or perhaps we should call it "upgrade/downgrade by association". Such a model has the causality wrong, and is therefore rightly viewed as unjust. While a statistical model with group predictors may give accurate predictions in aggregate, it is not appropriate for making individual decisions. The entire effort is ill-conceived.

Kaiser

TD: I hear that argument, and I'm grasping at a response to it because the entire industry of predictive modeling (or its other monikers, superviesd learning, AI) is guilt by association. The entire recommendation systems (collaborative filtering) industry is also guilt by association. Predictive policing is guilt by association. Clinical trials too. And I've also been preaching that a shortest-path solution for the average driver does not yield the shortest path for each driver. It's a deep question.

Ian Watkins

The predicted grades from teachers were not necessarily an example of 'grade inflation'.

If, in a class of 10 pupils, you know that 4 are capable of achieving an A/A* then you should predict their grade at that level. You also know that 1 of them is likely to mess up and not achieve their grade (this could be through misreading a question, or a change in home circumstances, etc) as historically you achieve 30% A/A* grades. Which of your A/A* pupils are you going to downgrade arbitrarily?

Another issue with the algorithm is that it assigned grades to pupils at schools from failure (ungraded) upwards, meaning that ungraded was filled first. If the algorithm decided that there was a non-zero chance of ungraded at a school then someone was given a U. This also leads to downgrading as the lower achievement grades are filled before the higher ones.

And the question that I don't think I've seen asked, why are so many pupils failing to achieve the grade they're capable of when faced with exams? Are exams the correct method for pupil assessment?
Why is it that grades are inflated this year, when you could say that grades are deflated in previous years because of exams.

Kaiser

IW: All discussion of grading always come back to whether pupils or people should be graded at all. What is the purpose of grading?

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Working...
Your comment could not be posted. Error type:
Your comment has been saved. Comments are moderated and will not appear until approved by the author. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.

Working...

Post a comment

Comments are moderated, and will not appear until the author has approved them.

Your Information

(Name is required. Email address will not be displayed with the comment.)

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR, Wired.

See my Youtube and Flickr.
Numbers Rule Your World:
Amazon - Barnes&Noble

Numbersense:
Amazon - Barnes&Noble

Search3

  • only in Big Data

Next Events

Jan: 10 NYPL Data Science Careers Talk, New York, NY

Past Events

Aug: 15 NYPL Analytics Resume Review Workshop, New York, NY

Apr: 2 Data Visualization Seminar, Pasadena, CA

Mar: 30 ASA DataFest, New York, NY

See more here

Courses

R Fundamentals, Principal Analytics Prep

Numbersense: Statistical Reasoning in Practice, Principal Analytics Prep

Applied Analytics Frameworks & Methods, Columbia

The Art of Data Visualization, NYU

Signed copies at McNally-Jackson, NYC

Excerpts: Numbersense Ch. 1, 7, 8. NRYW

Junk Charts Blog



Link to junkcharts

Graphics design by Amanda Lee

Community