Matura in Poland - exam results - skewed statistics

[Borek]

This is ridiculous.

I couldn't decide where to post it, as it is a little bit about politics and a little bit about stastics, so there are reasons to post in other forums as well. It can be even a brain teaser (see the question at the end of the post).

For those of you who don't know - matura is a serious exam ending stage in education, usually taken when you are 18-19 years old. Matura consist of several separate exams - compulsory Polish language, mathematics, other modern language, plus additional exams, to be selected by student.

Exam is graded outside of schools, by selected teachers who don't know the students (that was changed several years ago, as earlier grading was done in schools, which was criticized as being not objective - teachers could treat students differently). After changes number of students passing matura dropped down enormously - before it was around 95%, this year number dropped to record low 75%.

Central commission responsible for matura published the initial report about last year results (pdf in Polish). On page 16 there is a plot of results of the Polish language exam - as graded by the independent examiners. Note that grading language exam (when part of the exam is a short paper on the given subject) is never perfectly objective.

So - can you guess what was the number of points necessary to pass?

 

[Borek]

I think you missed one important part - those grading have no idea where the exams come from. There is one exam and one key to answers used throughout all parts of Poland.

Edit: Take a look at the page 9 of the linked document, lower table - percent of passing by voivodeship. There are regional differences, but they are small. Now take a look at page 10 - upper plot. Differences between school types. Almost threefold differences.

 

[Hells]

What is ridiculous? The test itself or the number of points needed to pass? From what I know, the level of education is so uneven that they have to give lower passing points. Some of the eastern parts really have to catch up with the western ones. They should be looking to Germany, how they did it.

According to the charts, some passed that shouldn't have passed. There's no reason to expect a jump at 20 otherwise, it should have followed a smooth bell-curve. There's no way that's coincidentally.

 

[Willowz]

I think you missed one important part - those grading have no idea where the exams come from. There is one exam and one key to answers used throughout all parts of Poland.

Edit: Take a look at the page 9 of the linked document, lower table - percent of passing by voivodeship. There are regional differences, but they are small. Now take a look at page 10 - upper plot. Differences between school types. Almost threefold differences.

I'm not an expert. So, looking at document there is a pretty even average in each district. So, how many points were needed? 21? Out of 70?

 

[Willowz]

According to the charts, some passed that shouldn't have passed. There's no reason to expect a jump at 20 otherwise, it should have followed a smooth bell-curve. There's no way that's coincidentally.

I don't think they want to hold up too many people in schools. So they give them one extra point.

 

[Borek]

I don't think they want to hold up too many people in schools. So they give them one extra point.

You don't need to pass matura to leave the school. You leave the school - with matura, or without. In one case you are given a school certificate, in the other case you get school certificate and matura certificate.

 

[Proton Soup]

the shift in that curve above doesn't appear near enough to account for 20% difference.

edit: well, i guess i should say the discontinuity does not appear to account for 20%. that doesn't mean that there isn't also a bias that shifts the entire mean to the right.

 

[Willowz]

You don't need to pass matura to leave the school. You leave the school - with matura, or without. In one case you are given a school certificate, in the other case you get school certificate and matura certificate.

EDIT: Sorry for the mistake. The schools aren't held up but I think it is better economically. So, that people get into the workforce quickly.

 

[berkeman]

I don't think they want to hold up too many people in schools. So they give them one extra point.

I think it's either that (the graders trying to help those who are close to passing on their own), or else maybe some students just want to pass the test, but don't care about their score beyond passing. So they take the first part of the test, see that they made the 21 points, and turn the test in. Show-offs!

 

[Willowz]

I think it's either that (the graders trying to help those who are close to passing on their own), or else maybe some students just want to pass the test, but don't care about their score beyond passing. So they take the first part of the test, see that they made the 21 points, and turn the test in. Show-offs!

Also. Just so that it doesn't look that bad. After you fail at matura you have 5 tries (5 years or maybe this has changed). So you get quite a lot of people who want to pass it, if they didn't pass it the first time. So, that spike might also partially account for those who are barely passing it the second/third/ect time. [If you don't pass it the first time it's not likely you'll get a very high score the second or third time, but just enough].

EDIT: I don't know if that graph also included the people who were taking the test another time.

 

[lisab]

Nice detective work, Borek !

Where I work, we employ auditors to go into production facilities to observe and audit onsite labs. That curve is just the kind of thing they look for to see if there's any 'fudging' going on.

 

[Borek]

Nice detective work, Borek !

Credit is due somewhere else (I am not even sure where), I just copied it here.

Similar effects can be seen in some other exam results (mostly humanities - languages, history), but nowhere it is as pronounced as here.

 

[BobG]

I think this is human nature. I wonder if you plotted the reported speed of drivers receiving speeding tickets, how many drivers were going 2 mph over the speed limit. I bet the frequency spikes up pretty quickly at 6 mph.

They probably hate to fail a person by 1 point. The fact that they never have to confront the person prevents a spike on the other side. (I have seen the occasional instructor, that when confronting a student that's failed by 3 or 4 questions, tosses out a couple questions just to prove they're a nice guy, but not enough questions for the student to pass.)

Even stranger are a few other phenomena that would seem like they'd be free of any of those types of quirks, such as the yard-line that a football is spotted on. (I'm not sure if the graph will display, since it doesn't seem to display on the computer I'm using now, but does on my home computer).

http://www.advancednflstats.com/2008/10/whats-frequency-kenneth.html. The football is spotted on, or very close, to one of the solid white lines on the field than any of the areas between the lines. (If one were a football coach deciding whether to challenge the spot of the ball, this would be something to consider.)