How accurate is PISA at determining "smartness"

[Jason M]

Came across vital signs it indicates that PISA Programme for International Student Assessment of the OECD shows that Singapore is the world's smartest nation.
USA ranked 20th .

 

[Sensatus]

These tests are averages of the entire student population/tested population. Considering the USA has many millions more students than Singapore, of course ours is going to be decreased. Anyways tests really don't matter, they are used as a marker and that's it, many of the most brilliant people in our country suck at tests, good test scores do not always equal high intelligence or as you put it "Smartness", plus there will always be outliers to the general population in terms of intelligence. Singapore and many other asian countries integrate how to take a test in all aspects of academics, it is not so much about innovation and conceptualizing as it is about scores. Essentially tests like the ACT and SAT and the PISA are all Pseudo-Intelligence tests. The only type of intelligence required is that of choosing answers and seeing a pattern.

 

[billy_joule]

Considering the USA has many millions more students than Singapore, of course ours is going to be decreased, its all averages my friend.

That is not how averages work...
If anything, having a larger sample size increases the confidence in the outcome.

The purpose of PISA is the comparison of education attainment across the world. I haven't seen a PISA test but I'm damn sure they would go to great lengths to make sure it measures what they want; the quality of the education the participant has received, rather what they don't want; their IQ. This a huge international effort to compare and ultimately improve education outcomes, an IQ test would not achieve that.

 

[Sensatus]

That is not how averages work...
If anything, having a larger sample size increases the confidence in the outcome.

What I meant was that depending on the sample size there are going to most likely be a disproportionate amount of people testing poorly compared to those testing well. Let's say 10 kids out of 100 get the highest possible score and then the rest get ok to mediocre scores, that is going to bring down the average significantly. My original point even though I may not have done it justice, is that intelligence cannot be judged by a test and in every country especially one the size of the US is that, like with IQs, there is a disproportionate size difference between those with higher intelligence and those with lower intelligence. With a larger country that disparity shows itself much more.

 

[billy_joule]

What I meant was that depending on the sample size there are going to most likely be a disproportionate amount of people testing poorly compared to those testing well.

Are you saying USA does poorly simply because their sample size is larger?
So some systematic error in the sampling method that increases as the sample size increases?

 

[Evo]

Came across vital signs it indicates that PISA Programme for International Student Assessment of the OECD shows that Singapore is the world's smartest nation.
USA ranked 20th .

That video doesn't work. Here are the rankings from 2012, which seem to be the latest ones being used.
Shanghai, China was #1 in all categories. http://www.businessinsider.com/pisa-rankings-2013-12

It seems there was a calculation error in US schools in the 2009 tests, so it's possible that error is still a problem, too soon to tell, it is being looked into.

• A sampling error in the U.S. administration of the most recent international (PISA) test resulted in students from the most disadvantaged schools being over-represented in the overall U.S. test-taker sample. This error further depressed the reported average U.S. test score.

http://www.epi.org/publication/us-student-performance-testing/

Also, as it has been pointed out, it's not rating intelligence, it's rating test scores, if kids are trained to take certain tests, they are likely to score higher on those tests. The US has been fighting to stop or reduce test training and focus on learning. We've had recent threads on it.