The Limitations of IQ Tests: An Insider's Perspective

[Zero]

Yes, I have declared it, so it must be true...the whole concept of an IQ test is sort of silly, and should be ignored. I should know, I've been subjected to about a dozen professional IQ tests over the years, and countless online tests(scores ranging from 150-170). The thing is, there's a pattern to the tests, a way of looking at them, that makes them easy to score high on after you have taken a few of them.

I guess they serve some sort of purpose, but I think their usefulness is very limited. Go ahead and attack me now...

 

[selfAdjoint]

Yes, your IQ is high enough that ordinary IQ tests, and certainly those online ones, don't measure you accurately. And the fact that you find you can "look" at them a certain way and see how to ace them confirms it. I ran into this attitude all the time when I was studying math in graduate school. The organ cannot perceive itself; if you are smarter than the people who designed the test, it's not going to work.

But this just means you are about 1 in 5 or 10,000 on the IQ distribution. For 99.9% of the population IQ tests are valid and consistent.

How do you do on the Raven matrices?

 

[Zero]

Originally posted by selfAdjoint
Yes, your IQ is high enough that ordinary IQ tests, and certainly those online ones, don't measure you accurately. And the fact that you find you can "look" at them a certain way and see how to ace them confirms it. I ran into this attitude all the time when I was studying math in graduate school. The organ cannot perceive itself; if you are smarter than the people who designed the test, it's not going to work.

But this just means you are about 1 in 5 or 10,000 on the IQ distribution. For 99.9% of the population IQ tests are valid and consistent.

How do you do on the Raven matrices?

I think I should have been a little clearer...I think the ability to do well on a standardized test is a combination of inherent ability and familiarity with standardized tests. IOW, if you test a group of people of approximately equal intelligence, and the variable is experience with standardized testing, the more experienced at testing itself will tend to score higher.

(BTW, I scored in the low 150s when I did the Raven testing...I don't know exactly where that puts me.)

 

[Evo]

If a person scores high on an IQ test it is a safe assumption that the person is intelligent. However, if a person does *not* score high on an IQ test it is *not* an accurate indicator of intelligence.

There are so many factors that can prevent a high IQ score for a person of high intelligence...illness, stress/anxiety (relationship problems), lack of sleep, malnutrition, etc, and some people just don't test well...

This is why IQ scores cannot be relied upon as a true indicator of intelligence because the tests fail to recognize a lot of high intelligence individuals. I remember reading recently of a nobel prize winning scientist that scored horribly on IQ tests.

I think too much emphasis is being put on IQ scores and not enough on what a person's abilities are. A person with a high IQ isn't necessarily going to do anything worthwhile with their life, while a person with a lower IQ can go on to achieve great things. This happens all the time.

I also scored very high on IQ tests requested by my school when I was 11. My results blew everyone away and caused quite a stir, but I can't hold a candle to the level of knowledge of most of the people on this forum. I never applied myself.

 

[Zero]

Thanks again, Evo...that was part of what I was thinking. Obviously a high IQ means something, but a lower one doesn't mean as much as people think it does. Plus, from my own experiences, those little factors you mentioned have a lot to do with the variations I saw in my own test results. My situation on the day of any given test could drop me 10-15 points.

 

[NateTG]

IIRC, 150-170 puts you about four standard deviations up, so something like 1 in 5000, but, since it's off the chart, the result essentially worthless beyond 1 in 100 or 1 in 1000.

If you want to discredit IQ, Flynn's effect is a much better place to start than a small number of self-selected samples.

Flynn's Effect is that the average IQ is rising. The average rate is about 3 IQ points per decade or about 10 per generation, and is most pronounced on tests like the Raven matrices.

Another reason to bring this up is that the meaning of the result obviously depends on the sample that the test was normalized against, so if you were taking older tests, your score would be "artifically inflated."

 

[Zero]

Originally posted by NateTG
IIRC, 150-170 puts you about four standard deviations up, so something like 1 in 5000, but, since it's off the chart, the result essentially worthless beyond 1 in 100 or 1 in 1000.

If you want to discredit IQ, Flynn's effect is a much better place to start than a small number of self-selected samples.

Flynn's Effect is that the average IQ is rising. The average rate is about 3 IQ points per decade or about 10 per generation, and is most pronounced on tests like the Raven matrices.

Another reason to bring this up is that the meaning of the result obviously depends on the sample that the test was normalized against, so if you were taking older tests, your score would be "artifically inflated."

Well, I'm glad to have you educated types around to help me out...I'm not exactly a polymath, but I do tend to get around, but I do so as a layman in most fields. Mostly, this thread is an intuitive reaction to a lot of the bandying about of IQ scores as though they have great significance.

 

[russ_watters]

I'll jump on this one too. I have what was considered a high enough IQ to put me in the "gifted" ("special" ) program in elementary school. I also did great on the SAT. However, I have a bad short term memory and short attention span. As a result, my grades in school were always pretty mediocre.

For me, other factors outweighed IQ and I did not live up to my supposed potential schoolastically.

 

[Jikx]

You know as they say, you're the most intelligent at being yourself.

 

[kuengb]

IQ tests are definitely obsolete.

You can discuss about their "accuracy", but that's not the point. They are a useless try to satisfy men's wish to measure themselves on all levels. To classify people by "objective data", going so far that little children in school get their future perspectives on a piece of paper after having passed through an IQ test. Already the concept is ridiculous: to describe the most complicated structure in the whole world with a single number? Please. In addition, IQ depends only on the "logic" part of the mind.

 

[dyslexic]

IQ tests

Very much enjoyed reading the replies. Having taken many such tests over the last 3 decades, I can, in a small way, relate. While the number results were interesting, they were only of value when I was taught how to use this intelligence, and more importantly, what to do with it. Starting school in 1960, there was not much that was known about various, shall I say, learning styles. Thus, several labels were applied. Not until some 20 years later was accurate testing available to render a corrected and accurate diagnosis. What I have learned though (as has been previously mentioned by others on this thread) is that what is most important is what one does with the intelligence they have. I am constantly amazed and humbled by the people I have met over the years and what they have accomplished--truelly remarkable in many cases. The more one understands the mind, how it works, what it is capable of, the greater the results. With informed teachers, mentors and self-determination a person can reach suprising results.
I work in field that I was told I would never be able to. I was also told I would never get accepted to any college ever. Well, after 9 years I graduated college, and work in the career I was told I had no chance of success in (been 20 years now in that career!)
So, are I.Q. tests bunk? They are, if one is not taught what to do with the information. If a person is taught what to do with the information from the testing, then I.Q. tests can be a road map to learning and applying that knowledge to the best of ones abilities.

 

[Dooga Blackrazor]

I agree what what's being said. ICQ tests are based on different information than what's taught to everyone amongst other things. If your smart it doesn't mean you'll be sucessful anyway. IQ is only a certain category I think. I used to score really High on ICQ tests as a kid now I do horribly. Age affects it right now so it's hard to say what it is. But I'm doing reat in school so that's all that really matters.

I'd say wait till your in your career and such to get an accurate IQ test done. But really it's not very signifigant. I'm getting over my IQ phobia. I used to take them really seriously and get upset. It's just a test though. All you can do is apply your own talents and do your best. Hapiness is what matters I think. :)

 

[hitssquad]

IQ test construction

Originally posted by Dooga Blackrazor
ICQ tests

What is an "ICQ test"?

are based on different information than what's taught to everyone amongst other things.

The information portions of IQ tests such as the Wechsler and the Stanford-Binet are based on information that tends to consistently differentiate well between members of the population. Statistical reliability and validity are the goals in IQ test construction, and test items that fail to meet standards in either of those realms are culled during the test construction process.

For further reading on this subject, the best textbook available on the subject of IQ test construction is Arthur Jensen's Bias in Mental Testing (1980), which is listed as a "citation classic" by the Institute for Scientific Information (ISI; publisher of the Science Citation Index {SCI} and the Social Science Citation Index {SSCI}).

 

[Chen]

Originally posted by Zero
IOW, if you test a group of people of approximately equal intelligence, and the variable is experience with standardized testing, the more experienced at testing itself will tend to score higher.

First of all, how will you find a "group of people of approximately equal intelligence"? But that's besides the point, which is that level of experience one can achieve depends on one's intelligence. Even if two different people study for the IQ test together (in the sense of learning to know the questions, time management, whatever) it doesn't mean they will both get the same high mark. Sure, their marks will probably be higher than the marks they would get if it wasn't for the studying, but neither the increase nor the mark itself will be identical.

(English is not my native language so I may sound a little confusing, my fault. )

 

[selfAdjoint]

Perfectly clear and correct, Chen. Arguments that assume you know the intelligence and then validate the intelligence tests are begging the question. And it is absolutely true that a more intelligent observer gets more out of studying that another. Studying is in itself an intelligence test!

 

[Chen]

Doesn't the IQ test claim to determine the intellectual potential of a person? Even if it's accurate at determining that, having a lot of potential doesn't guarantee that you will take advantage of it, and someone with less potential could take advantage of all of it and do better than you. I would think that the "amount" of potential you put to good use depends a lot more on your characteristics and other environmental factors, and even then we are dealing with human beings and the unexpected.

 

[hitssquad]

Indifference of culture-loading to reliability and validity

Originally posted by Chen
First of all, how will you find a "group of people of approximately equal intelligence"?

Test a population and split it randomly in half. The two resulting populations will be equal in g minus the variation from true randomness of selection for the split.

their marks will probably be higher than the marks they would get if it wasn't for the studying, but neither the increase nor the mark itself will be identical.

This was Arthur Jensen's point when he said that the fact that a given test can be taught-to does not disqualify that test as an effective discriminator.


--
I have not found in the literature any defensible proposal for a purely objective set of criteria for determining the culture-loadedness of individual test items, and perhaps none is possible. This is not the same as saying that there are not objective measures for determining test bias, a topic to be taken up shortly. As we shall see, one can determine with objective statistical precision how and to what degree a test is biased with repect to members of particular subpopulations. But no such objective determination can be made of the degree of culture-loadedness of a test. That attribute remains a subjective and, hence, fallible judgment. Because there is no a priori basis for assuming that all subpopulations are equal in the ability that a particular test is intended to measure, items cannot be ordered on the culture-loading continuum simply according to how they discriminate among various subpopulations.
--
Arthur Jensen. Bias in Mental Testing. p375.

 

[hitssquad]

The IQ test as a tool for diagnosis

Originally posted by Chen
Doesn't the IQ test claim to determine the intellectual potential of a person? Even if it's accurate at determining that, having a lot of potential doesn't guarantee that you will take advantage of it, and someone with less potential could take advantage of all of it and do better than you.

This is why IQ tests are used to help diagnose learning disabilities, personality disorders, and other impediments to success that may be sensitive to targeted remedies. The more disabled a person is, the more useful -- from a standpoint of addressing the disabilities -- it is to know that person's IQ.

 

[dyslexic]

I.Q. testing

Again, very much enjoy reading the replies. In 1983 Dr. Howard Gardner wrote a fascinating book, "Frames of Mind", in which he discussed the Seven Intellingences. They are: Logical/Mathematic,
Verbal/Linguistic, Visual/Spacial, Bodily/Kinesthetic, Musical, Intrapersonal, Interpersonal and Naturalist (this was recently added).
He can be contacted at Harvard University at: www.harvard.edu/Pls/HG.htm.[/URL] There are also several websites that address this, just search under The Seven Intelligences.
Dr. Gardner's study of how people learn and use intelligence can be very helpful as he focuses not so much on numbers, but what intelligence is and that all persons posses strengths in each of the categories. In addition he explains how one can use this information.
Hope this will be helpful.

 

[hitssquad]

Howard Gardner

Originally posted by dyslexic
In 1983 Dr. Howard Gardner wrote a fascinating book, "Frames of Mind", in which he discussed the Seven Intellingences.

--
GARDNER'S SEVEN "FRAMES OF MIND" AND MENTAL MODULES

Howard Gardner has been perceived as a critic of g theory and of tests that mainly reflect g, such as the IQ. I suspect that this is partly, if not largely, the basis of the popularity accorded Gardner's views, especially in educational circles, as many teachers feel desperate over the wide range of individual differences displayed in their classes. If a child has a low IQ and is doing poorly in school, there are, according to Gardner's theory, $$^{[32]}$$ several other kinds of "intelligence" in one or more of which the child may excel. Two of the seven "intelligences" claimed by Gardner -- linguistic and logical-mathematical -- would considerably overlap the conventional IQ. The remaining five "intelligences" are spatial, musical, bodily-kinesthetic, and two kinds of personal "intelligences," intrapersonal, or the perception of one's own feelings, and interpersonal, or the perception of others' feelings, motives, and the like (also called "social intelligence"). As exemplars of each of these "intelligences" Gardner mentions the following famous persons: T. S. Eliot (linguistic), Einstein (logical-mathematical), Picasso (spatial), Stravinsky (musical), Martha Graham (bodily-kinesthetic), Sigmund Freud (intrapersonal), and Mahatma Gandhi (interpersonal). In an interesting book $$^{[33]}$$ Gardner gives biographical analyses of each of these famous creative geniuses to illustrate his theory of multiple "intelligences" and of the psychological and developmental aspects of socially recognized creativity. When I personally asked Gardner for his estimate of the lowest IQ one could possibly have and be included in a list of names such as this, he said, "About 120." This would of course exclude 90 percent of the general population, and it testifies to the threshold nature of g. That is, a fairly high level of g is a necessary but not sufficient condition for achievement of socially significant creativity.

Gardner's seven "intelligences" were not arrived at through the factor analysis of psychometric tests, but are identified in terms of several kinds of categorical criteria, such as the extent to which an ability can be impaired or preserved in isolation by brain damage, the existence of idiots savants and prodigies in the particular ability, a common set of information-processing operations, a distinct developmental history, evolutionary plausibility, type of encoding in a symbolic system, modular or domain-specific abilities revealed by laboratory tasks, and the finding that psychometric tests such as IQ have low correlations with at least three of Gardner's seven "intelligences."

The boundaries of these criteria seem vague or elastic and one can easily imagine other "intelligences" that could be admitted by such criteria. Why is there no "sexual intelligence" (Casanova) or "criminal intelligence" (Al Capone)?

Some of Gardner's seven "intelligences" clearly correspond to well-identified group factors, such as linguistic (or verbal), logical-mathematical (or quantitative reasoning), and spatial. Tests of these abilities are all highly g loaded, and many elements of musical aptitude have been found to be moderately g loaded (see Chapter 8, p. 223). Other of Gardner's "intelligences" are not yet quantified or measurable in a way that makes it possible at present to assess their g loadings or their place in the factor analytic hierarchy. Some may not meet the criteria of mental abilities as set forth in Chapter 3, but are rather products of psychometrically identified abilities and certain personality traits (see Chapter 14, pp. 572-578). The completely nonquantitative nature of Gardner's theorizing about "intelligences" makes it impossible to assess their relative importance in terms of variance accounted for in the total range of human variation or in terms of their predictive validity in real-life situations.

As interesting as his theory of "multiple intelligences" may seem from the standpoint of literary psychology, in which Gardner has no betters, it is hard to see that it contributes anything substantively new to the taxonomy of abilities and personality discovered by factor analysis.

In fact, it is hard to justify calling all of the abilities in Gardner's system by the same term -- "intelligences." If Gardner claims that the various abilities he refers to as "intelligences" are unrelated to one another (which has not been empirically demonstrated), what does it add to our knowledge to label them all "intelligences"? All of them, of course, are abilities (as defined in Chapter 3), several qualify as group factors, and at least three of the seven are known to be substantially g loaded. To assign to the remaining traits the label "intelligences" makes no more sense to me than regarding chess-playing ability an athletic skill. (After all, playing chess requires some little physical activity, and chess players are jokingly called "wood pushers"). Bobby Fisher, then, could be claimed as one of the world's greatest athletes, and many sedentary chess players might be made to feel good by being called athletes. But who would believe it? The skill involved in chess isn't the kind of thing that most people think of as athletic ability, nor would it have any communality if it were entered into a factor analysis of typical athletic skills. Gardner's analogous extension of the ordinary meaning of "intelligence" probably serves more to make people feel good than to advance the science of mental ability.

In summary, I find nothing in Gardner's writings that could be considered a technically meaningful or coherent criticism of g theory. Gardner is at his best in writing about persons with some unusual accomplishment to illustrate his theory of different kinds of "intelligences." Galton, in his Hereditary Genius (1869), recognized that a high level of general ability is a necessary but not sufficient condition for outstanding achievement. Besides an above-average level of g, an exceptionally synergistic combination of special abilities or talents and personality traits is always found in the kinds of outstanding exemplars of Gardner's several kinds of "intelligences," such as the famous persons mentioned above. Most psychomtricians would probably agree with the criticism of Gardner's theory of "multiple intelligences" in a recent textbook $$^{[34]}$$ on "intelligence": "I have argued that a consideration of several sources of evidence used by Gardner to establish the existence of independent intelligences may be used to support the existence of a superordinate general intelligence factor. Thus I find his taxonomy to be arbitrary and without empirical foundation. Neither his rejection of a superordinate general factor [g] nor the specific subset of intelligences that he postulates appears to have a firm theoretical or empirical basis" (p. 40).
--
http://www.questia.com/PM.qst?a=o&d=24373874. pp128-130.



32. Gardner, 1983. This is the main exposition of Gardner's theory of "multiple intelligences."

33. Gardner, 1993.

34. Brody, 1992.

dyslexic wrote
Dr. Gardner's study ... can be very helpful as he focuses ... on ... that all persons posses strengths

Then he is what is known as a motivational speaker.

 

[6626068]

Originally posted by Evo
I remember reading recently of a nobel prize winning scientist that scored horribly on IQ tests.

That makes sense. IQ tests only measure a narrow range of mental faculties namely pattern recognition (ie. Raven Progressive Matrices). There are so many facets of "intelligence" that I feel IQ tests just totally skimp out on.

 

[Kerrie]

okay, i didn't want to read through every post here, but i completely agree with zero's initial point that IQ tests are bogus...they only measure a small perspective of intelligence, over a broad range of talents (creativity, musical ability, artistic ability, social ability, etc)...unfortunately, the "IQ" or typical book smart has become the indicator of real intelligence in society.

i was given all sorts of geomety and puzzle tests when i was 8...in school i was put into TAG programs, and i became intimidated by school and education because of the "push" to do my best...i agree with doing your best, but pusing a child into it is almost discouraging the free and creative use of their own talents...although i was labeled as TAG, i never finished college because of the pressure to do my best perhaps, but i still use my brain in fun activities--such as PF :)

 

[Kerrie]

hitsquad...i have read up some on Gardner's theory of multiple intelligences...very fascinating...

 

[hitssquad]

Is the general factor of mental ability actually a specific factor?

IQ tests are bogus...they only measure a small perspective of intelligence

Do you mean

1. IQ tests don't largely measure the g factor?

or

2. the g factor -- instead of being the common source of variance in a broad range of abilities -- is itself actually a narrow ability?

or

3. the g factor is the common source of variance in a broad range of abilities, but its variance accounts for little variance in success compared with that accounted for by variance in ability-profile?

unfortunately, the "IQ" or typical book smart has become the indicator of real intelligence in society.

Children rescued from severely adverse conditions (such as years continuously isolated in dark attics with no toys or other intellectual stimulants) have been found -- though they had, initially upon rescue, zero literacy and almost no ability to communicate -- to have normal or higher IQs. What do you mean by the "IQ" or typical book smart?

 

[Adam]

From the replies here it seems very few, if any, have actually done a real IQ test. They do not measure only a small range of mental faculties. They do not only measure pattern recognition capabilities.

 

[motai]

My personal experience with IQ tests have been frustrating. I remember back in elementary school I was never accepted into the "gifted" program and it rather demoralized me, as I knew that I could have done better. According to them I was unable to succeed in higher-level courses. They were wrong. Some of the people that I knew in the "gifted" program weren't so gifted at all, some eventually fell into the wrong crowd and resorted to juvenile-like behavior.

I also think that the IQ test establishes a negative effect on study habits. Those with self-proclaimed high IQ test scores will think that they can slide through school while doing little to no work (i.e. "My IQ score is high so I don't have to study or do my homework").

 

[dyslexic]

IQ Tests

Just finished reading the latest replies to the I.Q. question. HitSquad, very much enjoyed your dissertation concerning Dr. Gardner, well written. I have read and taught extensively about g factor (I am a former special education high school teacher), and indeed this must be taken into account when doing workups on students. Typically we administered a battery of written and oral tests to get a detailed analysis of the student (this often took many months). Special Education tests are usually administered twice during high school. Once this was accomplished, we then worked up an Individual Education Plan. The I.E.P., when used correctly, and with parent involvement / support, is a wonderful tool for the student and the teachers involved. Usually when a student is about a sophomore in High School they are prompted to start writing the I.E.P. themselves with some guidance from teachers and parents. By the time the student is a senior they should be able to write the I.E.P. themselves and present it at the I.E.P. meeting at the beginning of the school year. Unfortunately this does not always happen, nor do most parents and students know much about this program or section 504 of the education laws. I.E.P.'s can be written for students in Special Education or Gifted Programs. College students can even write a type of I.E.P. under section 504. I know this is somewhat off the beaten path of the tread, but thought this may be in some small way helpfull to the overall I.Q. question.
If a person is to be tested for learning ability, or some lack thereof, one must look at what we called the 'Fund of Knowledge' of the individual. If one is given several types of I.Q. Tests (each test is designed to test specific brain function to a more or less degree), pyscomotor ability, cognitive function, application of knowledge, retrieval and storage ability,etc., the researcher, teacher and student/client can get a fairly accurate picture of how their specific mind works. My task was to assist the students with developing their strong areas, and using those strong areas to compensate for areas that were weak. An individual test by itself was of use, but only gave me information about specific areas of the students brain. Some tests, such as for g, give a broad overview (sometimes much more specific) of the individuals brain functions. Couple this with MRI results and other such tests, it is truly amazing what one can learn! Of course this type of testing can be very expensive.
However, despite all of the advances made in the last 20 years regarding I.Q., brain function, learning styles; there are still many areas that are not adequately addressed, and far too many people fall through the educational system (note some of the replies to this forum). Some are lucky and make or find their own path through pure stuborness, will to succeed, what have you. Some are just overwhelmed by circumstances (I was also a crisis/suicide counselor for many years in adolescent/adult mental health).

In my own case I did not read above the 3rd grade level until I was a junior in high school. A dedicated English teacher taught me how to read and most importantly how to interpret and decipher words, sentences, etc. I was accepted to a junior college. There another English teacher taught me how to write so that others could understand (still not too good at this) a little of what I was trying to write. Later, I was accepted to a private university and after 9 years of college finally graduated. It was not until I was teaching special ed that I finally found out that I also was not too different from my students. Up to that point I was given the usual diagnosis for students that do not perform well in school (not much was known about learning disabilities/differences in the 60's 70's). As with many, my I.Q. testing was totally baffling, as I expected it to be much, much lower than what the tests indicated. I still read about 25 words a minute, have little or no math ability (yet understand quantum physics and the mathematics that goes with it as if were basic adding/subtraction. My mentors have not yet figured out why I can). Cannot spell without an electronic dictionary, stutter quite a lot when not working in my profession (teaching pilots about Human Factors, advanced aircraft systems, weather, flying advanced aircraft, all the usual pilot stuff). Yet, I would not trade my situation for another--I enjoy my challenges to a great degree, even though at times the challenge is very difficult, but there are also many more advantages than challenges.

So to conclude this rather lengthy disertation. To obtain a true picture of I.Q. a battery of tests should be taken. Once the results are in, one must seek guidance as to how to interpret the information so that maximum benefit of the information can be used. Finally, one must learn to apply the intelligence so as to be of benefit to themselves and society. It is a life long quest.
Again, I hope this was of use to someone, and pardon for the long reply, I generaly don't write lengthy replies-in future I will keep my replies shorter.

 

[hitssquad]

Chapter 5, Challenges to g

HitSquad, very much enjoyed your dissertation concerning Dr. Gardner, well written.

I can't take credit for it since I cut-and-pasted it from Chapter 5 ("Challenges to g") of Arthur R. Jensen's 1998 book The g Factor (as noted in my post).

 

[selfAdjoint]

Dyslexic, your testimony belies the claim you made that you have trouble with writing. It was wonderful! Clear, passionate and cogent.