New IQ test

peter smith

Hello smart folks!

I am developing an IQ test and is currently bringing all the tasks together, for one complete test. There will be no time limit on the test and it will be "culture fair", i.e. un-biased in terms of education, cultural upbringing etcetera.

I am going to organize the tasks in three levels: novice (1), difficult (2) and advanced (3). I am not quite sure yet how to divide the tasks and would therefore like to investigate this further with your help. I think that probably ten to fifteen volunteers would suffice.

I have put up one of the tasks on the web and would like you to take a look at the task and replace the question mark with your own answer to complete the set. I will then use the 10-15 first answers to appreciate the appropriate levels of difficulty for the tasks.

I can reveal that a friend of mine actually guessed his way to the correct answer for this task, but without adequate reasoning behind his choice. Probably due to the fact that there are only some few figures that are likely to fall in the question marks place, but in fact there is of course only one correct answer.

Send your reasoning behind your choice of answer to me at petersmith.de (gmail), or send me a PM here on the forum. If too many succeed to solve the task, I will probably have to revise the test. But don't hold back. ;-)

http://www.flickr.com/photos/33083694@N08/3080228094/

I appreciate your help.

/Peter S.

davee123

That may not be the best way to go, unless you have a baseline for the amount of people who actually tried to solve it. That is, I expect that you'll only hear from people who have actually solved the question, without knowing how many actually tried. I believe that generally, IQ's are supposed to be relative to the population, assigning the 'middle' score to an IQ of 100. Hence, to determine a question's relative level, you would want to know what percentage of people were able to solve the problem, not simply how many correct answers were received. Maybe you're farming here for a brighter cross-section of users (since I tend to think smarter people are the type to hang out on a physics & math forum), but I'm hoping that your sample set of participants is sufficiently random across the board to be of more use than entertainment purposes. Admittedly, that's the purpose of many IQ test nowadays anyway, but just saying.

DaveE

Evo

You can't create an "IQ" test, you would just be creating a "brain teaser" list for enjoyment, at best.

peter smith

You might be correct DaveE. I thought (maybe a bit naively) that people might also think that they've solved the problem (when not) and send their rational to me, or be honest enough to tell me if they find it too difficult for them to solve. :-/

I hope people will to this. But I can't be sure. In the mean time I will investigate other alternatives as well.

peter smith

Even so, let's call it a "brain teaser", I will still be norming it. I believe some of the tasks in the "teaser" will be highly difficult, which the norming might show later on. I will probably make a first norming based on the 100 or so hand-ins, after I've published the complete test/teaser.

davee123

You might get a few who would send an incorrect rationale, but by and large, I think 99% of the people who looked and didn't see a solution (myself included) won't send you any sort of email at all. In theory, you could count the number of times the image was viewed, and use that as a baseline, but that number's also similarly flawed-- you don't know of it's a googlebot scraping your page, a genuine viewer, someone browsing your folder, or a repeat viewer who's gone back and looked at your image a dozen times.

Of course, now the email's been removed, so that makes it even less likely that you'll get a response, although you could put the email up on the Flikr page if you really want to (just generally iffy practice to expose your email that way).

The only way to get really accurate numbers from an online style survey is to go through a lot of overhead. The tendency of the web is that if it isn't horrifically simple and easy, you probably won't get a good response. Most people will just breeze by and browse. In this case, there are about 16 possible answers, so turn it into a little submission form where people can submit their answer, along with a text box for reasoning. And have some code on the page for attempting to detect repeat visitors, bots, and so forth. It's not fool-proof, but it would probably work well enough to get a rough baseline.

The way they make real IQ tests would be to get people to physically take the test rather than online. Online is a very anonymous medium, and subject to a lot of potential error.

But as stated, making an IQ test probably isn't what you're trying to do. IQ tests are generally used for placement into educational groups, identifying students who need extra attention, one way or another. They're designed to verify if a student is outside the norm, in order to prevent normal students from being educated too slowly or too quickly, while also allowing non-normal students an alternative education that's better suited to them.

In the "adult" world, IQ tests are little more than fancy merit badges-- sort of like admission to Mensa. Doesn't mean anything (since you can't legally discriminate based on intelligence), but people who score well use it as evidence of superiority. People who don't score well are either defensive about it, or simply forget about it all together. It has some degree of use in psychological studies, probably, but it's not really practical.

DaveE

Evo

How will you know how long they took, if they had the help of others, etc...?

ƒ(x)

Just a tip, normally IQ tests have directions

peter smith

Hi again. As I wrote in the first post, there will be no time limit. But I cannot know for sure if people get help from others.

So far I have gotten two e-mails, both with incorrect answers. I believe about 1 in 10, or fewer, will get this task correct. So far so good.

peter smith

I believe you have many good points DaveE. I am currently using viewing statistics, the amount of answers sent to me and the amount of correct answers as parameters for the initial base. But remember the base of what: I am only going to use the data at this stage to determine level of difficulty.

I believe that 1 in 10 or fewer will get this task correct. So far so good (0/2). The level of difficulty for 1 in 10 or better will be put to "novice". Perhaps even fewer than 1 in 10 will solve this task, but it is yet too early to tell.

davee123

I think you'll want to open this up to a much wider audience if that's the scale you're looking for. In order to more-or-less accurately judge "1/10", I'd guess you'd need 2+ correct responses. Ideally (by my sense of stats), 5 correct responses out of 50 tries. As of now, the Flickr page says it's gotten 47 views (maybe 5-7 being me, checking both the problem and the Flickr status, etc). And I'd assume that both users Evo and f(x) also looked at it. Speaking from web experience (I'm a web developer, and I've seen some hit stats on stuff I've posted here before), I'd guess maybe 10-30 genuine views, and 0 correct answers yet? That's just a guess, of course, but something. Certainly, more than 2 people have looked at the problem. But regardless, I'd recommend posting it elsewhere to get a higher volume of responses.

But otherwise, if 1/10 is "novice", that's pretty difficult. I would encourage you to make it easier, if nothing else than for the sake of the people taking your test. If someone who's 90th percentile in intelligence takes your test and is rated "novice", their instinct reaction may be to hate your test and proclaim that it's unfair. Or they'll simply dismiss it as a worthless test and ignore it. A lot of creating tests of this nature is making sure that people FEEL at least some degree of accomplishment when they take it. If they do, they're more likely to try and spread the experience to others.

I would shoot more for something like <50/100 as novice, <20/100 as tricky, <5/100 as hard, and <1/100 as expert. Or something similar. I guess I still don't understand what you're trying to do with this test, beyond come up with really difficult questions. That's something I know I saw a lot of from teenagers and such, and have seen online from obvious question-writing-novices. Their goal is often merely to stump people with a question, not come up with something that's fun or interesting to solve. For the record, coming up with difficult questions is pretty easy. But if nobody can solve them, that's a failure on your part as a question writer, because you haven't made the question realistically solvable.

The only reason I suspect that may be the case is that you've only seemingly posted a really difficult question-- you haven't posted easy and/or medium level questions to get baselines on those (which generate more balanced statistics, and are therefore better suited to getting statistics on). It's sort of like creating a series of control groups in a science experiment. If you want to trust your statistics for the really difficult stuff, you have to balance that against the easy stuff in order to make sure that you're not totally out of whack. For instance, if you had an easy problem that got 0/2 responses, and a medium question that got 0/2 responses, all you've proven is that you don't know if the questions are any harder than each other or not.

DaveE

ninjaDUDE

timing it would make it more simple to evaluate scores