IQ Tests

For a bit of fun I did the “IQ Tickle test” a year ago and just received the “detailed results” for free – of course I wasn’t going to pay for them when I did the test. While psychometricians may claim that a test is valid in some sense, the tests often have certain problems, and this IQ test demonstrates some of these limitations. I’ve been working as a test developer for a year, and so I’m more aware than previously of the pitfalls of tests (eg, a test can only capture very limited aspects of anything).

When I was a kid, I loved IQ tests, but of course, as you get older you become aware that things are much more complex than you thought they were as a kid, and in any event, there are all the problems associated with giving any credence whatsoever to test results, eg, what does a certain test result actually mean?

One question on this test in particular irked me:

John likes 400 but not 300; he likes 100 but not 99; he likes 3600 but not 3700. Which does he like:

 
    900
    1000
     1100
     1200
 

(The arrow was my response and the tick is the “correct answer”.) The “answer” is supposedly 900. Why? Well, apparently all the numbers that John likes are squares of integers, those he doesn’t like are not squares of integers. This sounds fine at first glance, so ok, but this is only one possible answer.

The other three responses could equally well be “correct”, and this can be seen by turning the question into the following one which, to me, seems almost logically equivalent:

Given three points on a sheet of graph paper with x and y axes, which of the four following points also lies on a polynomial passing through the three given points and NOT passing through three other specified points.

(A difference between this problem and the Tickle problem is that points on an x-y plane need two pieces of data to describe them, while only one piece of data is given in the Tickle problem. Nevertheless, the principle is not dissimilar.)

Of course, any mathematician will immediately say “using Lagrange interpolating polynomials, I can, in principle, construct a polynomial passing through all three given points and also any one of the four potential responses and not passing through any of the points so specified”. 

The “answer” of 900 appears to be somewhat arbitrary.

In addition to this logical problem, there is also a problem in the way the Tickle question is constructed. Is John’s dislike of 300 related to his liking of 400, or does he just dislike the number 300 anyway, ie, are the numbers 400 and 300 specially connected in some way? This is unclear in the question. If I were writing the question I would say something like:

“John likes some numbers and dislikes others. Some of the numbers he likes are 400, 100, 3600. Some of the numbers he dislikes are 300, 99, 3700. Which of the following numbers does John also like?”

A problem with this wording is that the “pattern” of squares is not broken by the non-squares, so I would guess that more people would choose the “correct” answer.

I couldn’t work out a relation between the numbers he liked and the numbers he disliked, and so I guessed the answer (very usual behaviour for test takers). I was very surprised to learn that the “answer” was connected to squares of integers. Perhaps the “simplest” answer is what is required in these things. But what does “simplest” mean, and why is the “simplest” answer the correct one?

These are the sorts of questions that need to be raised in the construction of any test, and often they are, but can they be resolved in a satisfactory way?

Advertisements

41 Responses to IQ Tests

  1. Steve Edney says:

    Sacha, you are clearly thinking too hard about this! I agree that you could have some other generating function to get the others but these patterns are meant to be the most obvious.

    I often find this on these sorts of questions if they have some visual pattern, if you look at them hard enough you can detect some other “correct” answer on the basis of differnet criteria.

  2. Sacha says:

    Hey Steve, maybe I am. But this leads to the question of what the “obvious” or “simplest” pattern is – and to me a pattern involving squares is not any more obvious or simple than a pattern involving some other polynomial – they’re just polynomials – what makes one more “simple” than another?

    What you mention about visual patterns is interesting. If there is potentially more than one “correct” answer but this is not recognised, what then is the value of the question?

  3. […] Sacha Blumen thinks too much about an IQ test question. […]

  4. David says:

    Hm, I don’t know Sacha. I’m not a mathematician and I read the tickle question you presented once, thought about it for 5 seconds and got the right answer. I had to read your alternative question several times before I worked out what you were getting at. I’ll take your word that you can construct a polynomial that makes any of the answers correct – but is it as simple as y = x-squared? Can you explain it in words? I don’t think an answer like “John likes numbers that are 33.7 less than 2.58 times the square of an integer” would convince me.

  5. blumensacha says:

    What makes one polynomial “simpler” than another? You’d need some measure of simplicity and you can make up all sorts of measures – perhaps by “simple” one means a canonical measure. People learn about y=x^{2} pretty early on so maybe that’s why it’s seen as simple.

    Alternatively, why should any “pattern” underly the numbers 400, 100, 3600 (and not underly 99, 300 and 3700) ? By saying that he doesn’t like 99, 300 and 3700, I suppose that the question is implicitly asking you to say what is it about the good numbers that makes them good, and what is it about the bad numbers that makes them bad. The thing is, you can write down a very large number of “patterns” which coincide with these good numbers and miss the bad numbers.

  6. David says:

    Well here are two measures of simplicity – you can work it out in your head, you can express it in a simple sentence. Are there any patterns other than ‘They’re all squares of integers’ that you can see just by looking at the numbers and then express in a simple sentence? Maybe, but I can’t think of any.

  7. Sacha says:

    They’re all non-zero odd integers multiplied by 100.

  8. David says:

    What? No they’re not.

  9. Sacha says:

    Obviously they’re not! How about non-prime integers multiplied by 100.

  10. David says:

    I’m probably being annoyingly nitpicky but that obviously couldn’t be the pattern because that would make 3 of the 4 answers correct and you need to find a pattern that only fits one of the choices.

    I take your point about the construction of these tests Sacha – I’ve certainly seen questions where there’s more than one answer that seems to make just as much sense depending how you look at it – but I just think this example isn’t one where there’s a problem: there’s a straightforward answer to this question. Maybe it’s biased in favour of people who oversimplify and don’t appreciate the complexity of patterns – I’m not sure – but I think it’s a fair and well constructed question.

  11. Sacha says:

    The point I’m making is that it is possible to construct a very large number of “patterns” underlying the data, whether in your head, on paper, by using Lagrange interpolating polynomials or perhaps otherwise, and that there is not, then, one unambiguous answer.

    There may be an answer (eg 900) that many people select – but is this a question which sees whether you respond in the same way that many other people respond, or is it asking you to ascertain “the” regularities underlying data ? Any reasonably trained mathematician/scientist knows that given a finite collection of data, you can’t specify “the” rule underpinning it.

  12. Gummo Trotsky says:

    That question irked me too. Maybe that was because I couldn’t get past thinking “Who cares? This John guy is obviously some kind of wingnut so whatever his motivations for liking 100 and not 99, I’m just not interested.”

    Maybe the question was testing for the ability to ignore John’s obvious psychopathology long enough to identify its underlying “logic”. Was there a follow up question on how you would treat John’s peculiar mental condition?

  13. Sacha says:

    *laugh* – no there wasn’t.

  14. Tel says:

    Hi Sacha,

    Horrible question, the “obvious” answer depends on the assumption that John is as mathematically naive as the writer of the question.

    This month’s issue of the AustMS gazette has a nice article by Imre Bokor:

    Problem 1. 1,2,3,x,… Find x.

    He goes on to give examples which show the question is ill-formed. The question in the intellignece test and many questions like it are ill-formed and thus do not have a solution.

    See Bokor’s article, well worth the read

    http://www.austms.org.au/Publ/Gazette/2006/Mar06/bokor.pdf

  15. Sacha says:

    Hi Tel, thanks – I’ll check out Imre’s article.

    Agree with you about the questions in the “IQ test” – one could be forgiven for thinking that the “correct” response to many of the questions reflects a very limited imagination or knowledge – so are many of the questions in these sorts of tests really just seeing if you respond the way the question writer thinks you should respond – ie, there is no underlying logic to it?

  16. kriz says:

    wat are they answers to all the questions on the tickle iq test?

  17. Sacha says:

    Nup, not going to write them down – there’s even less point to it then.

  18. wickedpygmy says:

    perhaps one needs an IQ of a certain level for the answer to be obvious?

    Is that not the purpose of the test?

  19. Sacha says:

    The problem, wickedpygmy, is that there is no obvious answer – the supposed answer relies on the reader discerning only a small number of possibilities (say even one), not a multitude of possible answers as is actually the case. This is explicitly shown for a related problem in the very good article that I linked to in the following post:

    https://blumensacha.wordpress.com/2006/03/20/tels-comment-on-my-iq-tests-post/

    If these sorts of questions are about people answering in the “obvious” or “most simple” way amongst all the (infinite in this case) possibilities, then they are about people being able to determine what is meant by “obvious” or the “most simple”. What do these terms mean?

    A number of people have said to me that a polynomial of degree n is more simple than a polynomial of degree > n, but when pressed for why, do not give a reason. By simple, perhaps they mean that the fewer the number of zeroes of the polynomial, the “simpler” it is – but this is the number of zeroes of the polynomial – it’s not whether one is more “simple” than another. To me a polynomial of degree 3 is not “simpler” than one of degree 4.

    What is obvious to person A may be far from it for person B. What this question appears to be about moreso is whether or not the person taking the test thinks like the person writing the test. But the test purports to be something else.

    As Dr. Bokor in the article Tel and I linked to writes, this is all about ill-posed questions, ie, questions that are written in such a way that there is not enough information given to answer the question uniquely. That’s why it’s a shocker. Tel’s e-mail summed it up.

  20. Patrick says:

    I agree with your view entirely, I thought the pairs of numbers were especially attatched to each other in some way. I think that it shows more the way that your mind works rather than its capability which seems a fundemental flaw in these sort of IQ tests; everyones mind works in different ways and shouldn’t different answers to these questions with different possibilities thus show how your mind works rather than your intellect?

  21. Sacha says:

    Yes Patrick – and you could design a test question to try and do this, especially if you’re interested in evaluative tests, ie ones designed to give information about thinking processes rather than testing for a “correct” answer.

    Questions that your comment generates are: what value do these sorts of very limited tests have? What value can be placed on them? Why are these tests ascribed a certain status vis a vis other sorts of tests?

  22. Michael says:

    Excuse me for adding my two cents in.

    IQ tests are fundamentaly tests of pattern recognition.

    But…. Sometimes the test writers screw up. They make assumptions w/ regard to the patterns to which an individual has been exposed.

    If you have never seen a pattern you are unlikely to recognize it.

    Another of the questions on that (and other) test(s) is anagramatical.

    Ask an individual, deprived of certain basic knowledge and however brilliant, they won’t be able to decypher ‘England’ if they have never heard of the place.

    That same individual may have a slim chance of divining the mathematical relationships in the question at hand, but it is not bloody likely if they have never been exposed to the concept of ‘roots.’

    Any question is open to interpretation, but a question based upon assumption is flawed from its’ initiation.

  23. I found this thread when that question had me stumped. The preceding questions were quite dull and straightforward, but that one stopped me. As far as I am concerned, that question beat me fair and square. The test maker did not blow it, you and I did.

    Squares of integers. That is clearly the most elegant solution. Positive integers are the most fundimental concept in math. A square is one of the most basic geometric shapes. Squares are fundamental to natural law; for example, the force of gravity, or electromagnetism, decreases with the square of the distance.

    Multiplication is generally known by the third or fourth grade. As general knowledge, it is fair game for an IQ test. Even if you don’t know the squares as such, a number being multiplied by itself is intrinsically striking. And an earlier (dull) question was a series of squares.

    “Using Lagrange interpolating polynomials, I can, in principle, construct…” is not universal knowledge; it is specialist knowledge. An IQ test designed for the general population is invalid, if it requires specialist knowledge. When you take an IQ test, if your reasoning requires specialist knowledge, then your reasoning is wrong. If simpler reasoning using more basic knowledge would arrive at another answer, then that is the correct answer.

    If a phenomenon can be described by more than one theory or equation, the simplest is the best. Thus saith Occam’s Razor.

    I was factoring the numbers, looking for factors John didn’t like, counting the factors, looking at the shapes of the numerals, the letters contained in the English names of the numbers, etc. None of that worked, but squares do, simply, and beautifully. If you don’t see that, then you don’t truly appreciate Mathematics.

  24. Sacha says:

    Sorry ‘Nother David, I disagree with you.

    What is meant by “simplest”? This is not cut and dried. What is simplest to one person is not necessarily simplest to another, and so setting one meaning of “simplest” is problematic. In addition, I disagree with the notion that “the simplest is the best”. This seems to me to be arbitrary. “Best” in what sense?

    The problem is not the existence of IQ tests, but rather the meanings that are assigned to them. It’s fine to use questions in an evaluative sense, as in, certain responses might mean certain understandings, but you can’t say that there is one correct answer if there is more than one correct answer.

    In my current job I help to construct maths tests, and so I’m very aware of these problems.

    A good article to read on this is Igor Bokor’s “Mathematics” vs. Mathematics at http://www.austms.org.au/Publ/Gazette/2006/Mar06/bokor.pdf

  25. Eric says:

    There is an issue regarding regarding differential abilities to arithmetically process capability and to conceptualise higher level mathematical concepts. The two are not the same. Being low in the former and high in the latter can create some very weird and seemingly inexplicable results for a student .

  26. BeenThere says:

    Tickle Super and Classic (much lower score ceiling) IQ tests can only be for practice due to the ability to basically do what you want with them repeatedly. If you score the max after several attempts I believe it proves some capacity. But go on from there for the real thing. If you scored low try it more often, maybe when you are completely rested with coffee in hand. There’s no time restraint, The best open net tests are those that simply allow you to research and study as much as you want to. It would be impossible to use Tickle in any official way. The business entrepreneur test was an eye-opener. I totally failed. But went from 152 to 168 by answering the character string question the way they wanted. Because the many tests are free and cover so much ground take ‘m, that is if you can ignore the provocative adds sufficiently. Finally, my yearning subordinates, in a pro neuropsych test lab situation (beyond face-to-face) your capacities really come out with highly accurate results. At home alone, people tend to be too laid back or nervous (especially in timed tests). Does your savvy not come out best when in a challenging people environment? Never pay unless you just want one group’s interpretation and ego massaging. It is first and foremost about mass internet sales and people getting rich. Tickle presents itself as PhD certified all over the net. To many academics that means “Piled higher and Deeper” following the acronyms, B.S. and M.S.=more of the same. The expertise is not merely in who compiles the tests but how it is securely certified. If is for fun and a little self-revelation for the general public – well it’s the internet street vendor world, no? But I like some of their testing and have benefitted free of charge, discounting the crude ads. It’s like so many money machines with academics believing they are sincere in renting out their degree qualifications to entrepreneurs who will stop at nothing. So, try to PAY nothing and get all you can out of it for free. Some of the tests can help in revealing aspects of one’s self. I finally got a human at Tickle recently. The person made more blatant grammatical errors than sentences. S/he must have been at home drinking warm milk at 3 A.M.

  27. Sacha says:

    Hi BeenThere – yes, there probably is some “doing the test again” effect – you would expect that people could learn something about the test if they’ve done it before – generally you want to keep the test unknown to a person before they do it, otherwise they can learn about it and presumably other people won’t know anything about it before they do it and so the test’s effectiveness as a measuring instrument is diminished.

    I have a PhD but I wouldn’t advertise any test I write as “PhD certified”! This phrase a bit silly and doesn’t seem to mean much.

  28. Alison says:

    I completely agree with you. That question bothered me so very much. Hahah.

  29. leo says:

    i made my own solution… and i think it pretty valid, although its kind of random… but it worked for me:

    400 — 3600
    100 — X

    so (100 x 3600) / 400 = 900…
    but also:
    400 / 100 = 3600 / 900
    3600 / 400 = 900 / 100
    and it goes on and on and on blablabla.. i hope it works for u too… leo

  30. Sacha says:

    Ahh – Leo, this just demonstrates the problems with the question in the test. Why is your “rule” less valid than the supposedly correct one? And saying that, “The test developer must know,” doesn’t cut it.

  31. leo says:

    whos the developer? how can i contact him?

    and: what does: “doesnt cut it” mean? :P, im argentinian… thats why i dont understand it (A)

    thx

  32. Sacha Blumen says:

    The test developer might work for Tickle, or Tickle might have bought the test off a another company or individual. You could e-mail Tickle to ask who they are.

    “doesn’t cut it” means that it’s not a good enough reason.

    Cheers,
    Sacha

  33. […] likes 400 but not 300; he likes 100 In March last year I wrote this post on a question that irked me on the IQ Tickle test and on IQ tests in general, and since then, my […]

  34. Ameer says:

    Here’s the way we think:

    John learned adition and subtraction but not yet “integers,” “multiplications,” & “squares.” he recently learned the number 1000, a very nice round number and made a rule is his life:

    “When comparing any set of numbers john likes the one closest to 1000”

    So the answer is obviously 1000

    Thank you =)

    p.s. my pharmacy friend thinks that the evolution of learning about math fits this way thinking. hahah

  35. Michael says:

    so it doesnt matter what he dislikes???

    BTW I put 1000 cause it seemed to me that the guy liked simple numbers and 1000 was the most simple even rounded number.

    If I dont like 99, 3700 and 300, and if I am asked what I like better 900,1000, 1100, or 1200, I think I would like 1000 most simple number to concieve out of those 4 choices.

  36. Ameer says:

    thanx michael, although “simplest even round numer to concieve” isn’t by any measure meaningful lol, i know what u mean. To answer the question, if those idiots who think themselves to be intellegent for chosing 900 cant understand that this question “as it stands” fails to have one answer, then they must ask themsleves:
    when the question itself emphasizes stes “400 but not 300” etc. why is their answer lacking that emphasys?

  37. saleh says:

    hi hardly need the all the answers of tickle iq tests

  38. kittyklix says:

    Hi Sacha

    This thread goes on a while so I’ve only read the first ‘few’ postings (few being subjective of course!). I too feel the question was very obscure.

    I have to agree with your criticism of the Tickle test, although not from your more mathematical perspective. In my H opinion if a test question can be answered correctly by selecting any of the answers and is only marked correct for the ‘most obvious’ answer who’s intelligence is it testing exactly? Those people that jump at the obvious!!

    Sadly our school system has deteriorated for largely the same reason: unconventional thought or even deep thought of any kind is looked upon with scorn in this country. Sadly it would appear that those who challenge the status quo or who question consensus belief are still regarded as fools.

    Keep thinking
    xx

  39. Lupu says:

    lol, u all suck. he likes the squares

  40. depe says:

    why did u answered 1200?

  41. lneely says:

    I literally got pissed at this question once, writing, “Who cares!?!” in big letters on top of the question.

    What has ALWAYS caught me on that question as far back as I can remember was exactly that! Who the hell /cares/ what numbers he likes? Isn’t the act of liking something a subjective one? Is this even a real question?!

    At least now, long after I will ever need an IQ test, I can read their minds and realize what they’re asking for. ;p

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: