# Do academic personality tests prematurely abstract things?



## tailcalled (May 5, 2021)

By "academic personality tests", I mean personality tests in the style of those that test the Big Five or HEXACO, such as NEO-PI or HEXACO-PI. Lemme explain what I am suspecting is their problem, and how I suspect one could fix this.

A typical set of questions from such a personality test might be like this:








(Borrowed from the IPIP-NEO test.)
The way the test would score these is that it would add up all the questions under the "+ keyed" section, and subtract all of the questions under the "- keyed" section. And this set of questions is... fine. It does fairly well at measuring how often one exhibits anxious feelings in one's life. But is that what we want to know when we take a personality test that is supposed to tell us our Anxiety factor?

The trouble is that one might be anxious for many reasons. Maybe one has a bad disease, and the possible outcomes of that disease leads to a lot of worry. But would this worry _really_ be part of one's Personality? Obviously it depends on how you define "personality"; it's certainly part of how one feels and acts on average. But one might be more interested in a test that tells you Deeper things about _why_ you feel and act in the way you do; which is not so affected by your environment, but instead e.g. tells you how big of a tendency you have to become anxious _relative to_ the anxiety-inducing situations you face.

This is obviously hard to do for a "dumb" test that entirely relies on asking you a fixed set of questions and adding them up, but it seems to me that the academic tests are usually not really _trying_, and that if one changed the style of the test, one could do better.
Basically, the way these personality scales work is that they ask you a bunch of questions, and then your response to this question is partly determined by your personality, and partly random measurement error. When they then average up the responses, the random measurement error gets divided away in the average, while the personality component hopefully shows up in all of the questions, and so does not get averaged away. So this way, personality tests reduce or eliminate measurement error.

But you could use this principle to reduce specific environmental factors that make you anxious too, and emphasize relative anxiety. If for instance you had a test that went as follows...:


> How much do you worry about...
> 
> ... your performance at your job or in school?
> ... whether you look ugly?
> ...


Then the average score would be your tendency to worry across a whole bunch of different contexts, which would seem to get more at a Deeper notion of Anxiety than just "oh, how much time do you spend worrying about anything?". If there is some contextual factor, then that contextual factor would have less chance of influencing _all_ of these different contexts, and therefore would tend to disappear when averaging all the questions.

I call this "avoiding premature abstraction" because the standard academic tests are very abstract. They ask whether you worry about "things", with the "things" being left unspecified. But ultimately when people answer these questions on academic tests, they don't have magical access to the abstract Anxiety, so they aren't going to answer based on abstract information; instead they're going to answer on the basis of concrete information, like how much they've worried about whichever things they happen to have worried about (for contextual reasons). So it might be better to bring these contextual elements front and center in the test, so they can be reduced through averaging, rather than letting them be implicit.


----------



## DOGSOUP (Jan 29, 2016)

Found your previous thread about using games to test personality very cool.
And here you really hit the nail on the head.

I personally struggled with this where referring to "things" leads to - oh something has been on my mind a lot lately, a trait or a reaction to a situation - then I am likely to report that as a very relevant part of my personality in a test, even if it is not really a longterm/repeated thing. I'll remember all the 3 times I have been angry this year and regardless of what those situations were end up with a result "what an explosive personality you have!"


----------



## tailcalled (May 5, 2021)

> Found your previous thread about using games to test personality very cool.
> And here you really hit the nail on the head.


Thanks! 😀

To an extent, there's also overlap between the two ideas; if there is some situational/contextual factor that has a lot of effect on your behavior, then presumably games or other objective tests have no chance of measuring this factor, because they cannot recreate your unique real-life situation. However, they might have a much better chance of measuring situation-independent factors. So a prerequisite for making objective personality tests is to get something like this to work, I think.


----------



## BigApplePi (Dec 1, 2011)

tailcalled said:


> But ultimately when people answer these questions on academic tests, they don't have magical access to the abstract Anxiety, so they aren't going to answer based on abstract information; instead they're going to answer on the basis of concrete information, like how much they've worried about whichever things they happen to have worried about (for contextual reasons). So it might be better to bring these contextual elements front and center in the test, so they can be reduced through averaging, rather than letting them be implicit.


Seems to me a "concrete" test versus an "abstract" test is going to be "better" if it, pardon my way of saying this, is if it knows as much about the topic of personality as the topic itself it is going to test. I think of myself and anxiety. I am pretty laid back in looking at PerC topics and discussions but I do have an anxiety area which could easily be missed: anxiety on taking a train today when I usually drive. That is a "neurotic" area (travel flexibility) easily missed. The more thorough a test, the better.


----------



## Infinitus (Jul 12, 2019)

I see what you’re saying. Although I thought the point in the abstraction/vagueness is for it be open-ended and not lead the applicant. In your example, that they couldn’t possibly think of every cause of one’s anxiety, which could be irrational phobias or certainly things the test maker couldn’t have even imagined. Thus by having it open-ended there’s more possibility for the applicant to attribute and factor in these subjective or uncommon variables.

Since it’s a tool, and one can control the way you use it, one could simply answer the questions with your concerns in mind. So intelligently answering from your baseline state, accounting for and discounting temporarily illness, and not factoring them into your responses. Granted, this isn’t a solution to your concern with the vagueness of the test questions, per se, but it should account for, or allow for a more accurate reflection of your baseline, long term personality, rather than factoring in transient experiences.

I must confess, I already do this when I answer such questionnaires, but I guess each has their own method of answering.


----------



## BigApplePi (Dec 1, 2011)

@Infinitus. Open-endedness might be okay as long as it is not too open ended. True, one can't cover everything, but one can break down openendedness into general categories, can't they? A compromise needs to be reached as when somebody goes too far and gets banned. They need to perhaps find a way to apologize and mend their ways so they can be talked to. What do you think?


----------



## 8080 (Oct 6, 2020)

tailcalled said:


> How much do you worry about...
> 
> ... your performance at your job or in school?
> ... whether you look ugly?
> ...





> when we take a personality test that is supposed to tell us our Anxiety factor?


The Five (or Six: HEXACO-PI-R) Factor Model personality inventories are primarily designed for _statistical personality research with as many subjects as possible_, which is why the time and thus the number of questions = items is limited; NEO-PI-3: 240 questions (30-40 minutes), NEO-PI-FFI: 60 questions (15 minutes), HEXACO 200-item version: unspecified, 100-item version: 20 minutes, 60-item version: 12 minutes. 

*Trait-anxiety*: A *personality trait* referring to individual differences in the likelihood that a person would experience *state anxiety* in a stressful situation.
*State-anxiety*: A *transitory* emotional response involving unpleasant feelings of tension and apprehensive thoughts.

As usual, there are special inventories for clinical purposes, for example:

40-item State–Trait Anxiety Inventory (STAI)
State Trait Anxiety Inventory (Everything You Need To Know) - PsychReel

21-item Beck Anxiety Inventory (BAI)
Beck Anxiety Inventory (A Complete Guide) - PsychReel

What Is Trait Anxiety, and How Does It Compare to State Anxiety?



> I call this "avoiding premature abstraction" because the standard academic tests are very abstract. They ask whether you worry about "things", with the "things" being left unspecified. But ultimately when people answer these questions on academic tests, they don't have magical access to the abstract Anxiety, so they aren't going to answer based on abstract information; instead they're going to answer on the basis of concrete information, like how much they've worried about whichever things they happen to have worried about (for contextual reasons). So it might be better to bring these contextual elements front and center in the test, so they can be reduced through averaging, rather than letting them be implicit.


As with the other factors, the task requires the subjects to be able to describe their personality _in a generalised way_ with their answers.

The original forced-choice MBTI questions also included T/F questions that seemed unclear, even nonsensical to MBTI thinkers, but this obviously did not matter, as they then answered these questions in the negative as desired. This is just to point out that unclear or apparently nonsensical questions can have the desired filter effect.

I believe that thinkers in particular are annoyed by vague questions. They would like to answer all questions seriously, but how is that possible if you don’t know exactly what the question is?










I find the -keyed items unproblematic, because there are clarifying qualifiers (_not easily, most of the time)_ that help to keep out _state anxiety_. 

The +keyed item “*Worry* about *things*.” makes sense for people who often say “I worry about things.” But everyone else is wondering if it should read: “I _easily/often_ worry about things.” And then there is the question of what is meant by _things_. Inanimate objects only? There is still the _thing_ in _something_, but it is clearer: “I _easily/often_ worry about something.” If that is what is meant. 

“Fear for the worst.” Again, why is _easily/often/always_ missing?

“I am afraid of *many *things.” I am afraid of avalanches, amputation, colds, inflation, sharks, shrapnel, snakes, superfluous spaces, parcels with damaged goods – do my 9 items qualify as “many things”? 

“Get stressed out easily.” Fine.

“Get caught up in my problems.” Again, why not _easily/often/always?_

Let us now look at the anxiety facet of the *Emotionality *factor = domain of the *HEXACO-PI-R*: “The _Anxiety_ scale assesses a tendency to worry in a variety of contexts. Low scorers feel little stress in response to difficulties, whereas high scorers tend to become preoccupied even by relatively minor problems.“

*Anxiety items: 11, 35R, 59R, 83* [R= Reversed] Source: The HEXACO Personality Inventory - Revised

_11: „I *sometimes* can't help worrying about *little things*._

Sometimes I use letter clips when I submit official documents, and I ask myself: “May I use the _pink_ letter clip so that I am finally rid of it, or might the recipient interpret the colour in an undesirable way?” Is that worrying? I think so. But what does “I can’t help” mean? Am I helpless? Sleepless, because my dangerous pink letter clip? But perhaps I don’t worry about _little things_ because the potential problems caused by a pink letter clip etc. are _big things_?

35R: “I worry a lot less than most people do.” Fine.

59R: “I rarely, if ever, have trouble sleeping due to stress or anxiety.” Fine.

_83: I get *very anxious* when waiting to hear about an *important decision*. _

What is the difference between _very_ _anxious_ and _anxious_? If you are waiting in a waiting room for the result of an HIV or cancer test, and you feel uncomfortable, is that already _very anxious_? Probably not. Is the decision whether there is an empty table in a restaurant an _important_ decision from the point of view of the questioner? Possibly, if it is a first date.

Unlike insects, humans and their products rarely appear perfect.


----------



## Infinitus (Jul 12, 2019)

BigApplePi said:


> @Infinitus. Open-endedness might be okay as long as it is not too open ended. True, one can't cover everything, but one can break down openendedness into general categories, can't they? A compromise needs to be reached as when somebody goes too far and gets banned. They need to perhaps find a way to apologize and mend their ways so they can be talked to. What do you think?


I figure that's the quandary with knowledge gathering; too specific restricts the dataset and overall scope, too broad lacks specifics and precise detail. Intelligence tests, for example, suffer from a similar problem of open-endedness, to the point people argue they don't accurately reflect practical, real world, intelligence, and thus propose categorisation of different types of intelligences. Even then, I suspect that scoring high in different types of intelligences would generally correlate with above average IQ for most, in which case IQ wouldn't be inaccurate (as with these categorisations of aspects of personality), but can see where the lack of categorisation could indeed overlook individuals with, let's say, specialised intelligences. Or applied to personality, specific idiosyncrasies. But in my opinion, one should get the more generalised system correct before further complexifying.

As for the rest, personally, I'm a huge fan of communication, I've never refused to speak to anyone in my entire life, despite of anything. Then you run into the issue of if someone will even properly listen to what you say, I know I'm guilty of stubbornly believing my perspective and ways are the only way. Or if they even think there is a problem requiring help. In general, I love the idea of redemption though, I've clung onto that notion at many points in my life. Strive to do & be better, whether one believes you've done anything wrong or not. That's my default state, regardless. I don't know if that answers your question or not.


----------



## BigApplePi (Dec 1, 2011)

Infinitus said:


> ... knowledge gathering; too specific restricts the dataset and overall scope, too broad lacks specifics and precise detail.


Thoughts. The way to get at knowing something is to take samples directly of that something I suppose. Using a test on a complex human is indirect and crude. How can we demand results?



Infinitus said:


> I'm a huge fan of communication, ...


I guess communication goes best with there is an exchange ... both parties. I sent this professor an email and he replied. After that he said furthance would be slow as he was very busy. That means I have to work on getting it right the first time else I won't have anything to offer him. If he wants me to fit into his needs and not make errors, that discourages me to communicate with him. I'd rather he be more receptive.


----------

