Have you ever helped a friend by filling out a form? Perhaps it measured your attitude towards a social issue. Or maybe you’ve applied for a job and faced a mandatory behavioural assessment. If so, you’ve likely encountered a self-report assessment. These self-report assessments are common in various settings, from research to recruitment.
Any test or survey in which individuals themselves report about their beliefs, attitudes, feelings, or competence, is a self-report measure. Often used in research and educational contexts, organizations too have started using psychometric tests in the recruitment and selection process, to hire candidates with desirable abilities and traits.
Traditional personality or behaviour assessments often claim “no response is right or wrong.” This seems advantageous when you’re trying to present your best self during selection. It’s natural to want to meet an organization’s requirements for skills, abilities, and values. You face two choices in these assessments. You can tailor your responses to match what you think the organization wants. Or, you can answer based on how capable you believe you are at performing tasks. Both approaches are human nature when aiming to be seen as a good hire.
This raises the question – how reliable are self-report measures to choose the right fit for a specific role?
Inadequacies in Self-Report Assessments – Identifying Gaps in Measurement
As Eva and Regehr put it, “The fundamental cognitive limitation in the ability of humans to know themselves as others see them restricts the usefulness of self-assessment results“.
When someone responds to a question, to what extent does their response truly represent who they are in reality? Let’s try to understand the gaps in self-report measures.
Social Desirability Bias
Assessments that measure typical performance or behaviour when performing differ significantly from actual performance of tasks which are directly observable. Self-reported competence is neither free from bias, nor an objective measure of observed performance.
Self-report test results are also often subject to Social Desirability Bias, or an individuals’ tendency to overestimate the likelihood that they would carry out a desired action.
Consider a job application statement: “I’m a team player.” A job seeker will likely choose “Agree” or “Applicable to me” if the role requires teamwork. They might do this regardless of their actual teamwork skills. This tendency leads to flawed reports on individual behaviors and traits. Self-report assessments often fail to capture genuine characteristics.
Random Responses and Higher Dropout Rates
Self-report measures have also been criticized for careless responses by the test-takers or lack of effort in responding to the questions. When asked to fill out a long questionnaire with 50 or more questions, it is natural for one to lose interest mid-way, or try to get done with it without paying much attention to the questions.
While psychometrically sound pre-employment assessments do seek to measure job-relevant skills, abilities, and organizational fit, the dilemma of opting for traditional assessments that have shown real world outcomes over newer presumably less reliable methods still remains.
We need a better solution than self-report assessments. The ideal tool would measure the same traits, but without bias. It should be relevant to the role and engaging for participants. And most importantly, it should capture what people actually do, not what they claim they would do.
Shifting from Self-Report Assessments to Data-Driven Alternatives
HR technology has advanced rapidly. Organizations now prioritize data-driven processes in personnel selection. These changes have prompted a reconsideration of self-report pre-employment assessments. The focus has shifted to new evaluation methods. Companies now seek measures that can predict candidate performance based on evaluation outcomes. These new approaches aim to assess observable performance and behaviour.
These requirements made way for gamification in the organizational context, particularly in personnel selection. Gamification is nothing but the inclusion of game-based elements in a non-game context.
Why would one choose a gamified assessment over already established self-report assessments?
- The elements of games can be applied to the process of recruitment and selection to assess specific skills, abilities and traits, while simultaneously enhancing candidate experience and engagement.
- Unlike traditional self-report measures with just statements that individuals respond to, game-based assessments allow them to experience the task virtually, and decide on behaviors they want to exhibit when performing various tasks.
- The use of game technology in the creation of game-based evaluations has the advantage of recording nuanced changes in behaviour. This can be done by observing use of specific traits and abilities across numerous tasks. Self-report measures often fail to tap into such nuanced aspects.
- In recent years, studies on game-based assessments in employment showed significant psychometric properties, especially proving good construct and predictive validity when compared to traditional psychometric assessments.
- In comparison to paper and pencil or digital text-based formats, video-based assessments have been said to be more content valid and more likely to predict future work performance.
Apart from assessing reliability and validity of game-based assessments, several studies also tapped candidate reactions and experiences with playing a game rather than taking a seemingly long self-report behavioural assessment.
To make it even more interesting, in a recent study, game-framing was tested in individuals to observe their reactions. Participants were asked to take a general cognitive ability test labelled as a ‘game’ without changing any of its contents. This elicited positive reactions of enjoyment and motivation in the test-takers, depicting how perceived gamification can also add significant value to the selection process.
So, are we game yet?
Improvisation in the way pre-employment assessments are taking shape today is not to replace traditional self-report measures or question their validity in the age of tech, but to revamp the personnel selection process using both psychometrics and technology.
When playing a game, you don’t worry about what norms you need to comply with – or which action of yours would get you the job – you just play, and that’s all we want you to. Our assessments at PerspectAI curate enticing gaming experiences for candidates and organizations.
Make the shift from your traditional self-report tests to fun, interactive assessments with PerspectAI today!
References
Collmus, A. B., & Landers, R. N. (2019). Game-framing to improve applicant perceptions of cognitive assessments. Journal of Personnel Psychology.
Dang J, King KM, Inzlicht M. Why Are Self-Report and Behavioral Measures Weakly Correlated? Trends Cogn Sci. 2020 Apr;24(4):267-269. doi: 10.1016/j.tics.2020.01.007
Georgiou, K., & Nikolaou, I. (2020). Are applicants in favor of traditional or gamified assessment methods? Exploring applicant reactions towards a gamified selection method. Computers in Human Behavior, 109, 106356.
McCord, J. L., Harman, J. L., & Purl, J. (2019). Game-like personality testing: An emerging mode of personality assessment. Personality and Individual Differences, 143, 95-102.
Montag, C., Dagum, P., Hall, B.J. et al. Do we still need psychological self-report questionnaires in the age of the Internet of Things?. Discov Psychol 2, 1 (2022). https://doi.org/10.1007/s44202-021-00012-4
Newry, R. (2018, March 7). Why traditional assessment providers are frightened of Game-Based Assessments. LinkedIn. Retrieved September 3, 2024, from https://www.linkedin.com/pulse/why-traditional-assessment-providers-frightened-game-based-newry/
Van de Mortel, T. F. (2008). Faking it: social desirability response bias in self-report research. Australian Journal of Advanced Nursing, The, 25(4), 40-48.
