Vi har de test vi vill ha...

Ibland har jag undrat över varför så många användare fortsätter att använda test med en validitet som är okänd, eller vars validitet är känd att ligga omkring noll. Det är en gåta. Ett skäl kan vara att användare helt enkelt inte känner till att man ska ställa krav på validitet, inte vet vad det är, och inte ens förstår vad en korrelationskoefficient är. Ett annat och lite intressantare skäl är att man vill ha test som ger diffusa resultat. Testet ger information som kan tolkas lite som man vill, så man är helt fri att dra de slutsatser man önskar. Ett exempel är MBTI, som bygger på Jungs luftiga spekulationer och mäter dimensioner vars betydelse för arbetslivet ingen har kunnat dokumentera. Men det verkar spännande! Alla har vi en liten amatörpsykolog i oss, och vi smickrar oss själva genom att tro att vi kan förstå människor, skåda in i själens djup, och gör träffsäkra prognoser. Ju mera information vi får, desto säkrare känner vi oss på vår sak - det får gärna vara 30-40 personlighetsskalor. Ingen kan dra vettiga slutsatser utifrån sådan information, särskilt som testkonstruktören säger: "Inget är bra eller dåligt, allt måste tolkas av testanvändaren". Populärt nonsens!

Job interest and performance: a revised view

Is job interest of any importance to job performance? It seems very likely that it should be, but as pointed out by Nye et al. (Nye, Su, Rounds, & Drasgow, 2012), "interest measures are generally ignored in the employee selection literature" (p. 384). Part of the reason seems to be that previous meta-analytic work reported a very low correlation between interest and performance, only about 0.1 (Hunter & Hunter, 1984). However, Nye at al. criticized the often cited meta-analysis published by Hunter and Hunter and conducted a very extensive new analysis of the relation between interest and performance. They came up with a different conclusion: for studies where the interest scales matched the character of the jobs, the estimated correlation was 0.36, after correction for measurement errors and indirect range restriction. They concluded that interest should be considered in selection contexts. 

This is not the only example showing that earlier meta analyses of the effectiveness of predictors of job performance may be quite misleading. A recent publication on integrity tests by van Iddekinge et al. (2012) showed that earlier meta analytic work (Ones et al., 1993), cited by Hunter and Hunter, grossly over-estimated the validity of integrity tests.

The recent Nye at al. work  is undoubtedly very important. However, even stronger results can probably be obtained with specific interest measures. Vocational interest does not measure interest in a specific job, but in a class of jobs. In the UPP test, we measure routinely interest in the specific job under consideration, either in selection or in various types of follow-up. As an example, data from a study of employees in customer service in a finance company (Sjöberg, 2010) was re-analyzed. The correlation between job (not vocational) interest and supervisor rated performance on core job tasks was 0.55, after correction for measurement error and indirect range restriction. The specific interest measure is proximal to job performance, while vocational interest is distal, hence it should be expected to have a lower correlation.

What creates interest (Sjöberg, 2006)? For a given task content, optimal challenge may be the answer to the question. Interests are also probably somewhat elastic, i.e. you may develop a new interest under favorable circumstances (support, optimal challenge). Maybe one should try measure not only interest but also potential for developing interest. In a selection situation, it must be expected that interest scores are contaminated with impression management, and there is a need to correct for that factor. Alternatively, indirect measurement can be attempted, such as knowledge of facts. People who are strongly interested inform themselves about a job or area of study, hence know more. I tried this idea in the selection of applicants to the Stockholm School of Economics, with some success.

References

Hunter, J. E., & Hunter, R. F. (1984). Validity and utility of alternative predictors of job performance. Psychological Bulletin, 96, 72-98.

Nye, C. D., Su, R., Rounds, J., & Drasgow, F. (2012). Vocational interests and performance: A quantitative summary of over 60 years of research. Perspectives on Psychological Science, 7(4), 384-403.
Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (1993). Comprehensive meta-analysis of integrity test validities: findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology Monograph, 78, 679-703.
Van Iddekinge, C. H., Roth, P. L., Raymark, P. H., & Odle-Dusseau, H. N. (2012). The criterion-related validity of integrity tests: An updated meta-analysis. [doi:10.1037/a0021196]. Journal of Applied Psychology, 97(3), 499-530.

Sjöberg, L. (2006). What makes something interesting? (Review of the book, Exploring the psychology of interest by Paul J. Silvia). PsycCRITIQUES, 51 (46, Article 4), No Pagination Specified.

Sjöberg, L. (2010). UPP-testet och kundservice: Kriteriestudie. (The UPP test and customer service: A criterion study). Forskningsrapport 2010:6. Stockholm: Psykologisk Metod AB.
Sjöberg, L. (2010/2012). A third generation personality test (SSE/EFI Working Paper Series in Business Administration No. 2010:3). Stockholm: Stockholm School of Economics.
Click here,