360 degrees judgments as criteria of a personality test

In a current project we tested 166 middle level managers in health care with the UPP test (Sjöberg, 2010). 360 degrees dimensions (Arvonen, 2002) were related to the dimensions derived from the test. Results:

Correlations between 360-degrees judgments of managers and matching UPP test variables (N=166), corrected for measurment errors in the criteria and range restriction due to indirect selection
	360 degrees judgment dimensions
Judges	Relations	Structuring	Change
Superior manager	0.66	0.21	0.39
Colleagues	0.65	0.26	0.38
Direct reports	0.66	0.40	0.51
Self judgment	0.52	0.37	0.64

The test had a very high level of validity in these data, both in relation to the external criteria provided by independent judges of 3 types and levels, and self judgments.These values are somewhat unique for a personaliy test, but see also Hogan and Holland (2003) about the need to matcg predictors and criteria content-wise, an example of an old principle in psychological research (Sjöberg, 1980).

It is interesting to note that "Structuring" is not quite as strongly related to personality as the other two dimensions,  which may be because that dimensions reflects more of "can do" aspects than "will do". It should therefore be related more strongly to ability. Note that personality as measured by the UPP test is unrelated to ability, implying that a combination of personality and ability should be ideal for prediction.

Note that the correlations have NOT been increased by applying an exploratory multiple regression model, a common enough trick. 

A state-of -the-art article on correcting for measurement errior and range restriction is Hunter, Schmidt & Lee (2006).



References

Arvonen, J. (2002). Change, production and employees. An integrated model of leadership. Stockholm: Department of Psychology, University of Stockholm.

Hogan, J., & Holland, B. (2003). Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology, 88(1), 100-112.

Hunter, J. E., Schmidt, F. L., & Le, H. (2006). Implications of direct and indirect range restriction for meta-analysis methods and findings. [doi:10.1037/0021-9010.91.3.594]. Journal of Applied Psychology, 91(3), 594-612.


Sjöberg, L. (1980). Similarity and correlation. In E.-D. Lantermann & H. Feger (Eds.), Similarity and choice (pp. 70-87). Bern: Huber.Click here.

Sjöberg, L. (2010). A third generation personality test. Stockholm: Psykologisk Metod AB. Click here.