Tuesday, May 11, 2021

You want to work as what? The real problem with career-matching tests

You want to work as what? The real problem with career-matching tests


A review has just been published by the Association of Market and Social Research Organisations and the Statistical Society of Australia looking into the pollsters underestimating the Liberals’ chances of winning the last election. It observes that the polls were “likely to have been skewed towards the more politically engaged and better educated voters with this bias not corrected”.

They mean they listened to too many Labor voters. Similarly in the UK, the pollsters didn’t see Brexit coming.

One line of argument after the recent high-profile misses or failures in predicting political outcomes is that polls are just not that accurate. This makes a lot of sense. The world is complex and subject to continual change, and some of it can be relatively sudden and dramatic.

The Reserve Bank of Australia is hardly a bastion of extremist anarchist ideology, yet their discussion paper from 1983 called Are Economic Forecasts Accurate includes these warnings: “the legitimate criticism of the accuracy of economic forecasts is that they are only good at predicting the predictable.

“When the movements of economic variables are within the range of recently observed movements, forecasting accuracy can seem to be quite good.

“When movements are outside the range of recent experience, forecasts look poor.”

COVID-19, anyone?

This brings me to predictions about our careers. Our appetite for a certain future is clearly evident in our seemingly insatiable consumption of careers “tests” that hold out the promise of matching our supposedly carefully measured interests to possible future occupations. Some of these things will even rank the order of very best possibilities.

Putting aside the observation of one colleague in the testing industry — that, taken as a whole, including of all the free internet quizzes, “90 per cent of tests are crap” — even the high-quality interest tests struggle when it comes to prediction. Most good-quality studies show very modest correlations between measured interests and subsequent occupation, and over a period longer than a couple of years, there is a negligible relationship.

It is about time we focus on exploring, using curiosity, conducting experiments and remaining open to new avenues.Credit:Kerrie Leishman

Imagine you are 100 per cent uncertain about a future occupation. The best instruments might reduce that uncertainty by between 5 per cent (typically) and 16 per cent (to be very generous). If you think about getting a job your interests didn’t predict as being like getting the flu, and getting a job that was predicted as avoiding the flu, then measured interests reduce your chances by a tiny amount, whereas the real flu vaccine reduces your chances of flu by an estimated 59 per cent.

You take a vaccine to increase certainty about your future health. A vaccine that reduces your chances of infection by less than 20 per cent may be useful across society as a whole in reducing disease burden, but at the individual level, I doubt people would see the value. Luckily, per World Health Organisation, most routine childhood vaccines are 85 per cent to 95 per cent effective.

To top it off, Dr Jo Earl at Macquarie University reported in a 2019 study that “people may be better off taking well‐designed jobs than holding out for matched interests”.

Interests tests do have a place and can be useful in provoking further thought about options. However, the inevitable emphasis on the results and matches tends to breed a false sense of certainty that might be the opposite of what is helpful.

It is about time we focus on exploring, using curiosity, conducting experiments and remaining open to new avenues. In other words, focus more on possibilities than dubious predictions and just direct your feet to the sunny side of the street.

Jim Bright, FAPS is Professor of Career Education and Development at ACU and owns Bright and Associates, a Career Management Consultancy. Email to opinion@jimbright.com. Follow him on Twitter @DrJimBright

Most Viewed in Business


Source link

Leave a Response