One of the key questions in the IT market in 2026 is this: if a junior with AI completes tasks almost as quickly as a senior, and an inexperienced analyst gets ready-made SQL and a polished dashboard from the model, then why pay for experience? There is a well-supported rationale for this question. In a study of 5,172 support employees, an AI assistant increased productivity by an average of 15%; less experienced employees gained in both speed and quality, while the most experienced gained slightly in speed at the cost of a small decline in quality. In an experiment with 453 specialists who wrote professional texts, completion time fell by 40%, quality rose by 18%, and the inequality between workers narrowed. In other words, on routine tasks, LLMs bring weaker performers up to the level of stronger ones.
But what if the conclusion that “LLMs have leveled the playing field between novices and experts” is not a property of the model, but a consequence of how we compare them? For example, what if the IT hiring process has become such that an LLM-equipped rookie with zero experience can easily outcompete a senior for an open position?
What is expertise
It turns out that LLMs level the playing field between novices and experienced professionals not in general, but in a specific domain: on routine tasks within the model’s capabilities and where the result does not require human validation. Beyond this domain, the gap remains and sometimes even grows. Why? Cognitive psychology provided the answer long before LLMs. Expertise is, first and foremost, the content of long-term memory. An expert differs from a novice not by having a larger working memory, but by having encountered similar tasks many times and storing ready-made solutions for them.A novice has to build the solution step by step within limited working memory. Accumulated knowledge is the main—and perhaps the only alterable—source of differences between people.
Schemas manifest externally in three ways. First, the expert recognizes meaningful configurations: a chess player recognizes at a glance a position from a real game, but not a random arrangement of pieces, and the same effect has been observed in programmers. Second, the expert classifies problems by their deep structure, whereas a novice classifies them by superficial features: expert physicists grouped problems according to the law needed to solve them, while students grouped them according to what was depicted in the problem statement. Third, the expert immediately sees the first step of the solution. A method built on this feature showed a correlation of up to 0.92 with the full knowledge test and reduced testing time by a factor of 2.5–4.9.
Now it is clear why an LLM does not replace expertise. All three indicators amount to the ability to recognize what is before you and understand whether it is properly structured. This is exactly what is needed to check the model’s answer and notice that the task has gone beyond its capabilities. The model generates text and code, but the schemas for evaluating them remain in the human mind—or are absent.In other words, an LLM replaces execution, while expertise is, above all, recognition.
Hiring as an equalizer
Let us start with the typical hiring process:
the job description is written with the help of an LLM, often by someone who does not know the subject area;
61@@62@@63@@resumes are tailored to applicant tracking systems (ATS), and the ATS screens them based on keyword matches;69@@screening is conducted by a nonexpert: what are you looking for, why are you leaving, tell us about the company;
- 74@@then come general interviews in the STAR format
- 80@@ followed by technical interviews, testing only one small area of the candidate’s expertise
Now let us go through all the stages and examine exactly what is assessed at each one.
The job opening and ATS. The workflows required for a role are determined by the class of tasks a person will encounter. If the job opening is written by someone who does not know those tasks, they describe the role through superficial markers: the tech stack, years of experience, trendy terms. This is the same categorization of a novice by outward signs, only on the employer’s side. The ATS then compares the resume’s keywords with the job opening’s keywords—surface against surface. Both an expert and a novice will write an equally keyword-tailored resume.
Screening. A nonexpert cannot distinguish a deep answer from a superficial one and does not even realize this.In experiments with unstructured interviews, participants formed confident impressions of interviewees even when the interviewees answered randomly, and the interviews themselves reduced predictive accuracy compared with objective data alone. Subjective confidence is generally a poor indicator of accuracy: reliable intuition develops only in a regular environment with repeated feedback. At this stage, the fluency of the narrative matters more than scoring rubrics.
Interviews. The STAR format (questions about past behavior) is not inherently weak: with scoring scales describing levels of responses, its validity in a meta-analysis reached 0.63. But candidates choose for themselves what to talk about, so without such scales interviews largely measure storytelling ability.
Technical tasks. Algorithmic tasks, system design, and live coding introduce another source of noise: performance is confounded with anxiety about being watched. Moreover, such tasks are routine by nature, meaning they are highly likely to fall within the model’s capabilities. Therefore, when using an LLM in an interview (whether legally or not), the outcomes for a novice and a senior will be identical at best.
This creates an unfortunate loop. Each stage measures either surface-level traits, impressions, or performance on a routine task. This is precisely the area where a novice using an LLM and an experienced specialist resemble each other. No stage measures recognition, verification, or work beyond the model’s capabilities. Thus, when it comes to hiring, a novice using an LLM and an expert look identical not because the model put them on equal footing, but because hiring looks only where they are equal.
An honest caveat is needed here.If the role really consists almost entirely of routine tasks within the model’s capabilities, the devaluation of experience in it is genuine, and hiring has nothing to do with it. Yet somehow, even for writing CRUD and styling buttons, the market has always wanted to hire a naturally gifted engineer.
The Price of Leveling
If hiring does not distinguish between a novice with an LLM and an expert, the decision looks rational: hire whoever is cheaper. There are two types of consequences.
First, direct errors. A novice hired for a role requiring verification will make mistakes where the boundary of the model’s capabilities has been crossed, without realizing it. This follows from data on tasks beyond the model’s capabilities and on the illusion of competence, although there are no direct measurements of this kind of hiring error.
Second, a broken pipeline. Experts are made only from novices. As early as 1983, Lisanne Bainbridge described the “irony of automation” in industry: automation takes away from the operator the very practice from which the skill needed to monitor it and step in when it errs grows. Extending this to LLMs is an analogy, but a direct one. A novice hired as a model operator does not build their own schemas.
Economists describe automation not as the replacement of jobs, but as the redistribution of tasks: technology displaces labor from some tasks and creates new ones where it is needed again. If hiring evaluates people solely by the tasks the model has already taken over, it systematically underestimates those capable of doing the remaining tasks.
The circle closes: hiring does not see expertise, the market does not pay for it, novices grow into operators, and expertise genuinely becomes scarcer.
Hiring adaptations
The good news is that the psychology of personnel selection has proven tools, and the theory of expertise suggests how to target them. Below are adaptations, grouped by the strength of the evidence.
Adaptations with a strong evidence base.
TAG150@@Job analysis instead of a list of keywords. The job description is written from a list of role tasks compiled by an expert: 5–10 typical tasks, each marked according to where it falls relative to the model’s capabilities — the LLM performs it; the LLM performs it and a person checks it; the task is beyond the model’s capabilities.
Structured interviews with scales. Structured interviews are called the best method because, of all selection methods, they most accurately predict how a person will perform.
What makes this possible:
The same questions for everyone. Candidates’ answers can be compared with one another rather than with the interviewer’s mood.
Questions are built from job analysis. They ask about what will actually be needed in the role
Scales with described response levels. Assessors’ ratings of a candidate diverge less.
TAG180@@
TAG182@@Less room for impressions. In unstructured interviews, people confidently “understand” even random answers, and this makes the prediction worse.The structure eliminates this effect.
Work samples on real-world tasks. A work sample is an assignment that reproduces part of real work: the candidate performs the work rather than describing it. Its relationship with subsequent performance is moderate but consistent. The work sample can be assembled from a list of role tasks.
Why the method works:
It measures behavior on a task from the actual work, rather than self-description or impressions.
In the language of expertise theory, a work sample directly taps domain schemas rather than the ability to describe them.
However, this method also has caveats:
The work sample partially overlaps with general abilities. Its correlation with general cognitive ability tests is 0.32. In other words, the work sample measures not only expertise.
The work sample assesses only the schemas included in the assignment. A narrow work sample provides a narrow slice, so it should be assembled from a list of real tasks for the role.
A candidate can complete the take-home work sample with an LLM (hello to everyone who loves standardized test assignments within the LLM’s capabilities). Then it no longer distinguishes between a novice working with the model and an expert.
Adaptations grounded in theory but not yet tested in hiring.
Recognition probes. Short “first-step” tasks: a log fragment, diff, or architecture diagram with the question “What is happening here, and what would you do first?”, along with an expert answer key. Or sorting 6–8 short problem descriptions with explanations, as in studies in which experts classify tasks. This method works well in training; it has not been validated in IT hiring.
Probes for evaluating LLM responses. Within a live trial: find an error in a plausible solution generated by the model and solve a task beyond its capabilities. This directly measures what distinguishes an expert from a novice using the model, but there is not yet any validity data for such probes.
Broad sampling of schemas. One mandatory area and one that the candidate chooses as their strongest. This reduces the risk that the interview will assess a narrow area that is not the candidate’s strongest. Explaining or changing the solution live verifies that the candidate did the processing themselves.
Less extraneous load from the format. If the live-coding result is confounded by anxiety about being observed, the format should be changed—for example, allowing the candidate to solve the task without an observer present and then discussing it afterward.
The cost of these adaptations should be specified upfront. They require expert time at every stage, from compiling the task list to developing the scales and scoring the probes.But the alternative—hiring that does not see expertise—ultimately costs more.
Conclusion
The framework we have assembled reads as follows. LLMs do indeed narrow the gap between a novice and an experienced professional, but only on routine tasks within their capabilities, and only in producing the result. Expertise, however, consists of recognition schemas needed to verify an answer and detect when it goes beyond the model’s limits. Conventional hiring measures surface-level indicators, impressions, and performance on routine tasks—precisely what a novice with an LLM and an expert have in common. Therefore, it is hiring, not the model, that puts them on equal footing.
This yields three conclusions. First: the thesis that “LLMs have devalued experience” is, to a significant extent, an artifact of measurement rather than a property of the technology. Second: selection psychology has tools with a robust evidence base—job analysis, structured interviews with rating scales, work-sample tests, and mechanical aggregation of ratings—and these are sufficient for hiring to begin seeing expertise again.Third: Recognition and response-verification tests for an LLM are the most direct way to measure what the model does not provide, but they still need to be validated.
For companies, the conclusion is practical: as long as hiring puts a novice on an equal footing with the model and the expert, today’s savings on experience turn into tomorrow’s shortage of reviewers. For specialists, the conclusion is just as concrete: value grows where a person knows how to recognize and verify, not where they get the answer faster than anyone else.
The article was written with the participation of Claude Code (Anthropic). The author selected and studied the research, formulated the article’s main idea, and wrote a draft of the material. Claude Code checked for errors, edited the text, and prepared the infographic.
What to read
Expertise and schemas
Chase W. G., Simon H. A. Perception in chess // Cognitive Psychology. 1973. Vol. 4, No. 1. P. 55–81 — a classic study of position recognition: experts remember meaningful, rather than random, configurations.
Chi M. T. H., Feltovich P. J., Glaser R. Categorization and Representation of Physics Problems by Experts and Novices // Cognitive Science. 1981. Vol. 5, No. 2. P. 121–152 — experts classify problems by solution principles, while novices classify them by surface features.
McKeithen K. B., Reitman J. S., Rueter H. H., Hirtle S. C. Knowledge organization and skill differences in computer programmers // Cognitive Psychology. 1981. Vol. 13, No. 3. P.307–325 — the effect of recognizing and organizing knowledge, replicated with programmers.
Sweller J. Cognitive load theory and individual differences // Learning and Individual Differences. 2024. Vol. 110. Article 102423 — expertise as the content of long-term memory and the main modifiable source of individual differences.
Kalyuga S., Sweller J. Measuring Knowledge to Optimize Cognitive Load Factors During Instruction // Journal of Educational Psychology. 2004. Vol. 96, No. 3. P. 558–568 — the “first-step method”: correlation with the full test of up to 0.92 while reducing completion time by a factor of 2.5–4.9.
Kalyuga S., Ayres P., Chandler P., Sweller J. The Expertise Reversal Effect // Educational Psychologist. 2003. Vol. 38, No. 1. P. 23–31 — why assistance useful to a novice hinders experienced users.
AI, productivity, and testing
Brynjolfsson E., Li D., Raymond L. Generative AI at Work // The Quarterly Journal of Economics. 2025. Vol. 140, No. 2. P. 889–942 — 5,172 customer support workers: +15% on average, with the largest gains among the less experienced.
Noy S., Zhang W. Experimental evidence on the productivity effects of generative artificial intelligence // Science. 2023. Vol. 381, No. 6654. P. 187–192 — 453 specialists: 40% less time, 18% higher quality, and less inequality between workers.
Dell'Acqua F., McFowland E., Mollick E., et al. Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of Artificial Intelligence on Knowledge Worker Productivity and Quality // Organization Science. 2026. Vol. 37, No. 2.P. 403–423 — 758 consultants: a performance gain within the model’s capability boundary and −19% correct decisions beyond it.
Prather J., Reeves B. N., Leinonen J., et al.>The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers // Proceedings of ICER 2024. P. 469–486 — how generative AI accelerates strong novices and creates an illusion of competence among weak ones.
Goddard K., Roudsari A., Wyatt J. C. Automation bias: a systematic review of frequency, effect mediators, and mitigators // Journal of the American Medical Informatics Association. 2012. Vol. 19, No. 1. P. 121–127 — what excessive reliance on automation depends on and what reduces it.
Lee H.-P., Sarkar A., Tankelevitch L., et al. The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers // Proceedings of CHI 2025. P. 1–22 — trust in AI is associated with less critical thinking.
Validity of selection methods
Sackett P. R., Zhang C., Berry C. M., Lievens F. Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range // Journal of Applied Psychology. 2022. Vol. 107, No. 11. P. 2040–2068 — revisiting validity: structured interviews rank first, with validity estimates 0.10–0.20 lower than previous ones.
Roth P. L., Bobko P., McFarland L. A. A meta-analysis of work sample test validity: Updating and integrating some classic literature // Personnel Psychology. 2005. Vol. 58, No. 4. P.1009–1037 — validity of work-sample tests: 0.26 uncorrected and 0.33 corrected.
Kuncel N. R., Klieger D. M., Connelly B. S., Ones D. S. Mechanical versus clinical data combination in selection and admissions decisions: A meta-analysis // Journal of Applied Psychology. 2013. Vol. 98, No. 6. P. 1060–1072 — mechanical aggregation of ratings predicts better than holistic judgment, even expert judgment.
Interview and interviewer judgment
Levashina J., Hartwell C. J., Morgeson F. P., Campion M. A. The Structured Employment Interview: Narrative and Quantitative Review of the Research Literature // Personnel Psychology. 2014. Vol. 67, No. 1. P. 241–293 — a review of what makes up the structure of the interview and what is known about each of its elements.
Taylor P. J., Small B. Asking applicants what they would do versus what they did do: A meta-analytic comparison of situational and past behaviour employment interview questions // Journal of Occupational and Organizational Psychology. 2002. Vol. 75, No. 3. P. 277–294 — validity of past-behavior questions (0.63) and agreement between raters and scales (0.77–0.79).
Dana J., Dawes R., Peterson N. Belief in the unstructured interview: The persistence of an illusion // Judgment and Decision Making. 2013. Vol. 8, No. 5. P. 512–520 — confident impressions formed from random answers and worsening prediction after an unstructured interview.
Kahneman D., Klein G. Conditions for intuitive expertise: A failure to disagree // American Psychologist. 2009. Vol. 64, No. 6. P.515–526 — when intuition can be trusted and why subjective confidence does not indicate accuracy.
Behroozi M., Shirolkar S., Barik T., Parnin C. Does stress impact technical interview performance? // Proceedings of ESEC/FSE 2020. P. 481–492 — technical interview performance at a whiteboard is confounded by anxiety from being observed.
Automation, learning, and labor
Bastani H., Bastani O., Sungu A., et al. Generative AI without guardrails can harm learning: Evidence from high school mathematics // Proceedings of the National Academy of Sciences. 2025. Vol. 122, No. 26. Article e2422633122 — +48% and +127% with access, −17% after disabling it, the role of guardrail prompts.
Bainbridge L. Ironies of automation // Automatica. 1983. Vol. 19, No. 6. P. 775–779 — automation deprives the operator of the practice needed to control it.
Acemoglu D., Restrepo P. Automation and New Tasks: How Technology Displaces and Reinstates Labor // Journal of Economic Perspectives. 2019. Vol. 33, No. 2. P.3–30 — automation as task redistribution: displacement and restoration of labor.
