Currently reading Shen & Tamkin (2026) on “How AI impacts skill formation”

In the preprint “How AI impacts skill formation” by Shen & Tamkin (2026), study participants are learning a new Python library either in a control condition or with an “AI assistant”, and researchers observe how the participants prompt AI during learning and measure completion times and quiz scores after learning. They find that “AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation“.

Overall, using AI did not lead to a significant performance boost — some participants relied on AI to code for them and then just copied and pasted, never asking questions to understand the generated code, and those could complete tasks very fast. Other people spent a very long time writing prompts and asked AI a lot of questions, and those participants were very slow. And using AI in different ways had different effects on learning. The authors find that “high scorers only asked AI conceptual questions instead of code generation or asked for explanations to accompany generated code; these usage patterns demonstrate a high level of cognitive engagement“. They also find that “[t]he erosion of conceptual understanding, code reading, and debugging skills that we measured among participants using AI assistance suggests that workers acquiring new skills should be mindful of their reliance on AI during the learning process.”

The authors also conclude that “Together, our results suggest that the aggressive incorporation of AI into the workplace can have negative impacts on the professional development workers if they do not remain cognitatively engaged. Given time constraints and organizational pressures, junior developers or other professionals may rely on AI to complete tasks as fast as possible at the cost of real skill development.“* And it is easy to imagine what it will lead to if no or little human skills are developed, be it in school, higher education, entry level jobs… So I would argue that it is not just workers acquiring new skills that need to be mindful, but also teachers and employers who should all value learning and skill development over fast task completion. I think we need to remind ourselves that in really almost all cases in life, the process is the point…

Shen, J. H., and Tamkin, A. (2026). “How AI impacts skill formation”. arXiv preprint https://arxiv.org/pdf/2601.20245v2