Key points
- Generative AI is achieving incredible outcomes but possibly at the cost of a deterioration of our expertise.
- I cataloged some of these degradations in 2003 with regard to the use of information technology.
- Current AI may make us worse at recognizing patterns, searching for data, and improving our mental models.
- The challenge is to understand the basis of expertise in order to safeguard and enhance it.
Current versions of artificial intelligence have created so much well-deserved excitement. However, along with all that progress has come a wave of apprehension about the destructive potential of using AI. Fears that AI could be harming us. Fears that it might be making us stupid.
I wrote about this possibility in 2019, in a post I titled “The Second Singularity,”and I am not happy to see my predictions coming true.
However, this post asks a different question. Now I am looking back to 2003 and the original publication of my book The Power of Intuition. This was before large language models (LLMs) such as ChatGPT and Claude and the generative AI technologies. My book examined different facets of intuition/expertise. Chapter 16 was titled “Smart Technology Can Make Us Stupid.” The smart technology I was writing about was simply the 2003-style information technology, not current LLMs, but the mechanisms I identified seem quite relevant today.
In that chapter, I wrote, “Information technologies are intended to make us more productive, but their use carries a penalty. The computer programs, the decision support systems, the search engines, databases, and shareware we use can diminish our intuition and expertise. They can make us stupid.” I worried that the 2003-era technologies could disable our expertise, or at least slow down our acquisition of expertise, or, more ominously, teach dysfunctional skills that interfered with expertise.
But how would that work? In 2003, I identified several possibilities.
Recognizing patterns. That’s something we are good at—spotting patterns in data, seeing trends, seeing anomalies, spotting nonevents. But information technology “helps” us by doing the analyses for us, since the high data flows make it impossible for us to examine all or even most of the data. So we are removed from direct access to the data and removed from our ability to spot patterns.
For example, in one project we conducted on weather forecasters, a technologist at a weather forecasting conference explained how his system helpfully smoothed the data in order to highlight trends. Most of the audience appreciated this assistance, but the experienced forecasters in the room winced—they wanted to see the noise because that was where turbulence was going to affect the weather picture. The smoothing was removing a critical cue.
Searching for data. That’s another important human ability—to adaptively search through data. However, information technologies often organize the search space in advance and define the basic level of detail. But in complex settings there is no basic level of detail, and the appropriate level of detail depends on what you are searching for and what you are finding. The relevant data depend on how you are interpreting the situation, a phenomenon called the relevance realization process (Vervaeke et al., 2012)—skimming through the large amounts of data to identify what matters. Information technologies can reduce their users to passive recipients of data rather than active sensemakers.
Mental models. As we gain expertise, we are acquiring richer mental models about the phenomena we are interacting with. But information technology can weaken our mental models by keeping us tethered to the original categories we started our searches and analyses with. Decision-makers may have to set up categories in advance, and subsequent events may tie them into these categories rather than allowing and supporting the evolution of categories based on new experiences.
The story behind the data. As we gain expertise, we learn more about how the data were collected, and we become sensitive to limitations in the data collection process. However, information technology often makes it difficult to trace the story of the data and to appreciate how the data were collected and synthesized—the pedigree of the data. Borders et al. (2024) presented a mental model matrix that highlighted the importance of understanding the limitations and boundary conditions of a system in order to work effectively with it. However, if we don’t know the pedigree of the data, we will struggle to identify boundary conditions.
Adaptability. This is a critical function of expertise—being able to pivot based on discoveries about what is really going on in a situation. Unfortunately, many information technologies can go off the rails if we don’t follow the prescribed procedures for interacting with them or if we don’t follow the steps that have been laid out. As a result, it can become more difficult for us to improvise when new and unexpected conditions arise. When we stray from the prescribed path, we are likely to be bombarded with error messages, discouraging us from adapting.
Artificial Intelligence Essential Reads
Is It Real or Is It AI? Who Can Tell Anymore?
What El Niño Can Teach Us About AI Doomsday Fear
Active stance. With expertise comes an active stance of taking charge and making things happen. Yet information technologies can shift us into a passive stance, unthinkingly accepting the recommendations of the system. Here the link to LLMs is particularly strong, as users relying on LLMs contentedly accept their products, often without doing any kind of due diligence.
Conclusion
These are the attributes of expertise that information technologies seemed to jeopardize, in my 2003 analysis. Fast-forward two decades. Most of these attributes also seem threatened today by the spread of generative AI. I leave it to readers to gauge how closely the generative AI dysfunctions map onto the earlier information technology problems. At the very least, I hope this post will be helpful in unpacking what may be contributing to today’s generative AI dysfunctions. The challenge is to better understand the basis of expertise in order to find ways to safeguard and enhance it.
Borders, J., Klein, G., & Besuijen, R. (2024). Mental model matrix: Implications for system design and training. Journal of Cognitive Engineering and Decision Making,18(2), 75-98.
Klein, G. (2004). The power of intuition. New York, NY: A Currency Book/Doubleday. Hardcover edition titled Intuition at Work (2003).
Vervaeke, J., Lillicrap, T.P., & Richards, B.A. (2012). Relevance realization and the emerging framework in cognitive science. Journal of Logic and Computation, 22(1), 79–99. https://doi.org/10.1093/logcom/exp067
