Generative artificial intelligence tools have captured the interest of businesses and organizations. According to McKinsey & Co.’s “State of AI in 2023,” one-third of those surveyed said their organizations are regularly using generative AI in some capacity. In addition, most companies with more than 1,000 employees using AI expect to accelerate their rollout or investments of the technology in the near future. 

In these early stages of AI-human collaboration, there is a question of how the technology can match humans in productivity and creativity. There’s no doubt that AI language models like ChatGPT can execute a host of different tasks ranging from summarizing the latest research to writing compelling outlines. Yet, there has been minimal research conducted on the measurable impacts of AI-assisted tools on team performance. 

“Marketplace Morning Report” host David Brancaccio spoke with Kian Gohar, AI researcher and CEO of Geolab, a leadership-development firm. Gohar and his co-research partner, Jeremy Utley at Stanford University, studied the impacts of AI-assisted tools on team ideation, workflow and performance.  

Their research was published in Harvard Business Review’s current issue. The article, “Don’t Let Gen AI Limit Your Team’s Creativity,” details how employees need to approach AI as an ongoing conversation partner, rather than an oracle, to maximize results. Beyond the study, Gohar and Utley also created a five-step process for successfully implementing AI in the workplace. 

The following is an edited transcript of their conversation.

David Brancaccio: So, you designed this experiment, and you had actual humans take on different tasks. Some of them were just working on the problem, standard style, traditional. Others also had access to generative artificial intelligence system.

Kian Gohar: Exactly. We were really interested in measuring the quantity of ideas they developed and the quality of ideas. As results came in, we were shocked AI-assisted teams only had modest gains compared to their human-only counterparts. This was because these generative AI models are designed to give us the most likely answer to a potential question. We’ve had decades of conditioning to type in the perfect term in a search engine to get the right answer. Humans have a bias to settle for acceptable or good enough answers. Most teams are repeating the same behavior with AI and predictably getting average answers. And this is sort of what we call the tyranny of the search box. It’s not the tech that is the problem. It’s how humans use the tech. That’s the problem.

Brancaccio: I’m hearing that there may be some learnings from your research that might inform even individual use.

Gohar: Yeah, absolutely. I’d recommend talking to AI like you would to a colleague and give the AI context about the topic. We really need to have more chat and less bot.

Brancaccio: You’re going to have people in companies at top levels, you’re going to have teachers going, “How do I get the training to figure out how I’m supposed to interact with these AI devices?”

Gohar: The school systems are probably most likely the ones who are going to train younger generations on AI soft skills. But if you’re in the workforce, we’re seeing a big expectation gap. Chief HR officers are trying to teach people how to use it, but they can’t do it at scale. This leaves employees left on their own to experiment on learning AI. Now, as part of our research, we developed a conversational AI drill coach that trains all employees on conversational AI at scale. We’re seeing teams 10x their team’s AI skills in just 10 days.

Brancaccio: It sounds like you might be able to train yourself as some sort of facilitator. It’s like in most families, there’s the go-to IT person. It may be that each organization has a few go-to AI people. But what are some of the questions that you have going forward?

Gohar: We’re trying to figure out is how do teams learn to use these AI skills most effectively so that they become habits rather than being a chore? Like most good things, you don’t become an Olympic champion overnight. You go to the gym and train for years until you get better at it. And that’s where we are right now with AI.