News

Survey Takes the Temperature of Developers' AI Outlook

• By David Ramel
• 06/26/2023

Stack Overflow's extensive developer survey for this year, which included, among other things, a look at how coders are embracing cutting-edge generative AI technologies, yielded high praise for the AWS CodeWhisperer coding assistant.

AWS CodeWhisperer was behind tools like GitHub Copilot in the survey rankings, but it did earn the third spot on the list of AI developer tools, which positions the tool as a leader among cloud-hosted offerings. This achievement is noteworthy for a tool that went GA only last month after a year-long preview phase.

Stack Overflow's annual report, widely regarded as one of the foremost developer surveys in the industry, reached out to approximately 90,000 individuals worldwide. It aimed to gain insights into their preferred tools and programming languages, encompassing a comprehensive array of professional and personal details.

With the recent debut of ChatGPT, GitHub Copilot, GPT-4 and all the rest, Stack Overflow naturally added an AI section for the 2023 report.

AI highlights of the report as presented by SO include:

• 70 percent of respondents are already using or plan to use AI tools in their development process: 44 percent use AI tools now and 26 percent plan to soon.
• 77 percent feel favorable towards using AI tools as part of their development workflow -- but we do see some differences between groups.
• 33 percent see increasing productivity as the most important benefit of using AI tools as part of a development workflow.
• 42 percent trust the accuracy of the output of AI tools used in their development workflow.

One perhaps surprising finding is that while developers are keenly interested in AI tech, they aren't really using that many AI different tools in their work.

"It's early days in the hype cycle for these newer AI technologies," StackOverflow said in a June 14 blog post. "We expect that a little more time may need to pass before we see developers using more AI tools more broadly. Another factor that may be contributing to the slow adoption of AI tools into the development process is that professional developers don't highly trust the accuracy of AI tools. Of those using or planning to use AI tools, only 3 percent highly trust the accuracy they provide, and twice as many (6 percent) highly distrust them. While the majority (39 percent) are somewhat trusting of AI tools, this response tells us that AI tools still need to prove their usefulness."

Among those tools, as mentioned, GitHub Copilot was the far-and-away leader, mentioned by nearly 55 percent of all respondents, followed by Tabnine (nearly 13 percent) and AWS CodeWhisperer (a little over 5 percent). After those three, no other tool outranked the "Other" category.

As noted, the survey did show that 70 percent of respondents were using or planning to use AI tools, adding an interesting data point to compare with a recent GitHub report that found a "staggering" 92 percent of U.S. developers were using AI tools.

Looking forward, SO also asked developers how they thought AI tools would evolve their workflow over the next year, finding that 77 percent of respondents felt that the next year will look somewhat or very different for writing code with AI -- with 75 percent feeling the same about debugging code with AI. Updating the tools and evolving the learning curve will likely benefit those currently using AI tools or planning to soon, SO said.

The sprawling survey goes on to cover much more than AI, of course, delving into areas such as developer profiles, technology used (and admired/desired), community participation and much more, but AI was clearly the shining new star in this report.

"This year, we went deep into AI/ML to capture how developers are thinking about it and using it in their workflows," SO said. "Stack Overflow is investing heavily in enhancing the developer experience across our products, using AI and other technology, to get people to solutions faster. Stack Overflow Labs is where we're sharing all we're doing -- check it out for a deep dive on AI/ML insights as well as see what we're experimenting with so far."