Could AI make data science obsolete?

Generative artificial intelligence (gen AI) is paving the way for everyone to become their own software developers. But at the same time, AI may render many extraordinary skills unnecessary.

That's the word from Thomas Davenport of Babson College and Ian Barkin, a venture capitalist, in their latest book, All Hands on Tech: The AI-Powered Citizen Revolution. For starters, they point out that with low-code and no-code tools, robotic process automation, and now AI, the gates of software development are open to all. 

Also: AI scams have infiltrated the knitting and crochet world - why it matters for everyone

"Technology is no longer owned by any one department of function," they explain. "Data and its analyses are no longer the property of only the PhDs and the hard-core number crunchers. From now on, all employees have the ability to be system designers, data analysts, coders, and creators."

Davenport and Barkin note that generative AI will take citizen development to a whole new level. "First is through conversational user interfaces," they write. "Virtually every vendor of software today has announced or is soon to introduce a generative AI interface."

"Now or in the very near future, someone interested in programming or accessing/analyzing data need only make a request to an AI system in regular language for a program containing a set of particular functions, an automation workflow with key steps and decisions, or a machine-learning analysis involving particular variables or features."

As the authors mention, part of this future -- not quite formed yet -- are specialized bots designed to perform specific types of work. "There are digital workers from RPA vendors and other start-ups that claim to perform an entire job, although our investigations thus far suggest that they really perform just a few tasks and are certainly less flexible than human workers."

Also: Google survey says more than 75% of developers rely on AI. But there's a catch

This includes nascent software development bots, which vendors claim are "able to write software programs from start to finish," Davenport and Barkin state. "Our guess is that for the next several years, these bots will be capable of making human citizens more productive but won't replace them."

Gen AI will feel like the ultimate research assistant or programmer, they added, "because it is generating code to this analysis. It will elicit what you want, work very quickly, and allow you to change your mind infinite times in specifying your app, automation, or model."

"Gen AI will also make it easier to find existing models, features, or software components that you can use to begin your citizen project," they conclude. 

Also: How ChatGPT scanned 170k lines of code in seconds, saving me hours of work

Looking beyond these early starts, with the growth of AI, RPA, and other tools, "some citizen developers are likely to no longer be necessary, and every citizen will need to change how they do their work," Davenport and Barkin speculate. Gen AI will assume much of this work, including generating application code, automations, and data science analyses.

Dominic Ligot, CEO and CTO of CirroLytix, echoes Davenport and Barkin's observations in a recent HackerNoon article, noting how he enabled semi-technical individuals in a class to leverage data science tools:

"The participants, primarily CISOs who typically don't code, found the exercises, crafted with AI's assistance, to be intuitive and hands-on. My goal was to immerse them in working directly with data and code. They especially appreciated the chance to explore manually what modern cyberthreat surveillance and SIEM platforms typically automate, gaining insights into the processes happening 'under the hood.'"

At the same time, Ligot also suggests citizen developers and data scientists may not necessarily need technical skills, as AI takes on much of this work. "My key takeaway from the class was surprisingly counterintuitive: data science, as we know it, will eventually be replaced by AI," he said.  

Also: Technologist Bruce Schneier on security, society and why we need 'public AI' models

"The rise of AI-driven tools capable of handling data analysis, modeling, and insight generation could force a shift in how we view the role and future of data science itself," said Ligot. "Tasks like data preparation, cleansing, and even basic qualitative analysis -- activities that consume much of a data scientist's time -- are now easily automated by AI systems."

"What's worse (or better, depending on where you stand) is that AI is faster, more accurate, and less prone to human error or fatigue."

Still, getting to the point where development and data science are delivered seamlessly via AI will take time, Davenport and Barkin clarify. "It seems likely in the future that gen AI and conversational AI broadly will be the front end to all citizen applications," they say. "That's possible today with many tools, but it takes at least a modicum of sophistication to create prompts that will get you the first cut at an app, a data analysis, or an automation workflow that you want. That's also true with code generations, and it's one reason why experienced coders tend to have better luck than inexperienced ones." 

Also: The best AI for coding in 2024 (and what not to use)

However, they continued, "within a year or two, it will be possible to have an iterative discussion with a gen AI interface about a machine-learning analysis."