If you’ve checked your LinkedIn feed over the last few years, you’ve probably noticed the tech industry’s wild roller coaster ride. First, the pandemic led to a huge wave of tech hirings—promptly followed by the Great Resignation—and then huge rounds of layoffs.
In all, there were a staggering 1,405 rounds of tech layoffs which affected nearly 220,000 individuals. Even tech titans such as Amazon and Meta were seen cutting thousands of jobs and Twitter was reported to have made late night cuts to their data scientist teams.
At the same time, the U.S. Bureau of Labor Statistics (BLS) reports that there will be a 36 percent job growth rate over the next decade for data scientists. This rate is immensely higher than most other roles they track—where the average growth rate is five percent—and is even higher than the 31.4 percent growth they predicted for data scientists only back in 2020.
The BLS also predicts a growth rate of almost 28 percent by 2026 for the number of roles requiring data science skills. Yet, most companies have been struggling to fill these roles.
So, is the current shortage of data scientists simply a classic Economics 101 case study of low supply and high demand, is there an upcoming AI Winter—or is something deeper brewing below the surface? There seem to be some differing opinions, so let’s dive deeper into three factors that could be influencing the data scientist shortage.
Comedians used to joke about people inflating their job titles to feel more important or to get a salary bump. These days—in the midst of a great global recession—the theory has arisen that companies are actually deflating job titles to fight economic uncertainty.
A recent Interview Query analysis focusing on job posting data notes a curious trend for “data science” roles:
Apps such as Blind—which provide an anonymous forum and community for verified employees to discuss issues—echo these sentiments. Many posts ruminate over cases of “data science teams being downgraded to analysts” and SQL retrievers and verify the theory that the data scientist role is getting split up into different names, such as data analyst or machine learning engineer.
This may be bad news for experienced data scientists, but it’s good news for employers trying to save a buck and for young analysts trying to break into the field. Glassdoor confirms that—in the U.S. job market—data analysts now make about 30 percent less than data scientists.
The analysis makes three final notes:
Data Science is an extremely difficult and interdisciplinary field—and some worry that all of their academic prowess does not always translate well into real business value and ROI. Also, the data science skill set takes years to acquire and is long and complex:
The real challenge is finding a competent data scientist who can clean and augment the data
and then put models into production. A lot of data science knowledge is built on carefully planned and well-organized courses. Specific knowledge of a business problem, however, can’t be taught in the classroom.
Professionals who truly understand how to query and connect to databases, implement an object store and containerize models, and transform them into APIs and embed them into edge devices are in limited supply. In short, enterprises need people who can apply practical applications to their datasets.
It’s one reason why Gartner has pinpointed AI engineering as a top strategic technology trend—to put more focus on operationalizing AI models. Universities are improving, but it would still help if they would give their budding data scientists more practical work to put their intricate theories into practice.
In addition, once data scientists get started, they often feel siloed, especially with the increase in remote work. Senior data scientist Daliana Liu, who worked as a senior data scientist and senior machine learning instructor for Amazon Web Services (AWS), offered these comments on the reality of working in the field:
“I felt there’s a gap between what I learned in school, and what I actually do, and I also feel very insecure sometimes,” she said. “I didn’t know a lot of other data scientists who worked in the industry, so I wished I could have a community and talk to them.”
So far you’ve only heard the macroeconomic and employer perspectives, respectively. But what do many data scientists actually think? Many believe that the data scientist shortage doesn’t fall on their shoulders—but rather on enterprises that fall short in the following areas:
ML Studio makes AI accessible for organizations that need to solve real-world business problems by providing clients with a robust and ready-to-use End-to-End AI platform that doesn’t go off budget.
Contact us for a personal demo.