As machine learning becomes core to every product, engineering teams will restructure.

In the past, the core engineering team & the data science/machine learning teams worked separately. The core engineering team ships the product & focuses on reliability.

For most companies, the data science team analyzed data & wrote machine learning models to support the business functions : sales, marketing, customer support. These teams operated downstream of the data warehouse. But they are movin’ on up to work on products in front of the data warehouse.

ChatGPT has catalyzed an expectation in users’ minds that all products should converse with users & should be rather intelligent.

To accomplish this vision, companies will merge the data science team into core engineering.

Data scientists’ expertise will be required to optimize the end-user experience : determining the right data sets to use to train models, understanding the best prompts to elicit germane answers (prompt-engineering), & model monitoring. They will need to be part of core engineering to achieve those goals.

The forces behind this change have been brewing for some time. Data science teams’ increased budget faces pressure from the office of the CFO to justify the investment. LLM-features should contribute directly to revenue via upsell & market share, quieting questions.

Also, the challenges of pushing models into production have frustrated data science teams. In this new world, data scientists will commit directly to production code.

Organizational change will require behavioral change. Machine-learning enablers - startups that sell ML toolkits - have shifted sales tactics to seize the day. Their ideal customer profile is no longer data science teams but core engineering teams.

That fundamental shift - selling to the core engineering & data science teams - creates new opportunity for vendors to unify the two groups by providing platforms & infrastructure that enable a new form of collaboration.