Machine Learning / Artificial Intelligence Tools

Positioning for the ETR Observatory on ML/AI was determined by ETR’s two core, syndicated surveys. Full methodology and graphic explanation are available on our About the ETR Observatory page.

In late 2022, OpenAI made headlines when it made ChatGPT widely available, an interface where users could interact with a kind of chatbot to produce sophisticated AI-generated responses. Though in development for several years prior, the arrival of ChatGPT caught the attention of the press, business, and popular culture in a way no other previous generative AI or large language model (LLM) tool had. ChatGPT has been called a “Sputnik moment” in how it has launched a new AI race, and pundits have been quick to polarize into the two typical camps we see anytime a new technology breaks onto the scene: those seeing the technology as a revolutionizing force for utopian transformation, and those seeing it as a serious threat to the established order and calling for regulation. In a recent ETR Insights interview, the IT manager for a midsize municipal government remarked that this enthusiasm will ultimately be good for AI in general: “One of the good things about OpenAI, ChatGPT, and things like that is the increase of public awareness. People that I wouldn’t expect to be asking me about AI are now asking about AI […] which is great because it gives everyone a chance to have those types of discussions and potential use cases.” Just a few months in, and we already see many companies weaving ChatGPT and similar products into their enterprise tech offerings, such as Microsoft with its $10 billion investment in OpenAI and its rollout already of ChatGPT in its Microsoft 365 products under the Copilot branding.

As with any new technology, the hype around generative AI and LLMs will give way to more evenhanded discourse and practical achievements at scale tempered by rational controls. Until then, however, the hype for generative AI in general and for OpenAI in particular remains high. ETR added OpenAI to the February 2023 ETS, and right away it posted the highest evaluation rate in the entire survey. It remained atop the heap in the May 2023 survey, with a whopping 46% evaluation rate, which tracks plans to evaluate and current evaluation indications. Despite the high evaluation rates captured in the ETS, most organizations have still not shifted to utilization of generative AI tools, suggesting that clear business use cases are not yet evident for many businesses. In the April 2023 Macro Views survey, ETR asked respondents for what business use cases they were considering generative AI and LLM tools. More than half indicated they were not even evaluating these tools yet, but among those who were considering generative AI, customer support, text and data summarization, and code generation and documentation were the most common uses.

II. ML/AI at Enterprise Scale with Robust MLOps Capabilities As an organization shifts from experimentation with its first data science projects to full-scale deployment of ML/AI, the focus becomes ML model management and reuse, monitoring, governance, and ultimately automation. Many of the vendors that provide these capabilities – collectively called MLOps, which draws from DevOps principles of continuous delivery – are leaders in ETR’s spending intentions data, demonstrating deepening investment in complete and scalable platforms. The big three cloud platforms offer these complete capabilities in the form of Microsoft’s Azure Machine Learning, Google’s Vertex AI, and Amazon SageMaker, and these platforms are striving toward greater ease of use and accessibility. The VP of Business Intelligence and Analytics for a midsize financial services enterprise remarked in a recent ETR Insights interview that he liked Azure ML because it was “self-service” and did not require “a DevOps competency” to navigate. He said with “Azure, I was able to get my data scientists, plug them in, and just let them start working with the team, and we were able to go from training to deployment in a couple of weeks.” Microsoft, AWS, and Google have the highest Pervasion in the ML/AI sector, according to TSIS data. Microsoft’s Net Score looms large in the sector at 67%, followed by AWS at 55% and Google at 47%. Large legacy vendors like Oracle and IBM also offer full MLOps capabilities through Oracle Machine Learning and IBM Watson Machine Learning, respectively. Both have low Net Scores in recent TSIS cycles and Pervasion under 20%. Oracle has slowly gained ground in Pervasion in recent surveys and held relatively steady with single-digit Net Score, while IBM Watson has dropped considerably in Net Score into negative territory and is showing contracting Pervasion. Some smaller vendors, such as Domino, focus on enabling robust MLOps and collaboration without organizations having to worry about managing infrastructure. Domino has seen Net Sentiment increase year-over-year in the ETS, but it remains low in absolute terms at just 8% as of the May 2023 survey.

Open-source platforms Anaconda and TensorFlow offer support for ML model deployment and end-to-end management in both free licenses and in paid tiers with added support and storage. Anaconda provides a way to manage versions of Python and packages and deploy projects, while TensorFlow offers a more complete MLOps tool set but is known for its specialty in training for deep learning use cases. In a recent ETR Insights interview, the VP and Chief Data Architect for a large consumer goods company noted his organization uses platforms like Google and Databricks “for a certain class of problems,” but will turn to “more generic tools like Python and TensorFlow to build more complicated regression models.” In another interview, the CIO and CISO for a nonprofit medical research institute added that TensorFlow is “a solid name” with a “great community ecosystem,” an important attribute for the success of any open-source project. May 2023 ETS data shows TensorFlow and Anaconda with the third and fourth highest Net Sentiment in the ML/AI sector, trailing stand-out OpenAI as well as Databricks. TensorFlow’s Net Sentiment declined a bit from May 2022 levels, down four percentage points to 34%, while Anaconda’s Net Sentiment has remained steady at 28% for multiple surveys.

III. Democratizing Data Science to Speed Time to Business Value Several ML/AI companies have tried to simplify the complexity of data science and offer shortcuts for organizations to see business value in data science programs quickly. Often these vendors use terms like “democratizing data science” in their marketing materials, and this view of democratization might include low-code interfaces for building, training, and deploying models or ready-to-use pre-trained ML models fit for a variety of common business use cases.

H2O.ai is one such vendor, focusing on ML applications accessible to business users who do not have expertise in tuning models, model deployment, or infrastructure. Similarly, DataRobot offers automated machine learning, or AutoML, an approach to ML that uses AI to parse the signal from the noise in a large data set to fit models to clear business problems. C3.ai and Dataiku have similar offerings, with C3.ai focused on speed to business value and Dataiku known for its finesse as a data preparation tool and ease of collaboration between technical and business users. In a recent ETR Insights interview with the Head of Data Science and Analytics for a large healthcare/manufacturing enterprise, all four of these tools – H2O.ai, DataRobot, Dataiku, and C3.ai – were appraised as being “kind of a fun thing” to explore in proofs of concept, primarily because of their accessibility to business users. However, he noted that “our purists – our data scientists in the company who I could purists – don’t want to use these tools. ‘I want to use my Python and R, and all my open-source capabilities to build the models.’” However, he cited the organization’s leadership, who wanted shareability and a common platform where everyone could understand and learn from it, ultimately pushed for alignment on one of these easier-to-use platforms. In another interview, a data scientist for a large financial institution pointed out that the “positive things” about this class of tools is that “if the use cases are relatively straightforward, all of these tools are good,” adding that they offer “opportunities to get quick models and results.” However, he noted the “critical challenge in terms of the work product is that it is very difficult to customize because it’s a black box.” The trade-off with tools accessible to non-technical users is that it is often difficult to get to a technical understanding of what the system is producing, which is limiting.

Indeed, mature data management and preparation discipline is a key feature in enterprise-scale ML/AI programs, and this explains why Databricks continues to garner strong spending intentions and rave reviews from ETR’s community of IT decision makers. Databricks’ data lakehouse architecture, which blends the strengths of the data warehousing and data lake paradigms into a practical all-in-one data management solution, has brought the company not just into prominent standing as an ML/AI platform but also as a data warehousing alternative for more run-of-the-mill business analytics use cases. In an ETR Insights interview, the VP and Principal Software Engineer for a large IT services enterprise expects Databricks “to stay at the number one spot for whatever level of ML/AI” an organization needs or “whatever data processing tasks,” noting that “Databricks seems to be the way to go and the way to run your models for any of your analysis.” He adds that he “sees them getting bigger and bigger in the space” and taking on Snowflake and the big three cloud platforms for data warehousing too.

Finally in this class of ML/AI tools are vendors like ElectrifAi and Hugging Face. ElectrifAi’s value proposition focuses on pre-built ML and natural language processing (NLP) models ready for deployment in popular business use cases. Hugging Face, on the other hand, showcases the power of its community and the wealth of resources community members are willing to share with each other, from fully trained models to data sets. In an ETR Insights interview, a data scientist from a large financial services enterprise said “Hugging Face is a must in NLP. Their models, the pretrained models, are extremely helpful. An amazing product based on [a technique for NLP pre-training] BERT, for mostly NLP purposes in using neural networks.” With both ElectrifAi and Hugging Face, the aim is for businesses not to have to reinvent the wheel when it comes to ML/AI and realize quicker time to business value.

IV. Tools with Unique Value A cluster of other ML/AI vendors can be categorized based on the kind of unique value or specialized niche use case they offer to the broader ML/AI landscape. Anyscale, which emerged from a university lab, focuses on scalability, helping organizations make the leap from development to enterprise-level, sustainable production on ML/AI. Despite the name, it is not to be confused with Scale AI, a vendor focused on providing high quality training data for training ML models and AI applications. Further into a niche use case is Labelbox, a collaborative platform for labeling training data in a variety of formats, including text, images, and video. Lastly, Character.ai allows users to develop their own chatbot characters and personas and hold full conversations with these different characters powered by generative AI. Still in its early stages, the tool could have a number of uses from customer service chatbots to digital assistants to therapy and counseling.

As with any enterprise IT tool market, ML/AI tools have evolved to fit the needs of a variety of organizations that are all maturing their advanced analytics initiatives at different rates. What is clear, however, is that the early stages of exploration in ML/AI have, for many organizations – and especially larger enterprises – reached a point of widespread enterprise-scale adoption, requiring robust end-to-end data architectures and sophisticated management and governance programs. Smaller organizations or those just now dipping their toes in the waters of data science are finding early value in free, open-source offerings or tools with pre-built, pre-trained ML models ready to accelerate time to business value. And with the broad awareness of generative AI technology in just the last several months, vendors and organizations alike are scrambling to imagine new possibilities with AI to leverage the power of LLMs.

To respond to these organizational stages, the ML/AI vendor landscape is dotted with big players offering full MLOps capabilities for mature data science operations, with large public cloud players like Microsoft, AWS, and Google leading the way alongside open-source stalwarts TensorFlow and Anaconda and popular open libraries and languages like Jupyter Notebook, Pyton, and R. Other vendors have built reputations for strength in particular areas, like Databricks and Dataiku, which offer sophisticated data management setups that feed data science use cases as well as more line-of-business-focused reporting and self-service business intelligence use cases. A separate set of vendors aim for democratizing data science, with off-the-shelf, turnkey ML solutions alluring to business executives and less technically-savvy business users. Still, yet another group of vendors remain in a niche, specializing in nuanced techniques like model training, chatbot refinement, or rapid scaling that specific industries may find more appealing at the moment.