Maintaining the AI momentum

According to a recent PwC survey, 61% of U.S. CEOs expect AI to change how their business generates value, making generative AI a top investment priority for most organisations. However, enterprises must turn AI hype into reality to realise this potential. The good news is that companies are making progress in this regard.

PWC Survey

Databricks State of Data & AI Report

Based on Databricks' recent State of Data + AI report, companies have moved 1,342% more models from experimentation to the real world. Additionally, the number of experimental models has grown by 134%, indicating that companies are not slowing down on their data and AI ambitions. The current challenge is how to maintain this momentum.

However, some companies are finding it difficult to align their expectations. Many executives naturally expect an immediate return on these AI investments. Additionally, due to concerns that competitors are advancing faster in AI, companies are neglecting to consider long-term strategies.

PWC Survey

PWC Survey

That’s not the right approach. Companies that only focus on the present will find themselves constantly chasing after new trends. Generative AI is all about long-term strategies, and CIOs need to change the company’s AI mindset from seeking quick wins to aiming for long-term business transformation.

Here are the three main areas CIOs and other technical leaders should concentrate on to sustain the AI momentum.

1. Data Infrastructure

The priorities and focus areas for AI are evolving almost as rapidly as the technology itself. This progress requires relevant computing power, and GPUs are the dominant computing platform for accelerating machine learning workloads. They are foundational for today's generative AI era.

Three technical reasons:

• GPUs employ parallel processing.

• GPU systems scale up to supercomputing heights.

• The GPU software stack for AI is broad and deep.

The bottom line is that GPUs outperform CPUs in technical calculations, doing so faster and with greater energy efficiency. This makes them ideal for AI training and inference and for a wide range of applications that utilize accelerated computing. But GPUs are both powerful and expensive. The challenge lies in keeping GPUs busy with fast data from a data infrastructure standpoint. Data storage, processing, and retrieval costs can be substantial. Moreover, integrating GenAI with cloud-generative AI services by creating new storage silos can incur additional costs and management complexity. Customers prefer to leverage their existing data sources to build GenAI applications, as this allows them to control costs and management complexity while utilizing established data management capabilities and processes.

Many companies are now turning to Retrieval-Augmented Generation (RAG) to address this need, with data lakes and data warehouses serving as pivotal platforms that promise to enhance data integration, scalability, and analytical capabilities. Retrieval augmented generation (RAG) is the primary technique for enhancing large language models (LLMs) with enterprise data. For instance, companies use RAG to provide LLMs with domain-specific knowledge from user manuals or support documents to ensure that chatbots powered by LLMs give accurate and relevant responses.

However, balancing cost optimization with performance is a significant challenge for enterprises implementing RAG systems. Integrating data from multiple sources into a cohesive knowledge source that RAG systems can query is a complex task. The need for real-time access to structured enterprise data for generative AI has become evident.

2. Foundational Models

Foundation models like GPT-4, BERT, DALL-E 3, CLIP, and Sora are cutting-edge AI advancements. They are trained on large amounts of unlabelled or self-supervised data to develop broad knowledge and language understanding. These models can be fine-tuned for specific tasks and have a broad understanding of language, images, and other data types, making them highly versatile.

Advantages:

- Efficiency and Cost-Effectiveness: Training a custom model from scratch requires significant computational power, time, and expertise. Foundation models, being pre-trained, significantly reduce these costs and can be deployed much faster.

Access to Cutting-Edge Technology: Foundation models are often developed by leading AI research organisations and incorporate the latest advancements in the field. Businesses can leverage these models to stay competitive without investing in costly research. They can also take advantage of good architecture so that pre-trained models are pluggable, a key feature in a rapidly evolving industry.

- Scalability: Foundation models are designed to be highly scalable, allowing businesses to apply them across various tasks and industries with minimal modifications.

Many companies are using RAG and fine-tuning techniques to enhance AI model performance and reduce operational costs. These tools narrow the scope of AI outputs, increasing trust and autonomy in AI initiatives. RAG frameworks connect Generative AI models to private data, improving responses by providing context and factual knowledge. Agentic RAG uses intelligent agents to verify precise outputs, allowing for better handling of advanced queries with higher reliability. In product development, the focus shifts from model creation to application and fine-tuning, requiring integration into existing systems and customization for specific tasks.

Therefore, businesses should focus on the following steps:

• Identify use cases: Understand where AI can add the most value and select appropriate foundation models for these tasks.

• Set quality criteria: Establish testing criteria that define success. Seeking 100% accuracy is unachievable. However, improving manual human activities is very achievable. Being realistic about defining your PBOs (positive business outcomes) is crucial.

• Invest in talent: Although foundation models reduce the need for extensive data science teams, businesses still need skilled professionals to fine-tune and implement these models effectively. Understanding the technology you’re integrating is a must.

• Ensure ethical AI use: As AI becomes more embedded in business processes, ethical considerations around data privacy, bias, and transparency become increasingly important. Businesses must establish guidelines and monitoring practices to ensure responsible AI use and adherence to their ethics policy.

3. Prioritizing Accuracy in Use Of AI

AI is revolutionizing workflows and reshaping how organisations approach data. However, many companies are still in the testing and experimentation phase. This is because, currently, many models are not as accurate or reliable as they need to be. Some companies have already experienced the repercussions: for example, one airline's chatbot offered a refund that did not align with the company's official policy—and the court ruled in favour of the customer. Another instance is a car dealership's chatbot offering a car to a customer for one dollar.

Maintaining AI momentum for businesses requires a strategic approach that involves continuous investment in technology, talent, and training. Companies should prioritise the integration of AI into daily operations to enhance efficiency and decision-making processes. This is why businesses must focus on maintaining strong data infrastructure and practices. They should prioritize AI applications that align with their overall business strategy and develop a workforce ready for the era of change. Additionally, gathering data and feedback on AI performance helps refine its capabilities, which drives sustained interest and engagement across the organisation.