When generative AI is not appropriate in business
Last week, artificial intelligence (AI) made headlines in the US due to a sharp increase in the share prices of companies involved in AI. For example, Nvidia, a listed company on the Nasdaq index (NVDA), started the rally on 24 May 2023 with a share price of $305.38 per share that increased to $428.25 (R7778) on June 15.
Nvidia joined the six-member US trillion dollar club when its market cap increased over the past 8 months by $727 billion from $286.35bn on October 12, 2022, to $1.053 trillion on June 15. To put this achievement in perspective, it is about 24% larger than South Africa’s total gross domestic product (GDP) in 2022. Nvidia’s P/E ratio stands at an amazing 221.67.
This performance of Nvidia is partly due to the forecast of a 50% sequential leap in quarterly sales by the chip manufacturer but also due to the current global frenzy created by generative AI. Unlike its generative predecessors, such as Midjourney and DALL-E 2, ChatGPT seems to have caught the interest and imagination of the mainstream and experienced record downloads since its announcement in December 2022.
ChatGPT represents significant progress in generative Artificial Intelligence (AI). Generative AI is a type of artificial intelligence that creates new data in the form of images, text, music, or other forms of content. The process involves using a set of algorithms, such as deep learning, to generate new data that is unique and varied. Generative AI is the next generation in the development of AI-enabled solutions and is a rapidly evolving field and a powerful tool for creating new and innovative content.
The generated data can be used in a variety of applications, such as in art, media, entertainment and business, with new developments and applications being discovered all the time. The widely popular ChatGPT is one such example of generative AI that can handle a variety of conversational tasks such as question answering, dialogue generation, and text completion.
It is these developments that sparked a massive interest in AI companies and chip manufacturers. The growing role of AI in our society and in business should not be underestimated. But although I have highlighted the value and benefits of AI for business many times in the past, it is important to remember that at the core of generative AI (and ChatGPT is no exception) is a probabilistic model when generating new content.
When generating responses, probabilistic models try to infer or guess what the “answer” should be based on examples from its large training dataset. These models learn the probability distribution, patterns and structures from the training data and then use that knowledge to generate new examples.
Once the model is trained, it can generate new content by sampling from the learned distribution. However, since the learned distribution is an approximation of the true distribution of the data, the generated samples are not guaranteed to be perfect representations of the real data. Instead, they are probabilistic approximations that capture the main statistical properties of the training data.
The probabilistic or inference-based nature of these models, therefore, can render results that may sometimes be surprising. The probabilistic nature means that each time you generate content using a generative AI model, you are likely to get different results. For example, if you train a text generation model and ask it to generate a sentence multiple times, you might get slightly different sentences each time, even if the overall theme or style remains the same.
It is, for instance, well-known that, currently, ChatGPT makes mistakes, particularly related to factual queries, which is a direct result from it being a probabilistic model. Some critics even mentioned that ChatGPT sometimes “hallucinate.” Therefore, within settings where accuracy is a key requirement, generative AI is likely to be significantly less impactful.
While most real-world environments are not deterministic and generative, AI has shown great potential in various applications. There are certain scenarios in business for which accuracy is important and where the probabilistic nature of generative AI is not very suitable or appropriate.
Critical decision-making: For critical decision-making processes, where the outcomes can have significant consequences for the business, relying solely on the probabilistic outputs of a generative AI model may not be advisable. The uncertainty inherent in the generative AI's outputs can introduce risks unsuitable to a business. Instead, deterministic models that provide more certainty or models that combine probabilistic outputs with human judgment may be preferred.
Legal and compliance: Legal and compliance-related matters in business have increased and require precision and accuracy. Generative AI's probabilistic nature means that the generated outputs may not always be reliable or legally compliant. In situations where legal implications are involved, deterministic models or expert human judgment may be preferred over generative AI, or a careful and expert human eye is needed.
Customer service and support: Although AI, in general, can be of great help in customer service and support, businesses generally strive for consistency and reliability when it comes to customer interactions. The probabilistic nature of specifically generative AI can introduce variability and unpredictability in customer service scenarios, potentially leading to inconsistent or unsatisfactory experiences. In these cases, deterministic rule-based systems or human-operated customer support may be more suitable.
Safety-critical systems: In industries where safety is paramount, relying solely on probabilistic generative AI models may introduce unacceptable risks. Safety-critical systems, such as autonomous vehicles or medical devices, require deterministic behaviour and precise control. The probabilistic outputs of generative AI may not meet the stringent safety requirements, making other approaches, such as rule-based systems or deterministic control algorithms, more appropriate.
Financial planning and forecasting: Financial planning often requires precise and accurate predictions to make informed decisions. Generative AI's probabilistic nature means that its generated outputs may not consistently match the ground truth, leading to unreliable financial forecasts. In these cases, deterministic forecasting models or traditional statistical methods that provide more reliable estimates may be preferred.
Knowledge domains: Within constantly changing knowledge-based domains that require high levels of accuracy, such as the legal profession, the probabilistic nature of generative AI can be problematic. Generated AI answers will have to be cross-checked versus the latest developments to ensure that the answer is still accurate since some generative AI applications have a bias toward the past rather than recent events.
Tax: Non-deterministic tax preparation software would create numerous problems. In the case of tax, a traditional (deterministic) type of expert system would be better suited to assist businesses.
Without doubt, there will be continued improvements and refinements in the generative AI models to make them more suitable for environments requiring more certainty and consistency. However, a purely deterministic perspective may cause businesses to overlook the opportunities generative AI represents. Businesses often fall into the trap of using a traditional perspective of software versus recognising that generative AI represents a different class of ‘’probabilistic products’’, which are defined by non-deterministic outputs and emergent behaviours. Generative AI may, for instance, unlock a whole new category of product design - probabilistic products.
It is thus important that businesses select their AI carefully.
Professor Louis C H Fourie is an Extraordinary Professor in Information Systems University of the Western Cape.
BUSINESS REPORT