If the screwdriver were invented by the tech industry today, then it would be widely deployed for a variety of tasks, including hammering nails. Since the debut of ChatGPT, there has been growing fervor and backlash against large language models (LLMs). Indeed, many adaptations of the technology seem misappropriated, and its capabilities are overhyped, given its frequent lack of veracity. This is not to say there are not many great uses for an LLM, but you should answer some key questions before going full bore.

Will an LLM be better or at least equal to human responses?

Does anyone like those customer service chatbots that don’t answer any question that isn’t already on the website’s front page? On the other hand, talking to a person in customer service who just reads a script and isn’t empowered to help is equally frustrating. Any deployment of an LLM should test whether it is equal or better to the chatbot or human responses it is replacing.

What is the liability exposure?

In a litigious society, any new process or technology has to be evaluated against its potential for legal exposure. There are obvious places for caution, like medical, law, or finance, but what about an LLM-generated answer that directs people to a potential policy or advice that is misleading or unallowed? In many places, bad company policies or management have meant human-generated responses resulted in class action lawsuits. However, an improperly trained or constrained LLM could generate responses for a large number of users and create unintended liability.