Artificial Intelligence (AI) is rapidly transforming the way legal information is accessed and processed. From drafting documents to conducting research, AI tools can save significant time and improve efficiency. However, growing numbers of incidents are demonstrating the dangers of relying on AI-generated information without proper verification.

While these risks have become particularly evident in the legal profession, they extend far beyond attorneys and law firms. Businesses and members of the public who rely on AI-generated legal advice may expose themselves to significant financial, legal, and reputational risks.

Generative AI tools do not truly "understand" questions, documents, or legal principles. Instead, they generate responses by identifying patterns in large volumes of text and predicting what words are most likely to come next. The result is often a convincing and authoritative-looking answer, even when the information provided is incomplete, inaccurate, or entirely fabricated.

This phenomenon is commonly referred to as an AI "hallucination", where an AI system generates information that appears credible but is factually incorrect.

Consider the following observation by an expert software engineer contained in an expert report filed in court:

"Contrary to popular opinion, computers are in fact not intelligent at all. A computer (machine) has no more intelligence than, using an analogy, a hammer. A computer appears intelligent because many humans have converted their knowledge and expertise into instructions that the machine executes automatically and therefore appears at least as intelligent as the humans who created those instructions. In reality, machines simply execute these instructions with no more insight, motivation, or intelligence than a hammer displays when driving a nail into wood."

Recent reports have highlighted instances where legal practitioners unknowingly relied on AI-generated case law that did not exist. In one widely publicised South African example, a junior advocate was reportedly referred to the Legal Practice Council for investigation after an AI tool generated fictitious legal authorities that were subsequently included in court documents.

Such incidents serve as an important reminder that AI-generated information, no matter how convincing it may appear, should never be accepted at face value.

The risks for Legal Practitioners
For legal practitioners, the consequences of relying on unverified AI-generated content can be severe.

Where fictitious authorities, incorrect legal principles, or inaccurate factual information find their way into legal documents, practitioners may face:

• Professional embarrassment and reputational damage;
• Judicial criticism;
• Disciplinary proceedings by professional regulatory bodies;
• Professional negligence claims from clients;
• Financial losses arising from unsuccessful litigation or defective legal advice; and
• Loss of client trust and confidence.

Attorneys and advocates owe their clients a duty to apply independent professional judgment. This duty cannot be delegated to an AI system. Regardless of whether research is conducted by a junior candidate attorney, a legal researcher, or an AI platform, the legal practitioner remains responsible for ensuring the accuracy of the advice and documents ultimately provided to clients or filed with a court.

The risks for businesses and the general public
The dangers of using AI are not limited to legal professionals.

More and more individuals and businesses are turning to AI platforms for advice on contracts, employment disputes, debt collection, estate planning, property transactions, tax issues, and other legal matters. While AI tools can provide useful general information, they cannot replace advice tailored to a person's specific circumstances.

A member of the public who relies solely on AI-generated advice may:

• Sign an unenforceable contract;
• Misunderstand their legal rights or obligations;
• Miss important legal deadlines;
• Take action based on outdated or incorrect laws;
• Enter into unlawful or financially risky transactions; or
• Abandon valid legal claims because of incorrect information.

In many situations, the cost of acting on incorrect advice can far exceed the cost of obtaining proper professional guidance in the first place.

AI is a tool, not a professional advisor
AI undoubtedly offers significant benefits when used appropriately. It can assist with research, document drafting, information gathering, and administrative efficiency. However, it should be viewed as a tool rather than an authority.

The fundamental mistake many users make is assuming that AI-generated content is inherently accurate simply because it is presented confidently and in a professional format. AI systems do not exercise judgment, understand context in the same way humans do, or assume responsibility for the consequences of their advice.

The responsibility for verifying the accuracy of information remains with the person using the tool.

As AI technology becomes increasingly integrated into professional and everyday life, both legal practitioners and members of the public should exercise caution when relying on AI-generated information.

For law firms, robust AI governance policies and mandatory verification procedures are essential safeguards against professional and legal risks. For businesses and individuals, AI should be regarded as a starting point for gathering information rather than a substitute for qualified professional advice.

In the legal field, as in many other professions, technology can assist decision-making, but it cannot replace expertise, experience, accountability, and professional judgment. The growing number of AI-related errors serves as a timely reminder that convenience should never come at the expense of accuracy.

Disclaimer: This article is the personal opinion/view of the author(s) and does not necessarily present the views of the firm. The content is provided for information only and should not be seen as an exact or complete exposition of the law. Accordingly, no reliance should be placed on the content for any reason whatsoever, and no action should be taken on the basis thereof unless its application and accuracy have been confirmed by a legal advisor. The firm and author(s) cannot be held liable for any prejudice or damage resulting from action taken based on this content without further written confirmation by the author(s).