AI’s Legal Missteps: A Wake-Up Call for Lawyers Everywhere

_{^Introduction}

A recent South African High Court case has raised concerns about the use of artificial intelligence (AI) in legal practice. The case involved a candidate attorney’s notice of appeal, which cited fabricated case law. Upon review, the presiding judge found that most of the references did not exist, prompting an inquiry into the attorneys’ conduct and a referral to the Legal Practice Council for investigation (Marot, 2025). The judgement condemned reliance on unverified AI outputs as “irresponsible and unprofessional”, highlighting the need for practitioners to ensure the accuracy of materials submitted to the court. This incident underscores the risks of unchecked AI use and the importance of accountability when integrating new technologies.

South Africa’s challenges with AI in legal contexts are not isolated. In 2024, the Supreme Court of British Columbia addressed a similar issue where a lawyer cited fabricated cases generated by ChatGPT. Although the cases were withdrawn before the hearing, opposing counsel spent significant time attempting to verify the non-existent references. While the court refrained from imposing special costs, it warned against the irresponsible use of AI tools, stressing that legal professionals must understand the limitations of such technologies (Maphumulo, 2024). These cases reflect a broader, global issue: balancing technological innovation with professional and ethical standards.

_{^{Understanding AI and Generative AI}}

Artificial Intelligence simulates human intelligence to perform tasks like problem-solving, learning, and decision-making. Generative AI (Gen AI), a subset of AI, creates text, images, audio, and video by analysing patterns in extensive datasets. Tools like Thomson Reuters’ CoCounsel and Harvey AI are tailored for the legal sector, automating tasks such as legal research, document review, and drafting (Steele, 2024).

Gen AI tools promise to streamline legal workflows. For instance, CoCounsel claims it can review thousands of documents in minutes, condense complex information, and draft event timelines with remarkable speed. Similarly, Harvey AI offers features like summarising contracts, analysing documents, and recording billable hours (Steele, 2024). By handling routine tasks, these tools enable solicitors to focus on complex and strategic work.

However, the limitations of Gen AI are significant:

• Accuracy Issues: Gen AI tools can produce “hallucinations”—plausible but fabricated outputs. This undermines the credibility of legal work and can lead to serious consequences, as evidenced by recent court cases (Steele, 2024).
• Privacy Risks: AI tools may expose clients’ confidential or privileged information, particularly if the tools retain or share data for training purposes (Steele, 2024).

_{^{Ethical and Professional Responsibilities}}

The cases above illustrate the ethical duty of legal professionals to verify all materials submitted to the court. AI should be viewed as a tool, not a replacement for professional judgement. The South African judgement explicitly stated that AI cannot absolve practitioners of their responsibilities (Marot, 2025). Delegating critical tasks to AI without proper verification risks professional negligence.

Ethics are paramount in legal practice. AI usage must align with standards that prioritise client confidentiality and ensure the accuracy of submissions. The risks of reputational harm, legal penalties, and client mistrust highlight the need for cautious AI adoption.

_{^{Guidelines for Responsible AI Usage}}

To integrate AI responsibly, legal practitioners should adopt the following practices:

• Verify AI Outputs: Treat AI-generated content as preliminary input. Cross-check all research, citations, and drafts against authoritative sources to ensure accuracy.
• Protect Client Confidentiality: Use AI tools that guarantee data security. For example, CoCounsel ensures client data is processed securely and not used for training models (Steele, 2024).
• Obtain Client Consent: Seek explicit consent before using AI tools, particularly for sensitive matters. Address AI usage in confidentiality agreements to safeguard client interests.
• Invest in Training: Equip legal professionals with knowledge about AI’s capabilities and limitations. Training should emphasise ethical considerations and best practices for AI usage.
• Select Reliable Tools: Choose professional-grade AI solutions like CoCounsel or Harvey AI, which are designed specifically for legal applications and prioritise accuracy and confidentiality.
• Implement Internal Policies: Develop firm-wide policies on AI usage, including guidelines for data security and verification. Regular audits can ensure compliance and identify areas for improvement.

_{^{The Role of Regulation}}

Regulatory oversight is crucial to govern AI usage in the legal sector. Governments and professional bodies must establish clear guidelines addressing data privacy, accuracy standards, and accountability for errors. As Steele (2024) highlights, the inherent risks of Gen AI demand stringent measures to protect sensitive information.

Marot (2025) emphasises that regulatory frameworks should ensure AI tools complement rather than replace legal expertise. Collaboration between technologists, legal professionals, and regulators is essential to create standards that encourage innovation while maintaining professional integrity.

_{^{Broader Implications of AI in Law}}

The adoption of AI in the legal sector has wide-ranging implications. On one hand, it can democratise access to legal services by reducing costs and increasing efficiency. Small firms and sole practitioners, who often lack extensive resources, could leverage AI tools to compete with larger firms (Steele, 2024). On the other hand, the unequal distribution of advanced technologies risks widening the gap between large and small firms, potentially exacerbating inequalities in access to justice.

Marot (2025) notes that the rise of AI necessitates changes in legal education and training. Law schools must prepare future practitioners for an AI-enhanced landscape, teaching them to critically evaluate AI outputs, integrate technology into their practices, and address ethical dilemmas.

Bias in AI systems is another critical issue. Since AI learns from historical data, it may inadvertently perpetuate systemic inequalities. Addressing these biases is essential to ensure fairness and transparency, a concern echoed by Maphumulo (2024). By fostering equitable AI systems, the legal profession can better serve diverse communities.

_^Conclusion

AI integration in the legal sector offers transformative potential but also significant risks. Recent cases underscore the consequences of unverified and irresponsible AI use, reinforcing the need for innovation balanced with accountability. Legal professionals must adopt rigorous verification processes, prioritise confidentiality, and invest in ethical AI practices.

By navigating these challenges thoughtfully, the legal sector can harness AI to enhance efficiency, accessibility, and effectiveness. However, realising this potential requires steadfast adherence to ethical standards and a commitment to justice.

_^References

• Steele, M. (2024). Pros and cons of gen AI in the legal sector. Bizcommunity. Retrieved from https://www.bizcommunity.com/article/pros-and-cons-of-gen-ai-in-the-legal-sector-144346a
• Maphumulo, M. (2024). Cautionary tale for attorneys: Don’t rely on ChatGPT to supply case law. Bizcommunity. Retrieved from https://www.bizcommunity.com/article/cautionary-tale-for-attorneys-dont-rely-on-chatgpt-to-supply-case-law-127256a
• Marot, T. (2025). How unverified research and AI shaped a high court judgement. Financial Institutions Legal Snapshot. Retrieved from https://www.financialinstitutionslegalsnapshot.com/2025/01/10/how-unverified-research-and-ai-shaped-a-high-court-judgment/