AI ‘cannot yet rival divorce lawyers’, says firm

Artificial intelligence (AI) chatbots such as ChatGPT and BARD cannot yet rival the advice provided by divorce lawyers, one firm has claimed.

A recent study – conducted by SG Murphy Solicitors – started by asking ChatGPT3 and BARD various questions relating to divorce in Northern Ireland and the UK.

It revealed that, although ChatGPT3 fared better overall than BARD, both chatbots provided “insufficient detail and incorrect data” enough times to cause concern for anyone using them in place of an expert.

The researchers at SG Murphy asked each AI chatbot questions, including:

• What are the grounds of divorce in Northern Ireland?
• I want to leave my husband but don’t want to get a divorce. We live in the UK. What can I do?
• What happens to my pension in a divorce in the UK?
• I have been married for six months in Northern Ireland. Can I get a divorce?

They also asked the two AI platforms to draft a couple of documents, including a matrimonial separation agreement and a divorce petition. The goal was to see if AI could give clear, factually correct answers, and to see how these answers compared to what would be expected from a competent divorce lawyer.

More specifically, ChatGPT3 provided insufficient detail in 50% of its responses. In comparison, responses from BARD did not meet the required level of detail that we would expect to be given to a new client from an experienced divorce lawyer 70% of the time. What’s more, it’s said both AI chatbots generated answers that were factually incorrect 20% of the time.

Shane McCann, Managing Director at the firm, says:

“This is an unacceptable level of accuracy. With both AI chatbots being freely available to the public, this inconsistency poses a real threat to misinformed individuals who may take these answers at face value, to save both time and money…

… Ultimately, we do not feel that chatbots like ChatGPT3 and BARD will take over from experienced lawyers any day soon. The responses are usually not detailed enough, and the level of incorrect answers is far too high.

That said, we do see the value of using the AI tools for initial research by members of the public. AI is already capable of competing with divorce lawyers for certain simple tasks, including creating legal documents at a basic level.”