An AI platform purpose-built for legal professionals outperformed ChatGPT and Google’s Gemini when tested on legal accuracy, scoring 30% higher than the mainstream generative AI models.
Legal AI research assistant LawY was tested on a broad range of typical legal questions across multiple areas of law, achieving 86% overall accuracy. In comparison, popular off-the-shelf AI tools Gemini 2.5 Pro and ChatGPT 4.1 achieved just 57% and 54% respectively.
The assessment was carried out following reports that 61% of lawyers in the UK are now using AI, up from 41% earlier this year. Only 6% say they have no plans to adopt AI, down from 16% six months ago.
LawY’s assessment highlights the importance of choosing specialist AI tools – benchmarked against trusted legal sources – for legal purposes.
Gareth Walker is CEO of LEAP, provider of legal practice management software and integration partner of LawY. He believes the challenge for law firms and legal professionals is not whether lawyers will use AI, but how and whether they can do so safely, effectively and strategically. He commented:
“The legal sector is at a turning point. While AI adoption is accelerating, culture and confidence are struggling to keep pace. Firms that lean on generic tools without clear frameworks risk serious missteps.”
LawY says its aim is to enable lawyers to make informed decisions about the AI research tools they choose, and stresses that confidence in legal tools must be evidence-based and benchmarked against objective standards. The company has committed to publishing further evaluations to ensure users have ongoing clarity on the performance of its AI tools.
The company has also shared advice on how firms can adopt AI safely. As well as choosing sector-specific tools with verifiable results, LawY says firms should embed a firm-wide AI strategy; invest in training that equips teams to critically assess AI outputs and use them responsibly; and prioritise client outcomes by ensuring the technology delivers tangible benefits, greater efficiency, lower costs and more accurate advice.
LawY’s evaluation and methodology are available to view at https://www.lawy.ai/accuracy#results
One Response
Full transparency please: how did it do against Gemini Deep Research?