If you’re an in-house lawyer, you’ve probably seen the headlines: AI will transform legal work. Legal tech startups are raising millions. Tools promise to review contracts in seconds, flag risks, and make lawyers redundant.

Meanwhile, your team’s still buried in NDAs, with limited time to test what actually works.

That’s why a new independent study by Legal Benchmarks is worth a closer look. It tested legal-specific AI tools against general-purpose ones – like ChatGPT and DeepSeek – on real in-house legal tasks.

The headline? Most legal AI tools didn’t outperform the generalists.

The surprising results

The researchers evaluated how well AI tools handled 18 common legal tasks – from reviewing clauses to advising on risk.

ChatGPT and DeepSeek got 12 out of 18 right (66.7%). That’s as accurate – or more – than most legal-specific tools tested.

So what does that mean for busy in-house teams?

• You don’t necessarily need expensive, complex platforms to start using AI effectively.
• General-purpose tools can already lighten the load on many everyday tasks.
• Legal-specific tools aren’t a silver bullet – they still require human oversight.

But wait – before you delegate your next SLA review to ChatGPT…

Accuracy isn’t the only concern.

Even if a general AI tool gives a correct answer, can you trace its reasoning? Can you ensure confidentiality? Can you audit it later?

The study found that human input remains critical – to frame questions properly, interpret ambiguous responses, and apply judgment to grey areas.

In other words, AI might be able to do the job – but it still needs supervision.

So which type of tool should you use?

Here’s a practical way to think about it.

General-purpose AI (like ChatGPT, Claude or DeepSeek):

• Great for quick research, summarising, rewriting or brainstorming.
• Works well for internal comms, FAQs, or first drafts of low-risk documents.
• Fast, flexible – but with variable accuracy and no built-in safeguards.

Legal-specific AI tools:

• Built for particular use cases (e.g. contract review, due diligence).
• Often offer better auditability, integrations, and compliance features.
• Can support legal review – but rarely replace it.

The takeaway: use the right tool for the right job, and always keep a lawyer’s brain in the loop.

What this means for in-house teams

This research is a reality check – and a relief. You don’t have to overhaul your tech stack tomorrow. You can start small, experiment wisely, and focus on AI that actually helps.

It’s also a reminder to stay sceptical. The legal tech space is noisy, and claims often outpace reality. Just because a tool is “for lawyers” doesn’t mean it’s better than what’s already in your toolkit.

And finally, it’s an invitation: to help shape what legal AI becomes, by using it, testing it, and sharing what works.

Final thought

AI isn’t replacing in-house lawyers anytime soon. But it is changing how we work – and it can make your day a bit less overloaded.

Start where you are. Test what’s useful. Keep what helps.

Because the future of legal AI isn’t just about the tools – it’s about how we use them.