Can new AI bot answer your climate questions? We tested it out

ChatNetZero aims to fact-check green plans - and outperform ChatGPT

Users of ChatNetZero can ask the AI bot to crunch through thousands of long-winded climate plans. Getty

A world alarmed by climate change gets its first look on Friday at an AI chat bot that aims to cut through green waffle.

The National was given early access to ChatNetZero to test the beta version of a “potent fact-checking and anti-greenwashing tool”.

It puts thousands of glossy climate plans under an AI’s sceptical eye – and is blunt when it finds them wanting.

Unveiled during New York Climate Week, part of the preparations for Cop28 in Dubai, it is billed as more up to date and transparent than existing AI bots such as ChatGPT.

There is also an algorithm meant to avoid “hallucinations”, when an AI bot confidently states something as fact that is totally wrong or impossible.

The beta model has limitations too. Unlike ChatGPT, you can’t reply with a follow-up question, and the underlying data is so net zero-specific that the bot can lack context or veer into the jargon it is trying to cut through.

The large language model is trained on data from the Net Zero Tracker project and was developed by AI company Arboretica and research lab Data-Driven EnviroLab. A strapline says it is “helping to demystify net zero”.

Angel Hsu, the research lab’s founder and a contributor to the Intergovernmental Panel on Climate Change, said the app was a tool for investors and citizens putting green plans “under an ever-brighter spotlight”.

“We – the public, governments, civil society and businesses – need credible information to decipher who is doing what and doing it in a robust and meaningful way,” she said.

“That’s where ChatNetZero comes in. By providing a gateway to the world’s largest living database on the integrity of net zero commitments, we hope to increase transparency and, in turn, boost ambition and real action.”

We tried asking it some questions.

Climate basics

First, an easy answer about the UN’s top science panel that shows the key features of ChatNetZero.

It is concise, less than half as long as ChatGPT’s 469-word response to the same question.

In every line there is a link to a citation. On ChatGPT you just have to hope it is telling the truth.

And it is up to date: the final version of the latest IPCC report came out in March but ChatNetZero can quote accurately from it.

Unlike ChatGPT, you don’t see ChatNetZero typing out its answer word by word. You do, though, see it “checking sources”, “generating answer” and finally “checking hallucination”.

Without follow-up questions you don't get a complete chatbot experience like with ChatGPT or Bard. ChatNetZero sticks to facts and references and there is none of the interplay that can make some AI bots feel eerily human. For an app designed to get to the point, that is no bad thing.

Fact-checking

UK Prime Minister Rishi Sunak had fact-checkers scrambling to their desks on Wednesday in a speech in which he postponed some of Britain’s green commitments.

Mr Sunak said Britain was ahead of its fellow G7 countries in its real and planned emissions cuts.

ChatNetZero backs him up in this answer and gives specific figures.

That said, it doesn’t recognise the term G7, which isn't climate specific, so you have to tell it exactly which countries you mean. It’s an example of how artificial “general intelligence” is still a thing of science fiction.

On another of Mr Sunak’s claims, ChatNetZero misread the question and repeated its previous answer. Developers say the beta release can struggle when more than five entities, in this case countries, are named.

Corporate plans

This is where ChatNetZero is in its element, sifting through climate plans from the world’s 2,000 biggest publicly listed companies by revenue. It also claims to cover every city of more than 500,000 people.

Although it ducks subjective questions like whether a company has a “good” climate plan, it is confident enough to give a clear-cut answer here. A Google search takes much longer to find this information.

There is advice to policymakers, too, for example on how cities and regional governments can set net zero targets even if they do not control all the utility grids flowing through their territory.

Hallucinations

Arboretica co-founder James Zhang says ChatNetZero users can have “confidence in the veracity of its responses”. A future release will connect the app to live information online.

We did find one mistake, though.

Was Benin decades ahead of its time? No, this answer is way off the mark. To be fair, though, it seems to arise from a mishap in the underlying data that ChatNetZero is merely regurgitating.

At least the citation lets us make that check ourselves.