What we learned building an AI governance analyst
Hathor Nodes
Blockchain Insights
Running a validator means reading every governance proposal, including the 40-page software upgrades and the parameter tweaks hidden in JSON. This year we built an AI assistant to help. Here is an honest account of how it went.
Our goal was not to automate voting. We vote ourselves, and we publish our reasoning. The goal was to cut the time spent getting from “new proposal” to “we understand what it does and who it affects”, so we could spend that time on the judgment calls.
Four versions, one lesson
We measured each version against a set of past proposals where we already knew the right answer: what changed, what it costs, and what the risks are. A summary counted as accurate only if an analyst would sign it unchanged.
Summary accuracy by version
Share of summaries an analyst approved without edits, past proposals.
- v1, prompt only. Fluent, confident and wrong about a quarter of the time, usually on the details that matter, like amounts and parameter values.
- v2, with the full proposal text. Much better, but it still could not tell what a parameter change meant without knowing the current value.
- v3, with on-chain parameters. Giving it tools to read the live chain state was the biggest single jump. “Raises the max validator set from 150 to 175” beats “increases the validator set”.
- v4, with analyst review. The assistant drafts, a person checks and edits. Nearly every published summary is now right, and the corrections feed back into our evaluation set.
Time saved
The time savings grew as the team learned to trust the drafts and focus their review on the risky sections.
Minutes per proposal
Average analyst time per proposal, manual vs AI-assisted, over 20 weeks.
First month, assisted
65 min
per proposal
Latest month, assisted
23 min
per proposal
What we would tell anyone building one
- Give the model the chain state, not just the text. Most errors were missing context, not bad reasoning.
- Build an evaluation set from past decisions before you start. Without it you are just judging vibes.
- Make the assistant quote its sources. Reviewers move much faster when they can click through.
- Keep the vote human. AI is a great reader; accountability belongs to the validator.
We now offer this as part of our governance intelligence work for DAOs and validators.