governance-server
BIP300 / BIP301 mining infrastructure · Rust · github.com/LayerTwo-Labs/governance-server

governance-server

A JSON-RPC service that hands mining pools getblocktemplate templates built by the CUSF enforcer, accepts their submitblock calls, records every template and submission in SQLite, and shows it all on a dashboard with a JSON twin for every page.

~7,000 lines e2e regtest passing 24 unit tests passing open access, no login clippy -D warnings clean cargo fmt clean 3 open TODO(verify)
Overview

Architecture

One process, one shared template cache. Pools never talk to the enforcer directly, and the service never asks bitcoind for a template: if the enforcer is down or its template is too old, pools get an error instead of a template that would drop the sidechain commitments.

flowchart LR
  pools["Pools and anyone checking
no login"] ops["Operators
browser / scripts"] prom["Prometheus"] subgraph gs["governance-server"] rpc["rpc.rs + clients.rs
JSON-RPC 1.0 + 2.0
limits per client IP"] cache["TemplateCache
watch channel
last 64 templates"] trig["triggers.rs
BMM poll + ZMQ"] dec["decode.rs + policy.rs
coinbase decode
NoopAnnotator"] side["sidechains.rs
snapshot every 30 s"] net["network.rs
network numbers every 15 s"] store["store.rs
one writer, read-only reader"] dash["dashboard
HTML pages + /api JSON"] end enfrpc["Enforcer JSON-RPC
:8122"] enfgrpc["Enforcer gRPC
:50051"] btc["bitcoind
RPC + ZMQ"] db[("SQLite")] pools -->|"getblocktemplate
submitblock"| rpc ops --> dash prom -->|"/metrics"| dash rpc -->|"wait_for_change"| cache trig -->|"request_refresh"| cache cache <-->|"GBT 2.0 + long-poll"| enfrpc cache --> dec rpc -->|"submitblock"| enfrpc rpc -.->|"also_submit"| btc enfgrpc -->|"GetSeenBmmRequests"| trig btc -->|"ZMQ sequence"| trig enfgrpc --> side btc -->|"headers, mempool"| net net --> dash cache -->|"TemplateSeen"| store rpc -->|"Submission"| store side -->|"Snapshot"| store store --> db db --> dash cache --> dash

Solid arrows are always on. The dashed bitcoind submit only runs when bitcoind.also_submit = true. The dashboard, /api and /metrics can be moved to a separate admin_listen port so pools never see them.

Upstream clients

upstream/enforcer.rs speaks JSON-RPC 2.0 through jsonrpsee and always asks for {"rules":["segwit"],"capabilities":["coinbasetxn","longpoll"]}. upstream/grpc.rs is a tonic client generated from the enforcer's own protos. upstream/bitcoind.rs is used for chain info, headers and the optional parallel submit.

Event bus

Request handlers and background tasks never touch the database. They send TemplateSeen, Submission and SidechainSnapshot events on a channel; a single writer task batches them. A slow disk cannot stall a pool's request.

Shutdown

Ctrl-C cancels one token. Listeners stop accepting, parked long-polls are answered with the current template, the writer flushes, and the process exits. The e2e test checks the long-poll part.

Config

TOML file plus GOVSRV_* env overrides (__ for nesting). Unknown keys are errors. Startup refuses an invalid dashboard password hash, zero limits, and an upstream long-poll timeout of 30 s or less. governance-server check validates without starting.

Request path

A pool calls getblocktemplate

  1. Parse. rpc.rs accepts a JSON-RPC 1.0 or 2.0 envelope, or a batch, and answers in the same version. It is written directly on axum because jsonrpsee's server rejects 1.0 requests.
  2. Identify the client. There is no login. The client is the connecting IP address: one IPv4 address, or one IPv6 /64. Behind a proxy listed in trusted_proxies, it is the rightmost X-Forwarded-For entry the proxy added; the header is ignored from anyone else, so a client cannot choose its own identity.
  3. Proposal mode. mode: "proposal" is passed straight to the enforcer.
  4. Long-poll or not. With a longpollid, the request parks on the cache's watch channel until the template changes or 30 s pass. A client may hold 8 long-polls at once (4096 across everyone). Without a longpollid, the call counts against the client's rate limit (60/min, burst 10). Either limit answers error -32005. Batches over 16 requests are refused.
  5. Serve. The cached template is returned with the service's own longpollid (id:seq). If the cache has not been confirmed by the enforcer within max_template_age_s (65 s), the pool gets error -9 instead.
Why the triggers exist

Getting a new BMM bid into the template fast

The enforcer's own long-poll only wakes when a new block arrives (cusf-enforcer-mempool, lib/server.rs lines 909–921). A sidechain BMM bid that lands in the mempool mid-block would sit unseen until the 30 s long-poll timeout. Every second of that is a chance to mine a block without the bid, which loses the sidechain its block.

So the cache refreshes on four triggers: startup, the upstream long-poll returning, a change in the enforcer's GetSeenBmmRequests list (polled every second over gRPC), and a block-connect from bitcoind's ZMQ sequence feed. Any trigger fetches a fresh template. If its content is unchanged, the cache just renews its timestamp; if it changed, parked pool long-polls wake at once.

sequenceDiagram
  participant U as Sidechain user
  participant B as bitcoind
  participant E as Enforcer
  participant T as triggers.rs
  participant C as TemplateCache
  participant P as Pool (parked long-poll)
  U->>B: M8 bid tx enters mempool
  B->>E: mempool sync
  T->>E: GetSeenBmmRequests (every 1 s)
  E-->>T: new bid in list
  T->>C: request_refresh(Bmm)
  C->>E: getblocktemplate
  E-->>C: template with M7 accept in coinbase
  C->>C: decode, annotate, fold, publish
  C-->>P: long-poll wakes with new template
Request path

A pool calls submitblock

  1. Limit. 30 submissions per minute per client, burst 10, checked before the block is decoded.
  2. Match. The block's coinbase is compared with the last 64 templates. exact means the coinbase sidechain outputs and the tx list match; commitments means the sidechain outputs match but the pool changed the tx list. No match is still submitted and recorded.
  3. Submit. Sent to the enforcer, and in parallel to bitcoind when also_submit is on.
  4. Combine. The enforcer's answer goes back to the pool, with one exception: if bitcoind already accepted the block, the enforcer then reports duplicate, and the pool is told it was accepted. Both raw answers are stored.
  5. Record. A Submission event with the client address, template id, match kind, block hash, height, both results and latency.
Coinbase

What the decoder recognises

decode.rs classifies every coinbase output and every template transaction. Encodings were taken from the enforcer source, not the BIP text, and the tests run on coinbases captured from a real enforcer.

MessageMeaningWhereTag bytes
M1Propose a sidechaincoinbase OP_RETURND5E0C4AF
M2Ack a proposalcoinbase OP_RETURND6E1C5DF
M3Propose a withdrawal bundlecoinbase OP_RETURND45AA943
M4Bundle votes (subversions 00–03)coinbase OP_RETURND77D1776
M7BMM accept (h* commitment)coinbase OP_RETURND1617368
M8BMM request / bidmempool tx, output 000BF00
TreasurySidechain deposit UTXO (CTIP)tx outputOP_NOP5/8 01 <slot> 51

The decoded result feeds the TemplateAnnotator hook in policy.rs. Today it is NoopAnnotator, which accepts every template unchanged. The future policy feed will plug in here and can add a governance field or reject a template.

Persistence

SQLite: fold repeats, share transaction lists

The enforcer returns the same template many times. Storing each copy would be wasteful, so templates are folded: a repeat with the same content key only moves last_seen, curtime_last and poll_count. The key covers prevhash, version, bits, height, the full coinbase (so a new M7 or M2 is a new row) and the txid list.

templates

One row per distinct template: header fields, fees, weight, commitment summary (e.g. m2:1 m7:1), decoded coinbase outputs as JSON, and what triggered the first fetch. Kept 90 days.

txsets + template_txs

Content-addressed transaction lists. Two templates that differ only in the coinbase share one list. Kept 3 days.

submissions

Every submitblock: client address, matched template, match kind, block hash, enforcer and bitcoind results, latency.

sidechain_snapshots

Periodic sidechain state from gRPC, kept 30 days. Per-client usage is in memory only (at most 4096 clients), not stored.

One writer task flushes every 10 s and prunes every hour. The dashboard reads through a separate read-only connection pool.

Operators

Dashboard and metrics

Server-rendered askama pages sharing one stylesheet in the same style as this document, with a small script for auto-refresh and the light/dark switch. Every data page has a JSON twin under /api, so anything on screen can be scripted. This document is served at /architecture inside the dashboard layout.

PageJSON twinShows
//api/miningMining overview, public: network numbers, ways to mine, the live pool list, eCash facts, resources
/dashboard/api/overviewNetwork numbers (also at /api/network), current template, upstream health, recent clients with addresses shortened (IPv4 /24, IPv6 /48)
/templates/api/templatesFolded template history with commitment summaries
/templates/{id}/api/templates/{id}Decoded coinbase, transactions, and a diff against the previous template
/sidechains/api/sidechainsEach slot: status, ack progress, CTIP, withdrawal bundles and votes, recent BMM, live bids
/submissions/api/submissionsEvery block submitted, by shortened client address, with match kind and results
/catalog/api/catalogAPI catalog: every endpoint with parameters, the server's current limits, example curl calls and JSON-RPC error codes
/architecture—This page, compiled into the binary

Prometheus metrics on /metrics:

govsrv_gbt_calls_total            govsrv_template_changes_total
govsrv_rate_limited_total         govsrv_template_height
govsrv_long_poll_waiters          govsrv_template_tx_count
govsrv_submit_results_total       govsrv_upstream_errors_total
govsrv_auth_failures_total        govsrv_upstream_latency_seconds
Code map

Modules

PathLinesResponsibility
src/main.rs238run, check, hash-password; wires tasks and shutdown
src/config.rs287TOML + env loading and validation
src/upstream/660Enforcer JSON-RPC, enforcer gRPC (tonic), bitcoind clients
src/template/634Cache, refresh loop, triggers, dedup key, template view
src/rpc.rs447Downstream JSON-RPC, long-poll, rate limits, submit matching
src/decode.rs462Coinbase and transaction classification
src/policy.rs53TemplateAnnotator trait and NoopAnnotator
src/clients.rs384Client identity, rate limiters, long-poll slots, usage table
src/auth.rs117Argon2 basic auth for the optional dashboard login
src/network.rs547Network numbers from bitcoind; eCash 288-block retarget estimate
src/registry.rs150Public pool list from the pool registry
src/sidechains.rs149Sidechain snapshot task
src/store.rs576SQLite writer, reader, folding, pruning
src/dashboard/2,067Pages, JSON twins, formatting, API catalog, mining overview
src/metrics.rs179Prometheus registry
tests/regtest.rs691End-to-end regtest test
proto/—Enforcer .proto files, copied unmodified
docs/upstream.md—Enforcer interfaces and encodings, with source links

Main crates: axum 0.8, jsonrpsee 0.26 (client only), tonic 0.14, bitcoin 0.32, sqlx 0.8 (SQLite), askama 0.14, figment, governor, argon2, zeromq, prometheus.

What I learned building it

Findings

Upstream
Enforcer long-poll wakes on new blocks only.

New BMM bids would wait up to 30 s. Handled with the BMM poll and ZMQ triggers.

Upstream
Live BMM bids appeared to need -txindex on bitcoind.

Without it the enforcer did not pick up an M8 bid until restarted; with it, under 500 ms. Seen on Core v29.2 only, so marked TODO(verify). If it reproduces on a supported Core, it is an upstream bug that costs BMM blocks.

Upstream
curtime can be below mintime.

A block stamped with curtime is rejected as time-too-old. Pools must use max(curtime, mintime); documented in the README.

Upstream
The gRPC server is Connect RPC that also speaks plain gRPC. tonic was tested against it.
Upstream
A wallet-less enforcer refuses templates without coinbasetxn, and the enforcer's JSON-RPC needs content-type: application/json. Don't run it with --gbt-cache-lifetime-s: a cached template hides new bids.
Fixed
GOVSRV_CONFIG broke config loading.

The env prefix treated it as a config key. It is now ignored by the loader.

Fixed
duplicate for a good block with also_submit.

bitcoind stored the block first, then the enforcer said duplicate. The pool is now told it was accepted.

Against the brief

Changes from the brief

  • Downstream server on axum, not jsonrpsee. jsonrpsee's server rejects JSON-RPC 1.0, which the brief requires accepting. jsonrpsee is still the upstream client.
  • Decoder landed one commit early, in the template-cache commit, because the policy hook needs decoded coinbases.
  • Your choices applied: trigger on SeenBmm poll + ZMQ; tonic + prost for gRPC; coinbasevalue passed through as the enforcer returns it.
  • Not built, as asked: the policy feed. Only the no-op hook exists.
Verification

Testing

Unit tests (20)

Coinbase decoding against real captured fixtures (M1 proposal, M2 ack, M7 BMM accept), template folding, shared and changed tx lists, pruning, JSON-RPC envelopes, submit result combining, config parsing, per-client limits, IPv6 grouping and X-Forwarded-For handling.

End-to-end regtest

Starts bitcoind and the enforcer, runs the service, and mines every block through it with a small pool client that swaps the payout output the way a pool would. It proposes and activates a sidechain, places a BMM bid and checks it reaches a template, submits blocks, checks the batch limit and the shortened client addresses, and checks shutdown answers parked long-polls.

REGTEST_BITCOIND=/path/to/bitcoind \
REGTEST_ENFORCER=/path/to/bip300301_enforcer \
cargo test --test regtest -- --nocapture

Skipped unless both paths are set. scripts/regtest.sh up|down|cli starts the same stack by hand; scripts/capture_fixture.py captures new coinbase fixtures.

Needs you

Open items

Before relying on this

  • Run the e2e test on a supported Core (30.4, 31.2 or 32.1). It has only run against v29.2 with the enforcer's version check skipped; downloading the patched release was blocked by a permission check. This also settles the -txindex question.
  • Licence. The enforcer repo has no licence file, and its .proto files are copied into proto/.
  • Withdrawal-bundle ID byte order. Shown as the enforcer returns it, consistent with the coinbase votes; not yet compared with how explorers show it.
  • Set trusted_proxies to your reverse proxy's address in production. Otherwise every client behind the proxy shares one set of limits.
History

Commits

3e8b734docs: upstream enforcer interfaces (RPC, gRPC, encodings)
7389cdcupstream clients: enforcer JSON-RPC, enforcer gRPC (tonic), bitcoind
7d70a1dtemplate cache + downstream JSON-RPC
48ec412persistence: SQLite store with a single writer and a read-only reader
3dc5edcdecode: real enforcer coinbase fixtures and capture script
7ca1edbdashboard: overview, templates, template detail, submissions
dafb3b9sidechain view: slots, ack progress, ctip, bundles, recent BMM, live bids
4c4a3e1tests, metrics, docs: regtest end-to-end test and README
8980457docs: architecture overview page
82fbb73rpc: open access with per-client limits, no pool logins

The build stages and the changes after them. For anything newer, see git log.