Interoperability + Governance

Standards and Governance

Specification, compatibility vectors, semantic version policy, and contributor governance for a portable benchmark ecosystem.

Claim Bundle Spec v1.0

Canonical bundle contract for independent submissions and compatible evaluators.

Open claim bundle spec

Compatibility Vectors

python -m automechinterp_evaluator.cli reference-vectors

Checks canonical tier-classification behavior and helps third-party implementations validate compatibility.

Protocol Version Governance

Semantic versioning and no-silent-drift policy for gate and threshold changes.

Open migration policy

Holdout Stress Governance

Roadmap for hidden stress families and authorship separation to reduce benchmark-target overfitting.

Open holdout governance plan

Multi-Lab Contributor Program

Roadmap from single-team benchmark to independent multi-author evidence at release time.

Open multi-lab roadmap