Community Submissions · AutoMechInterp

Submission Contract

Required bundle files:

protocol.yaml
hypothesis.jsonl
evaluation_result.json
manifest.json

Optional: cross_model_results.json for transfer evidence.

Formal spec: Claim Bundle Specification v1.0.

Minimal End-to-End Commands

python -m automechinterp_evaluator.cli evaluate --bundle /path/to/bundle --output result.json
python -m automechinterp_evaluator.cli report --bundle /path/to/bundle --output stage_gate_report.md

python -m automechinterp_evaluator.cli submission-review \
  --bundle /path/to/bundle \
  --reruns 3 \
  --output-json submission_review.json \
  --output-md submission_review.md

How Review Output Changes Workflow

Tier	Decision	Next Step
`cross_model_confirmed`	Ready to share	Include as cross-model claim
`single_model_confirmed`	Ready to share	Include as single-model claim and collect transfer evidence
`causal_tested_unstable`	Hold	Run robustness and sensitivity follow-up
`suggestive`	Hold	Run confirmatory split and multiplicity-aware rerun
`rejected`	Do not publish	Address failed gates before resubmission

Bundle-Level Community Demonstration

python main/run_community_submission_demo.py

Outputs are written under main/output/community_submissions, including aggregate workflow decision-change rates and per-claim remediation actions.

Multi-Lab Expansion Roadmap

Contributor governance and externalization plan: