Circuit discovery has a credibility problem. A method can point to a subgraph and call it "the circuit behind X," but faithfulness (does this subgraph actually explain the behavior) and actionability (can you actually do something with it) are two different questions, and most tools only answer the first one, if that.
CircuitKit is built to answer both. It ships 13 discovery algorithms across three maturity tiers (2 stable, 2 experimental, 9 research-tier), so you can pick a method that matches how much risk you're willing to take on. Every discovered circuit then runs through a 6-pillar faithfulness evaluation suite: causal patching, ablation, stability, robustness, comparison against baselines, and generalization testing. A circuit that survives all six is one you can actually trust to act on.
That's where CircuitKit stops looking like a research tool and starts looking like a deployment tool. Once a circuit is discovered and scored, five intervention modules let you use it:
- Structural pruning. Cut the model down to the identified circuit.
- Circuit-aware quantization. Mixed-precision, 3 or 4-bit, with protection tiers so the circuit that matters doesn't get degraded along with everything else.
- Knowledge editing. ROME and MEMIT-style edits, applied at the circuit's own components.
- Activation steering. Adjust behavior at inference time. No retraining.
- Circuit-restricted LoRA. Fine-tune only the circuit, not the whole model.
CircuitKit runs on production-scale models (Llama-3, Gemma, Qwen), handles GQA, RoPE, and chat-template specifics, and integrates with lm-evaluation-harness for benchmarking after an intervention. The GPT-2 quickstart runs on CPU in a few minutes, so you can see the whole discover-evaluate-intervene loop before touching a GPU.
The methodology behind CircuitKit is grounded in Lexsi's own research, published as "Faithfulness Is Not Actionability: An Extrinsic Audit of Circuit Discovery for Model Compression" (EMNLP Findings 2026).
Who it's for
- Interpretability researchers who need faithfulness scores, not just attribution heatmaps.
- ML engineers who want to compress or edit a model based on a mechanism they can verify, not a guess.
- Safety teams who need to locate and act on a specific behavior (a bias, a capability, a failure mode) inside a production model.
Links:
- arXiv: https://arxiv.org/abs/2607.19317
- Docs: https://lexsi-labs.github.io/CircuitKIT/
- License: Lexsi Labs Source Available License (LSAL) v1.1 — free for research, evaluation, education, and audit; commercial use requires a separate license
- Authors: Pratinav Seth, Hem Gosalia, Aditya Kasliwal, Vinay Kumar Sankarapu