Verification
Nine scenarios that take a freshly onboarded repository from “the gate is installed” to “I have watched it pass, watched it block, and watched it refuse to be waived”. Run them once, on a throwaway branch, before you trust the thing.
A merge gate that never runs looks exactly like a merge gate that finds nothing. Both are green. The difference only shows up on the pull request you needed it to stop, which is the worst possible moment to discover it.
This is the drill that tells the two apart. It takes about twenty minutes of mostly waiting, it runs entirely on a branch you delete afterwards, and it ends with you having seen every path the gate has.
Before you start
redline inithas run and its pull request is open or merged — see Onboard a repository.- You can push a branch and open a pull request on the repository.
- You know the repository's rung. A fresh onboarding is
observe, and the first four scenarios assume it.
Every push below also triggers whatever CI your repository already runs. On expensive runners — macOS, large self-hosted — that adds up, which is why the nine scenarios are arranged into three pushes and six edits to the pull request body or its labels. Body and label edits re-run the gate and nothing else, because the gate's caller listens for edited and labeled and a typical CI workflow does not.
Step 1 — the static check
Before provoking anything, ask Redline what it thinks. This reads the checkout and the host and contacts no pull request.
$ npx redlinegate verify ok gate-machinery .github/workflows/redline.yml publishes redline-gate / gate ok review-ownership every owner in CODEOWNERS resolves ok artifacts-current rendered artifacts match the standards version FAIL pending-admin an administrator must still enable: merge-policy
gate-machinery green means the workflow that would run the gate exists and publishes the right check name. It does not mean the gate has ever run. That is what the rest of this page is for.
Step 2 — the throwaway branch
One fixture file carries two of the nine scenarios: a TODO with no ticket, which core/untracked-todo reports at HIGH, and enough added lines to cross the 300-line ADR threshold.
mkdir -p .redline-gate-test
{
echo "// Throwaway fixture. Nothing here is built or shipped."
echo "// TODO: decide whether the scanner should follow symlinks"
for i in $(seq 1 400); do echo "// filler line $i"; done
} > .redline-gate-test/scenarios.js
git checkout -b redline/gate-scenarios
git add -A && git commit -m "test(redline): gate scenario fixtures"
git push -u origin redline/gate-scenariosOpen it as a draft pull request, and give it a body with no ## Launch readiness heading at all. Base it on whichever branch carries the gate — the onboarding branch if that has not merged yet, otherwise your default branch.
The nine scenarios
Each row is one edit. After each, wait for redline-gate / gate to finish and compare against expected. The results here are from a real run on a real repository, not a description of what the workflow ought to do.
1 — Nothing is filled in
The state you just pushed: no checklist heading, 400+ added lines, no labels, rung observe.
| Check | Expected | Why |
|---|---|---|
PR checklist | FAIL | No ## Launch readiness section. The message names the missing heading, not a box. |
ADR required… | FAIL | Over the 300-line threshold with no docs/adr/ link and no no-adr label. |
Deterministic policy | pass | The TODO is reported at HIGH. observe sets the floor at BLOCKER, so a HIGH is recorded and does not fail the job. |
gate | PASS | Two process checks are red and the aggregate is green. That is what observe means — measured, not yet enforced. |
If gate is red here, your repository is not at observe. Check rung: in .github/workflows/redline.yml.
2 — Apply the no-adr label
Label only, no push. Expected: ADR required… flips to pass; the checklist stays red.
3 — Add the heading, leave one box unticked
Paste the ## Launch readiness section from .github/pull_request_template.md into the body and untick exactly one line. Expected: PR checklist still fails — but on a different message, naming the unticked line rather than the missing heading. Two distinct failures, which is the point of checking both.
4 — Tick everything
Expected: all six jobs green, aggregate green. This is your baseline: from here, every red you see is something you caused.
5 — A BLOCKER, and the rung that acts on it
Two changes in one push. Append to the fixture:
// eslint-disable-next-line no-unused-vars
export function suppressed() {}and set rung: block-blocker in .github/workflows/redline.yml.
| Check | Expected | Why |
|---|---|---|
Deterministic policy | FAIL | A suppression with no ticket is core/type-checker-suppression, a BLOCKER, and the floor is now BLOCKER. |
gate | FAIL | At a blocking rung a failing process check stops the merge. |
This is the scenario worth doing even if you skip the rest. The same finding was reported and waved through in scenario 1; only the rung changed. If the aggregate does not go red here, your ladder is decorative.
6 — Exempt it, properly
Apply the redline-exempt label and add a filled-in block to the body:
## Redline exemption
- reason: deliberate BLOCKER planted on a throwaway branch to record what the gate does at a blocking rung
- until: 2026-11-30
- scope: policyExpected: Deterministic policy stays red — an exemption records who accepted a finding, it does not delete it — and gate goes green with a warning naming the scope: “Process checks failed but a valid exemption is recorded: policy. A reviewer is accepting this deliberately, with a reason and an expiry.” See Exemptions for the field rules.
7 — The label on its own
Delete the block, keep the label. Expected: the aggregate still passes, and says exactly why it should not have: “This exemption has no valid ## Redline exemption block. It is being accepted this time; once this repository moves to require, it will not be.”
That is exemption-enforcement: warn, the default. Repositories move to require one standards version after the block is introduced, so nobody's open pull request is failed by a rule that did not exist when they opened it.
8 — Stand a job down
Set stand-down: policy in the caller and push. Expected: Deterministic policy reports skipped, and the aggregate reads the skip as a pass.
Getting this wrong is how a repository is punished for narrowing the gate to match tooling it already runs. If your aggregate fails on a skip, do not stand anything down until it is fixed.
9 — Prove the security floor cannot be waived
Keep the redline-exempt label and the valid block from scenario 6. Then make the secret scan fail, by appending a step to the secrets job in .github/workflows/redline-gate.yml:
secrets:
# ...the shipped job, unchanged...
- uses: trufflesecurity/trufflehog@<pinned sha>
with:
extra_args: --results=verified
# SCENARIO 9 ONLY. Deleted with the branch.
- name: Force the secret scan to fail
run: exit 1Do not plant a credential to test this. The scanner runs with --results=verified, so a fake secret is ignored and a working one means you have committed a real, live credential to a branch — and to every fork and cache that ever sees it. Failing the job directly tests the thing actually in question: what the aggregate does with a hard job.
Expected: gate fails, with a valid exemption sitting right there on the pull request: “Redline gate failed on security checks (not label-exemptible, and not on the enforcement ladder): secrets”
Record it
| # | Condition | gate |
|---|---|---|
| 1 | Nothing filled in, rung observe | PASS |
| 2 | no-adr applied | PASS |
| 3 | Heading present, one box unticked | PASS |
| 4 | All ticked | PASS — all six green |
| 5 | BLOCKER + rung: block-blocker | FAIL |
| 6 | + valid exemption block | PASS, exemption recorded |
| 7 | Label with no block | PASS, warned |
| 8 | stand-down: policy | PASS, job skipped |
| 9 | Secret scan failing, exemption applied | FAIL |
Paste the table into the onboarding pull request. It is the evidence that the gate was watched rather than assumed, and it is what a later reviewer needs when the gate behaves in a way nobody expected.
Clean up
Close the pull request unmerged and delete the branch. Nothing in it was ever meant to land: the fixture, the promoted rung, the stand-down and the forced failure are all instruments. Leaving the branch alive leaves an edited gate one merge away from a repository that trusts it.
What scenarios 5, 8 and 9 also told you
Each of those three changed the gate's own behaviour — the rung, the stand-down list, the secret-scan job — from the branch the gate was judging, and the gate honoured every one. That is not a bug. It is what a vendored gate is: a pull_request workflow runs from the pull request's own head commit.
The rung and the stand-down list behave that way on an org-hosted gate too, because they are caller inputs and the caller lives in your repository. Scenario 9 is the one that does not: with --gate-source org the job definitions live in <org>/.github and a pull request here cannot reach them.
The mitigation for a vendored gate is the /.github/workflows/ entry in .github/CODEOWNERS, and it does nothing until the branch ruleset is applied — which is admin work a non-admin redline init leaves in pendingAdmin. If you ran the drill and that ruleset is not yet in place, scenario 9 is exactly what anyone with push access can do on purpose.
When to run it again
- After promoting a rung. Scenario 5 is the promotion's only real proof.
- After moving between
--gate-source localandorg. The required check name should not move; this is how you find out that it did. - After standing a job down or adding one — see adding a check of your own.
- After an admin applies the ruleset. Until then no check is required, so nothing you saw here was actually blocking a merge — it was reporting a failure that a human could click past.