Objective verification (verifiers)
This page is generated from docs/guide.en.md · Edit this page on GitLab
A review says what it thinks of the code. A verifier says what happens when you run it: it is your test commands, Mergerie prepares the repositories for them and reads their verdict. The two complement each other — a 9/10 score on an MR whose integration tests break means nothing any more once you know it.
What it changes in practice: a badge on each merge request (✓ verified, ✗ 2 tests broken, ⚠ base red, ⟳ stale), and above all the ability to verify together merge requests from different repositories that are only worth anything together — the front-end MR and the API one that only pass when combined.
What a verifier is
You give a list of commands (npm ci, then npm test), Mergerie runs it in the prepared repository and the verdict comes from the exit codes. Nothing to write, nothing to install.
A second family existed up to 1.2: an executable committing to a JSON contract. It has been removed — it asked you to write and maintain a program to get what three lines of commands give. A verifier of that family still on file stays visible in the settings, marked as such, and refuses to run: rewrite it as a list of commands, then delete it.
The order matters — npm ci before npm test — and is corrected in one click: each row carries its rank and two arrows to move it.
A line can run only once. In front of each command, a scope: in each repository (the default) or once — docker compose up -d after the repositories are checked out has no business running three times. A once line also says in which repository it runs (its worktree, or its in place folder), picked with search; left empty, the first repository of the verification. The list order is kept and the repositories run inside it: npm ci in each, then compose up once, then npm test in each, then compose down. A failed once line stops everything; it is skipped when a command already failed before it (the log says so); and if it points at a repository that is not part of the verification, that is an error with its name, not a guessed folder. The report and the log mark the line and the repository it ran in, the card shows a 1× badge, and where it runs is part of what you approve (see below) — the line travels with the shared verifier.
Such a verifier can cover several repositories. Be careful about what “cover” means: declaring a repository only announces that this verifier knows how to test it. At run time, the list only runs in the repositories actually targeted by the verification — launched on a single merge request, it only touches that one's repository, even if the verifier covers five. On a batch, it is replayed in each of them, one after the other, and the verdict is the AND — everything must pass. That is what you want when several projects are tested the same way: a single verifier instead of an identical one per repository. Inside a repository, the first failing command stops the following ones (they depend on it); from one repository to the next we carry on, because they are independent and knowing that two are broken beats stopping at the first.
When several repositories are tested, the failures are prefixed with the repository (grp/lib › cart › discount): without that, two projects each having a test of the same name would be indistinguishable — and the base/head comparison would confuse them.
What a verifier can say, and what it cannot
The list stops at the first command that fails: after a failed npm ci, the output of npm test is just noise. The report then shows the sequence — which command, which code, how long, and its output.
That leaves the question of test names, which is what makes the verdict causal. Mergerie looks for them in this order, never guessing:
- The JUnit report file, if you declare one (the JUnit report field, a path relative to the repository — so it is read in each of them when several are tested). It is the pivot format:
pytest --junitxml,jest-junit,phpunit --log-junit, Surefire,go-junit-report… The most reliable, and it does not suffer the log's truncation. - TAP in the output, recognised on its own — see below.
- Nothing. Then it is the command that is blamed: the badge shows
✗ failed: npm testinstead of announcing a number of tests nobody knows. And since the delta's key becomes the command, a command already red on the base still gives⚠ base red— the regression stays honest.
In that last case, the report additionally shows what is new compared to the base: the lines present at the head and absent from the base run. It costs nothing — both outputs already exist — and it often points straight at the regression, including behind an opaque make test.
Finally, if the exit code and the test report contradict each other (exit 0 with red tests, or the opposite), the verdict follows the exit code and the report reports the contradiction. It is almost always a genuine defect of the test command, and hiding it would do you a disservice.
TAP: nothing to declare
Many runners write TAP as soon as their output is not a terminal — which is always the case here, since Mergerie runs the commands through pipes. It is then recognised and parsed with no configuration at all: node --test, mocha (--reporter tap), vitest (--reporter=tap), pytest-tap, prove…
The traps are handled: sub-tests are not counted twice (only the failing leaves are kept, with their full name suite › test), a # TODO is an expected failure and does not count, a # SKIP is neither, a Bail out! gives an error and not “zero failures”, and the plan (1..43) acts as a control: if the output was truncated, the report says so instead of presenting a partial list as exhaustive.
The Parse TAP switch exists for the day an exotic output would trigger the detection by mistake.
npm: command not found
The commands' environment is minimal (PATH, HOME, LANG, MERGERIE_VERIFY=1, with no token at all), and the PATH is the server process's. Launched from a terminal where nvm is loaded, everything works. Launched by a service or a desktop launcher, npm will not be found — that is what the Environment variables field is for: one KEY=value per line, for instance a full PATH.
How the roles are shared
Mergerie does all the git. Your commands do no checkout and know no branch: they run in directories already positioned on the right commits and answer “do the tests pass”. That is what lets the same list serve in a throwaway worktree as well as in your own working directory.
Declarative coverage ≠ actual checkout. In Settings → Verifiers, declaring a repository only says “this verifier knows how to test that repository”. Only the repositories actually targeted by a verification are prepared. The other covered repositories configured in place are read read-only and appear as context in the report (with a ⚠ if they are off their default branch or modified): a green obtained thanks to a neighbour left on an old branch must not go unnoticed.
Two modes, repository by repository
- worktree (the default) — Mergerie creates a detached
git worktreeunderdata/worktrees/, removes it after the run, and collects the orphans at startup. Nothing you have on your machine is touched. - in place — the run happens in your working directory. Useful when the test environment cannot be recreated (a local database, containers already warm,
node_modulesinstalled). Three guard rails, in this order: explicit consent to tick, the repository's identity checked (the directory'soriginmust be the repository's), and a flat refusal if TRACKED files have been modified — never an automaticstash. Untracked files do not block: they are in no commit, the detached checkout does not touch them and the restore leaves them where they are — requiring the opposite forbade in place mode to any directory carrying a.env.localor an artefacts folder, that is to say to almost all of them. They are counted and noted in the run's log: they stay there during the tests and may weigh on the result. (If one of them bears the name of a file of the branch under test, git refuses the checkout by itself: the verification fails instead of overwriting your file.) During the run, your directory is on a detached commit: do not develop on it. It is put back on its original ref in every case, including on a timeout or a job crash; if the restore fails, the report carries a “Manual restore required” banner — that never drowns in a log.
The directory is picked rather than typed: if you have declared local directories (Settings → Repositories), a selector lists all their git projects — with search as you type, because a root commonly holds dozens — and fills in the absolute path. A hand-typed path remains possible for a directory outside any declared root, but it is a typo you only discover on the first run, and that costs the run.
The “Test the directory” button answers while the form is still in front of you: directory recognised, current branch, and the two possible reservations — tracked modifications (which would get the run refused) and untracked files (which do not block, but will be there during the tests).
node_modules, and why the base is sometimes red
node_modules strategy. A fresh worktree has no dependencies installed. Two answers: a symlink from a shared cache (fast, but assumes the lock has not changed), or an install in the worktree (slow, but faithful). The choice is yours — it lives in your commands. An ln -s "$CACHE/node_modules" "$dir/node_modules" does the job as long as you invalidate the cache when package-lock.json changes.
FAQ.
- Why is my base red? The base run replays the same suite on the target branches, before your changes. If it already fails, the verdict is
⚠ base redand nothing is blamed on your branch — which is the point. Unticking “Also run the base” removes that second run: the verdict still comes out, but it is no longer causal, and the report says so. - Is the base replayed every time? Yes, even when no commit has moved. A cache keyed on the set of SHAs did exist and did save time; it assumed the environment had not moved either, which nothing lets you check — and it was wrong in both directions: a red base fixed outside git (a service restarted, a migration applied) stayed stuck and blocked the MR on a stale “base red”; conversely, a base green that had become false made your branch carry the blame for a failure that did not come from it. A verification therefore costs two runs — that is the price of a verdict that does not lie. To pay for only one, untick “Also run the base”: the verdict is then marked non-causal.
- git does not work in my test container. In a worktree,
.gitis a pointer file to the main repository, not a folder. Mount the clone (data/clones/…) in the container too, or do not call git from the tests. - “Manual restore required”. The in place mode could not put your directory back on its branch: it stayed on a detached commit. The message says which directory and which ref to expect — a
git checkout <ref>is enough once whatever was blocking is cleared. - The verdict is
⟳ stale. It covers commits that are no longer the branch's. It is kept, not erased (it is dated): run the verification again.
Verify together, and fix
The Verify button is present on the merge requests to review as well as on those already reviewed (in the list and in the report panel): a review is an opinion, a verdict is a fact, and the second keeps all its value once the first has been given. A click opens a confirmation announcing what is going to run — the commands, the repository, the mode, the timeout — before launching anything. It appears even when a single verifier covers the repository: running commands on your machine deserves a screen, not a silent click.
A battery that starts by itself. A checkbox Run on every new merge request of the covered repositories on a verifier: as soon as a merge request appears at discovery, all the automatic verifiers covering its repository start, with no click. Only a new merge request triggers them — a known one is seen again at every sync, and re-running it every time would keep the battery running on everyone forever.
Re-running when the verdict goes stale. A second checkbox, Re-run when a verified merge request gets new commits: a green given on commits that are no longer the latest is worth nothing. It is SEPARATE from the first one because it is a different appetite — on a branch that moves ten times a day, that is ten batteries. Unticked, the badge simply says “stale” and you re-run it yourself.
⚠ Five verifications at most per discovery run, and that cap is a setting (Settings → Verifiers, just above the list — next to the checkbox that triggers them). On a Monday morning discovery can bring back fifteen merge requests; fifteen functional batteries saturate the machine for an hour and block the queue shared with reviews. Beyond that, the merge requests keep their Verify button and the server log says what did not start — a silent cap would read as “everything was verified”. 0 means “no limit”, a choice that should be made on purpose. Verifications of the same repository queue up instead of being refused: they will never run at the same time, but none is lost.
Publishing the verdict on the merge request
Two paths, and they do not blur into one.
By hand, after reading it. From the report, Publish as a comment opens the pre-filled body — exactly the one automatic publication would send — in an editable field. A confirmation names the merge request before sending, and once published the screen says so (date, recipients) instead of offering the button again as if nothing had happened: that is what stops the same verdict being posted twice on someone's merge request. If publishing fails, the text stays on screen — you never lose what you have just written.
By itself, if the Publish the verdict as a comment box is ticked on the verifier — as long as the base run is green. It is the base that gives the verdict its meaning:
| Base | Head | Published? |
|---|---|---|
| green | red | yes — “it passed before, this branch breaks it” |
| green | green | yes — “verified, and it holds”: on a merge request you are about to review, a written green beats a badge you have to go and find |
| red | — | no — it is not this branch's doing; writing it on ITS merge request would blame it for what someone else broke |
| absent | — | no — without a base run we do not KNOW whether it was already red, and publishing would assert what we have not checked |
The server log says why nothing went out: silence reads as “published”. Unticked by default: writing on other people's work is a decision. Publishing by hand stays available in every case — there, a human decides, with the text in front of them.
And the Jira ticket, if you ask for it. The forge comment addresses whoever reviews the code; the ticket is read by QA, the project manager, support — the people who ask “is it tested?” without ever opening the forge. A checkbox in Settings → Jira (off by default, like “Notify Jira”) makes the merge request's ticket receive a comment when a verification breaks. Same guards as the forge comment: only “the base passed, the branch breaks”, never on a green run nor on an already-red base, and once per ticket even when the batch holds five merge requests of the same one. Jira being unreachable does not call an acquired verdict into question: the failure is noted in the job log.
And when the merge request is merged. A second checkbox in Settings → Jira, unticked by default as well, comments the ticket as soon as Mergerie sees the merge request merged — through discovery or Refresh from the forge. The text is the team's template ({url} required, plus {iid}, {project}, {title}, {branch}, {target}, {key}) or, when empty, the shipped message in the interface language. One comment per ticket: the first merged merge request posts it, each following one of the same ticket rewrites it with the full list (one template entry per merge request, in merge order) — five merge requests of one ticket do not make five comments. If Jira no longer lets that comment be edited (deleted, or no longer yours), a new one takes over the list and is the one followed from then on. A merge request is announced once, however many paths see the same merge; nothing for a merge request closed without merging, nothing without a ticket key in the branch or the title, and nothing from a Period catch-up — ten days of tickets need not learn at once what the team already knows. When Jira refuses, the marker is released and the next discovery round retries.
A red verdict leaves a todo behind. The notification goes by and is forgotten; the badge assumes you reopen Reviews. Between the two, nothing carried “there is a broken test on !218” through to the next morning. A todo is therefore created per merge request (running the same verification three times updates the same row rather than stacking three), and a green run closes it — ticked, never deleted: what got fixed today is read back under the done ones.
What else the report says. When the test output carries durations (vitest's # time=, node's duration_ms, JUnit's time attribute — until now thrown away with the rest of the comment), the report folds the five slowest tests at the bottom: a slow test is almost always GREEN, which is exactly why nobody notices it. And a red test already green on this very code in another run is marked flaky: there is then nothing the branch broke to look for. It takes at least two runs on the same commits to say so — one proves nothing, and doubting a test that never lied would cost more than silence.
And “Investigate” from the red. A failed verification report carries an Investigate button next to Fix: the two do not address the same moment — a red test whose trace you do not understand is not fixed, it is searched for first. It opens the incident investigator with the broken tests and their log excerpts as the request. The same button sits on a Jenkins build console, at the exact place where you READ the trace, and only appears when the console actually holds one.
The comment template can be edited (it appears under the box). What each field produces:
| Field | What it produces |
|---|---|
{verdict} | the verdict line, with the verifier's name — **integ**: ✗ 2 test(s) broken by this branch. On a branch verification, “by this branch” disappears |
{tests} | the broken tests, one per line, with their message when the output gives one. Cut past twenty, and it says so. Empty when everything passes |
{commandes} | the commands that failed, with their exit code (prefixed with the repository when there are several). It answers what {tests} leaves open when the output names the tests: which ones broke, yes — but which command did. Empty when nothing failed |
{commits} | the commits actually tested: one per repository, repository · branch @ sha. That is what makes the verdict checkable |
{mentions} | the people declared on the verifier (People to ping when it breaks), only on a red verdict. Empty on a green one |
{verificateur} | the verifier's name alone — useful if you write your own verdict sentence |
{date} | the date the comment is published (17/08/2026), not the date of the run |
{heure} | the time it is published (14:12) |
Pinging someone. The People to ping when it breaks field takes handles (@amady @bruno) or a group (@my-team, which ages better than a list of people), written as is wherever you put {mentions}. The forge resolves the mention and sends the mail — Mergerie only writes it. Two things to know: it must be the handle, not the numeric id (GitLab does not resolve @42), and you will not be notified of your own mentions, since the comment is posted with your token. Nothing is mentioned on a green verdict: pinging someone to say all is well is the surest way to end up in a mail filter.
Everything else is written as is, and an unknown field stays visible rather than disappearing: a typo should show in the preview, not turn into a hole in the comment. An empty block leaves no extra blank line. Under the field, “See an example comment” shows YOUR template rendered on sample data — composed by the same engine as the real comment, otherwise the preview would end up lying. Left empty, the default template is used, and it then benefits from later improvements.
Verifying a branch — or a tag —, with no merge request
Back from holiday, several merge requests have been merged: the question is no longer “what does this branch break?” but “is develop still green?”. The Verify a branch button lives in the Git tab and on every verifier's card. One row per covered repository, each on its default branch, picked from a searchable selector — an active repository lines up hundreds of them. The last branch verified is remembered. Run answers at once: cloning or fetching each repository, reading the tag and resolving the commit happen in the job, line by line in the log — an unknown ref puts the verification in error with its name and repository, in the log and the report, instead of holding the dialog.
A tag as well as a branch. In front of the ref, a Branch / Tag selector: “does the 3.2.0 shipped on Friday still pass with today's services?” is the same question, asked about a ref that does not move. The verifier checks out the commit the tag points to in every ticked repository (not the annotated tag object, which has its own SHA), and the report, the brief and the comment announce the target as a tag. The kind is remembered per repository, with the last tag and the last branch each on their own side: going back to Branch finds the previous branch again. A red run on a tag offers no Fix: a tag does not move, there is no branch to put a correction on.
You tick the repositories you want, not the whole coverage. Every row carries a checkbox, ticked to start with: a verifier covering five repositories can therefore verify just one, without having to supply the other four branches — and a row whose default branch cannot be read (an unreachable repository) no longer blocks the others from starting. The selection is remembered per verifier: “only api-core” is a habit, not a whim of the day. The filter above the list hides rows without ever unticking them — what is ticked goes out, visible or not.
Two things change meaning, and the tool deduces both from the absence of a merge request:
- the causal double run switches off — on an integration branch, the branch IS the base; leaving it on would run the battery twice to compare
developwithdevelop; - attribution disappears: nothing is “broken by this branch”, what is red is red. The report and the comment word it differently.
There is no merge request card to carry the badge: the result lives in the verification history and in the morning brief, where the line reads “repository · branch”.
Seeing what ran, even when it is green. A See the verifiers' results button on the merge request opens the details of each verifier that ran on it — verdict, tested commits, broken tests, and the command by command breakdown with exit codes and output. It appears as soon as a result exists, not only in red: “it is green, but what exactly ran?” is a fair question, especially when you did not see the run start. A stale result (the merge request's SHA moved since) says so. Each red block keeps its own Fix (AI session) button — with several reports, a single button at the bottom would not say which one it fixes.
While it runs, you can see it. A verification takes minutes: the button turns into Verifying… with its spinner, disabled — so a second click cannot start the same one again — and the card carries the "in progress" marker (the pulsing bar), the same one a review uses. The state comes from the server, not from the page: it survives a tab change, a re-sort of the list and a reload, and it clears itself at the end, the list refreshing to show the verdict. ⚠ A review on the same merge request does mark the card, but does not spin the Verify button: a spinner pointing at the wrong command would be worth less than no spinner at all.
In Reviews, tick several merge requests: the action bar offers Verify together and Create a batch (a batch is named, persisted, and re-verifiable with one button from AI Dev). Two MRs from the same repository are refused — we would not know which code was tested.
On a ✗ verdict, the report offers “Fix (AI session)”: a single coding session covering all the repositories of the batch — not only the one where the test broke, because the cause of an integration failure is often elsewhere. The prompt carries the facts (broken tests, messages, excerpts, tested commits) and the working branches are the MRs' own: the push updates them in place. After the push, you run the verification again by hand — no automatic chain.
Two verifications at the same time?
It depends on what is running. A single-repository verification is contained in its directory: another can start in parallel, as long as it does not target the same repository — re-running the same thing would not give a second opinion, only a run that waits for the first to say the same thing.
A multi-repository verification, on the other hand, is an integration run: it brings up a complete environment, often containers on fixed ports and databases. That one blocks everybody, and gets blocked by everybody — two in parallel would return reds that teach nothing.
In both cases the refusal is immediate and says which of the two reasons applies.
Dry-run mode: it only concerns the AI agent. A verification stays real if it is configured. In demo mode, on the other hand, no command is run at all: the verdict is simulated.