Quality inspectors

Weak and cheap agents write quickly, but in different ways. The inspector is a strong model who reads the diff of someone else's work, gives a grade and explains what is wrong. Evaluations are accumulated in the context of “agent and model”: it is clear who to trust what and what adjustments to give in instructions.

The inspector does not rule anything: his role is review, and the policy prevents it from writing files and commands regardless of the mode (see Control).

How to launch

Button. The task header says “Quality”. Pulse takes the last agent job in this task, raises the inspector session on the strongest available model and gives it a diff. A chip with a rating appears next to the button; click opens a report with findings.

Automatically. Settings → Quality → “Automatic”. The check starts when the agent has completed the entire queue of turns and there are no other ongoing sessions in the task. The same work is not checked twice, the sessions of the inspectors themselves are not checked, and the task budget is taken into account.

In a swarm. Add the roles “Frontend Inspector” and “Backend Inspector” to the composition (ready composition is “With inspection”). Then the architect can give them tasks along with everyone else, and their reports are returned to him and influence the next wave of edits.

What does it evaluate?

Six axes, each from 0 to 5:

Axis What's watching
Correctness does the code do what was asked without any bugs or edge holes?
Readability names, structure, compliance with project style
Tests Are the changes covered by checks, have they been run?
Security secrets, input validation, dangerous calls
As part of the task are there any unnecessary edits and amateur activities?
Comments whether the non-obvious has been explained, whether the documents have been updated

In addition to assessments, the inspector returns:

  • verdict — edits can be accepted/needed/remade;
  • findings - “file:line - problem - how to fix” with criticality (critical, important, trifle);
  • adjustment — one phrase of what to demand from this agent next time.

Who checks

The default inspector is Claude Code on the strongest available model (Opus or Fable). The agent can be changed in the settings; The Pulse model selects itself from the agent's cache of options.

The role of the inspector is selected based on the composition of the changed files: the majority of client files are “Front-End Inspector”, server files are “Back-End Inspector”, mixed ones are “Quality Inspector”. You can assign a role manually.

Agent comparison

Settings → Quality → “Who writes better”: table for the key “agent + model” - average score, number of checks, averages for each axis, critical findings and last adjustment. The leader is highlighted. Below are the latest reports in their entirety, with a disclosure of the findings.

This is what a real test looks like: Claude Opus 5, as a frontend inspector, looked at the work of GigaChat-2-Max and gave 0.5 out of 5 with a verdict of “remake” - non-assemblable token module, non-existent props for common components, 543 files node_modules in the commit. Correction to the agent: “demand proof of assembly - output of tsc and tests in the report.”

Settings → Quality: when to check (automatically, inspector, inspector role, who to check) and threshold before merging in a swarm; taken 10/08/2026

What to consider

  • Checking costs money. The inspector is a strong model, and each inspection is a separate session. Turn on auto-check consciously; The budget of the task stops it in the same way as regular work (see. Control).
  • Inspector material — diff with the contents of new files, a list of files, the task text, what the agent himself said about the work, task notes and repository memory. The large diff is cut off, the inspector reads the files himself.
  • The assessment is subjective. It is useful for comparing agents to each other on your tasks, and not as an absolute metric. The more checks, the more meaningful the table.
  • The inspector may be wrong. His findings are a reason to look at the code, not a verdict. Read the report, not just the score.

Reviewer in reviews (agent on Pulse)

The same inspector, but not in the window, but in checks of the repository of Pulse itself: for each merge request, run the model with the reviewer’s profile, comments with comments. The axes, verdict and response format are exactly the same as described above, so the score from the checks and the score from the window are comparable.

There is one difference, and it determines everything else: in the window the inspector looks at the work of his agent, and in the checks - someone else's code, which may be hostile. The diff is written by the person whose work is being checked, and it may contain text addressed to the model: in a comment, in a README, in a test, in the description of a merge request. The model will read it along with the code.

What's been done about it

Risk What has been done What's left
The text under review commands the reviewer From the reviewer no tools: one request, diff inside it as data, back - JSON. The untrusted part is fenced off with a tag with a random salt, which the diff does not know and therefore cannot close Bad remark. Cannot be closed: this is a property of the model, not a setting
The instruction is slipped file name The file name is entirely chosen by the author of the thread, so the names are INSIDE the labels along with the diff: outside, where the system prompt promises the model “these are instructions from the project,” only the numbers that the reviewer himself counts remain. The name itself is checked against the same phrases as the diff lines (delimiters are read as spaces), and the attempt is called to the person A name written together will not be caught by the list of phrases - just like the new wording in the text
The task from the fork reaches the key The request pipeline from the fork goes in the fork itself, where there are no project variables; beyond this, the task itself refuses to work when the branch project is not equal to the run project —
A reviewer changes something in the repository Comment token - with role Reporter: He physically cannot push. Only parsed response fields go outside, the raw text of the model never —
The model's response becomes a GitLab command A comment line starting with a slash is a shortcut command (/merge, /approve). A slash at the beginning of a line is escaped, mentions @name wrapped in code —
The reviewer is pulling people's leg file name @ is a regular file name character, and the list “didn’t fit into the budget” is printed in the comment. Paths undergo the same cleaning as the model text: the mention is wrapped in code, backquotes are replaced (otherwise the name would come out of the code), a line break in the name does not break the line —

The last lines are not a trifle: without them, “a reviewer without the right to write to the repository” would receive this right through his own comment, and “untrusted data is fenced off” would cease to be true for exactly that part of the request that the author of the thread chooses most freely.

What cannot be closed, and this is the owner’s choice

.gitlab-ci.yml taken from the branch being checked. Anyone who can push a branch into a project can rewrite the reviewer's assignment and take away the model key. No code in the repository will interfere with this - the issue is solved by one setting in GitLab, and it has two outcomes:

Variable PULSE_REVIEW_KEY Merge Request Reviewer Key available
protected (default) doesn't work, says "no model key" only to protected branches and tags - that is, a night run on main
unprotected works on every request anyone who can write in the thread of this project

There is no third option: “a reviewer for each merge request” and “the key is not available to the merge request branch” are incompatible on GitLab. Until the choice is made, the first option applies - the reviewer remains silent and says why, rather than pretending to work.

What is mitigated regardless of choice: the code of the reviewer himself retrieves the task from the target branch, so the branch being checked does not execute its version of the script next to the key. And not in the task npm ci: installation would execute postinstall from the branch being checked, that is, someone else's code next to the key. The reviewer is satisfied with the standard Node library.

What the reviewer says about himself

Each comment ends with a "What the reviewer didn't see" section: files that didn't fit into the budget, passed as generated, and - always - the rest of the repository code, history, build and test results. Without this section, the absence of comments on the file would read as “the file is OK,” although the file might not be in the request at all.

Diff is cut to fit the budget only whole files: the hunk cut off in the middle looks like a complete piece of code, and the model draws confident conclusions from it about what it did not see. The cost of the solution: a file larger than the budget does not get reviewed at all - and this is said out loud. The code goes ahead of the documentation: on a live change, the entire budget was otherwise eaten up CHANGELOG.md, README.md and a snapshot of the code, but fifty source files were not looked at.

Comments found don't paint the purlin in any color: the reviewer is not the gate, the person decides. The color expresses something else - whether you looked at the code or not. GitLab doesn't have a neutral exit, so it's compiled from the exit code: 0 - looked, 78 - didn’t look (warning during the run), the rest - the reviewer himself broke down. A green “not tested” would read “checked, everything is fine” about untested code.

What does the owner need to have?

B Settings → CI/CD → Variables project:

Variable What is this
PULSE_REVIEW_API OpenAI-compatible endpoint address, https:// (by http the key is not sent)
PULSE_REVIEW_MODEL model name
PULSE_REVIEW_KEY key; masked, and the decision about protected - according to the table above
PULSE_REVIEW_TOKEN project token with the Reporter role - only to leave a comment
PULSE_REVIEW_ENABLED=0 switch if the reviewer is fed up

GigaChat is not supported by this script: it needs OAuth and the NCA root of the Ministry of Digital Development (see. docs/gigachat.md) is a separate vehicle, and there was no live passage along that road.

How is this verified?

It is easy to write more about a reviewer than has been verified, so it is said here what exactly has been proven by the files.

  • Clean part (src/shared/ciReview.test.ts): launch decision, diff budget, search for calls to the model, prompt assembly and response disinfection - checks on functions.
  • End-to-end verification (src/shared/ciReviewScript.test.ts): real scripts/ci/review.mjs is run by a child process on top of a temporary git repository, and instead of the model and GitLab on the loop there are fake servers that record every request. This is where the statement comes from, which is not given by any check on functions: in case of five different failures (fork, disabled by the owner, inappropriate source, no key, nowhere to write) the model server doesn't receive any requests, and the task exits with code 78. It is also checked that the request does not contain a field tools, that the file name with the “indication” remains inside the labels and at the same time is named to the person by the find, that the name with @ does not turn into a mention, and that the answer is not in the format and gives “not checked”, and not a green leaf with a comment.
  • What tests cannot prove: that the real model will obey the prompt frame. This is a property of the model and is listed in the risk table above as open.

The file with end-to-end verification is located in src/shared/, and not next to the script, for a prosaic reason: vitest is configured to src/**/*.test.ts, and the check next to the script would never run - that is, it would look like it was written without checking anything.

Where is the code

Layer File
Launch decision, diff budget, prompt, response disinfection src/shared/ciReview.ts (+ ciReview.test.ts)
I/O: git, one model request, comment scripts/ci/review.mjs (+ through src/shared/ciReviewScript.test.ts)
Quests reviewer and night reviewer .gitlab-ci.yml
Same for GitHub - never performed .github/workflows/review.yml

See what goes into the model without sending anything:

node --experimental-transform-types --import ./scripts/ts-loader.mjs \
  scripts/ci/review.mjs --base=main --head=HEAD --dry-run

The reviewer intentionally not included to the list of mandatory checks (src/shared/contributorCommits.ts): mandatory verification is a gate, and the model’s opinion cannot be a gate.

Where does this lead?

The accumulated assessments and adjustments are the material for the next step: correct the agent’s instructions (it is being versioned, see below). Memory and context), change the model of the role in the swarm, or stop giving this agent tasks that it consistently fails.