A single rating next to a candidate's name says little. You cannot see what went into it, what tipped the balance or which part of it you might disagree with. If AI rates applications in your recruiting, it is worth knowing what the rating consists of and who can change it.
A recruiter reading an application checks several things at once: where the candidate has worked, what they can do, when they can start, how much they want to earn. Unless each of these is written down separately, what remains is a general impression and a short note.
With a handful of applications that is enough. With several hundred, simple questions become hard to answer later: why this person went forward and that one did not, and whether both were checked for the same things. A manager who sees only the final result can trust it or not. They cannot check it.
It is the same with AI, only on a larger scale. A model that returns one number or the single word “fits” leaves you asking on what basis. That is why it is worth breaking the rating down into parts that can be named, read and corrected.
The SmartyTalent virtual recruiter rates every application against the same sixteen criteria. The list is the same for every candidate and falls into three groups.
The first group says what the candidate has done so far. The second says how well prepared they are. The third says whether the job on the proposed terms is possible for them and matches what they are looking for. These are three different questions, and it is good when the answers do not get mixed up.
An example: a candidate with a lot of experience who can only start in six months is a different case from a candidate who is available at once but lacks a required licence. A single rating could make the two look equal. Separate criteria show the difference, and you know straight away what to discuss with the manager.
It is also worth noticing what is not on the list. The list of criteria has no name, gender or age on it.
The list of criteria is shared, but jobs differ. So in every project a criterion has a set importance: it is required, preferred or optional. A language may be required in one job and optional in another.
The first category needs the most attention. In any recruitment process a must-have requirement has the greatest effect on what happens to an application, so you need to know who set it. In our product a must-have requirement comes only from three sources: from the manager's words, from the law or from the ReCops team, the people who supervise the virtual recruiter's work. The AI's assumptions are marked.
The manager's words follow a set path. The manager writes an email or answers a few questions. From this comes a list of requirements, split into must-haves and nice-to-haves, which you approve before the job opens.
Why the caution? AI that helps write down requirements sometimes assumes things, for example that a given position needs a degree in the field. If such an assumption quietly becomes a must-have requirement, applications start being decided by something none of you said. The marking lets you see the assumption and make a conscious decision about it.
Splitting the rating into criteria also changes how the model works. The model rates one criterion at a time and adds a reason in plain language to each rating. The final rating is set by a fixed rule in the code, not by the model.
This separation has a practical effect. The model does not combine sixteen ratings into one. A program does that, following fixed rules. As a result a person sees each rating separately, can change it and can bring a rejected person back into the process.
For the manager this means a different conversation about a candidate. Instead of “the system gave a low rating”, they hear, for example: industry experience was rated low because the application does not mention this industry; the other criteria are fine. If the manager knows that the candidate does know the industry, that one rating can be changed. There is no need to argue with the whole result.
A rating is not yet a decision. After every rating the virtual recruiter presents a recommendation, and it only proposes a rejection. A person approves it.
That the list of criteria has no name, gender or age on it is only the start. Whether such traits affect the result by another route has to be checked with a test. The test is simple: the same CV and the same job, one trait changed, that is, the first name, gender, year of birth or citizenship, and a comparison of the ratings. The pass rules are written down before the first run of the model.
In our tests, on synthetic data, such comparisons found three problems (model test report, October 2026). The candidate's first name, email address and phone number were being passed into some of the ratings. This data was removed the same day. An answer from the form about shift work was not being passed into the rating. This was fixed before the test was repeated. The rating of the right to work depended on the name of the status. Now what counts is whether the person can legally take up this job.
The result, also in our tests, on synthetic data: in the three pairs measured, the variant received the same final rating as the base CV in every run, in 10 runs out of 10. A pair is a base CV and its variant with one trait changed.
This result has a limitation that has to be given together with it. There was one base CV for each trait. The result concerns the pairs measured, not all candidates. We do not yet have measures for whole groups of candidates.
We write about this because it is a good pattern for a question to any vendor. Not “does your AI rate fairly”, but: what did you test, what did the test find and what did you fix.
Film (in Polish, with English subtitles): “16 criteria: what the virtual recruiter checks”
Splitting a rating into criteria does not make the decision. It puts what is known about an application in order, so that it can be checked and corrected, and a person decides about the candidate.