Skip to content

5 questions to ask an AI recruiting vendor

Five questions for an AI recruiting vendor, with pointers on how to recognise a good answer before you sign the contract.

You are choosing an AI tool for recruiting, and every vendor tells you that its system is safe. An assurance cannot be checked, and after the contract is signed it is you who will explain to candidates, lawyers and the board how the system works. Below are five questions worth asking before you decide, and pointers on how to recognise a good answer.

1. How do you classify the system, and which documents will we get?

The EU Artificial Intelligence Act (AI Act), as a rule, classifies AI systems used in recruiting as high-risk systems. In that case the employer that uses the system has obligations too, not only the vendor. That is why the first question is about how the vendor itself describes its system and what you will get in your hands.

A good answer is a list of documents and deadlines. A warning sign is a bare assurance that everything is fine.

Make it easier for the vendor and name what you need. You can ask for the instructions for use, the training programme for the people who will supervise the system, a description of how the system uses the criteria, the way to suspend the system and the reporting channel, the retention period for the records of the system's operation and access to the history, and a template of the information for candidates. For each item, ask about four things: what works today, what is not there yet, by when it will be there, and in which document you can check it.

For example: we design and document the SmartyTalent virtual recruiter as a high-risk system. Before you decide, you receive from us the instructions for use, the competence pack and the training programme for supervisors, template information notices, a data processing risk analysis, the model test report, and the status of every safeguard with its implementation date. This article is not legal advice. It is a list of things you can ask any vendor for.

2. Who makes the decision about a candidate?

The answer “a person” is not enough. Ask for a list: what the system does on its own, and what a person decides. A warning sign is a general assurance without such a list.

A good list is specific and does not hide the exceptions. Ours looks like this. The virtual recruiter prepares everything for the decision. After the rating of an application, after the call and after the task, it only proposes a rejection and gives the reasons, and a person approves it. Only one situation closes an application without a person: an answer in the form that does not meet a must-have requirement agreed with the manager. Such a decision can also be reversed. When a candidate does not reply by the set deadline, for example to an invitation to a test, the virtual recruiter proposes closing the application and a person approves it.

So an honest list also includes what happens automatically. If a vendor claims that its system does nothing on its own, ask about knockout questions in the form and about candidates who do not reply to messages. Also ask who at your company will be able to change the system's decision, how they will be trained for it, and whether every stage can be switched to manual mode.

3. Where does candidate data go, and who processes it?

A CV, a recording of a call and the result of a task are the personal data of people who applied to you, not to the vendor. Behind the vendor there are usually further companies that also process this data.

A good answer includes a list of entities and countries and the basis for every transfer of data outside the European Economic Area. A warning sign: the list from the conversation differs from the list in the documents. If it does, ask which list is binding.

It is worth following up on three matters. First: whether the AI models learn from your candidates' data. Second: how long the data is kept and what happens to it after the contract ends. Third: what materials you will get for the data protection impact assessment and what the data processing agreement looks like. The legal basis, the information notice and the impact assessment remain on your side, so you will need these materials.

4. Does the system recognise emotions or analyse voice and facial expressions?

Emotion recognition in the workplace has been banned in the EU since February 2025. Here a good answer is short: no.

A “no” alone proves nothing yet. A warning sign is a “no” with no confirmation in the documentation. Ask the vendor to show you the place in the documents that describes what the system saves after a conversation with a candidate and what the rating of that conversation is based on. This question matters especially when the tool conducts phone or video calls.

5. How do you check that the rating does not discriminate?

A good answer describes the test, the result and what has not been measured yet. A warning sign is an assurance that the system rates everyone fairly, with no description of any test. A sentence like that cannot be checked.

We will show this with our own example. In our tests, on synthetic data, we give the model the same application with one trait changed, for example the first name or the year of birth, and we compare the ratings. We write down the pass criteria before the test starts. Such a test found two problems in our system, which we fixed. We do not yet have measures that would let us assess results for whole groups of candidates. That is what our next measurement will cover.

This answer has four elements worth looking for with any vendor: the method, criteria written down in advance, the result together with the problems found, and a limitation named plainly. A vendor that also shows its own mistakes and what it did about them tells you more than one that shows only successes.

Also ask what the system does when a manager asks for a candidate of a particular age or a particular gender. Make sure that the criteria entered into the system follow from the requirements of the position, and ask for a description of how the system uses them.

To check at your company

  • Do you have a list of documents with deadlines from the vendor, and not just an assurance?
  • Do you know what the system does on its own and what a person decides, and who at your company can change the system's decision?
  • Does the list of entities that process the data, as you heard it in the conversation, match the list in the documents?
  • Is the answer “no” on emotions, voice and facial expressions confirmed in the documentation?
  • Has the vendor described the test, the result and what it has not yet measured?

Film (in Polish, with English subtitles): “5 questions for an AI recruiting vendor”

Ask for the answers in writing, together with the documents. The better you know what the system prepares on its own, the more confidently you can say that at your company a person decides.

Book a 30-minute demo

Slawomir Kroczak

Slawomir Kroczak

The Co-Founder and CEO at SmartyTalent. With 19 years of experience in software development, I specialize in optimizing processes to enhance efficiency and drive growth.

Latest Articles

ReCops: who supervises the work of AI in recruiting

ReCops: who supervises the work of AI in recruiting

Who watches AI when it starts working in recruiting? The tasks of supervision, two models for running it and the questions worth asking a v...

The AI Act in recruiting: what HR should know before deployment

The AI Act in recruiting: what HR should know before deployment

Questions for an AI recruiting vendor and things to prepare at your company before deployment, for HR, lawyers and data protection officers...

The candidate is waiting for an answer. What silence in recruiting costs and how to shorten it

The candidate is waiting for an answer. What silence in recruiting costs and how to shorten it

No reply after an application costs the employer brand. Where silence in recruiting comes from and how to shorten it when the team is short...