The introduction of ChatGPT by OpenAI in November 2022 kicked off a frenzy of excitement around the future of AI as a technological driver of productivity, cost efficiency and innovation. Prior to this, EU lawmakers had already proposed a risk-based AI Act focussing on prohibiting certain AI systems and providing a regulatory framework for “high risk” AI systems.

Against this backdrop of innovation and regulation, businesses are eager to purchase and incorporate AI solutions into their products, services and workplace. Principles of good procurement apply to the procurement of AI solutions as they do to other tech solutions. The unique nature of AI should be specifically considered as part of any procurement process. In-house legal, compliance and procurement functions will also need to keep in mind compliance with the proposed AI Act, where their organisation is looking to roll out AI solutions.

With all of this in mind, we will offer in this article series some practical tips to organisations in relation to the procurement and deployment of AI solutions with each article focussing on a different stage of the procurement and deployment process, namely:

1. Initial due diligence and risk assessment
2. Contracting
3. Policies and governance

In advance of publishing each of these articles, we include below a brief flavour of their content.

Initial due diligence and risk assessment

Organisations will first need to identify the particular business needs and the commercial objectives underpinning the procurement process and be clear that an AI solution can meet these needs and objectives. The efficacy and proposed use of the AI solution will then need to be weighed against the identified risks arising from the AI solution and an assessment will be required as to how these risks can be appropriately mitigated. This will also feed into your organisation’s determination as to whether the AI solution is a high risk system under the AI Act, which will require particularly detailed scrutiny (e.g.  AI systems used to determine creditworthiness or eligibility for certain essential services such as internet connectivity).

The next step will be to conduct due diligence specific to the proposed provider of the AI solution. In this regard, we will identify a few key areas to focus on in our upcoming article (including alignment with your organisation’s ethical and sustainability values).

Contracting

In our second article we will cover some key areas to address in contracts for AI solutions including:

• IP ownership and licensing in the context of inputs, AI instructions, outputs and documentation
• Security
• Transparency and explainability
• Bias and discrimination
• Recordkeeping and logging
• Cooperation and training

Policies and documentation

Many organisations are developing AI policies to ensure that AI is not only good for business but also aligns with their organisation’s ethical and sustainable objectives. Such policies also help further employees’ understanding of the organisation’s use of AI and introduce some important guardrails in relation to that use. There are also certain key documentation requirements under the AI Act that organisations will need to address (e.g.  requirements around conformity assessments, risk assessments and incident logging and reporting for high risk AI solutions). Ensuring that your organisation has appropriately documented explanations for end customers around how third party AI solutions work will also be an important piece of the compliance puzzle in respect of both high risk and low risk AI systems.

Governance and oversight

With all the excitement around AI it is easy to forget that good governance processes are equally important to ensure that your organisation can realise the benefit of cutting-edge AI solutions while staying inside the regulatory lines. We saw in 2023, and continue to see, an acute focus by organisations across all sectors on developing mature and effective governance structures in relation to data and cyber-related laws. It is clear that governance will be a key element of the legal/regulatory approach towards AI. We will address this in more detail in our upcoming article on AI and governance.

We hope that this series of articles will help busy in-house legal, compliance and procurement professionals battling with the procurement of AI solutions in a rapidly evolving (and expanding!) regulatory landscape. Please do keep an eye out for the first instalment in this series on initial due diligence and risk assessment, which we will be publishing shortly.