Final week, the UN working group on enterprise and human rights formally offered its thematic report on the procurement and deployment of synthetic intelligence programs by States and companies (A/HRC/59/53, 14 Could 2025 — notice there’s additionally an govt abstract infographic).
The report focuses on actions to be taken to facilitate alignment of AI procurement and deployment with the UN’s Guiding Rules on Enterprise and Human Rights and addresses organisations procuring quite than growing AI. The report approaches procurement in broad phrases by encompassing each private and non-private procurement, and by considering the place and duties of States, enterprise and stakeholders. The report incorporates a collection of findings and suggestions.
findings on the Regulatory panorama
One of many report’s key findings is that ‘States are more and more shifting from voluntary pointers to binding laws on AI and human rights, corresponding to by the European Union AI Act and Council of Europe AI Conference. Nevertheless, there are vital gaps when it comes to rights-respecting procurement and deployment of AI programs, together with an absence of a human rights-based method, no consensus on key definitions, inadequate integration of the angle of the World South, the availability of broad exceptions and restricted involvement of civil society. Additional, enforcement gaps and loopholes are weakening human rights protections in present laws on AI and human rights.’ This requires a more in-depth look.
The report highlights that ‘Globally, there are over 1,000 AI-related requirements and over 50 AI governance initiatives primarily based on ethics, duty or security rules’. Though unsurprising, I discover this fascinating and speaks to fragmentation and duplication of regulatory efforts that create a fancy panorama. Given the repeated recognition that AI challenges transcend borders and the requires worldwide collaboration (eg right here and right here), there’s clearly a niche nonetheless to be addressed.
In that regard, the report stresses that ‘The dearth of consensus on key ideas corresponding to “AI” and “ethics” is resulting in inconsistencies within the regulation of AI programs and is especially problematic given the transnational nature of AI’, and highlights UNESCO’s Suggestion on the Ethics of Synthetic Intelligence because the kind of doc that might be used as a blueprint to advertise coverage coherence throughout jurisdictions.
Though the report identifies a latest shift from voluntary pointers to legally binding guidelines for AI programs, such because the EU AI Act or the Council of Europe Framework Conference on AI, it additionally highlights that ‘there’s nonetheless uncertainty concerning how one can handle sure loopholes within the EU AI Act’ and that the Framework Conference creates comparable challenges in relation to the numerous exemptions it incorporates, and the best way it provides signatory States discretion to set its scope of utility. Though the report doesn’t take an specific place on this, I believe it takes a small step to conclude that legislative motion must be much more decisive if the problem of upholding human rights and elementary values in AI deployment is to be met.
One other key discovering of the report is that ‘States are largely procuring and deploying AI programs with out satisfactory safeguards, corresponding to conducting human rights impression assessments as a part of human rights due diligence (HRDD), resulting in human rights impacts throughout the general public sector, together with in relation to healthcare, social safety, monetary providers, taxation, and others.’ This outcomes from the restricted rising approaches to AI procurement.
Certainly, specializing in the regulation of AI public procurement, the report highlights a collection of approaches to growing legally binding normal necessities for AI procurement and deployment, corresponding to in Korea, Chile, California, Lithuania or Rwanda, in addition to efforts in different jurisdictions to sort out particular features of AI deployment. Nevertheless, the report additionally stresses that these regimes are likely to have exemptions in relation to essentially the most controversial and doubtlessly dangerous areas for AI deployment (corresponding to defence and intelligence), and that the sensible implementation of these regimes nonetheless hinges on the restricted growth of generally understood requirements and guardrails and, crucially, on public sector digital expertise.
On the latter, the report clearly places it that ‘At present, there’s an imbalance in information and experience between States and the personal sector round what AI is, the way it works and what outcomes it produces. There may be additionally little area and time for procurers to interact critically with the claims made by AI distributors or suppliers, together with as they relate to potential and precise human rights impacts.’ Once more, that is unsurprising, however this renewed name for funding in capacity-building ought to make it abundantly clear that with inadequate state capability there will be no efficient regulation of AI procurement or deployment throughout the general public sector (as a result of, in the end, as we’ve not too long ago argued procurement is the infrastructure on which this regulatory method rests).
The report then covers intimately enterprise duty in relation to AI procurement and deployment and covers problems with relevance even in contexts of light-touch self-regulation, corresponding to due diligence, contextual impression assessments, or stakeholder involvement. Equally, the report finds that ‘Companies are largerly procuring and deploying AI programs with out conducting HRDD, risking opposed human rights impacts corresponding to biased choice making, exploitative employee surveillance, or manipulation of shopper habits.’
The ultimate a part of the report covers entry to cures and, in one other of its key findings, stresses that ‘Courts are more and more recognizing the human rights-related considerations of AI procurement and deployment, highlighting the pressing want for transparency and public disclosure for private and non-private sector procurement and deployment of AI programs, and the truth that present treatment mechanisms lack assets and enforcement energy, leaving communities with out efficient recourse for AI-related human rights abuses. Stronger authorized frameworks, public reporting obligations, and impartial oversight our bodies are wanted to make sure transparency, accountability and redress.’
The report thus makes the first level that a lot elevated transparency on AI deployment is required, in order that present cures will be successfully utilized by these affected and anxious. It additionally highlights how present cures could also be inadequate and, particularly, new ‘mechanisms can even have to be arrange, creating built-in approaches that acknowledge the intersectional nature of AI-related harms and their disproportionate impression on at-risk teams. Efficient redress for AI-related harms requires each sturdy institutional frameworks and deep understanding of how expertise intersects with present patterns of human rights violation and abuses, each of that are at the moment lacking’ (this largely chimes with my view that we want a devoted authority to supervise public sector AI use, and that preventative approaches have to be explored given the dangers of mass harms arising from AI deployment).
suggestions
In an effort to handle the unsatisfactory state of affairs doc within the report, the working group formulates a log checklist of suggestions to States, companies and different actors. Within the govt abstract, the next are highlighted as key suggestions to States.
Set up sturdy authorized, regulatory and coverage frameworks on AI: Develop and implement AI laws following a human rights-based method which are aligned with worldwide human rights legislation, making certain transparency and accountability in AI procurement and deployment and authorized certainty for all.
Mandate HRDD: Require public disclosure, HRDD, and safeguards for AI programs procured and deployed by personal and public sector actors, together with AI programs utilized in high-risk sectors like legislation enforcement, migration administration, and social safety.
Prohibit Dangerous AI Programs: Ban AI applied sciences incompatible with human rights, like mass surveillance, distant real-time facial recognition, social scoring and predictive policing.
Guarantee Entry to Treatment: Strengthen judicial and non-judicial mechanisms to handle AI-related human rights abuses, shifting the burden of proof to companies and authorities, and making certain satisfactory assets.
Promote AI Governance Collaboration: Construct world cooperation to determine widespread AI requirements, fostering interoperability and making certain the illustration of World South views.
Nevertheless, it’s value mentioning different suggestions included within the for much longer checklist within the report, as a few of them are immediately related to the precise activity of AI procurement. In that regard, the report additionally recommends that, with regard to AI procurement and deployment, States:
Present particular steerage to public sector procurement actors on a human-rights primarily based method to the procurement of AI programs; together with particular limitations, steerage and safeguards for AI programs procured and deployed in high-risk sectors and areas corresponding to justice, legislation enforcement, migration, border management, social safety and monetary providers, and in conflict-affected areas;
Present capacity-building for all stakeholders to know the technical and human rights dimensions of AI, and guarantee accessible, explainable and comprehensible details about the procurement and deployment of AI programs, together with by mandating public registration of AI programs deployed by each private and non-private entities;
Guarantee impartial oversight of AI programs and require the availability of clear documentation on AI system capabilities, limitations and information provenance;
Promote significant stakeholder session and participation in decision-making processes round AI procurement and deployment;
These suggestions will resonate with the maim necessities (in precept) relevant beneath eg the EU AI Act, or proposals for finest follow AI procurement.
Remaining remark
The report helpfully highlights the present state of affairs within the regulation of AI procurement and deployment throughout the private and non-private sectors. The problems it raises are well-known and lots of of them contain advanced governance challenges, together with the necessity for ranges of public funding commensurate to the socio-technical challenges introduced by the digitalisation of the general public sector and key personal market providers.
The report additionally highlights that, within the absence of satisfactory regulatory interventions, States (and companies) are creating a major stack of AI deployments which are merely not assured for related dangers and, consequently, are creating an put in base of probably problematic AI embeddings throughout the general public sector and enterprise. If something, I believe this needs to be a name for a renewed emphasis on slowing down AI adoption to permit for the event of the required governance devices.
