The use of facial recognition for surveillance, or algorithms that manipulate human behavior; will be banned under proposed European regulations on Artificial Intelligence AI.
The wide-ranging proposals, which were leaked ahead of their official publication, also promised tough new rules for what they deem high-risk AI.
That includes algorithms used by the police and in recruitment.
Experts said the rules were vague and contained loopholes.
The use of AI in the military is exempt, as are systems used by authorities to safeguard public security.
The suggested list of banned AI systems includes:
- those designed or used in a manner that manipulates human behavior; opinions or decisions …causing a person to behave, form an opinion or decide to their detriment
- AI systems used for indiscriminate surveillance applied in a generalized manner
- AI systems used for social scoring
- those that exploit information or predictions and a person or group of persons to target their vulnerabilities
European policy analyst Daniel Laufer tweeted that the definitions were very open to interpretation.
“How do we determine what is to somebody’s detriment? And who assesses this?” he wrote.
For AI deemed to be high risk, member states would have to apply for more oversight; including the need to appoint assessment bodies to test, certify and inspect these systems.
And any companies that develop prohibited services, or fail to supply correct information about them; could face fines of up to 4% of their global revenue, similar to fines for GDPR breaches.
High-Risk Examples of AI:
High-risk examples of AI include:
- systems that establish priority in the dispatching of emergency services
- systems determining access to or assigning people to educational institutes
- recruitment algorithms
- those that evaluate creditworthiness
- those for making individual risk assessments
- crime-predicting algorithms
Mr. Leufer added that the proposals should “be expanded to include all public sector AI systems; regardless of their assigned risk level”.
“This is because people typically do not have a choice about whether or not to interact with an AI system in the public sector.”
As well as requiring that new AI systems have human oversight; the EC is also proposing that high-risk AI systems have a so-called kill switch, which could either be a stop button or some other procedure to instantly turn the system off if needed.
“AI vendors will be extremely focussed on these proposals; as it will require a fundamental shift in how AI is designed,” said Herbert Swaniker, a lawyer at Clifford Chance.
Sloppy and Dangerous:
Meanwhile, Michael Veale, a lecturer in digital rights and regulation at University College London; highlighted a clause that will force organizations to disclose when they are using deepfakes; the particularly controversial use of AI to create fake humans or to manipulate images and videos of real people.
He also told the BBC that the legislation was primarily “aimed at vendors and consultants selling – often nonsense- AI technology to schools, hospitals, police, and employers”.
But he added that tech firms who used Artificial Intelligence “to manipulate users” may also have to change their practices.
With this legislation, the EC has had to walk a difficult tightrope between ensuring Artificial Intelligence is used for what it calls “a tool… with the ultimate aim of increasing human wellbeing”; and also ensuring it doesn’t stop EU countries from competing with the US and China over technological innovations.
And it acknowledged that AI already informed many aspects of our lives.
The European Centre for Not-for-Profit Law, which had contributed to the European Commission’s White Paper on AI; told the BBC that there was “lots of vagueness and loopholes” in the proposed legislation.
“The EU’s approach to binary-defining high versus low risk is sloppy at best and dangerous at worst; as it lacks context and nuances needed for the complex Artificial Intelligence ecosystem already existing today.
“First, the commission should consider risks of Artificial Intelligence systems within a rights-based framework – as risks they pose to human rights, rule of law, and democracy.
“Second, the commission should reject an oversimplified low-high risk structure and consider a tier-based approach on the levels of AI risk.”
The details could change again before the rules are officially unveiled next week. And it is unlikely to become law for several more years.
I am a freelance writer studying Mass Communication at the University of Karachi in the faculty of Media sciences. Also certified from Hubspot Academy as a Content Marketer.