We gathered in Amsterdam to discuss how tech (regulation) excludes minority groups

Press release *

Amsterdam, 17/03/2025  

On 17 March, we convened the panel, Excluded by Design: Tech Regulation’s Racial Bias, at the platform and physical meeting place for creation and social innovation, Pakhuis de Zwijger in Amsterdam (NL). In our programme, part of Pakhuis’ Week Against Racism, we welcomed a rich mix of speakers, including a politician from the pan-European party Volt (Marieke Koekkoek (Volt Nederland)), researchers (Petra Molnar (Refugee Law Lab and Centre for Refugee Studies) and Dr. Sangeeta Mahapatra (GiGA)), and a representative from civil society (Aljosa Ajanovic Andelic, European Digital Rights (EDRi)). During the discussion, we covered various topics, such as whether technology and algorithms used to process data could ever yield decisions in a neutral manner, the effect of surveillance technologies at the border, and how existing regulations, for instance, in the EU, often contain loopholes which inequitably affect minoritised communities.

The discussion did not only identify the issues consistent with the use and regulation of many technologies but also discussed a way forward, including challenging narratives that excuse fundamental rights violations as necessary for defending national security and narratives that position innovation and the regulation of technology as opposed to one another.

We discussed how the persons who are most affected by technology are often ‘excluded by design,’ and how to move towards their inclusion by intention.

Is there such a thing as neutrality?

Opening our conversation, our speaker, Mr. Ajanovic spoke about the place of techno-solutionism in the EU agenda. He reflected on this process of ‘throwing technology’ at a host of problems in the hope that tech would somehow resolve them. Ms. Molnar also scrutinised these kinds of policies and challenged the audience to think about whether a technological solution was always appropriate.

Despite the promise of these new technologies and the efficiencies of using data to inform decisions and make predictions, our speakers agreed that data is not neutral, nor are the technologies which use it or their impact. Even the decision to collect the data of certain individuals is made based on who authorities and others in power consider a potential threat, a verdict that carries with it all kinds of social, political, and other baggage. When looking at the question of neutrality, then, we were urged to also consider how data and technologies fit into wider sociopolitical systems. Mr. Ajanovic and Ms. Koekkoek shared examples of these biases, citing cases out of Denmark and the Netherlands, where diverse family structures or even migration backgrounds were coded as risk factors for identifying potential fraud, with devastating results.

Dr. Mahapatra followed up by adding that technology is also about values, power structures, and the infrastructure that moulds it. She encouraged us all to exercise agency in the development of these technologies in a way that represents our positive values. In other words, technology that is not ‘value blind’ but ‘value positive.’

Ms. Molnar shared her comparative perspective analysing the use of technologies at Greek borders, the Polish-Bellarussian border, the US-Mexico border, and in the occupied Palestinian territories, expressing the consistent narrative that certain groups are dangerous and a potential threat in order to justify the collection of their data and the use that data to track and control said populations.

Learning from developments elsewhere

Sharing her experience from her work in South and Southeast Asia, Dr. Mahapatra urged the audience to be aware of the potential future in Europe and elsewhere should countries continue to adopt technologies which have negative consequences for human and fundamental rights (especially those of minoratised communities). Mr. Ajanovic and Dr. Mahapatra reflected on how systems that tie access to basic services to the use of digital technologies can negatively affect those who are less digitally literate or from certain minority communities (e.g. those with irregular residency status, the elderly, and those with different abilities).

Dr. Mahapatra used the specific example of Aadhaar in India, a unique identifier, which allows users to access welfare programmes, subsidies, healthcare, public services, buy a SIM card for their mobile phone, and use many more resources. Its use has been scrutinised by rights groups because certain groups are often excluded, including the homeless. In addition, correcting errors with Aadhaar is difficult, and the collection and storage of massive amounts of personal data (including biometric data) presents additional risks.

Dr. Mahapatra suggested that instead of seeing such technologies for the harm that they can do to marginalised groups, marginalised groups should be included from the beginning in the process of developing and implementing these technologies. After all, these technologies have in the past created benefits, like eliminating the ‘middle man’ and allowing groups to access resources directly. Their inclusion in developing these technologies could thus mean improved access to the services provided to them. As just one example, Dr. Mahapatra spoke about the way that farmers in India were part of the design process of some disaster management technologies. All speakers agreed that democratising the process of these technologies’ development was key to abating potential negative effects.

Our speaker, Mr. Ajanovic, expressed his concerns that the regulation of technologies in the EU has created two kinds of rights systems, one for EU passport-holders, whose rights are protected and one for non-EU citizens, whose rights aren’t always protected. As an example, he cited how underaged migrants may have their fingerprint data stored in Eurodac upon arrival in the EU from as young as 6 years old whereas those in the EU have their data protected by GDPR (General Data Protection Regulation), which disallows children under 16 to from giving consent for the processing of their data.

Challenging strong narratives

We spoke about how the European Commission has recently changed its tune when it came to the regulation of various technologies, buying into the narrative that more regulation will result in the stifling of creativity, a popular narrative in US political spaces. Our speakers didn’t see it that way and argued that we can challenge these narratives by creating new, factual narratives and thinking about the way that we can implement norms and values important to us, like the protection of human and fundamental rights in regulations.

Overall, the strong feeling remained that narratives have an underestimated power. They can lead politicians and others to make decisions about data privacy and the use of potentially harmful technologies that differ considerably from their typical stance. For instance, even privacy-oriented Europeans who share norms like human and fundamental rights may find themselves excusing the use of technologies that disrupt privacy rights and may violate human and fundamental rights if they believe it is for security or used against persons likely to be a ‘problem.’

Who should challenge these narratives? According to our speakers, everyone, although it’s easier for some than others. As Ms. Molnar pointed out, the establishment of narratives is rooted in existing power structures. This makes it difficult for those at a power disadvantage to challenge them. Ms. Koekkoek saw a place for politicians to challenge these narratives and call them out for being untrue. Doing so can be made easier by incorporating facts and data, as per Mr. Ajanovic. Dr. Mahapatra, while acknowledging the power of governments and tech industries in creating and centering these narratives, shared examples of how civil society has been able to challenge pervasive narratives. Her examples come from Asia, where organisations like the Internet Freedom Foundation in India and SafeNet in Indonesia managed to provide alternatives for popular narratives, such as those that pit innovation against regulation.

By the end of the event, it was clear that, despite the confluence of difficulties plaguing the digital rights space, civil society, academics, and politicians were all working in their own ways to enact positive change.

The full event can be watched here.

For more information or comments, contact:   

Foundation The London Story Advocacy Team 

advocacy@thelondonstory.org

* The thoughts expressed during this event are those of the participants and no responsibility of the organization can be asserted from this piece.

We spoke about calling out false narratives and creating new ones. Ms. Koekkoek expressed her belief that we don’t always have to think in the same frame drawn by Silicon Valley and shared that she imagines a kind of EU-based Silicon Valley where the standard is that values, efficiency, and competition are not contradictory to each other but supportive of each other.