Humanitarian AI: A Literature Survey
Introduction
Artificial Intelligence (AI) systems offer transformative potential for humanitarian work. AI is a digital technology whose definition has been hard to pin down, with no unanimously agreed upon international definition. Generally speaking, it can be broadly understood as “a collection of technologies that combine data, algorithms and computing power.” [1].
AI technologies consist of “software (and possibly also hardware) systems designed by humans that, given a complex goal, act in the physical or digital dimension by perceiving their environment through data acquisition, interpreting the collected structured or unstructured data, reasoning on the knowledge, or processing the information, derived from this data and deciding the best action(s) to take to achieve the given goal.” [2]
This pre-Generative AI definition is useful as its incorporation of two elements; knowledge-based systems (which use existing knowledge databases to solve problems) and machine learning systems (algorithms or systems which improve their actions, performance, or knowledge through learning) apply broadly across technological use in the humanitarian sector today.
A Generative AI definition can be added to this broader classification; it refers to techniques or models which learn representations from data, using these representations to probabilistically generate novel and unique artifacts which resemble but do not repeat the original data [3]. Simply put, Generative AI allows generation of new content - text, images, audio, videos - by giving prompts to the system to create specific outputs. [4] This literature review covers all three aspects of AI mentioned.
Increasing integration of AI in humanitarian contexts has attracted significant attention in recent years, producing human rights based approaches to AI, AI governance and data protection frameworks, case studies of deployments, challenges and risk assessments. For the purposes of this guidance document, the following selective, brief literature review maps the aforementioned work across three categories of Humanitarian approaches:
Use Cases and Approaches to AI
Approaches to AI Governance
Data Privacy and Protection
These categories map not only work done in relation to AI but also historicizes AI as a digital technology which, despite seeming new in its current accelerated moment, has been experimented with for at least 8-10 years in the field. In looking at AI applications, and its governance and privacy frameworks that have been developed, it is important to understand AI as a technology with a much longer history [5]. Choosing to survey Data Privacy and Protection literally specifically acknowledges AI as a topic that is fundamentally about data. It underscores the role of data as foundational to training machine learning models, enabling their pattern-recognition capabilities and how its quantity, quality and variety can influence performance and accuracy of AI systems. This data in a humanitarian context is doubly personal, and sensitive, requiring robust privacy safeguards and secure collection, storage and processing directives.
Generative AI has the potential to be a revolutionary step in AI, but it is crucial to frame this innovation within its broader technological and practical contexts, particularly its assessment, evaluation, and piloting in the humanitarian sector [6] A grounded perspective on AI, and Generative AI in particular, demands an understanding of what is novel and what builds upon existing foundations. For this reason, the literature review included in this study encompasses works from both before and after the emergence of Generative AI, ensuring a comprehensive and nuanced exploration of its evolution and impact.
Humanitarian Approaches to AI
In recent years, the aid sector has leveraged AI to improve crisis response, management and assessment, used chatbots for education, community engagement and information provision, and deployed health-related AI applications. However, most of these uses have been one off or stuck in pilots.
AI for Crisis Response and Management, Assessment
AI has been used to predict disasters, analyze damage assessments, optimize resource allocations, and engage in informational services through chatbots. For example, AI/ML models’ ability to analyze geospatial imagery has seen this technology deployed to support early-warning systems and efforts related to climate adaptation [7] [8].
AI/ML systems have been also been used to assess damage to infrastructure caused due to conflict [9], natural disasters and other crises. [10]
AI and Chatbots
The humanitarian sector has experimented with chatbots for different use-cases for a decade now. These chatbots were mostly rule-based, operating on predefined rules and decision trees to respond to specific commands within a narrow programmed scope. Since these do not contain an AI component, they have been often labelled as low-level in the literature. There are also mid-level chatbots which use keywords to identify user intent. [11] Advanced or AI powered chatbots use natural language processing (NLP) and machine learning (ML) to understand and respond to user queries in a conversational manner. These underlying techniques allow them to handle unstructured inputs and learn from interactions [12]
The LLM powered chatbot or agent represents a significant advancement in Advanced AI chatbot technology. Relying on large-scale pre-trained transformer models, these chatbots feature the ability to contextually understand and generate novel text in response to prompts or user queries. The training of models on multilingual datasets also enable them to communicate in multiple languages. [13]
Since at least 2016, chatbots have been piloted in the humanitarian sector. For example, in the Philippines, USAID worked with the Filipino Central Bank to develop a Facebook and SMS chatbot to address customer complaints and reduce employee workload [14]
Pre-LLM advancements led to improvement in chatbots which were then used for education (IBM Watson Tutor), chatbots for emergency communication (Rescue from Rescue.io) and chatbots for mental health support (Tess from X2AI) [15]
Seeing the potential of chatbots, humanitarian organizations have also used AI to expand their use of chatbots to engage with communities and ask questions about their needs. [16] [17]
Humanitarian organizations have utilized chatbots to communicate with vulnerable and underserved communities. For example, Singapore Red Cross (SRC) partnered with KRDS Singapore and MullenLowe Singapore to release Ella, a healthcare bot for seniors [18], UNICEF has used U-report, a social messaging tool connecting millions of users, which also contains a mental health chatbot [19], Refugee Text, InfoAid and Refunite all use their platforms to assist refugees in unification, obtaining information on services in a new country and/or assisting in legal processes [20]
Chatbots are also being used to provide information to affected communities about their rights [21], location and availability of services [22], design of education plans [23], and seeking recourse [24].
They have also been used to provide educational information, e.g. Farm.ink provided information on local prices and buyers to African farmers through the African Farmers Club chatbot [25], FarmChat was created for Indian potato farmers to help address questions about plant protection, pests, weather and other concerns [26], Ask Marlo, a LINE chatbot created by UNAIDS Indonesia to help educate youth about HIV and AIDS [27]. Girl Effect developed two chatbots, Big Sis and Springbot to educate girls on sexual and reproductive health [28]
LLMs chatbots are already being piloted in different humanitarian contexts; predominantly in community engagement, provision of critical information to affected and vulnerable populations and monitoring and evaluation [29][30][31]
Finally, humanitarian organizations have increasingly used chatbots and conversational agents to gather evaluative data and feedback from the communities they serve. This is especially useful in situations where standard face-to-face surveys are either infeasible or cannot be easily performed.
AI for Health and Development
AI applications in healthcare, such as disease prediction and diagnostics, have been explored in low-resource settings. The technology has shown potential in predicting need and delivering higher-impact responses that are targeted and/or delivered more quickly [32] [33] . This shows promise in risk reduction and anticipatory action in public health [34] and epidemiological monitoring [35].
As Spencer [36] has documented, AI models and applications can predict location and impact of floods [37], if, where and how an epidemic of pandemic will spread [38] [39]; how viral outbreaks might mutate [40]; and levels of morbidity and mortality linked to these outbreaks [41] [42].
AI systems have also been used to improve diagnostics in low-resource and/or remote settings by facilitating screening for cervical cancer, tuberculosis, malaria and antimicrobial resistance [43] [44] [45]
Some organizations have drawn on AI/ML to design vaccination campaigns and improve dissemination by combining population estimates with predictions in viral outbreaks and mutations to improve the delivery of vaccines [46][47] [48].
Challenges and Risks
The literature on applications of AI in a humanitarian context also document associated challenges, risks and trade-offs. These have variously included issues related to trust, transparency, data privacy, algorithmic bias, and the digital divide, which can exacerbate inequalities. Some of the more prominent risks and challenges are summarized below:
Accuracy and Reliability
AI contains inherent risks to reliability and accuracy in humanitarian AI systems[49]. These include:
Unpredictability: Machine Learning (ML) and Deep Learning (DL) algorithms can learn and evolve in unpredictable ways. This may occur as a consequence of pattern identification and development of unforeseen conclusions based on them
Data Quality: The concept of Garbage In - Garbage Out (GIGO) is well understood in Computer Science. [50] It refers to the idea that poor input data will lead to poor outcomes. This may occur due to models trained on incomplete, unbiased or incomplete data
Generalization: One of the key risks posed by AI systems as a whole is their inability to generalize outside of their training data. This problem is exacerbated with degrees of novelty
Compounding Effects: AI systems can amplify and compound errors which can cascade through decision chains. Identification and trouble-shooting such errors is made harder by such systems’ opacity and black-boxed nature.
While the above mentioned issues affect all AI/ML systems, Generative AI systems add an additional complication. Generative AI systems such as Large Language Models (LLMs) regularly output plausible yet factually incorrect information and state these outputs with high confidence. This tendency of Generative AI is referred to as “hallucinations” [51]. It is simultaneously an open problem as well as an inescapable characteristic, given how LLMs are trained and created.[52] Such hallucinations give rise to inaccuracies, lack of reliability and misinformation. Additionally AI/ML models can degrade in performance (referred to as drift [53]) over time due to changes in parameters, weights or future training runs. They need substantial upkeep to maintain after pilot stages.
In a humanitarian context, even minor inaccuracies in information can have catastrophic outcomes for clients and users.
Data and Privacy Concerns
Use of Generative AI raises significant data and privacy considerations along a few different vectors. First , private AI companies confirm collection of all data that is sent by their LLMs, including conversations, personal information and other forms of sensitive information.[54] [55][56] There are no guarantees that this data will not be shared with third-parties and vendors for legal or business purposes.
This lack of privacy extends to use of proprietary algorithms that are engaged in internal processing and working of the agent. For example, the Signpost AI agent has an internal software tool that rechecks LLM output against Constitution AI rules to maintain ethical compliance, and is itself an OpenAI bot whose functionality depends on sending additional information back to the company.
There are no guarantees that such centralization of data may not be subjected to present and future use in training, fine-tuning, model evaluation during the AI development life cycle. Inadequate data policies for example have unintentionally contributed to humanitarian surveillance [57]. There is also a fear that AI systems’ tendencies towards centralization and extraction give these technologies the quality of anti-localization [58], leaving out local, national groups and communities.
There is also a hypothetical risk that a threshold volume of prompts sent to LLMs might be used in future training to potentially produce a competing humanitarian AI product.
Despite reassurances of anonymization and de-identification of sensitive data by third party systems and private platforms, research shows that similar methods can be used to do harm by ethnically, racially or demographically profile certain individuals and communities [59]
User Trust
There is evidence which shows user preference favoring humans over non-human chatbot interactions. For example, a report on humanitarian organizations’ past use of pre-LLM chatbots highlight that people “express a desire for personalized interaction” [60] There are also studies on AI chatbot-human interactions showing that people surveyed preferred interacting with human beings [61]. Poorly deployed chatbots have the danger of increasing user frustration, while decreasing trust in the institutions deploying them [62] and have generally been criticized for removing the human touch by replacing front-line workers.
User trust in a humanitarian organization is further tested in the environment of an AI-human interaction. Chatbot interfaces are associated with anthropomorphism which can manipulate users into trusting AI agents as all-knowing and which produce factual information. [63]
The role of informed consent and expectation management is fundamental to ensuring user trust. It is essential that users are explicitly informed that they are interacting with AI agents or bots. It is also important to inform users about the capabilities and limitations of chatbots. [64]
Biased, Harmful and Discriminatory AI
LLM output is intrinsically connected to the underlying quality of its training data and algorithms. Internet content makes up the bulk of this underlying data that Large Language Models have been trained on.[65] Such data is predominantly in English, and is culturally and epistemologically Western. English makes up the largest language (being at ~50% or higher) of text databases of crawled internet data used for LLM training. [66][67][68] As a result LLMs by default, assume anglophonic and Western contexts.
Due to its training data’s provenance, connected to the internet, LLMs are inherently trained on biased, stereotypical, misogynistic, discriminatory and harmful data. This also flattens descriptions of people from other parts of the world and does not represent diversity, complexity or heterogeneity. [69]
Given LLM optimizations for efficiency, performance and cost-savings, these models run the risk of exacerbating above-mentioned tendencies and do not have any no internal mechanism to foreground marginalized populations.
The Black Box Problem
Generative AI suffers from the Black Box problem; how Generative AI tools process and generate answers is not disclosed. Neither is the provenance or ownership of data that it uses to generate outputs. Only the inputs and the outputs are visible and everything inside the technology is a mystery or a black box.
There is general research consensus on how LLMs’ neural networks are trained on billions of words of language which are then used to “predict the next word.” [70] Why they are black boxes is because of the sheer complexity and scale of billions of parameters which make them difficult to pinpoint the machine’s decision-making process. They also often exhibit “emergent behaviors”, or capabilities that were not explicitly programmed.[71]
Explainable AI efforts (XAI) attempt to open this box to make such processes comprehensible for humans but such efforts have only had limited success thus far. [72]
This opacity poses the problem of trust in GenAI systems which can cascade down to second order unknown risks, open to legal, regulatory, accountability and responsibility challenges.
Cybersecurity
Generative AI is the new frontier of cyber-security. There is research that existing security and data infrastructures around Generative AI architectures are inadequate to hold back new threats.[73] In the agent space, such threats include a mixture of the old and the new.
Data Poisoning: the injection of harmful/biased information into the knowledge based used for retrieval
Prompt Injection Attacks: Input prompts which could manipulate agent into bypassing security/protection measures or reveal sensitive information
Information Leakage: agent might inadvertently reveal sensitive or confidential information from its knowledge base
Adversarial attacks: inputs designed to mislead the agent to potentially malfunction or produce harmful outputs
API Vulnerabilities: A agent accessible via API will be vulnerable to injection attacks or authentication bypasses
Denial of Service (DoS) Attacks: a technique through which a system is overwhelmed with service requests causing it to become unavailable
Data Breaches: User interactions with the agent could be intercepted or stored insecurely
Research indicates that existing cybersecurity infrastructures might not be able to cope with AI attacks powered by accelerating improvements in AI [74].
While Generative AI models such as transformers and LLMs can enhance security systems through attack simulations, vulnerability and anomaly detections, it also increases attack surface areas creating new avenues of attack making the whole AI supply chain including data and algorithms more vulnerable. [75]
Assessment of Public-Private Partnership Opportunities
Large technology companies and private actors are predominantly responsible for developing and deploying AI systems. Such systems are used in the humanitarian sector through either third party contracts or public-private partnerships.
Such partnerships raise the possibility that private or corporate interests might take precedence over the public interest; technology ends up searching for a case-use, rather than a case-use looking for a solution. Often in such situations, a profit-making interest may provide strong incentives to push for high-tech solutions where a low-tech or no-tech solution might have been more context-appropriate and cost-effective. [76]
Beyond market-driven incentives, research has also found that other factors might be at play driving humanitarian tech partnerships; these are often a mix of ethical values, corporate social responsibility, staff satisfaction, visibility, budget governance and fiscal incentives [77]. This broad set of motivations may lead to value misalignment between partners resulting in “aidwashing” [78], extractivism [79] or experimentation [80][81] on very vulnerable populations.
AI Governance
AI governance in humanitarian contexts focuses on creating frameworks that ensure alignment with principled humanitarian action by fostering ethical, inclusive, and accountable use of AI technologies.
AI Governance requires a combination of principles, policies, understanding of current standards and laws, and industry best practices when it comes to the design, development and use of AI tools.
AI Governance Frameworks: A Humanitarian Approach
AI governance represents a structured approach to addressing a litany of issues raised by proliferating AI including safety and impact concerns on individuals and societies, confusion over how the technology works, algorithmic and dataset biases, privacy rights violations and automation and dissemination of misinformation.
It constitutes an evolving integration of principles, legal frameworks, policies, procedural guidelines, standards, regulatory frameworks, industry best practices, and other methodologies systematically applied throughout the lifecycle of AI systems, including their design, development, deployment, and operational utilization.
Given AI’s continent-spanning and technological complexity and its potential for transformation, significant efforts have been taken to constitute AI governance. IAPP’s report on AI Governance in Practice offers us a look at the most prominent and consequential AI Governance Efforts taken place globally to date. These are divided into the categories of Principles, Laws and Regulations, AI Frameworks, Declarations and Voluntary Commitments, and Standards Efforts [82]:
Principles
OECD AI Principles
Laws and Regulations
EU Product Liability Directive, proposed
EU General Data Protection Regulation
Canada – AI and Data Act, proposed
U.S. AI Executive Order 14110
Sectoral U.S. legislation for employment, housing and consumer finance
U.S. state laws, such as Colorado AI Act, Senate Bill 24-205
China's Interim Measures for the Management of Generative AI Services
The United Arab Emirates Amendment to Regulation 10 to include new rules on
Processing Personal Data through Autonomous and Semi-autonomous Systems
Digital India Act
AI Frameworks
OECD Framework for the classification of AI Systems
NIST AI RMF
NIST Special Publication 1270: Towards a Standard for Identifying and Managing Bias in AI
Singapore AI Verify
The Council of Europe's Human Rights, Democracy, and the Rule of Law Assurance
Framework for AI systems
Declarations and Voluntary Commitments
Bletchley Declaration
The Biden-Harris Administration's voluntary commitments from leading AI companies
Canada's guide on the use of generative AI
Standards Efforts
ISO/IEC JTC 1 SC 42
The Institute of Electrical and Electronics Engineers Standards Association P7000
The European Committee for Electrotechnical Standardization AI standards for EU AI Act
The British Standards Institution and Alan Turing Institute AI Standards Hub
This is a complex landscape of ever-changing draft AI laws, regulations, standards and frameworks. One key theme that emerges from these documents is the need to conduct a thorough assessment of an organization’s specific AI risk profile and implement a risk-based strategy which would allow them to establish a resilient and adaptable AI governance framework capable of operating effectively across multiple jurisdictions. [83]
Governance and accountability in the humanitarian sector situates AI Governance firmly with a human-centered approach. AI governance in the humanitarian sector ensures AI is used responsibly and ethically. It involves setting rules and guidelines to align AI with laws, humanitarian principles, and ethical standards. This includes assessing risks, ensuring transparency, protecting data, and providing alternatives for those who can’t or won’t use AI. It also means training teams, involving legal experts, and addressing environmental and local impacts to ensure AI supports, rather than harms, humanitarian goals.
There have also been specific humanitarian efforts at drafting sector specific AI governance:
| Topic | Documents | Key Themes |
|---|---|---|
| Ethics and Principles |
Unite Paper - Ethical AI at the UN Ethics of Artificial Intelligence | UNESCO Artificial Intelligence: examples of ethical dilemmas | UNESCO Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-Based Approaches to Principles for AI Artificial Intelligence Principles for Vulnerable Populations in Humanitarian Contexts - World | ReliefWeb Ethics guidelines for trustworthy AI | Shaping Europe's digital future Ethics by design for artificial intelligence | AI and Ethics Nonhuman humanitarianism: when 'AI for good' can be Harmful |
|
| Frameworks and Governance |
A Framework for the Ethical Use of Advanced Data Science Methods in the Humanitarian Sector OECD Framework for the Classification of AI systems Governing AI for humanity : final report / Advisory Body on Artificial Intelligence Building a Responsible Humanitarian Approach: The ICRC'S Policy on Artificial Intelligence | International Committee of the Red Cross The Signal Code: A Human Rights Approach to Information During Crisis | Harvard Humanitarian Initiative GiveDirectly Responsible AI/ML Framework Participatory AI for Humanitarian Innovation (Nesta) USAID Artificial Intelligence Action Plan - USAID Digital Development Site |
|
| Guidance and Best Practices |
Harnessing the Potential of Artificial Intelligence for Humanitarian Action: Opportunities and Risks Machine Learning for Disaster Risk Management Humanitarian AI revisited: Seizing the potential and sidestepping the pitfalls Generative AI For Humanitarians VOICE Key highlights - 'Artificial Intelligence: disruption or opportunity for humanitarian aid' Handbook on data protection in humanitarian action | ICRC Responsible Data Handbook |
|
| Case Studies and Applications (chatbots and agents) |
Chatbots in Humanitarian Contexts Chatbots in the humanitarian field Chatbots: A Conversation with the Community (A Kazakhstan Case Study) Navigating Humanitarian AI: Lessons Learned from Building a Chatbot Proof-of-Concept | Refugee Solidarity Network Using Large Language Models for Humanitarian Frontline Negotiation: Opportunities and Considerations Artificial Intelligence, disasters and marginalised groups | PreventionWeb Chatbots against COVID |
|
Humanitarian AI initiatives must be anchored deeply in core humanitarian principles, ensuring that technological adoption is principled and people-centered. This requires maintaining transparency, conducting inclusive consultations, actively incorporating community perspectives, implementing robust accountability mechanisms, fostering collaborative partnerships, and intentionally seeking diverse expertise and viewpoints.[84]
Data Privacy and Data Protection
Data has increasingly become an important resource; data scientist Clive Humby coined the phrase “data is the new oil” almost two decades ago [85] while the Economist in 2017 called the most resource [86].
An understanding of AI is incomplete without understanding its relationship to data. The two are inextricably intertwined; the development and use of AI involves massive amounts of data, in many cases, personal data. In a humanitarian context, it can be useful to think of AI projects as Data Science projects [87]. This framing underscores not only the importance of data to AI but also of its protection and safeguarding in the specific context of humanitarian work. Data Privacy and Protection is one of the most crucial aspects to consider when creating governance policies and deciding on the development and application of AI tools.
Data Privacy
Data Privacy and Protection are often used interchangeably and while they are related and have some overlap, they differ in significant ways.
Data privacy revolves around the question of who has authorized access to handle one’s information (collection, processing , sharing, etc.) and the extent to which one can control this access (e.g. opting out of data collection). This term refers not just personal data but to any kind of data, if accessed by others, would be seen a violation one’s personal autonomy [88]
Privacy has often been understood as control over one’s own information but given the scale and magnitude of the loss of control faced by many today challenges this notion. Current frameworks and privacy regulations, however, still appear to operate on this principle of personal control.
The contextually contingent (e.g. sharing one’s location data to friends might be okay but that same data being collected by a company for advertising violates privacy) and relational (e.g. data is social and can appear in shared social media posts) nature of data further challenges the idea of privacy as personal control. [89]
Data Protection
Data Protection involves safeguarding personal information using procedural rights. It requires that data is handled equitably for specific purposes and collected on sound bases. Consent is the strictest basis and can allow people to withdraw it after the fact, whereas legitimate interest provides the most flexibility in terms of allowing entities to justify data processing on the necessity of this data on their activities [90].
Entities processing data must be respectful of individuals’ fundamental protection rights (e.g. providing notice upon collection of data, providing access to said collected data and the means to modify, delete or correct it. There is however a predilection for the assumption that acceptance is baked by default.
In the EU, there is a formal distinction between privacy and data protection in its European Charter of Fundamental Rights [91]. The two concepts often overlap and complement each other.
For example, when data is not considered personal falling outside of data protection rules (e.g. anonymized body scan information), privacy rights still come into play, given such information could still affect the person’s individual being. On the other hand, data protection regulations restrict and limit processing and handling of personal information, even in situations where privacy does not seem to be compromised.
A Brief Look at Two Regulatory Frameworks: FIPs and GDPR
FIPs
Fair Information Practices (FIPs) is a more than 50 year old set of principles which provides the framework for giving individuals due process for their personal information. [92] The FIPs , as part of the US federal code introduced five safeguard requirements regarding personal privacy as a means of ensuring “informational due process.” [93] FIPs focus on providing the individuals to know about, stop alternative use and correct information about themselves.
FIPs do not frame privacy as fundamental human rights as with the United Nations Universal Declaration of Human Rights [94] and the European Charter of Fundamental Rights [95] Instead they outline rules and obligations between individuals and the data processor. This framing is based on the assumption that the modern state require record keeping (and data collection) for its administration and working.
This initial framing was modernized through the OECD in 1980 and amended in 2013, expanding into eight principles [96]. The principles are:
Collection Limitation: Personal data collection should have limits; it must be gathered through lawful and fair methods and, when applicable, with the knowledge or consent of the individual concerned
Data Quality: Personal data should be pertinent to its intended purposes, and it must be accurate, complete, and regularly updated
Purpose Specification: The purposes for which personal data are collected should be specified at the time of data collection and the subsequent use limited to the fulfillment of those purposes
Use Limitation: Personal data should not be disclosed, made available, or used for purposes other than those specified under the Purpose Specification principle, except in the following cases: (a) with the consent of the data subject; or (b) as permitted by law
Security Safeguards: Personal data should be safeguarded by appropriate security measures to protect against risks such as loss, unauthorized access, destruction, use, modification, or disclosure
Openness: There should be a general policy of transparency regarding developments, practices, and policies related to personal data, including the primary purposes for its use and the identity of the data controller
Individual Participation: An individual should have the right: (a) to obtain from a data controller the data the controller has about them (b) to challenge data relating to them and, if the challenge is successful to have the data erased, rectified, completed or amended
Accountability: A data controller should be accountable for complying with measures which give effect to the principles stated above [97]
Despite being conceived long before the digital and information age, key components like collection limitation and purpose specification continue to impact today’s AI systems by limiting how broadly companies can repurpose collected data for one purpose to train or develop new AI systems.
The EU’s General Data Protection Regulation (GDPR) relies heavily on these principles, which we will look at now.
General Data Protection Regulation (GDPR)
GDPR is a comprehensive data privacy law which updated the 1995 Data Protection Directive and consolidated national data privacy regimes across EU member states. It was passed in 2016, and became enforceable in May 2018. It grants individuals or “data subjects” rights regarding the processing of their personal data, such as the right to be informed and a limited right to be forgotten, and guides how businesses can process personal information. [98]
It contains provisions which apply directly to AI systems even if the term artificial intelligence is not used. For example, Article 22 provides protections to individuals against decisions “based solely on automated processing” of personal data without human intervention also called automated decision making (ADM). [99] This enshrines individual rights not to be subjected to ADM where these decisions could produce adverse legal or significant effects on individuals.
GDPR’s articles on “Data Minimization”, “Purpose Limitation” and “Consent” are especially relevant to data protection principles in the case of AI systems.
Data Minimization: Article 5 of the GDPR on collected data states that it is “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed.” [100] This principle prescribes proportionality: entities should not blindly collect as much data as they want, particularly outside of the context they have provided for collection.
Purpose Limitation: Data “collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes.” [101] This underlines the significance of context and sets rules for re-using collected data for a different context
Consent: This is defined in Article 7 and Recital 32; consent must be “given by a clear affirmative act establishing a freely given, specific, informed and unambiguous indication of the data subject’s agreement to the processing of personal data relating to him or her, such as by a written statement, including by electronic means, or an oral statement.” [102] Consent is required for all processing, including multiple purposes: “consent should not be regarded as freely given if the data subject has no genuine or free choice or is unable to refuse or withdraw consent without detriment.” [103]
The GDPR also places transparency obligations where notices must be given to individuals when their personal information is processed and establishes rules granting individuals the right to access their own information and ensure that processing of their data is accurate. These principles are meant to curb unfettered data mining and processing that is commonplace in data-intensive AI systems. [104]
Data Privacy and Protection Issues Related to Generative AI in Humanitarian Contexts
The harms to individual autonomy, including the inability to make informed choices, the difficulty in correcting data, and a general lack of control over the collection and usage of personal information, are significant concerns. These issues are pertinent to AI-based systems, much like they were to the technological advancements witnessed during the past three decades of internet expansion.
Although these harms existed prior to the integration of AI into the consumer sector, commercial AI systems are likely to perpetuate and exacerbate these issues, while also introducing new forms of harms, such as identity-based risks, data aggregation and inference risks, personality and emotional state inferences, and exposure of previously unavailable or redacted sensitive personal information, misidentification and defamation. [105]
Moreover, the privacy risks and harms associated with AI systems extend beyond the individual level; they pose threats to groups and society as a whole in ways that cannot be effectively addressed through the exercise of individual data rights. An simple example is that relational inferences where even people people whose data is not included in training datasets are impacted. [106] [107]
These larger issues apply to the humanitarian context in a more specific manner. There are attempts to identify and recognize use-cases where AI can be deployed in the humanitarian sector. However, it is important to address existing algorithmic bias and data privacy related risks as a priority before any such deployments.
There are sector-specific concerns relating to AI, including surveillance humanitarianism [108], an over-eager techno-solutionism [109] and the potential rise of techno-colonialism. [110]
AI technologies have the potential to support humanitarian missions in three main dimensions: Preparedness, Response, Recovery. Refer to [111] for case study examples Yet AI comes at the expense of risks such as the following :
Data Quality: Input data may have poor quality (e.g. “Garbage In, Garbage out” [112]), difficult to procure due to constrained or low-resourced digital environments [113], and datasets may be historically biased [114]
Algorithmic Bias: AI systems may have assumptions, biases of humans baked into them [115], biases may also present themselves as a result of poorly representative data sets used for training [116]
Security Risks: Generative AI systems create known and unknown security risks related to data security and theft. The use of such systems elevate risk of data breaches, identity theft, data leaks and inadvertent disclosure of sensitive information on clients as well as staff [117]
Another risk here is that to Data Privacy. As shown, international and regional human rights law instruments recognize the right to privacy and have frameworks which are meant to regulate that. In the humanitarian context, consent, for example, may not be completely unambiguous and freely given due to the inherent imbalance between humanitarian organizations and those who access their assistance and services. Refusal to consent to the collection and processing of personal data will essentially result in the denial of humanitarian assistance. Additionally, humanitarian actors may face challenges in ensuring that recipients of such assistance fully comprehend the implications of consent, given the presence of linguistic barriers and the complexities associated with administrative and institutional frameworks. [118]
Furthermore, fully informed, specific and unambiguous consent may also be challenging in the case of Generative AI, as such data goes into the black box of Large Language models (LLMs). Even if anonymized, such growing collection of data may inadvertently lead to surveillance humanitarian mentioned above and inadvertently increase the vulnerability of those in need. [119]
Finally, data protection becomes crucial in collaborations between humanitarian organizations and technology companies where hasty and under-planned deployments might lead to neglecting the needs and experiences of their users.
Accordingly, it is essential that humanitarian organizations create clear guidelines for implementing AI in the humanitarian context specifically in the realm of data privacy and protections. It is also critical for humanitarians to develop, and operationalize strategies for data protection and privacy principles. This will require combining in Data Protection guardrails what is already applicable legally and regulatory frameworks-wise with AI risk anticipatory actions safeguarding fundamental humanitarian principles.
There a few key takeaways from the above-surveyed work:
Individual Autonomy Risks
AI systems pose risks to personal autonomy by disrupting an individual’s ability to manage their own information. Such systems create barriers to informed decision-making, making it challenging for individuals to understand how their data is collected, used and manipulated. The inability to correct personal information and the lack of meaningful control over data usage threaten personal agency, now amplified by AI’s entangled and opaque algorithmic processes
The Challenge of Consent
Truly voluntary consent becomes difficult to achieve due to inherent power dynamics embedded in systems which deliver humanitarian services. Individuals asserting their data rights and refusing data collection may find themselves losing critical support, negating the notion of free choice. Informed consent faces complications in the face of issues such as language barriers, and indecipherable administrative frameworks where a true comprehension of long-term implications due to sharing personal data might not be possible. This structural inequality complicates consent; making it less of a choice and more of a coercive requirement for accessing essential services.
Systemic Privacy Risks
Large scale collection of individuals and their data points create systemic risks which can affect broader communities. The ability of such systems to generate insights, inferences and predictions about individuals and groups means people can be categorized, and profiled without ever having explicitly provided their personal information. This aggregative and relational capability of AI systems in particular, pose a challenge to traditional notions of personal privacy and protection.
Data Protection in Black Boxes
Generative AI as opaque black boxes make tracking and understanding data use challenging. Protections offered by humanitarian practices such as deanonymization can be circumvented by cumulative data collection and generation of profiles which can potentially increase vulnerability of marginalized populations. The potential for unintended surveillance and sensitive information disclosure becomes heightened as these systems can aggregate and infer personal details in which existing data protection frameworks are ill-equipped to anticipate or prevent
Data privacy and protection requires a proactive multi-pronged approach which combines legal frameworks with humanitarian-centered strategies. The aid sector must develop guidelines which prioritize data privacy and protection before implementing AI technologies. This involves creating robust mechanisms which leverage existing regulations while anticipating new potential risks, center human dignity and ensure that a rush towards AI innovation in the sector does not overrun fundamental principles of humanitarian assistance not only to communities but staff and organizations themselves. The goal should be to develop nimble, principled strategies which can evolve alongside rapidly changing technological landscapes.
Bibliography
European Commission, White Paper on Artificial Intelligence: A European Approach to Excellence and Trust, COM (2020) 65 final, 2020, p. 2.
European Union High Level Expert Group on Artificial Intelligence, A Definition of AI: Main Capabilities and Scientific Disciplines, Brussels, 2019, p. 6.
Ask the Experts: The Impact of Generative AI, Such as ChatGPT | Gartner Webinars
Max Roser (2022) - “The brief history of artificial intelligence: the world has changed fast — what might be next?” Published online at OurWorldinData.org. Retrieved from: 'https://ourworldindata.org/brief-history-of-ai'
Misiura, Joanna. Verity, Andrej. 2019. “Chatbots in the Humanitarian Field - Concepts, Uses & Shortfalls - World | ReliefWeb.” https://reliefweb.int/report/world/chatbots-humanitarian-field-concepts-uses-shortfalls).
How AI can actually be helpful in disaster response | MIT Technology Review
From ‘paradise’ to hell: how a luxury residence in Turkey became an earthquake death trap
Misiura, Joanna. Verity, Andrej. 2019. “Chatbots in the Humanitarian Field - Concepts, Uses & Shortfalls - World | ReliefWeb.” https://reliefweb.int/report/world/chatbots-humanitarian-field-concepts-uses-shortfalls).
What Is an LLM and How Does It Relate to AI Chatbots? Here's What to Know - CNET
Paul, Amy, Craig Jolley, and Aubra Anthony. 2018. Reflecting the Past, Shaping the Future: Making AI Work for International Development. https://www.usaid.gov/sites/default/files/documents/15396/AI-ML-in-Development.pdf
Chatbots in humanitarian contexts: Learning from practitioner experiences
Chatbots in humanitarian contexts: Learning from practitioner experiences
Behance. "Red Cross - ELLA / The messenger bot with a heart - Martin Coppola." Behance, 24 Feb. 2024, www.behance.net/gallery/53783325/Red-Cross-ELLA-The-messenger-bot-with-a-heart#
"U-Report." 25 Feb. 2025, ureport.in/about
Refugee Text. 2016. http://www.refugeetext.org/.; Migration Aid. n.d. https://www.migrationaid.net/infoaid/.; Refunite. 2018. Refunite
IRAP and Marhub Launch ChatBot to Provide Legal Information to Refugees Seeking Assistance
From inaccessible to indispensable: Technology for teacher training in emergency contexts | INEE
"Farm.ink: Analysing livestock social media data for farmer chatbot | ICTforAg Learning Network." learningnetwork.ictforag.com/innovation/farm-ink-analysing-livestock-social-media-data-for-farmer-chatbot
FarmChat: Using Chatbots to Answer Farmer Queries in India - ICTworks
Curious about HIV/AIDS? Just Ask Marlo - Health - The Jakarta Post
Handforth, Calum and Kecia Bertermann. 2018. How Girl Effect built a chatbot. https://www.researchgate.net/profile/Calum_Handforth/publication/329924130_How_Girl_Effect_built_a_chatbot/links/5c23e8cb92851c22a3484a98/How-Girl-Effect-built-a-chatbot.pdf?origin=publication_detail
Kevin Cole. 2024. “Navigating Humanitarian A I: Lessons Learned from Building a Chatbot Proof-of-Concept.” Refugee Solidarity Network.
Spencer, Sarah. 2024. “Humanitarian AI revisited Seizing the potential and sidestepping the pitfalls”. Humanitarian Practice Network (HPN)
Spencer, Sarah. 2024. “Humanitarian AI revisited Seizing the potential and sidestepping the pitfalls”. Humanitarian Practice Network (HPN)
How can machine learning predict cholera: insights from experiments and design science for action research | Journal of Water and Health | IWA Publishing][Mathematical Models for Cholera Dynamics—A Review - PMC
New AI tool could help predict viral outbreaks | University of Oxford
New machine learning tool can help predict patients most at risk of COVID-19 | Imperial News
Computerized radiographic detection of tuberculosis in densely populated areas of Manila | Epicentre
Malaria Screening Gets “Smart” with Machine Learning] [Antibiogo | The MSF Foundation
How maps built with Facebook AI can help with COVID-19 vaccine delivery
Pizzi, Michael, Mila Romanoff, and Tim Engelhardt. 2020. “AI for Humanitarian Action: Human Rights and Ethics.” International Review of the Red Cross 102(913):145–80. doi: 10.1017/S1816383121000011
What is garbage in, garbage out (GIGO) ? | Definition from TechTarget
Note: A parallel narrative frames this same inherent hallucinatory quality of LLMs as desirable because it gives Large Language Models a creative facet, which has the potential to foster innovation
6 Displaced, Profiled, Protected? Humanitarian Surveillance and New Approaches to Refugee Protection
AccessNow. 2024. “Mapping Humanitarian Tech: Exposing protection gaps in digital transformation programmes”
Mani, Z., & Chouk, I. (2018). Consumer resistance to innovation in services: Challenges and barriers in the internet of things Era. Journal of Product Innovation Management, 35(5), 780–807
Chatbots in humanitarian contexts: Learning from practitioner experiences
How Tech Giants Cut Corners to Harvest Data for A.I. - The New York Times
LAION-5B: A NEW ERA OF OPEN LARGE-SCALE MULTI-MODAL DATASETS
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus
Generative AI like Midjourney creates images full of stereotypes - Rest of World
A jargon-free explanation of how AI large language models work | Ars Technica
https://www-tandfonline-com.libproxy.newschool.edu/doi/full/10.1080/08839514.2022.2037254
Generative AI in Cyber Security: New Threats and Solutions for Adversarial Attacks
Pizzi, Michael, Mila Romanoff, and Tim Engelhardt. 2020. “AI for Humanitarian Action: Human Rights and Ethics.” International Review of the Red Cross 102(913):145–80. doi: 10.1017/S1816383121000011
AccessNow. 2024. “Mapping Humanitarian Tech: Exposing protection gaps in digital transformation programmes”
Aidwashing Surveillance: Critiquing the Corporate Exploitation of Humanitarian Crises
"Do no harm: A taxonomy of the challenges of humanitarian experimentation." International Review of the Red Cross, 15 Feb. 2025, international-review.icrc.org/articles/do-no-harm-taxonomy-challenges-humanitarian-experimentation
Humanitarian experimentation - Humanitarian Law & Policy Blog
Ibid.
The clock is ticking to build guardrails into humanitarian AI
The world’s most valuable resource is no longer oil, but data
A FRAMEWORK FOR THE ETHICAL USE OF ADVANCED DATA SCIENCE METHODS IN THE HUMANITARIAN SECTOR
Ibid.
Gellert, Raphaël and Serge Gutwirth. "The legal construction of privacy and data protection." Computer Law & Security Review, vol. 29, no. 5, 1 Oct. 2013, pp. 522-30, doi:10.1016/j.clsr.2013.07.005.
Ibid.
Ibid.
Ibid.
Ibid.
Ibid.
[2310.07879] Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks
Keren Weitzberg, Margie Cheesman, Aaron Martin and Emrys Schoemaker, “Between Surveillance and Recognition: Rethinking Digital Identity in Aid”, Big Data & Society, Vol. 8, No. 1, 2021.
Duffield, M. (2016). The resilience of the ruins: towards a critique of digital humanitarianism. Resilience, 4(3), 147–165. https://doi-org.libproxy.newschool.edu/10.1080/21693293.2016.1153772
What is garbage in, garbage out (GIGO) ? | Definition from TechTarget
Christopher Kuner and Massimo Marelli, Handbook on Data Protection in Humanitarian Action, 2nd ed., ICRC, Geneva, 2020, p. 39; OCHA, above note 15, p. 10; ICRC, The Engine Room and Block Party, above note 52, p. 32.
Andrew Ferguson, The Rise of Big Data Policing: Surveillance, Race, and the Future of Law Enforcement, New York University Press, New York, 2017
James Zou and Londa Schiebinger, “AI Can Be Sexist and Racist — It's Time to Make It Fair”, Nature, Vol. 559, 2018
Ibid.
Cashless cash: financial inclusion or surveillance humanitarianism? - Humanitarian Law & Policy Blog