Hallucinations: Understanding AI Mix-Ups
What is AI Hallucination?
Imagine your AI assistant is like a really smart friend who has read a ton of books. Sometimes, when you ask it a question, it’ll try to give you the best answer it can based on what it’s learned. But just like your friend might sometimes mix up facts or make up a story, AI can do that too.
That’s what an AI Hallucination is. It’s when the AI gives you an answer that’s not quite right, either because it doesn’t have enough information, or because it made a mistake in how it processed the information.
Why do Hallucinations happen?
Hallucinations, also known as “AI-generated fabrications”, can occur due to several factors:
Lack of Contextual Understanding: The model may sometimes struggle to understand the context of a prompt or question, leading to responses that are irrelevant or misleading. This can be particularly problematic when dealing with complex or ambiguous queries where you assume background knowledge.
Data Quality Issues: If the data used to train an AI model is biased, incomplete, or noisy, it can introduce errors and biases into the model’s output. This can lead to inaccurate responses.
Model Complexity: More complex AI models with many parameters can be more prone to hallucinations. This is because these models are more likely to overfit to the training data and struggle to generalize to new examples.
Prompt and User Interactions: The way a user interacts with an AI model can also influence the likelihood of hallucinations. For example, overly complex or ambiguous prompts can confuse the model and lead to incorrect responses, as well as links generated.
It’s important to note that hallucinations are a common challenge in generative AI development, and we’re continuously working to address these issues and improve the accuracy and reliability of AI models in the humanitarian space to ensure AI is used responsibly and ethically.
What were our challenges with Hallucinations?
Our experience with AI hallucinations revealed a diverse range of issues, including:
Incorrect Information and Inconsistencies: Inaccurate contact details, information not sourced from our trained data, and irrelevant responses.
Prompt: “Only share information and resources in the context and never from other sources”, and “The AI should not generate external information from external sources from Google for example”
Broken Links: Incorrect or broken links to articles and services.
SignpostAI fixed this with a combination of prompts and a change to the ways we search for links in our databases vector search - Weaviate. To learn more about Weaviate, visit the link here.
Prompt: “Do not generate links or website paths that are not from your provided knowledge base”
Language Mismatches: Responses in a different language than the user's query.
Resolved by a combination a prompts and a feature that was added to the bot’s configuration “Translate to Detected Language”.
Each country can choose to enable or disable this for their instance.
The feature above enables the bot to detect the users language and respond back in the same language. This is especially effective if the countries instance offers more than one language on their website. Example websites for your reference: Bolo.pk and Refugee.Info.
Prompt: “Always generate a response and sources in the user's language. When the user asks a question in more than one language, you should ask them in which language they prefer to communicate.”
Below is an example of an English user prompt with an English bot response, with a Haitian reference at our early phases.
4. Procedural Errors: Outlining procedures not present in the bot's knowledge base.
At the early stages of building the bot, we were connected to the Signpost’s broad knowledge base instead of country specific data.
This resulted in a mix of different information. An example is that the bot pulled information from CuentaNos to Refugee Info enquiries.
By improving our search database to ‘Weaviate’, it allowed us to limit the bots' knowledge to the countries' websites focusing on Zendesk (articles), and Directus (services).
Examples of Hallucinations and The Magic of Prompts
To address these challenges, we focused on refining our AI model and data infrastructure. A significant breakthrough came from rebuilding the vector database (search engine) without content chunking. This enabled the bot to access complete articles rather than snippets, leading to more accurate and comprehensive responses.
Alongside that, our testing highlights the significant influence well-crafted prompts can have on AI responses. For instance, a seemingly minor change in a local prompt could dramatically alter the course of the response, even improving its accuracy.
Case 1 - Links in Prompts
We employ iterative testing to ensure prompts are effective. For example, a prompt instructing the AI to summarize and generate a response with a specific link (e.g., "When a user asks about support for building a house, summarize and generate a response including https://elsalvador.cuentanos.org/es/articles/8189425640989") yielded a specific paragraph. When we removed this prompt, the response shifted, prompting the user about their specific support needs yet remaining informative with accurate references.
This demonstrated how well-designed prompts can guide AI responses while maintaining accuracy and helpfulness. While a seemingly effective prompt might initially improve responses, it can inadvertently introduce new challenges. Overusing or misapplying prompts can lead to unintended consequences, such as increased hallucinations. It’s pretty sensitive!
To learn more about The Impact of Prompts on Performance, visit our blog.
Case 2 - Biased Output and Overconfidence
Bias and Overconfidence in AI is a common issue. AI models can express excessive certainty in their responses, even when those responses are incorrect, misleading or in our case, can be harmful. A simple phrase such as “Remember, you are not alone in this”, and “You’re not alone, there’s support available for you”. The bot may unintentionally downplay the user's situation by providing generic assurances without fully understanding their specific circumstances. It's important to remember that while we strive for helpful responses, we cannot guarantee specific outcomes.
See the image below to understand the type of questions we may receive, and detecting a biased response from the bot.
Case 3 - Response Structure and Correct Information
In this specific instance, a combination of technical enhancements and refined prompting techniques proved instrumental in mitigating hallucinations and improving overall performance.
The reconstruction of our vector database with patches to our code, coupled with the introduction of a default prompt (“Please always provide and share relevant and accurate articles, data and services information with a link in your response to the user, and only use the links provided in the context and never from other sources;”) emphasizing accuracy and relevance, significantly enhanced the bot's ability to provide informative and helpful responses in the format we wanted.
Below is a list of improvements expected and post vector database rebuild:
Ability to query using specific properties (domain, country, etc), and do the entire search in just one query.
Search results are complete articles instead of snippets, giving the bot a much better context to answer the questions. Resulting in an enhanced response of the bots due to the increased context.
Ability to feed more distinct articles in the bot.
Removed prevention of services from being considered in responses.
Enhancement made to the internal prompts that extracts search terms. This allows for the extraction of relevant data from the vector db, even when the input information is unclear.
New search mode that combines several different search results with different methods and curates the results using AI before being sent to the context.
Minimized the number of prompts needed to improve responses, and especially PFA (Psychological First Aid) which is important for our audience.
Confirmed it produced less hallucinations due to the bot being able to find relevant information from its knowledge base, as a result, sharing the right links that led to the right content.
Always responses with correct information, addresses, phone numbers, and links.
Update allowed the bot to give more precise responses
By prioritizing information sharing from the outset, Signpost Bot effectively guides users through their journey, fostering trust and encouraging repeat visits. This proactive approach demonstrated the importance of tailoring AI responses to meet user expectations and deliver maximum value.
Comparison Snippet 1 - “help me. asylum seeker, need advice”
Note: v2 is Claude and v3 is GPT.
Comparison Snippet 2 - “I need therapy support in Athens”
Note: v2 is Claude and v3 is GPT.
Comparison Snippet 3 - “I am 8 years old, I am alone.”
Note: v2 is Claude and v3 is GPT.
Analyze the three images carefully, assess the tone, information relevance, formatting, and the quality of any included links. This exercise will evaluate your attention to detail and understanding of the bot's role in providing accurate and helpful responses.
Think about which out of the two would you have chosen? What would your feedback be? What would you want to improve? Is the .v2 bot starting to get annoying when it keeps responding with ‘Hello’ at every message. Now, think beyond the scope:
Is this what you would’ve expected from a human moderator?
Do you unconsciously want the AI Bot to do a better job than a highly trained human moderator?
Are you trying to build an AI Bot to outperform the human moderator, or build a bot that can handle general queries, and if it’s unable to, triage to a human moderator? We’re proceeding with the latter.
Global Hallucination Prioritization: A Strategic Framework
Given the evolving nature of AI and the diverse needs of our country programs, we implemented a strategic approach to address the hallucination challenges we encountered. Rather than a piecemeal approach, we opted for a unified, global strategy.
To effectively address the challenges of AI development, we adopted an iterative approach, focusing on individual components rather than attempting to tackle all requirements simultaneously. This strategic shift allowed us to refine our approach incrementally and achieve better results.
We prioritized the following considerations:
Response Quality vs. Link Accuracy: We evaluated whether the accuracy of the response itself was more critical than the links provided. We determined that response quality requires that the links provided are controllable and verified.
Link Prioritization: We determined if functional links within the bot response were a higher priority than those in the references section.
Quality vs. Format: We weighed the importance of content quality over response format.
Our rapid testing cycles highlighted the need for a flexible approach. We recognized that our scoring mechanism might require adjustments to accurately reflect the bot's performance.
Key Decisions:
Prioritizing Accuracy: We initially focused on ensuring accurate responses, even if it meant sacrificing some links.
Prioritizing accurate responses over links in a humanitarian bot is crucial for several reasons.
Misinformation Prevention: Providing incorrect or misleading information can have severe consequences. It’s essential to ensure people receive the correct guidance and support, this means we cannot rely on crawling the open internet for sources
Trust Building: Accurate responses foster trust between users and the bot. Users will only reciprocate trust, when the information is validated by their human experience and provides helpful support.
Effective Decision-Making: Empowers users to make informed decisions about their situations, potentially leading to life-changing outcomes.
Legal and Ethical Implications: Providing inaccurate information can have legal and ethical consequences, especially in sensitive humanitarian contexts.
Deferred Customization: We postponed decisions about response structure until after the vector database rebuild in case 3.
While links can be valuable for providing additional context, ensuring the accuracy of the core response is paramount. If necessary, links can be provided in a separate section or upon request, allowing users to access additional information while prioritizing the accuracy of the primary response. Our bot's responses may not always include perfectly accurate links, we've found that the references section consistently provides reliable resources for users to explore.
Below is an example where links are not in the bots response but it is in the references section after bots response.
By adopting this strategic approach, we've laid the groundwork for ongoing optimization and empowerment of country teams to make informed decisions regarding AI customization when it comes to hallucinations and responses. With the mindset that hallucinations can still happen periodically even 1-2% of the time when you properly outfit your tools to sidestep hallucinations.
Conclusion
Hallucinations in AI, while a concern, can be mitigated to some degree through careful development, prompting, testing, and ongoing monitoring. In the humanitarian space, where accuracy and reliability are paramount, addressing hallucinations is crucial to ensure the effectiveness and trustworthiness of AI-powered solutions. By focusing on our prompting techniques and architecture, we can remain reliable and beneficial to our internal members and our external audiences. We highly recommend that any team building with Generative AI adapts and considers their own strategic framework and stance on mitigating hallucinations.
