Jodie Burchell

Lies, Damned Lies and Large Language Models

What if 40% of your LLM's answers are just plain wrong? Learn how to measure factuality and build more reliable AI applications.

Lies, Damned Lies and Large Language Models
#1about 2 minutes

Understanding the dual nature of large language models

LLMs can generate both creative, coherent text and factually incorrect "hallucinations," posing a significant challenge for real-world applications.

#2about 4 minutes

The architecture and evolution of LLMs

The combination of the scalable Transformer architecture and massive text datasets enables models like GPT to develop "parametric knowledge" as they grow in size.

#3about 3 minutes

How training data quality influences model behavior

The quality of web-scraped datasets like Common Crawl, even after filtering, directly contributes to model hallucinations by embedding misinformation.

#4about 2 minutes

Differentiating between faithfulness and factuality hallucinations

Hallucinations are categorized as either faithfulness errors, which contradict a given source text, or factuality errors, which stem from incorrect learned knowledge.

#5about 3 minutes

Using the TruthfulQA dataset to measure misinformation

The TruthfulQA dataset provides a benchmark for measuring an LLM's tendency to repeat common misconceptions and conspiracy theories across various categories.

#6about 6 minutes

A practical guide to benchmarking LLM hallucinations

A step-by-step demonstration shows how to use Python, LangChain, and Hugging Face Datasets to run the TruthfulQA benchmark on a model like GPT-3.5 Turbo.

#7about 4 minutes

Exploring strategies to reduce LLM hallucinations

Key techniques to mitigate hallucinations include careful prompt crafting, domain-specific fine-tuning, output evaluation, and retrieval-augmented generation (RAG).

#8about 4 minutes

A deep dive into retrieval-augmented generation

RAG reduces hallucinations by augmenting prompts with relevant, up-to-date information retrieved from a vector database of document embeddings.

#9about 2 minutes

Overcoming challenges with advanced RAG techniques

Naive RAG can fail due to poor retrieval or generation, but advanced methods like Rowan selectively apply retrieval to significantly improve factuality.

Related jobs
Jobs that call for the skills explored in this talk.

test

Milly
Vienna, Austria

Intermediate

test

Milly
Vienna, Austria

Intermediate

Matching moments

Understanding the problem of LLM hallucinations
00:02 MIN

Understanding the problem of LLM hallucinations

Martin O'Hanlon - Make LLMs make sense with GraphRAG

Addressing the key challenges of large language models
01:47 MIN

Addressing the key challenges of large language models

Large Language Models ❤️ Knowledge Graphs

Why web data is essential for training large language models
00:05 MIN

Why web data is essential for training large language models

How to scrape modern websites to feed AI agents

Explaining how large language models work and why they hallucinate
04:08 MIN

Explaining how large language models work and why they hallucinate

Innovating Developer Tools with AI: Insights from GitHub Next

Addressing the core challenges of large language models
00:27 MIN

Addressing the core challenges of large language models

Accelerating GenAI Development: Harnessing Astra DB Vector Store and Langflow for LLM-Powered Apps

Understanding the risks of large language models
00:48 MIN

Understanding the risks of large language models

Inside the Mind of an LLM

Understanding the limitations of large language models
01:03 MIN

Understanding the limitations of large language models

Knowledge graph based chatbot

Demonstrating LLM hallucinations with a basic chatbot
13:22 MIN

Demonstrating LLM hallucinations with a basic chatbot

Make it simple, using generative AI to accelerate learning

Featured Partners

Related Videos

See all videos
Creating Industry ready solutions with LLM Models58:00

Creating Industry ready solutions with LLM Models

Vijay Krishan Gupta & Gauravdeep Singh Lotey

Inside the Mind of an LLM27:11

Inside the Mind of an LLM

Emanuele Fabbiani

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge42:26

How to Avoid LLM Pitfalls - Mete Atamel and Guillaume Laforge

Meta Atamel & Guillaume Laforge

Large Language Models ❤️ Knowledge Graphs29:11

Large Language Models ❤️ Knowledge Graphs

Michael Hunger

Martin O'Hanlon - Make LLMs make sense with GraphRAG24:42

Martin O'Hanlon - Make LLMs make sense with GraphRAG

Martin O'Hanlon

Three years of putting LLMs into Software - Lessons learned26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

Give Your LLMs a Left Brain27:35

Give Your LLMs a Left Brain

Stephen Chin

What do language models really learn47:28

What do language models really learn

Tanmay Bakshi

Related Articles

View all articles
LM
Luis Minvielle
What Are Large Language Models?
Developers and writers can finally agree on one thing: Large Language Models, the subset of AIs that drive ChatGPT and its competitors, are stunning tech creations. Developers enjoying the likes of GitHub Copilot know the feeling: this new kind of te...
What Are Large Language Models?
DC
Daniel Cranney
How machine learning can help us tell fact from fiction
A decade ago, machine learning was everywhere. While the rise of generative AI has meant artificial intelligence has stolen the spotlight to some degree, it’s machine learning (ML) that silently powers its most impressive achievements.From chatbots t...
How machine learning can help us tell fact from fiction
CH
Chris Heilmann
Dev Digest 137 - AI'm not sure about this
Hello fellow developer, this is the 1st "out of the can" edition of 3 as I am on vacation in Greece going "whee are you cute" at donkeys. So, fewer news, but lots of great resources. Enjoy! News and ArticlesOpenAI has been the big topic winning in th...
Dev Digest 137 - AI'm not sure about this
KD
Krissy Davis
The Best Large Language Models on The Market
Large language models are sophisticated programs that enable machines to comprehend and generate human-like text. They have been the foundation of natural language processing for almost a decade. Although generative AI has only recently gained popula...
The Best Large Language Models on The Market

From learning to earning

Jobs that call for the skills explored in this talk.