Inside the Mind of an LLM

An LLM leaking private data isn't a bug, it's a core feature. Learn why deep learning models are fundamentally designed to memorize unique information.

#1about 7 minutes

Understanding the risks of large language models

LLMs are often used without understanding their inner workings, leading to factual errors and the generation of insecure code.

#2about 8 minutes

How large language models are trained

A four-phase process explains how models learn language through pre-training, are taught tasks, aligned with human preferences, and refined using reinforcement learning.

#3about 5 minutes

Why Llama 2 models think in English

Research on Llama 2 models reveals they use English as an internal representation for all tasks due to its prevalence in the training data.

#4about 4 minutes

Controlling LLM behavior with monosemantic features

By identifying and amplifying single-meaning concepts, or monosemantic features, it is possible to deterministically control a model's output on specific topics.

#5about 2 minutes

Why LLMs memorize and leak private data

Deep learning models inherently memorize unique outlier data from their training set, which explains why LLMs can leak personal information and pose a privacy risk.