Max Hausner & Yves Fauser

gRPC Load Balancing Deep Dive

Is your gRPC load balancer creating server hotspots? Learn why long-lived connections undermine autoscaling and how a simple Kubernetes configuration can solve it.

gRPC Load Balancing Deep Dive
#1about 4 minutes

An overview of gRPC fundamentals and its trade-offs

gRPC is a high-performance framework using Protobuf for efficiency, but it has limitations in browser support and tooling maturity compared to REST.

#2about 4 minutes

How gRPC streaming and HTTP/2 affect load balancing

gRPC supports various streaming patterns over persistent HTTP/2 connections, which can cause traffic hotspots with traditional Layer 4 load balancing.

#3about 3 minutes

Client-side versus infrastructure-based load balancing strategies

Choose client-side load balancing for low-latency internal services and infrastructure-based load balancing for external APIs that require a clear demarcation point.

#4about 7 minutes

Exploring different types of load balancing algorithms

A review of basic, load-based, and hash-based algorithms reveals that options like "least outstanding requests" can outperform simple round robin for uneven loads.

#5about 2 minutes

Why autoscaling gRPC services can be challenging

Long-lived streaming connections can prevent traffic from being distributed to newly scaled instances, making traditional CPU-based autoscaling ineffective.

#6about 4 minutes

Tools for functional and performance testing of gRPC

Use tools like grpcurl for functional API testing with proto files and ghz for comprehensive performance and load testing of your gRPC services.

#7about 3 minutes

Case study: Separating unary and streaming calls

A practical example shows how separating unary and streaming gRPC calls into different Kubernetes services and target groups solves uneven load distribution.

#8about 1 minute

Key takeaways for effective gRPC load balancing

Successfully load balance gRPC by being mindful of long-lived sessions, understanding client traffic patterns, and selecting L7-based algorithms when possible.

Related jobs
Jobs that call for the skills explored in this talk.

test

Milly

Milly
Vienna, Austria

Intermediate
.NET
TypeScript
+1

test

Milly

Milly
Vienna, Austria

Intermediate
.NET
TypeScript
+1

Matching moments

Exploring gRPC for high-performance communication
02:46 MIN

Exploring gRPC for high-performance communication

REST in Peace? What does the API protocol of the future look like? Or do we have it already?

Using a load balancer for multi-region deployments
01:28 MIN

Using a load balancer for multi-region deployments

Cloud Run- the rise of serverless and containerization

Building a load-balanced web app with CDK
07:04 MIN

Building a load-balanced web app with CDK

Program your infrastructure with CDK and TypeScript

Q&A on GraalVM, Project Loom, and gRPC
15:32 MIN

Q&A on GraalVM, Project Loom, and gRPC

Bootiful Spring Boot 3

Exploring alternatives to REST like gRPC and GraphQL
02:07 MIN

Exploring alternatives to REST like gRPC and GraphQL

API = Some REST and HTTP, right? RIGHT?!

Q&A: GraphQL bombs and gRPC for microservices
01:31 MIN

Q&A: GraphQL bombs and gRPC for microservices

The Great API Debate: REST, GraphQL, or gRPC?

Evaluating the pros and cons of REST, GraphQL, and gRPC
03:35 MIN

Evaluating the pros and cons of REST, GraphQL, and gRPC

The Great API Debate: REST, GraphQL, or gRPC?

A high-level overview of Kubernetes cluster architecture
02:57 MIN

A high-level overview of Kubernetes cluster architecture

Mastering Kubernetes – Beginner Edition

Featured Partners

Related Videos

See all videos
Designing gRPC APIs - The things they don't tell you27:17

Designing gRPC APIs - The things they don't tell you

Bastian Eicher

Fifty Shades of Kubernetes Autoscaling30:26

Fifty Shades of Kubernetes Autoscaling

Mario-Leander Reimer

A Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast Training and Inference on Kubernetes32:19

A Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast Training and Inference on Kubernetes

Kevin Klues

The Great API Debate: REST, GraphQL, or gRPC?28:16

The Great API Debate: REST, GraphQL, or gRPC?

Alexis Yushin

Demystifying application networking in the cloud20:39

Demystifying application networking in the cloud

Duan Lightfoot

Develop, test and run a communications application in a serverless cloud16:38

Develop, test and run a communications application in a serverless cloud

Filippos Kyprianou & Maksym Mednikov

Containers in the cloud - State of the Art in 202228:03

Containers in the cloud - State of the Art in 2022

Federico Fregosi

REST in Peace? What does the API protocol of the future look like? Or do we have it already?20:29

REST in Peace? What does the API protocol of the future look like? Or do we have it already?

Simon Auer

Related Articles

View all articles
CH
Chris Heilmann
All the videos of Halfstack London 2024!
Last month was Halfstack London, a conference about the web, JavaScript and half a dozen other things. We were there to deliver a talk, but also to record all the sessions and we're happy to share them with you. It took a bit as we had to wait for th...
All the videos of Halfstack London 2024!
CH
Chris Heilmann
Dev Digest 139 - Soft and hard queries
News and ArticlesLet's start with Amelia Wattenberger's excellent essay Bridging the hard and the soft talking about humans and data. Do you sometimes also miss aspects of the early web like a lack of big data and freedom to create without numeric go...
Dev Digest 139 - Soft and hard queries
CH
Chris Heilmann
Dev Digest 130 - The Diablo is in the details
News and ArticlesA mixed bag this week of security and AI news: Troy Hunt covers the “3 Billion People” national public data breach in detail, a vulnerability in Microsoft apps allowing to spy on Mac users rains on their smug parade following the Cro...
Dev Digest 130 - The Diablo is in the details

From learning to earning

Jobs that call for the skills explored in this talk.