Adnan Rahic
Making Data Warehouses fast. A developer's story.
#1about 3 minutes
The developer's struggle with data warehouse latency
High latency in applications built on data warehouses creates a poor user experience and presents a significant challenge for developers.
#2about 5 minutes
Differentiating between OLAP and OLTP database workloads
Data warehouses use OLAP for complex, low-volume queries on large datasets, contrasting with OLTP's high-volume, simple transactions.
#3about 3 minutes
Understanding the key factors of query latency
User-perceived performance is impacted by network delays and data scan times, making sub-second responses a critical goal.
#4about 7 minutes
Exploring BigQuery's caching and concurrency limitations
BigQuery's cache only works for identical queries and its concurrency is capped per project, impacting real-world application performance.
#5about 4 minutes
Benchmarking BigQuery's performance under concurrent load
Load testing reveals that BigQuery maintains a consistent query latency of around two seconds regardless of user concurrency up to its hard limit.
#6about 2 minutes
Introducing Cube as a semantic analytics API layer
Cube provides a semantic layer over data warehouses, enabling caching, pre-aggregations, and access control to build fast data apps.
#7about 3 minutes
Setting up a local Cube development environment
A local Cube instance can be configured using Docker Compose to connect to BigQuery and automatically generate data schemas.
#8about 5 minutes
How pre-aggregations dramatically improve query speed
Pre-aggregations act as materialized views that store condensed query results, reducing a query's response time from seconds to milliseconds.
#9about 3 minutes
Comparing benchmark results of Cube vs direct BigQuery
Benchmarks show that using Cube's pre-aggregation layer results in a nearly five-fold performance increase over querying BigQuery directly.
#10about 8 minutes
Answering questions on Cube's architecture and use cases
The discussion covers when to implement a caching layer, how Cube improves performance, and its utility for medium-sized databases.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
00:09 MIN
A DBA's journey to running SQL Server on Kubernetes
Adjusting Pod Eviction Timings in Kubernetes
28:34 MIN
Answering questions on data volume, challenges, and databases
Remote Driving on Plant Grounds with State-of-the-Art Cloud Technologies
27:05 MIN
Q&A on implementation details and technology choices
Challenges for omnichannel applications at ALDI: Data distribution and offline capabilities
34:48 MIN
Q&A on performance, parallelism, and organizational impact
Convert batch code into streaming with Python
43:15 MIN
Q&A on security, custom functionality, and performance
Anvil: Full Stack Web Apps With Nothing But Python
37:27 MIN
Overcoming challenges of data size and security
Web-based Information Visualization
00:43 MIN
Meeting modern application and data platform demands
Tomorrow's cloud data platforms - fully managed database-as-a-service (DBaaS)
00:05 MIN
Introducing Kusto for interactive analytics at Microsoft scale
From Tables to Graphs in Minutes: Supercharging Kusto Graph Analytics with AI-Powered Development
Featured Partners
Related Videos
24:22Database Magic behind 40 Million operations/s
Jürgen Pilz
27:27Things I learned while writing high-performance JavaScript applications
Michele Riva
28:12Scaling: from 0 to 20 million users
Josip Stuhli
50:06Single Server, Global Reach: Running a Worldwide Marketplace on Bare Metal in a Cloud-Dominated World
Jens Happe
26:53Enjoying SQL data pipelines with dbt
Matthias Niehoff
22:32Swapping Low Latency Data Storage Under High Load
George Asafev
29:25Modern Data Architectures need Software Engineering
Matthias Niehoff
30:02Lessons learned from building a thriving Vue.js SaaS application
Abdelrahman Awad
Related Articles
View all articles.gif?w=240&auto=compress,format)
From learning to earning
Jobs that call for the skills explored in this talk.
Full Stack Software Developer
Red Bull Media House GmbH
Elsbethen, Austria
Intermediate
Java
NoSQL
Docker
Angular
Hibernate
+6
Cube Academy - Full Stack Software Engineer (AI) Part-time
3 SIDED CUBE
Remote
€27K
Docker
MongoDB
Node.js
+5
Performance Analyst (Google and Power BI)
Digital Skills Ltd
€78K
Data analysis
Google Analytics
Microsoft Office
Google Tag Manager
Business Intelligence Workshop
Scalefree International GmbH