Adnan Rahic
Making Data Warehouses fast. A developer's story.
#1about 3 minutes
The developer's struggle with data warehouse latency
High latency in applications built on data warehouses creates a poor user experience and presents a significant challenge for developers.
#2about 5 minutes
Differentiating between OLAP and OLTP database workloads
Data warehouses use OLAP for complex, low-volume queries on large datasets, contrasting with OLTP's high-volume, simple transactions.
#3about 3 minutes
Understanding the key factors of query latency
User-perceived performance is impacted by network delays and data scan times, making sub-second responses a critical goal.
#4about 7 minutes
Exploring BigQuery's caching and concurrency limitations
BigQuery's cache only works for identical queries and its concurrency is capped per project, impacting real-world application performance.
#5about 4 minutes
Benchmarking BigQuery's performance under concurrent load
Load testing reveals that BigQuery maintains a consistent query latency of around two seconds regardless of user concurrency up to its hard limit.
#6about 2 minutes
Introducing Cube as a semantic analytics API layer
Cube provides a semantic layer over data warehouses, enabling caching, pre-aggregations, and access control to build fast data apps.
#7about 3 minutes
Setting up a local Cube development environment
A local Cube instance can be configured using Docker Compose to connect to BigQuery and automatically generate data schemas.
#8about 5 minutes
How pre-aggregations dramatically improve query speed
Pre-aggregations act as materialized views that store condensed query results, reducing a query's response time from seconds to milliseconds.
#9about 3 minutes
Comparing benchmark results of Cube vs direct BigQuery
Benchmarks show that using Cube's pre-aggregation layer results in a nearly five-fold performance increase over querying BigQuery directly.
#10about 8 minutes
Answering questions on Cube's architecture and use cases
The discussion covers when to implement a caching layer, how Cube improves performance, and its utility for medium-sized databases.
Related jobs
Jobs that call for the skills explored in this talk.
Hubert Burda Media
München, Germany
€80-95K
Intermediate
Senior
JavaScript
Node.js
+1
Matching moments
03:02 MIN
A DBA's journey to running SQL Server on Kubernetes
Adjusting Pod Eviction Timings in Kubernetes
07:21 MIN
Answering questions on data volume, challenges, and databases
Remote Driving on Plant Grounds with State-of-the-Art Cloud Technologies
04:13 MIN
Q&A on performance, parallelism, and organizational impact
Convert batch code into streaming with Python
03:43 MIN
Q&A on implementation details and technology choices
Challenges for omnichannel applications at ALDI: Data distribution and offline capabilities
05:20 MIN
Overcoming challenges of data size and security
Web-based Information Visualization
15:13 MIN
Q&A on security, custom functionality, and performance
Anvil: Full Stack Web Apps With Nothing But Python
04:17 MIN
Meeting modern application and data platform demands
Tomorrow's cloud data platforms - fully managed database-as-a-service (DBaaS)
02:23 MIN
Building a one-way data pipeline for analytics
From event streaming to event sourcing 101
Featured Partners
Related Videos
49:57Uncharted Territories of Web Performance - Andrew Burnett-Thompson and David Burleigh
Andrew Burnett-Thompson & David Burleigh
24:22Database Magic behind 40 Million operations/s
Jürgen Pilz
42:081, 2, 3... Fastify!
Matteo Collina
27:27Things I learned while writing high-performance JavaScript applications
Michele Riva
28:12Scaling: from 0 to 20 million users
Josip Stuhli
37:21The Data Mesh as the end of the Datalake as we know it
Mario Meir-Huber
22:32Swapping Low Latency Data Storage Under High Load
George Asafev
30:02Lessons learned from building a thriving Vue.js SaaS application
Abdelrahman Awad
Related Articles
View all articles.gif?w=240&auto=compress,format)
From learning to earning
Jobs that call for the skills explored in this talk.
nono
£75-120K
Intermediate
API
Node.js
Grafana
GraphQL
+9
Huxley Associates
Amsterdam, Netherlands
ETL
GIT
Google BigQuery
Continuous Integration
La Collective
Canton de Nantes-1, France
Remote
Intermediate
GIT
Python
Data analysis
Continuous Integration
LEVY PROFESSIONALS
Amsterdam, Netherlands
Remote
Tableau
Data analysis