WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE • Nov 30, 2020

Hate organising your photos? Try it with 5 Terabytes

Arto Liukkonen

How do you store and index 5TB of RAW photos without blowing your cloud budget? Discover a hybrid Raspberry Pi and AWS architecture costing under $15 a month.

Pause
Mute Enter Fullscreen
#1 about 4 min

Managing massive volumes of raw digital camera photography

A massive collection of raw photography files creates a need for an accessible and scalable backup solution.

#2 about 4 min

Evaluating commercial cloud storage costs for large archives

Comparing commercial storage options reveals that standard tiers are too expensive for multi-terabyte archives.

#3 about 5 min

Utilizing deep archive storage for affordable digital backups

Archiving raw files in deep storage provides incredibly cheap capacity with acceptable retrieval delays.

#4 about 3 min

Extracting preview images and metadata from raw files

Generating smaller thumbnails and copying metadata enables a browsing interface without fetching massive raw files.

#5 about 4 min

Using a local hardware device for image processing

A local script handles image resizing and synchronization to avoid native binary issues in serverless functions.

#6 about 4 min

Implementing automated object and face recognition for images

Integrating artificial intelligence services allows automatic categorization of occurrences and specific individuals across thousands of photos.

#7 about 3 min

Handling service rate limits and cloud execution costs

Throttling upload speeds prevents errors from strict quotas while free tiers help mitigate processing expenses.

#8 about 4 min

Demonstrating the final architecture and web interface workflow

Connecting local processing to cloud services powers a secure web application for browsing tagged photography galleries.

#9 about 2 min

Integrating the backup gallery with a portfolio website

Tagged image metadata automatically populates a public portfolio site using cloud storage triggers and serverless functions.

#10 about 2 min

Summarizing the total monthly cost of the serverless architecture

A combination of deep archiving and serverless compute reduces massive data storage expenses to mere dollars a month.

Matching moments

4:38 min

Architecting common use cases via serverless design

Oliver Arafat · LIVE

3:35 min

Integrating cloud capabilities in a hybrid architecture

Jens Happe Jens Happe · LIVE

5:08 min

Automating data collection and managing crowdsourced training image sets

Kris Howard · LIVE

3:29 min

Optimizing software architectures to minimize infrastructure resource drain

Hendrik Lösch Hendrik Lösch · WWC 2024

2:11 min

Integrating advanced cloud functionality into on-premise applications

Adam Bien · WWC 2021

3:58 min

Struggling with ungoverned data lakes and massive storage costs

Kateřina Ščavnická Kateřina Ščavnická · WWC 2025

More from WeAreDevelopers LIVE

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz thumbnail
5:43
5 days ago

Aliens in Quarantine, Python Deprecates Loops, and AI Buys Old Books - Zbyszek Tenerowicz

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Node and Package Security thumbnail
1:08:30
5 days ago

WeAreDevelopers LIVE - Node and Package Security

Chris Heilmann, Daniel Cranney, Zbyszek Tenerowicz

WeAreDevelopers LIVE - Streaming HTML thumbnail
1:10:26
12 days ago

WeAreDevelopers LIVE - Streaming HTML

Chris Heilmann, Daniel Cranney, Jad Joubran

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe thumbnail
1:02:34
19 days ago

WeAreDevelopers LIVE - CODE100 at WeAreDevelopers Europe

Chris Heilmann, Daniel Cranney, Justin Halsall, Werner Jeschko, Nicole Jeschko, Mirco Tomić

WeAreDevelopers LIVE - Web Performance in an AI World thumbnail
1:07:59
26 days ago

WeAreDevelopers LIVE - Web Performance in an AI World

Chris Heilmann, Daniel Cranney

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko thumbnail
5:55
about 2 months ago

FIFA Hack, Midjourney Does Ultrasounds, and Trampoline Traffic Lights - Nicole Jeschko and Werner Jeschko

Chris Heilmann, Daniel Cranney

Upcoming sessions on this topic

Open session

World Congress 2026 North America

Point. Ask. Answer. Building Vision into AI Live on Stage.

Kavya Sri Chennoju

Staff AI Engineer at Arm

Kavya Sri Chennoju
Open session

World Congress 2026 North America

You Can’t Re-Run Sunlight: Designing ML Data Architectures for Physical AI

An Phan

Senior Data Infrastructure Engineer @ Hippo Harvest

An Phan
Open session

World Congress 2026 North America

The $0.15/month Algo-Trader: Architecting High-Performance Serverless Trading Engines

Vishwak Thatikonda

Lead Full Stack Engineer at SDH Systems

Vishwak Thatikonda
Open session

World Congress 2026 North America

Optimizing consumption and cost of AI APIs in the enterprise

Derric Gilling

Vice President and General Manager, API Platform

Derric Gilling
Open session

World Congress 2026 North America

public void saveMoney(AI): The Developer's Guide to Unit Economics

Hrushikesh Pokala

Senior Software Engineer Lead at Equifax

Hrushikesh Pokala
Open session

World Congress 2026 North America

rm -rf: Horror Stories From Unsandboxed AI Agents (and How Docker Fixes This)

Rishab Kumar

Staff Developer Evangelist @ Twilio

Rishab Kumar

Related videos

How we built an AI-powered code reviewer in 80 hours thumbnail
28:07
WWC 2025 • 12 months ago

How we built an AI-powered code reviewer in 80 hours

Yan Cui

Reliable scalability: How Amazon.com scales on AWS thumbnail
22:24
WWC 2024 • almost 2 years ago

Reliable scalability: How Amazon.com scales on AWS

Florian Mair

Protector Of The Realm thumbnail
57:44
LIVE • about 3 years ago

Protector Of The Realm

Sani Yusuf

Machine Learning for Software Developers (and Knitters) thumbnail
49:45
LIVE • about 5 years ago

Machine Learning for Software Developers (and Knitters)

Kris Howard

Functions Triggers using Azure Event Grids in Azure Blob Storage thumbnail
47:38
LIVE • almost 6 years ago

Functions Triggers using Azure Event Grids in Azure Blob Storage

Menaka Baskerpillai

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML thumbnail
39:58
LIVE • over 5 years ago

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML

Linda Mohamed

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.