WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
WeAreDevelopers LIVE β€’ Nov 30, 2020

Hate organising your photos? Try it with 5 Terabytes

Arto Liukkonen

How do you store and index 5TB of RAW photos without blowing your cloud budget? Discover a hybrid Raspberry Pi and AWS architecture costing under $15 a month.

Pause
Mute Enter Fullscreen
#1 about 4 min

Managing massive volumes of raw digital camera photography

A massive collection of raw photography files creates a need for an accessible and scalable backup solution.

#2 about 4 min

Evaluating commercial cloud storage costs for large archives

Comparing commercial storage options reveals that standard tiers are too expensive for multi-terabyte archives.

#3 about 5 min

Utilizing deep archive storage for affordable digital backups

Archiving raw files in deep storage provides incredibly cheap capacity with acceptable retrieval delays.

#4 about 3 min

Extracting preview images and metadata from raw files

Generating smaller thumbnails and copying metadata enables a browsing interface without fetching massive raw files.

#5 about 4 min

Using a local hardware device for image processing

A local script handles image resizing and synchronization to avoid native binary issues in serverless functions.

#6 about 4 min

Implementing automated object and face recognition for images

Integrating artificial intelligence services allows automatic categorization of occurrences and specific individuals across thousands of photos.

#7 about 3 min

Handling service rate limits and cloud execution costs

Throttling upload speeds prevents errors from strict quotas while free tiers help mitigate processing expenses.

#8 about 4 min

Demonstrating the final architecture and web interface workflow

Connecting local processing to cloud services powers a secure web application for browsing tagged photography galleries.

#9 about 2 min

Integrating the backup gallery with a portfolio website

Tagged image metadata automatically populates a public portfolio site using cloud storage triggers and serverless functions.

#10 about 2 min

Summarizing the total monthly cost of the serverless architecture

A combination of deep archiving and serverless compute reduces massive data storage expenses to mere dollars a month.

Matching moments

4:38 min

Architecting common use cases via serverless design

Oliver Arafat Β· LIVE

3:35 min

Integrating cloud capabilities in a hybrid architecture

Jens Happe Jens Happe Β· LIVE

5:08 min

Automating data collection and managing crowdsourced training image sets

Kris Howard Β· LIVE

3:29 min

Optimizing software architectures to minimize infrastructure resource drain

Hendrik LΓΆsch Hendrik LΓΆsch Β· World Congress 2024

2:11 min

Integrating advanced cloud functionality into on-premise applications

Adam Bien Β· World Congress 2021

3:58 min

Struggling with ungoverned data lakes and massive storage costs

KateΕ™ina ŠčavnickΓ‘ KateΕ™ina ŠčavnickΓ‘ Β· World Congress 2025

More from WeAreDevelopers LIVE

WeAreDevelopers LIVE - Is CSS Catching Up to Flash? thumbnail
1:03:33
10 days ago

WeAreDevelopers LIVE - Is CSS Catching Up to Flash?

Chris Heilmann, Daniel Cranney, Bramus Van Damme

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along thumbnail
1:21:17
17 days ago

WeAreDevelopers LIVE - Project Chopin: Make Your Team and Agents Play Along

Chris Heilmann, Daniel Cranney, Krzysztof CieΕ›lak

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak thumbnail
4:23
18 days ago

Fake or News - AI Toothbrushes, Carrier Pigeon Tech & Life on Mars - Krzysztof Cieślak

Chris Heilmann, Daniel Cranney

WeAreDevelopers LIVE - Open Source Agent Toolkits thumbnail
1:00:38
24 days ago

WeAreDevelopers LIVE - Open Source Agent Toolkits

Chris Heilmann, Daniel Cranney, Eddie Jaoude

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude thumbnail
6:16
25 days ago

Fake or News - AI Music Detectors, Translating Ancient Languages & Tiny Heads in Teslas - Eddie Jaoude

Chris Heilmann, Daniel Cranney

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas thumbnail
5:46
about 1 month ago

Fake or News: AI Agents Hire Each Other, Spider-Man Ads in Cars & RAM Thieves - Marc Plogas

Chris Heilmann, Daniel Cranney, Marc Plogas

Related videos

How we built an AI-powered code reviewer in 80 hours thumbnail
28:07
World Congress 2025 β€’ about 1 year ago

How we built an AI-powered code reviewer in 80 hours

Yan Cui

Reliable scalability: How Amazon.com scales on AWS thumbnail
22:24
World Congress 2024 β€’ about 2 years ago

Reliable scalability: How Amazon.com scales on AWS

Florian Mair

Protector Of The Realm thumbnail
57:44
LIVE β€’ over 3 years ago

Protector Of The Realm

Sani Yusuf

Machine Learning for Software Developers (and Knitters) thumbnail
49:45
LIVE β€’ over 5 years ago

Machine Learning for Software Developers (and Knitters)

Kris Howard

Functions Triggers using Azure Event Grids in Azure Blob Storage thumbnail
47:38
LIVE β€’ almost 6 years ago

Functions Triggers using Azure Event Grids in Azure Blob Storage

Menaka Baskerpillai

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML thumbnail
39:58
LIVE β€’ almost 6 years ago

Leverage Cloud Computing Benefits with Serverless Multi-Cloud ML

Linda Mohamed

Related articles

From learning to earning

Jobs that call for the skills explored in this talk.