In this talk we will introduce how to use the popular cloud service Databricks for
hosting Apache Spark applications for distributed data processing in combination with
Apache Airflow, an orchestration framework for ETL batch workflows. After a brief
exploration of the Databricks Workspace and the fundamentals of Airflow we will take a
deeper look into the functionality Databricks provides in Airflow for orchestrating its
workspace. Afterwards, we will find out how to extend and customize that functionality to
manage virtually every aspect of the Databricks Workspace from Airflow.
The talk does not require any prior knowledge of Databricks, Spark or Airflow but it does
assume familiarity with the fundamentals of the Python programming language especially
object oriented programming and REST api requests. The actual distributed data processing
with Apache Spark itself is not the focus of this talk.

DeepFace is the most popular facial recognition library nowadays. Users can build and run facial recognition with a few lines of code. In this talk, we are going to unbox DeepFace to understand how deep face recognition working.

In March 2020 the world is completely blocked and people are lining up to shop or to the pharmacy or to buy basic necessities.

There have been many initiatives and among these I have created a worldwide map that allows anyone to check the estimated waiting times of supermarkets, pharmacies and places of interest.

In addition to this, I gave people the opportunity to check waiting times and correct them through a crowdsourcing mechanism.

All this, to be fast in development and in responding to requests, has exploited Redis with its geospatial indexes.

The opensource project has obtained more than 2Mln visits in about 3 months of life, until June 2020 when the pandemic slowed down.

In this talk we will see the architecture and the problems I encountered and solved with Redis.