Python/Spark Or Ai Developer

Hrbaires
A Coruña, Spain
1 day ago
Apply on www.buscojobs.com.es
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
€80,000 - €120,000
Working hours
Regular working hours
Languages
English

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Apache HTTP Server Microsoft Azure Data Governance Apache Hive Python (Programming Language) Microsoft SQL Server Cloudera SQL Databases Transact-SQL Large Language Models
+9 more
Apache Spark Electronic Medical Records Fastapi Pytest Data Lakes Pyspark Kubernetes Azure Synapse Analytics Docker

Job description

Location:Europe (Remote) - must work South Africa hours (UTC+2)Employment:Full-timeSeniority:Mid-Senior LevelSalary:$80,000 - $120,000 (estimate; client hasn’t disclosed range)About The CompanyA fast-growing privacy-preserving data collaboration fintech (Series A, backed by a global credit bureau and a major retailer), with teams spanning South Africa, the UK, the US, the UAE and Brazil. Banks and lenders use the platform to work with retailer and telco data without raw personal data changing hands - mainly to help people with no credit history access credit.The roleThe Data Team Is Migrating Legacy SQL Server / Azure Synapse Stored?procedure Pipelines Onto Open?source Apache Spark And Delta Lake. You WillRebuild T?SQL logic as typed, tested PySpark pipelinesProve the new pipelines match legacy outputsBuild FastAPI services that run and report on pipeline jobsAdd provider?neutral LLM features (local and hosted models) into data workflowsSpec-first codebase: design docs and ADRs, with tests and documentation shipped alongside every change.Core stackPython, PySpark, Spark SQL, Delta Lake, FastAPI, pytest, Docker, T?SQL, LLM integrationRequiredModern typed Python (3.10+) as primary production language, with type hints passing pyright/mypy and Pydantic models - writes maintainable, tested code, not only notebooks2+ years production Apache Spark / PySpark - DataFrame API and Spark SQL, including partitioning and performance tuningBased in Europe or South Africa, able to work South Africa hours (UTC+2)Nice to haveSQL Server / Synapse migration to Spark with validated output parityOpen?source or self?managed Spark (Kubernetes, EMR, Dataproc, Spark Connect)FastAPI, LLM integration, privacy/data governance (PII tokenisation, GDPR/POPIA), fintech/banking/credit backgroundSkills: apache spark,credit,fastapi,pipelines,apache,azure,fintech,data,sql,python,spark,english#J-*****-Ljbffr

Requirements

Skills: apache spark,credit,fastapi,pipelines,apache,azure,fintech,data,sql,python,spark,english #J-*****-Ljbffr

About the company

Rebuild T?SQL logic as typed, tested PySpark pipelines Prove the new pipelines match legacy outputs Build FastAPI services that run and report on pipeline jobs Add provider?neutral LLM features (local and hosted models) into data workflows Spec-first codebase: design docs and ADRs, with tests and documentation shipped alongside every change. Core stack Python, PySpark, Spark SQL, Delta Lake, FastAPI, pytest, Docker, T?SQL, LLM integration Required Modern typed Python (3.10+) as primary production language, with type hints passing pyright/mypy and Pydantic models - writes maintainable, tested code, not only notebooks 2+ years production Apache Spark / PySpark - DataFrame API and Spark SQL, including partitioning and performance tuning Based in Europe or South Africa, able to work South Africa hours (UTC+2) Nice to have SQL Server / Synapse migration to Spark with validated output parity Open?source or self?managed Spark (Kubernetes, EMR, Dataproc, Spark Connect)

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.buscojobs.com.es
Prepare application

Good distractions

Loading talks and stories from around this role…