Sr. Data Engineer with Python and Azure
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+7 more
Job description
You will collaborate with technical and business partners to translate data requirements into efficient workflows, maintain data quality, and improve pipeline performance.
Requirements
This hands-on role is ideal for someone with strong Python, Apache Spark, and SQL skills who enjoys solving complex data challenges and delivering reliable production solutions., * 5+ years of hands-on experience developing and maintaining ETL pipelines in production environments. \n
-
Strong Python programming skills for data processing, transformation, and automation. \n
-
Practical experience using Apache Spark to process large datasets. \n
-
Advanced SQL skills, including complex queries and data transformations. \n
-
Solid understanding of data engineering principles, including scalability, reliability, data quality, and performance optimization. \n
-
Experience troubleshooting production data workflows and resolving technical issues. \n
-
Strong communication, problem-solving, and collaboration skills. \n, n \n
-
Experience with Microsoft Azure data services, particularly Azure Data Factory. \n
-
Experience developing or running Spark workloads in Databricks. \n
-
Familiarity with Power BI and how data pipelines support reporting and analytics.
Benefits & conditions
n \n
-
Design, build, and maintain production ETL pipelines that extract, transform, and integrate large datasets. \n
-
Develop data processing workflows using Python and Apache Spark. \n
-
Write and optimize complex SQL queries to support data transformation, validation, and analysis. \n
-
Partner with cross-functional teams to understand source data, define requirements, and deliver usable datasets. \n
-
Implement data quality checks to improve accuracy and consistency across pipelines. \n
-
Investigate pipeline failures, troubleshoot processing issues, and resolve performance bottlenecks. \n
-
Improve the scalability, reliability, and maintainability of existing data solutions. \n
-
Follow engineering best practices for version control, code documentation, and collaborative development. \n
-
Participate in an agile team environment and communicate progress, issues, and technical decisions clearly. \n
\n
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦