Databricks Developer

Scigon Solutions

Franklin Park, United States

3 months ago

Apply on dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Compensation

$134,910.0 - $149,900.0

Working hours

Regular working hours

Job source

dice.com

Tech stack

Amazon Web Services Business Analytics Applications Microsoft Azure Cloud Computing Code Review Databases Information Engineering Data Governance Data Integration Extract Transform Load (ETL) Data Migration Data Security

+20 more

Data Warehousing Enterprise Architecture Framework Github Apache Hive Operational Databases Parquet Google Cloud Cloud Platform System Sql Optimization Apache Spark Reliability of Systems Data Lakes Pyspark Information Technology Integration Frameworks Cloud Migration Terraform Data Pipelines Databricks Programming Languages

Job description

Design, develop, and maintain scalable data pipelines and ELT/ETL workflows using Databricks and Apache Spark (PySpark/Spark SQL)
Build and optimize modern lakehouse architectures using Delta Lake and related data formats (e.g., Parquet, Iceberg)
Develop production-grade data workflows using Databricks Workflows and/or declarative pipeline frameworks
Implement data governance, security, and access controls using Unity Catalog and related enterprise frameworks
Support data migration and modernization initiatives, transitioning legacy data warehouse systems to cloud-based platforms
Apply medallion architecture principles (bronze, silver, gold layers) in scalable solution design
Implement CI/CD pipelines and infrastructure-as-code for Databricks environments (e.g., Terraform, Databricks Asset Bundles, GitHub Actions)
Monitor, troubleshoot, and resolve production data pipeline issues to ensure system reliability and performance
Collaborate with technical and business stakeholders to gather requirements and translate them into scalable data solutions
Participate in code reviews, testing, documentation, and deployment activities
Contribute to the continuous improvement of data engineering standards, tools, and best practices
Provide technical support during solution implementation and stabilization phases
Perform other duties as required

Requirements

Bachelor s degree in Engineering, Information Technology, Computer Science, or a related field (or equivalent experience)
5+ years of data engineering experience, with at least 4+ years focused on Databricks
Strong proficiency in Apache Spark (PySpark and/or Spark SQL) and advanced SQL
Experience designing and implementing solutions on AWS, Azure, or Google Cloud Platform
Proven experience in cloud modernization, data migration, and data warehousing initiatives
Strong understanding of data security, messaging patterns, ELT processes, and data integration architectures
Hands-on experience with cloud computing, database technologies, and data processing frameworks
Ability to prioritize tasks, manage multiple responsibilities, and contribute to high-performing, collaborative teams
Excellent verbal and written communication skills, with the ability to engage effectively across technical and business audiences
Highly analytical, detail-oriented, and self-motivated, with a strong sense of ownership and urgency
Demonstrated ability to quickly learn and apply new technologies to support enterprise data solutions, * Experience facilitating technical workshops with both technical and business stakeholders to drive solution alignment and value delivery
Familiarity with advanced analytics tools, programming languages, and data visualization platforms
Experience with data integration / EAI frameworks, * Databricks certification
Experience in business analytics design and development, including exposure to best-in-class platforms and tools
Working knowledge of business processes and enterprise systems
Strong skills in change management, communication, and business process improvement