Principal Data Engineer

We are seeking a highly experienced Data Engineer with deep hands-on expertise in Databricks and AWS to help design, build, and evolve enterprise-scale data solutions. This individual will work across data engineering, platform engineering, and architecture to develop scalable data pipelines, establish modern engineering patterns, and support the continued adoption of Databricks across the organization. The role can be aligned at the Senior or Principal level based on experience, with opportunities for a more senior candidate to provide broader technical leadership, influence architecture, and help establish enterprise Databricks standards., * Design, develop, and maintain scalable enterprise data solutions using Databricks, Spark, SQL, and AWS

• Build and optimize data ingestion, transformation, orchestration, and integration pipelines
• Develop reusable frameworks and engineering components that improve scalability, maintainability, and adoption across teams
• Design solutions using modern Databricks capabilities, including Delta Lake, Medallion Architecture, Unity Catalog, Databricks Workflows, and Databricks SQL
• Partner with architects, engineers, product owners, and business teams to translate requirements into scalable technical solutions
• Participate in architecture and solution-design discussions and provide technical recommendations
• Support modernization efforts involving migration from legacy ETL and data warehouse technologies to Databricks
• Establish and promote best practices around data quality, performance, security, governance, observability, and cost optimization
• Troubleshoot complex production and performance issues and implement sustainable solutions
• Build and support production-grade engineering practices, including automated testing, CI/CD, and infrastructure as code
• Mentor other engineers and contribute to technical standards, documentation, and engineering best practices
• For more senior candidates, provide technical leadership across Databricks architecture, platform adoption, modernization, and enterprise engineering standards

• 5-8+ years of experience in data engineering, platform engineering, data architecture, or related areas
• 3-5+ years of hands-on Databricks experience in production environments
• Strong hands-on experience with AWS cloud technologies, including services such as S3, Lambda, Glue, SQS, IAM, CloudWatch, and related services
• Advanced experience with SQL, Spark, PySpark, and distributed data processing
• Strong knowledge of the Databricks Lakehouse Platform, including:
• Delta Lake
• Medallion Architecture
• Unity Catalog
• Databricks SQL
• Workflows
• Notebooks
• Experience designing and developing enterprise-scale ETL/ELT pipelines and data integration solutions
• Strong understanding of data modeling, data quality, governance, and enterprise data architecture
• Experience optimizing Spark and SQL workloads for performance and scalability
• Experience with Terraform or other infrastructure-as-code technologies
• Experience with Git-based development and CI/CD practices
• Ability to independently solve complex technical problems while collaborating across engineering and architecture teams
• Strong communication skills with the ability to explain technical concepts to both technical and business stakeholders

Desired Skills:

• Experience with advanced Databricks capabilities such as Delta Live Tables, Asset Bundles, Serverless Compute, Lakehouse Federation, or Data Intelligence
• Experience leading or contributing to enterprise Databricks migrations or modernization initiatives
• Experience with streaming or real-time data technologies such as Kafka, MSK, or Kinesis
• Experience with AI/ML workloads within Databricks
• Databricks Professional or Solutions Architect certification
• AWS Solutions Architect or Data Engineer certification
• Healthcare or health insurance industry experience
• Familiarity with healthcare interoperability standards such as FHIR

Genesis10 is currently seeking a Senior / Principal Data Engineer for a 6+ month hybrid contract to hire opportunity with a Regional Health Insurance Provider located in Eagan, MN., Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals.

For contract roles, Genesis10 offers the benefits listed below. If this is a perm-placement opportunity, our recruiter can talk you through the unique benefits offered for that particular client. Benefits of Working with Genesis10:

• Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years
• The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years
• Access to an experienced, caring recruiting team (more than 7 years of experience, on average)
• Behavioral Health Platform
• Medical, Dental, Vision
• Health Savings Account
• Voluntary Hospital Indemnity (Critical Illness & Accident), For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all of our available career opportunities, please visit us at our website.