> Markdown version of [/jobs/ext/626136-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/626136-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Ivory Cloud LLC - **Location:** Rockville, MD, United States - **Experience:** Expert - **Salary:** $155,000.0 - $200,000.0 - **Contract:** Temporary contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Airflow, Amazon Web Services, Data Analysis, Microsoft Azure, Big Data, Software as a Service, Cloud Computing, Cloud Storage, Continuous Integration, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Systems, Data Warehousing, Github, Integrated Development Environments, Python (Programming Language), Standard Sql, SQL Databases, Azure Service Bus, Cloud Platform System, Feature Engineering, Azure Data Factory, Sql Optimization, Snowflake, Apache Spark, Git, Data Layers, Data Lakes, Pyspark, Semi-structured Data, Apache Kafka, Machine Learning Operations, Video Streaming, Data Pipelines, Databricks - **Published:** June 19, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=4bd4b580c9734c2d ## About the Role Do you have experience in Technical architecture?, Do you have a Bachelor's degree?, * U.S. Citizenship is required and non-negotiable * 2 days a week in the Rockville, MD office are required and non-negotiable. * Ability to pass multiple background and employment checks - federal and local. * Bachelor's degree Technology, Business, or Related Field, * 5+ years of hands-on data engineering experience (or equivalent depth), including production-grade pipelines. * Strong experience building solutions using Databricks, Spark/PySpark, SQL, and Delta Lake. * Solid understanding of Lakehouse and Medallion architectures and when to apply them. * Advanced SQL skills and experience supporting analytics, BI, or operational reporting workloads. * Experience developing and operating ETL/ELT pipelines on Azure or another major cloud platform. * Familiarity with Git-based development and CI/CD practices for data pipelines (Azure DevOps or GitHub preferred). * Strong understanding of data modeling (relational + dimensional) for both operational and analytical workloads. * Working knowledge of data quality, lineage, governance, and security concepts in cloud environments. * Note: This role focuses on building data products on Databricks, not platform administration., * Experience with Databricks features such as Unity Catalog, Delta Live Tables, Workflows, Lakeflow, SQL Warehouse, or Mosaic AI. * Experience using tools that schedule and run data pipelines, such as Data Factory, Synapse pipelines, Airflow, Dagster. * Experience using modern SQL-based transformation tools such as dbt, Databricks SQL (SQL Warehouse), or Delta Live Tables * Exposure to AI/ML workflows: Azure OpenAI, embeddings, vector stores, feature engineering, or model training pipelines. * Knowledge of streaming technologies such as Kafka, Event Hubs, or Kinesis. * Familiarity with modern data warehouses (Snowflake, Synapse, Redshift) and how they integrate with a Lakehouse. Technical Environment Candidates should have experience with several of the following: * Databricks * Apache Spark * PySpark * SQL * Delta Lake * Python * dbt * Airflow * Azure Data Factory * Kafka * Snowflake * Git * CI/CD Pipelines * Cloud Platforms (Azure, AWS, or GCP) ## Description We are seeking a highly capable Senior Data Engineer to design, build, and optimize modern data pipelines and analytics platforms. This role is focused on leveraging Databricks as a development environment for large-scale data engineering, transformation, and analytics workloads-not platform administration. In support of a Department of Energy client, you will work closely with business stakeholders, analytics teams, software engineers, and AI/ML practitioners to build reliable, scalable, and high-performance data solutions that support reporting, operational intelligence, and advanced analytics initiatives., * Design, develop, and maintain scalable data pipelines using Azure Databricks and Apache Spark. * Build and optimize batch and streaming data processing solutions. * Develop robust ETL/ELT frameworks for structured and semi-structured data from SQL, APIs, SaaS apps, and Data Live Lake Storage. * Implement transformation logic using PySpark, SQL, and Delta Lake within a Lakehouse / Medallion architecture. * Integrate and model data from SQL, SaaS systems, and cloud storage into unified, analytics-ready and AI-ready structures. * Partner with analytics, BI, and AI/ML teams to deliver trusted datasets, semantic layers, and AI-ready data (features, vector-ready patterns, embeddings). * Improve data quality, observability, lineage, governance, and security in alignment with Azure and enterprise standards. * Optimize data performance, storage costs, and processing efficiency in cloud environments. * Participate in architecture discussions and contribute to the evolution of the organization's data platform and engineering best practices. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)