> Markdown version of [/jobs/ext/1828242-software-developer-iii](https://www.wearedevelopers.com/jobs/ext/1828242-software-developer-iii). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Software Developer III

- **Company:** Seneca Resources
- **Location:** Arlington, VA, United States (Remote available)
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Business Analytics Applications, Microsoft Azure, Big Data, Cloud Engineering, Cluster Analysis, Code Review, Information Systems, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Database Development, Dimensional Modeling, Distributed Systems, Information Systems Security Architecture Professional, Python (Programming Language), Metadata, Microsoft SQL Server, Pair Programming, Performance Tuning, Role-Based Access Control, Power BI, Azure Active Directory, Cloud Services, Standard Sql, Azure Data Lake, SQL Stored Procedures, SQL Databases, Enterprise Data Management, Google Cloud, Azure Data Factory, Autoscaling, Apache Spark, Data Layers, Data Lakes, Pyspark, Information Technology, Star Schema, Data Pipelines, Serverless Computing, Databricks
- **Published:** July 13, 2026
- **Apply:** https://public-rest40.bullhornstaffing.com/rest-services/R0SYP/query/JobBoardPost?where=id=47410&fields=id,title,publishedCategory(id,name),address(city,state),employmentType,dateLastPublished,publicDescription,isOpen,isPublic,isDeleted

## About the Role

* Bachelor's degree in Computer Science, Engineering, Information Systems, or a related technical discipline.
 * 5+ years of Data Engineering, Data Platform Engineering, or Big Data development experience.
 * 5+ years of experience working with cloud platforms such as Microsoft Azure, AWS, or Google Cloud Platform (GCP).
 * Hands-on experience designing and implementing Databricks Lakehouse solutions in enterprise environments.
 * Strong expertise with Apache Spark, PySpark, Delta Lake, Databricks SQL, Delta Live Tables (DLT), and Databricks Workflows.
 * Experience migrating enterprise data warehouses from SQL Server or similar relational platforms to cloud-native analytics platforms.
 * Experience building scalable ETL/ELT pipelines and metadata-driven ingestion frameworks.
 * Strong knowledge of dimensional modeling, star schema design, enterprise data warehousing, and incremental data loading.
 * Experience optimizing Power BI datasets and Databricks SQL Warehouses for enterprise reporting.
 * Experience implementing Unity Catalog, Microsoft Entra ID (Azure AD), RBAC, row-level security, and column-level security.
 * Strong SQL and Python programming skills.
 * Experience leading technical projects, mentoring engineers, and conducting architecture reviews.
 * Excellent analytical, troubleshooting, documentation, and communication skills.

Preferred Qualifications

 * Experience with Azure Data Lake Storage (ADLS).
 * Experience designing enterprise-scale data governance and security architectures.
 * Knowledge of distributed computing and cloud-native analytics platforms.
 * Experience monitoring and optimizing Databricks cost and performance (DBU optimization).
 * Familiarity with enterprise data quality frameworks and metadata-driven architecture.
 * Microsoft Azure and/or Databricks certifications are highly desirable.

## Description

Seneca Resources is seeking an experienced Senior Databricks Migration Engineer to support a large-scale enterprise data modernization initiative. This is an excellent opportunity for a senior-level Data Engineer or Databricks Architect who is passionate about cloud data platforms, scalable analytics, and modern Lakehouse architecture. In this role, you will lead the migration of legacy SQL Server and Azure Data Factory (ADF) solutions to a modern Databricks Lakehouse platform. You will design high-performance data pipelines, optimize enterprise-scale analytics, establish governance and security standards, and mentor engineering teams through the transition from traditional SQL development to distributed Spark-based data engineering. This position offers the opportunity to influence enterprise architecture, drive cloud modernization efforts, and implement best practices that will shape the organization's future data platform., * Lead migration of legacy SQL Server stored procedures and Azure Data Factory (ADF) pipelines to Databricks Lakehouse using Delta Lake architecture.

 * Design scalable Lakehouse solutions utilizing Bronze, Silver, and Gold data layers.
 * Build reusable ETL/ELT frameworks using PySpark, Delta Live Tables (DLT), and Databricks Workflows.
 * Develop optimized dimensional models, star schemas, and Gold Layer datasets to maximize Microsoft Power BI performance.
 * Optimize Databricks SQL Warehouses for DirectQuery and Import mode workloads.
 * Implement advanced Databricks performance optimization techniques including Z-Ordering, Liquid Clustering, Data Skipping, Materialized Views, and file optimization.
 * Establish governance standards for cluster sizing, auto-scaling, serverless SQL compute, and cost optimization.
 * Create monitoring dashboards to analyze Databricks Unit (DBU) consumption and improve platform efficiency.
 * Design secure enterprise data governance using Unity Catalog, Microsoft Entra ID (Azure Active Directory), RBAC, row-level security, and column-level security.
 * Define Delta Lake partitioning strategies and storage optimization best practices.
 * Lead technical design sessions, architecture reviews, pair programming, mentoring, and code reviews.
 * Produce technical documentation, architecture diagrams, migration playbooks, and operational standards.
 * Drive knowledge transfer initiatives to ensure long-term operational success for internal engineering teams.
 * Collaborate with business stakeholders, architects, developers, and analytics teams to deliver scalable data solutions.
 * Communicate effectively with both technical and non-technical audiences while delivering projects on schedule.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [A Data Mesh needs Open Metadata](https://www.wearedevelopers.com/videos/505-a-data-mesh-needs-open-metadata)
- [REST, GraphQL, gRPC, and more: A comparison of modern API styles](https://www.wearedevelopers.com/videos/100247-rest-graphql-grpc-and-more-a-comparison-of-modern-api-styles)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [What’s the Difference between a Junior, Mid, and Senior Developer?](https://www.wearedevelopers.com/magazine/238-what-s-the-difference-between-a-junior-mid-and-senior-developer)