Senior Data Engineer

• Design, implement, and maintain ELT/ETL pipelines for efficient processing of source data in Azure Synapse and Azure Machine Learning (using SDK V1 and SDK V2)
• Review, maintain, and improve existing architecture and pipelines, including periodic audits to address bottlenecks, deprecated dependencies, and architecture drift.
• Establish quality controls for maintaining all pipelines, and introduce error handling, logging mechanisms, and validation checks.
• Incorporate source control for all pipelines and data analytics codebases to enable iterative code development while ensuring data architecture stability.
• Optimize the ingestion, processing, and storage of a wide variety of datasets and data types, including modern columnar formats such as Parquet.
• Design, implement, and maintain an efficient, secure, stable, and flexible data architecture that supports products and end-users, with all assets managed via source control.
• Develop self-service capabilities for analysts to query and export data for investigations and audits.
• Coordinate with data scientists to ensure the architecture efficiently supports machine learning algorithms and data pipelines in Azure Machine Learning.
• Assist with data products by providing highly skilled and authoritative expertise on data engineering methods and best practices, including code-first development approaches and modern pipeline design patterns.
• Develop robust standard operating protocols (SOPs) dictating the authoring, development, validation, publishing, execution, and monitoring of all data pipelines and assets in Azure environment.
• Provide detailed documentation of the data architecture, including data dictionaries, ER diagrams, and pipeline process maps.
• Maintain and expand the environment with additional datasets and services upon request, following a defined intake and testing process prior to production deployment.
• Stay current with emerging AI tools relevant to data engineering, and contribute to exploratory efforts evaluating automation and LLM-assisted capabilities.

Do you have experience in T-SQL?, Do you have a Bachelor’s degree?, * Five (5) years of hands-on experience in each of the following:

a. Maintaining SQL databases and conducting advanced operations in SQL and T-SQL.

b. Designing, implementing, and maintaining ELT/ETL processes in cloud-based data analytics environments.

1. Three (3) years of hands-on experience in each of the following:

A. Working in Azure Synapse and Azure Machine Learning, with the modern data stack. Certifications preferred (DP-203 or equivalent).

B. Manipulating data in Python. Pandas required. PySpark/Polars preferred. Experience developing reusable, modular code preferred.

Preferred experience:

• Implementing pipelines and infrastructure using code-first approaches (Python SDK, CLI, REST APIs, or IaC tooling)
• Implementing source control and CI/CD workflows
• Demonstrated familiarity with AI coding assistants and LLM integration patterns, * Bachelor’s (Required)

$90,000 - $110,000 a year - Full-time, Pulled from the full job description

• 401(k) 4% Match
• 401(k)
• Health insurance
• 401(k) matching
• Paid time off
• Vision insurance
• Dental insurance, * Base salary based on job, experience and skill level.
• Health benefits include medical, dental, and vision plans.
• 401(k) with 4% company match.
• 10 days Paid time off (PTO) and eleven (11) paid federal holidays.

Pay: $90,000.00 - $110,000.00 per year, * 401(k)

• 401(k) matching
• Health insurance
• Paid time off

Application Question(s):

• What is your work status: US Citizen, Permanent resident, H1B, OPT?
• Do you currently hold an active security clearance?
• What is your expected annual salary?