Data Cloud Engineer
Zachary Piper
Fort Meade, MD, United States
29 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.clearancejobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Compensation
$125,000.0 - $140,000.0
Working hours
Regular working hours
Job source
Tech stack
Agile Methodology
Amazon Web Services
Amazon S3
Data Analysis
Microsoft Azure
Big Data
Cloud Computing
Cloud Computing Security
Cloud Database
Cloud Engineering
Cyber Security
Information Systems
+51 more
Continuous Delivery
Continuous Integration
Data Validation
Information Engineering
Data Governance
Data Integration
Extract Transform Load (ETL)
Data Transformation
Data Migration
Data Security
Data Warehousing
Relational Databases
DevOps
Apache Hive
JSON
Python (Programming Language)
Meta-Data Management
Metadata Repositories
Performance Tuning
Query Optimization
Role-Based Access Control
Software Engineering
SQL Databases
Systems Integration
Unstructured Data
Enterprise Data Management
Data Logging
Cloud Platform System
High Performance Computing
Data Ingestion
Azure Data Factory
Apache Spark
Change Data Capture
SC Clearance
Data Lakes
Pyspark
Information Technology
Data Lineage
Low Latency
Deployment Automation
AWS Glue
Data Analytics
AWS Data Analytics
Cloud Migration
Cloud Integration
Software Coding
Restful APIs
Azure Synapse Analytics
Data Pipelines
Amazon Redshift
Databricks
Job description
- Design, develop, and maintain production-grade ETL/ELT pipelines, with a strong emphasis on hands-on, high-performance PySpark development and Spark SQL optimization.
- Build and maintain scalable data pipelines using Azure Synapse Analytics, Databricks, AWS Glue, and related cloud technologies.
- Develop ingestion frameworks for structured and unstructured data, including relational databases, flat files, JSON, nested data, and REST APIs.
- Implement full and incremental data loads, including CDC, late-arriving records, and rerunnable pipeline architectures.
- Design and maintain cloud data lakes, warehouses, and analytics-ready datasets supporting enterprise reporting and operational decision-making.
- Implement data quality and governance controls, including schema validation/enforcement, schema drift, RBAC, lineage, cataloging, and credential management.
- Monitor and troubleshoot pipelines for performance, latency, failures, logging, alerting, and operational reliability.
- Support CI/CD, automated deployments, rollback strategies, and environment promotions across cloud data environments.
- Collaborate with cybersecurity, cloud, DevOps, and software engineering teams to support RMF, STIG, FedRAMP, and DoD security requirements, data migrations, integrations, and Agile delivery., Keywords: Cloud Data Engineer, Data Engineer, Cloud Engineer, Data Cloud Engineer, Senior Data Engineer, Azure Data Engineer, AWS Data Engineer, PySpark, Spark, Apache Spark, Spark SQL, Python, SQL, ETL, ELT, ETL Pipelines, Data Pipelines, Data Engineering, Data Integration, Data Transformation, Data Ingestion, High-Performance Computing, Performance Optimization, Query Optimization, Azure, Microsoft Azure, AWS, Amazon Web Services, Azure Synapse, Azure Synapse Analytics, Synapse, Databricks, Azure Databricks, AWS Glue, Azure Data Factory, ADF, Amazon Redshift, Redshift, Amazon S3, S3, Data Lake, Data Lakes, Data Warehouse, Data Warehousing, Analytics, Data Analytics, Structured Data, Unstructured Data, Relational Databases, JSON, REST API, REST APIs, Flat Files, Nested Data, Full Loads, Incremental Loads, Incremental Loading, Change Data Capture, CDC, Late-Arriving Data, Rerunnable Pipelines, Data Quality, Data Validation, Schema Validation, Schema Enforcement, Schema Drift, Data Governance, Data Lineage, Data Catalog, Data Cataloging, RBAC, Role-Based Access Control, Credential Management, Pipeline Monitoring, Logging, Alerting, Troubleshooting, Pipeline Reliability, CI/CD, Continuous Integration, Continuous Deployment, DevOps, Automated Deployment, Deployment Automation, Environment Promotion, Rollback, Cloud Migration, Data Migration, Cloud Integration, Agile, RMF, Risk Management Framework, STIG, DISA STIG, FedRAMP, DoD, Department of Defense, DISA, Fort Meade, Secret Clearance, Active Secret, Security Clearance, Cleared Data Engineer, Secure Cloud, Secure Data Environment, Production-Grade Data Pipelines, Enterprise Data, Enterprise Data Infrastructure, Mission-Critical Data, Cloud Data Platform, Cloud Data Architecture
Requirements
- 10+ years of recent experience designing and operating scalable, production-grade data pipelines using Azure Synapse and/or Databricks, w/ a Bachelors degree in Computer Science, Data Science, Engineering, Information Systems, or a related field.
- Strong hands-on PySpark development experience is required, including demonstrated high-performance coding skills developing ETL pipelines; candidates without this experience will not be considered.
- Advanced experience with Spark SQL, Python, and SQL, including large-scale data transformation and optimization.
- Experience ingesting structured and unstructured data and implementing full/incremental loads, CDC, and rerunnable pipeline architectures.
- Strong experience with data quality, schema validation/enforcement, pipeline monitoring, logging, alerting, and troubleshooting.
- Experience implementing CI/CD practices and automated deployments for data pipelines and cloud environments.
- Experience with data governance, RBAC, and credential management in secure cloud environments.
- Hands-on experience with Azure and/or AWS data services, including Azure Synapse, Azure Data Factory, Databricks, AWS Glue, Redshift, and S3.
- Active Secret clearance*
Benefits & conditions
- Salary: $125,000 - $140,000 depending on experience
- Comprehensive Benefits: Medical, Dental, Vision, 401k Plan, PTO, Holidays, Sick Leave if required by law
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.clearancejobs.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦