Data Cloud Engineer

Zachary Piper
United States
11 days ago
Apply on www.clearancejobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$120,000.0 - $140,000.0
Working hours
Regular working hours

Tech stack

Application Programming Interfaces (APIs) Agile Methodology Amazon Web Services Amazon S3 Microsoft Azure Big Data Cloud Engineering Cyber Security Information Systems Databases Continuous Integration Information Engineering
+27 more
Extract Transform Load (ETL) Data Warehousing DevOps Apache Hive JSON Python (Programming Language) Operational Databases Oracle (Applications) Role-Based Access Control Software Engineering SQL Databases Unstructured Data Data Logging Data Ingestion Azure Data Factory Change Data Capture SC Clearance Pyspark Information Technology Deployment Automation AWS Glue AWS Data Analytics Cloud Migration Azure Synapse Analytics Data Pipelines Amazon Redshift Databricks

Job description

Design, develop, and maintain scalable cloud-based ETL/ELT pipelines using Azure and AWS technologies. Build and optimize large-scale data transformations using PySpark and Spark SQL. Develop data ingestion frameworks supporting structured and unstructured data, including databases, APIs, JSON, and flat files. Implement full and incremental data loading strategies, including Change Data Capture (CDC). Design and support enterprise data lakes, data warehouses, and analytics environments. Implement data quality, governance, and security controls including schema validation and RBAC. Monitor, troubleshoot, and optimize production data pipelines. Support CI/CD implementation and automated deployments. Partner with cloud engineering, cybersecurity, DevOps, and software development teams.

Requirements

Active Secret Clearance. Bachelor’s degree in Computer Science, Engineering, Information Systems, Data Science, or related field. 8+ years of data engineering experience. Strong hands-on experience with PySpark, Spark SQL, Python, and SQL. Experience with Azure Synapse Analytics, Databricks, and/or AWS data services. Experience building and supporting production-grade ETL/ELT pipelines. Experience with data ingestion, integration, transformation, and CDC. Experience troubleshooting production data environments and implementing monitoring and logging., Experience with Azure Data Factory, AWS Glue, Redshift, Amazon S3, and Oracle. Experience supporting federal or DoD cloud modernization programs. Familiarity with RMF, STIG, and FedRAMP. Experience working in Agile development environments.

Benefits & conditions

Salary: $120,000 - $140,000 depending on experience

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.clearancejobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…