> Markdown version of [/jobs/ext/4571595-data-engineer-with-gcp-and-pyspark](https://www.wearedevelopers.com/jobs/ext/4571595-data-engineer-with-gcp-and-pyspark). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer with GCP and Pyspark - **Company:** ApTask - **Location:** Irving, TX, United States - **Experience:** Experienced - **Salary:** $114,400 - $124,800 - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Artificial Intelligence, Audit Trail, Big Data, BigQuery, Code Review, Continuous Integration, Information Engineering, Data Governance, Data Transformation, Database Queries, Python (Programming Language), Machine Learning, Meta-Data Management, Operational Data Store, Cloudera, Simple Data Format, Data Streaming, Parquet, Apache Spark, Pyspark, Machine Learning Operations, Software Version Control, Data Pipelines - **Published:** November 4, 2026 - **Apply:** https://www.thejobnetwork.com/job/9df5ec39-4d3e-4ad3-ab62-2381a883710b/data-engineer-with-gcp-and-pyspark ## About the Role * 3 years (L3) / 6 years (L4) hands-on data engineering in production environments. * Strong SQL skills and proficiency building data transformations at scale. * Practical experience with Spark (preferably PySpark) and big data file formats (Parquet; familiarity with Iceberg concepts). * Experience delivering reliable, scheduled pipelines with monitoring, alerting, and incident response. * Understanding of data cataloging, metadata, and lineage concepts, and why they matter for governance and reuse. * Exposure to GCP data services (e.g., Dataproc/Spark, BigQuery). Deep infra setup knowledge is not required; collaboration with central cloud teams is expected. * Solid software engineering practices: version control, CI/CD, testing, code reviews, and documentation. Preferred: * Experience building reusable pipeline frameworks or platform components adopted by multiple teams. * Familiarity with Starburst/Trino for federated query across hybrid environments. * Knowledge of data quality frameworks, reconciliation, and auditability in regulated or enterprise settings. * Python proficiency; Java experience a plus (willingness to work primarily in Python). * Background in operational data environments with time-bound SLAs and prod support rotations. * Exposure to AI/ML model lifecycle platforms or MLOps concepts (platform enablement rather than model authoring). Soft Skills: * High ownership, bias for action, and clear communication with technical and non-technical partners. * Comfort operating in a fast-scaling organization with evolving scope. has context menu ## Description * Design, build, and harden production-grade data pipelines for ingest, curation, reconciliation, DQ monitoring, lineage, and notifications. * Contribute to and extend the generic pipeline framework so federated teams can onboard data consistently and at speed. * Engineer data transformations primarily using PySpark on GCP (Dataproc/Sparkflow) and BigQuery as the main query engine. * Implement best practices for schema management, performance, reliability, and cost efficiency across large-scale datasets (Parquet, Iceberg). * Integrate with centralized data cataloging and governance (e.g., Dataplex/Knowledge Catalog or equivalent), and support automated lineage harvesting. * Collaborate with platform, security, and central cloud teams; support hybrid access patterns using Starburst where needed. * Operate with an ownership mindset in an on-time, production SLAs environment, ensuring resilient, observable, and recoverable data flows.