> Markdown version of [/jobs/ext/3051583-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3051583-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Wells Fargo - **Location:** Charlotte, NC, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Agile Methodology, Airflow, Amazon Web Services, Application Frameworks, Automation of Tests, Microsoft Azure, BigQuery, Cloud Computing, Cloud Engineering, Cloud Storage, Continuous Integration, Directed Acyclic Graph (Directed Graphs), Data Validation, Data Security, Data Vault Modeling, Data Flow Control, Apache Hive, Identity and Access Management, Python (Programming Language), Open Source Technology, Cloudera, SQL Databases, Workflow Management Systems, Data Logging, Google Cloud, Data Ingestion, Cloud Monitoring, Apache Spark, Build Server, Git Flow, Performance Monitor, Data Management, Data Pipelines, Apache Beam - **Published:** September 24, 2026 - **Apply:** https://www.beyondcharlotte.com/job.asp?id=3402958974&tx=HT7468TYV&pt=1&aff=0B19D771-A501-4A5E-8338-2A822B784D54&utm_source=Job%20Feed&utm_medium=textkernel&utm_campaign=DE&utm_term=0B19D771-A501-4A5E-8338-2A822B784D54 ## About the Role * 5+ years of Database Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education * 5+ years of data management experience within Public Cloud (GCP, AWS, Azure) * 5+ years of hands on experience of Python or Java, plus Spark SQL for building data pipelines, libraries, and automation tooling. * 5+ years with orchestration tools (Cloud Composer/Airflow) and CI/CD (Cloud Build, Git-based workflows) for data workloads, * Experience with logging/monitoring stacks (Cloud Logging, Cloud Monitoring, error reporting, metrics dashboards * Experience with automated testing, data quality checks, and monitoring for pipelines and platform services. * Knowledge of cloud architecture principles (networking, security, IAM, reliability, cost management). * Experience with core GCP data services: BigQuery, Dataflow/Apache Beam, Dataproc, Pub/Sub. * Experience with Agile transformations and technology roadmaps. * Experience working with onshore and offshore teams. ## Description * Design and implement scalable, secure data platforms on Google Cloud using managed services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer).) * Build reusable frameworks and tooling (ingestion, transformation, quality, orchestration) that can be adopted by multiple product and domain teams. * Enable self-service data consumption and governance by standardizing patterns, templates, and platform capabilities rather than one-off pipelines. * Design logical and physical data platform architectures leveraging BigQuery, Dataflow/Apache Beam, Dataproc/Spark, Pub/Sub, and Cloud Storage. * Define and implement standardized ingestion, transformation, and serving patterns (batch and streaming) as reusable blueprints. * Optimize cost, performance, and reliability of GCP data workloads (partitioning, clustering, storage classes, autoscaling strategies). * Build opinionated data ingestion frameworks (e.g., config-driven pipelines, connectors, schema handling, error handling) on top of Dataflow, Dataproc, or Composer. * Develop shared transformation libraries in Python/SQL/Beam (e.g., common SCD patterns, data quality checks, masking/tokenization routines). * Provide orchestration capabilities via Cloud Composer or Cloud Workflows with reusable DAGs/templates and CI/CD integration. * Implement robust data modeling (dimensional, data vault, or canonical models) and semantic layers in BigQuery and related tools. * Enforce data quality, lineage, and observability using standardized metrics, validation rules, and monitoring dashboards. * Apply security and governance controls: IAM, VPC-SC, CMEK, row/column-level security, and policy-driven access patterns * Partner with domain data engineers, analytics, and ML teams to onboard use cases onto platform services and frameworks * Document patterns, runbooks, and best practices, and provide enablement through workshops and code examples. * Contribute to platform roadmap, tool selection, and evaluation of new GCP services and open-source components, Employees support our focus on building strong customer relationships balanced with a strong risk mitigating and compliance-driven culture which firmly establishes those disciplines as critical to the success of our customers and company. They are accountable for execution of all applicable risk programs (Credit, Market, Financial Crimes, Operational, Regulatory Compliance), which includes effectively following and adhering to applicable Wells Fargo policies and procedures, appropriately fulfilling risk and compliance obligations, timely and effective escalation and remediation of issues, and making sound risk decisions. There is emphasis on proactive monitoring, governance, risk identification and escalation, as well as making sound risk decisions commensurate with the business unit's risk appetite and all risk and compliance program requirements. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Why Git Still Matters](https://www.wearedevelopers.com/videos/100288-why-git-still-matters) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)