Lead Data Engineer

Wells Fargo
Charlotte, NC, United States
2 days ago
Apply on www.beyondcharlotte.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours

Tech stack

Java (Programming Language) Agile Methodology Airflow Amazon Web Services Application Frameworks Automation of Tests Microsoft Azure BigQuery Cloud Computing Cloud Engineering Cloud Storage Continuous Integration
+23 more
Directed Acyclic Graph (Directed Graphs) Data Validation Data Security Data Vault Modeling Data Flow Control Apache Hive Identity and Access Management Python (Programming Language) Open Source Technology Cloudera SQL Databases Workflow Management Systems Data Logging Google Cloud Data Ingestion Cloud Monitoring Apache Spark Build Server Git Flow Performance Monitor Data Management Data Pipelines Apache Beam

Job description

  • Design and implement scalable, secure data platforms on Google Cloud using managed services (BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer).)
  • Build reusable frameworks and tooling (ingestion, transformation, quality, orchestration) that can be adopted by multiple product and domain teams.
  • Enable self-service data consumption and governance by standardizing patterns, templates, and platform capabilities rather than one-off pipelines.
  • Design logical and physical data platform architectures leveraging BigQuery, Dataflow/Apache Beam, Dataproc/Spark, Pub/Sub, and Cloud Storage.
  • Define and implement standardized ingestion, transformation, and serving patterns (batch and streaming) as reusable blueprints.
  • Optimize cost, performance, and reliability of GCP data workloads (partitioning, clustering, storage classes, autoscaling strategies).
  • Build opinionated data ingestion frameworks (e.g., config-driven pipelines, connectors, schema handling, error handling) on top of Dataflow, Dataproc, or Composer.
  • Develop shared transformation libraries in Python/SQL/Beam (e.g., common SCD patterns, data quality checks, masking/tokenization routines).
  • Provide orchestration capabilities via Cloud Composer or Cloud Workflows with reusable DAGs/templates and CI/CD integration.
  • Implement robust data modeling (dimensional, data vault, or canonical models) and semantic layers in BigQuery and related tools.
  • Enforce data quality, lineage, and observability using standardized metrics, validation rules, and monitoring dashboards.
  • Apply security and governance controls: IAM, VPC-SC, CMEK, row/column-level security, and policy-driven access patterns
  • Partner with domain data engineers, analytics, and ML teams to onboard use cases onto platform services and frameworks
  • Document patterns, runbooks, and best practices, and provide enablement through workshops and code examples.
  • Contribute to platform roadmap, tool selection, and evaluation of new GCP services and open-source components, Employees support our focus on building strong customer relationships balanced with a strong risk mitigating and compliance-driven culture which firmly establishes those disciplines as critical to the success of our customers and company. They are accountable for execution of all applicable risk programs (Credit, Market, Financial Crimes, Operational, Regulatory Compliance), which includes effectively following and adhering to applicable Wells Fargo policies and procedures, appropriately fulfilling risk and compliance obligations, timely and effective escalation and remediation of issues, and making sound risk decisions. There is emphasis on proactive monitoring, governance, risk identification and escalation, as well as making sound risk decisions commensurate with the business unit’s risk appetite and all risk and compliance program requirements.

Requirements

  • 5+ years of Database Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education
  • 5+ years of data management experience within Public Cloud (GCP, AWS, Azure)
  • 5+ years of hands on experience of Python or Java, plus Spark SQL for building data pipelines, libraries, and automation tooling.
  • 5+ years with orchestration tools (Cloud Composer/Airflow) and CI/CD (Cloud Build, Git-based workflows) for data workloads, * Experience with logging/monitoring stacks (Cloud Logging, Cloud Monitoring, error reporting, metrics dashboards
  • Experience with automated testing, data quality checks, and monitoring for pipelines and platform services.
  • Knowledge of cloud architecture principles (networking, security, IAM, reliability, cost management).
  • Experience with core GCP data services: BigQuery, Dataflow/Apache Beam, Dataproc, Pub/Sub.
  • Experience with Agile transformations and technology roadmaps.
  • Experience working with onshore and offshore teams.

Benefits & conditions

  • This role is not eligible for Visa Sponsorship
  • Hybrid schedule (3 days in office, 2 days remote)
  • Work Transparently: You always deal in an honest, direct, and transparent way
  • Take Ownership: You embrace responsibility and find joy in having the answers
  • Learn More: You regularly self-educate and improve your skill set

About the company

Within COO Technology, Wells Fargo is seeking a Lead Data Engineer to help shape and scale our cloud-native data ecosystem. In this role, you will focus on Google Cloud Platform (GCP) services and frameworks, leading the design, build, and operation of reusable data capabilities that power analytics and AI at enterprise scale. The ideal candidate is passionate about standardized frameworks, self-service data platforms, and governance-by-design, enabling secure, reliable, and compliant data solutions across Google Cloud services.

The COO Technology group powers the firm’s most critical operations by modernizing and optimizing enterprise technology platforms that enable resiliency, regulatory excellence, data services, customer experience, and strategic execution across the Chief Operating Office., Wells Fargo maintains a drug free workplace. Please see our Drug and Alcohol Policy (https://www.wellsfargojobs.com/en/wells-fargo-drug-and-alcohol-policy) to learn more.

Wells Fargo Recruitment and Hiring Requirements:

a. Third-Party recordings are prohibited unless authorized by Wells Fargo.

b. Wells Fargo requires you to directly represent your own experiences during the recruiting and hiring process.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.beyondcharlotte.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

5:02 min

Mapping Git flow branches to application tester segments

Majid Hajian · LIVE

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

3:53 min

Introduction to git flow and clean feature branches

Johannes Haux · World Congress 2022

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy · World Congress 2024

Videos

See all

Related articles

See all