Senior Associate AWS Data Engineer
Role details
Job location
Tech stack
Job description
We are looking for a Senior Associate Data Engineer role focused on building and supporting scalable data pipelines on AWS. You'll work hands-on with PySpark, SQL, Python, Databricks, and AWS services such as Redshift, EMR, Glue, and S3, partnering with onshore and offshore teams in a client-facing environment.
Hybrid role based in Los Angeles (3 days onsite).
Responsibilities
-
Combine your technical expertise and problem-solving passion to work closely with clients, turning complex ideas into end-to-end solutions that transform our clients' business
-
Lead, design, develop and deliver large-scale data systems, data processing and data transformation projects that delivers business value for clients
-
Automate data platform operations and manage the post-production system and processes
-
Conduct technical feasibility assessments and provide project estimates for the design and development of the solution
-
Provide technical inputs to agile processes, such as epic, story, and task definition to resolve issues and remove barriers throughout the lifecycle of client engagements
-
Creation and maintenance of infrastructure-as-code for cloud, on-prem, and hybrid environments using tools such as Terraform, CloudFormation, Azure Resource Manager, Helm, and Google Cloud Deployment Manager
Requirements
-
Demonstrable experience in data platforms involving implementation of end to end data pipelines
-
Hands-on experience with Amazon Web Services Cloud Platform
-
Implementation experience with column-oriented database technologies (i.e., Redshift, Vertica), NoSQL database technologies (i.e., DynamoDB, Cosmos DB, etc.) and traditional database systems (i.e., SQL Server, Oracle, MySQL)
-
Experience in implementing data pipelines for both streaming and batch integrations using tools/frameworks like Glue ETL, Lambda, Spark, pysparkStreaming, etc.
-
Ability to handle module or track level responsibilities and contributing to tasks "hands-on"
-
Experience in data modeling, warehouse design and fact/dimension implementations
-
Experience working with code repositories and continuous integration
-
Data modeling, querying, and optimization for relational, NoSQL, timeseries, and graph databases and data warehouses and data lakes
-
Data processing programming using SQL, DBT, Python.
-
Experience with data processing platforms such as Databricks
-
Logical programming in Python, Spark, PySpark, Java, Javascript, and/or Scala
-
Data ingest, validation, and enrichment pipeline design and implementation
-
Cloud-native data platform design with a focus on streaming and event-driven architectures
-
Test programming using automated testing frameworks, data validation and quality frameworks, and data lineage frameworks
-
Metadata definition and management via data catalogs, service catalogs, and stewardship tools such as AWS Glue Catalog, OpenMetadata, DataHub, Alation, and similar
-
Code review and mentorship
-
Bachelor's degree in Computer Science, Engineering or related field.
Benefits & conditions
Pay Range: $103,000 - $154,000
-
An inclusive workplace that promotes diversity and collaboration.
-
Access to ongoing learning and development opportunities.
-
Competitive compensation and benefits package.
-
Flexibility to support work-life balance.
-
Comprehensive health benefits for you and your family.
-
Generous paid leave and holidays.