GCP Data Engineer

ASB Resources
Secaucus, United States of America
yesterday

Role details

Contract type
Temporary to permanent
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 156K

Job location

Remote
Secaucus, United States of America

Tech stack

Java
API
Agile Methodologies
Data analysis
Automation of Tests
Batch Processing
Big Data
Google BigQuery
Continuous Integration
Data Architecture
Information Engineering
ETL
Serialization
Data Systems
Software Debugging
Data Flow Control
Hadoop
Hadoop Distributed File System
Hive
JSON
Kerberos (Protocol)
MapR (Big Data)
Microsoft SQL Server
SQL Azure
MongoDB
NoSQL
Scrum
E2e Testing
Cloudera
SQL Databases
System Testing
Teradata
XML
Google Cloud Platform
Spark
PySpark
Apache Flink
Avro
Kafka
Data Management
REST

Job description

  • Design, code, test, debug, and document technology solutions for complex projects and programs.
  • Review and analyze large-scale technology solutions to align with tactical and strategic business objectives, as well as enterprise technological environments.
  • Evaluate and develop companywide best practices for engineering and technology solutions, ensuring alignment with industry standards and new technologies.
  • Collaborate with key technical experts and senior technology teams to resolve complex technical challenges and meet project goals.
  • Contribute to the development of Data Mesh capabilities to improve data accessibility across disparate platforms such as Hadoop, Teradata, and SQL Servers.
  • Support Data Science functions by preparing pre-work required for data exploration and analysis.
  • Support System Testing, User Acceptance Testing (UAT), and end-to-end testing activities; develop test automation for batch processes and APIs within a CI/CD environment.
  • Work with Agile methodologies, writing user stories, and participating in Agile ceremonies., Job Title: Data Engineer (GCP, Matillion) Location: Secaucus, NJ (Remote) Job Description: As a Data Engineer, you will be a key member of an agile team dedicated to designing…
  • 15 days ago

Requirements

  • Extensive experience designing and delivering data solutions in Google Cloud Platform (GCP), leveraging services such as DataProc, DataFlow, BigQuery, and other GCP tools.
  • Expertise in Big Data platforms (e.g., MapR, Cloudera) using technologies like HDFS, Spark, PySpark, and Hive.
  • Proficiency in event-driven solutions using Java, REST APIs, Apache Flink, and Kafka technologies.
  • Experience with relational and NoSQL databases, including MongoDB, SQL Server, Atlas, and Azure SQL.
  • Strong familiarity with wire formats such as XML, JSON, Avro, and CSV, along with serialization/deserialization techniques.
  • Knowledge of security protocols (e.g., Kerberos, JWT) and their integration into data platforms and solutions.
  • Solid understanding of Agile methodologies, including writing user stories and participating in ceremonies such as sprint planning and retrospectives., Required Skills & Qualifications * Strong experience in data engineering, ETL (preferably Informatica), and data architecture * Experience working with Salesforce platforms and…
  • 1 month ago

Apply for this position