Senior Data Engineer

Brooksource
New York, NY, United States
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$135,200.0 - $176,800.0
Working hours
Regular working hours
Job source

Tech stack

Java (Programming Language) Agile Methodology Amazon S3 Data Analysis Bash Shell Big Data Cloud Database Databases Data Architecture Data Integration Extract Transform Load (ETL) Data Warehousing
+23 more
Database Design Dimensional Modeling Apache Hadoop Apache Hive Python (Programming Language) Linked Data Machine Learning Microsoft SQL Server Oracle (Applications) Scrum Methodology Azure Data Lake Shell Script SQL Databases Talend Cloud Platform System Informatica Powercenter Apache Spark Vba Programming Language Data Management Restful APIs Looker Analytics Data Pipelines Programming Languages

Job description

We are seeking a highly skilled Senior Data Engineer to join our dynamic data team. In this role, you will be responsible for designing, developing, and maintaining scalable data pipelines and architectures that support advanced analytics and business intelligence initiatives. The ideal candidate will leverage expertise in big data systems, cloud platforms, and data modeling to enable efficient data management and integration across diverse sources. This position offers an exciting opportunity to work with cutting-edge technologies in a collaborative environment dedicated to innovation and data-driven decision-making., * Design, develop, and optimize large-scale ETL pipelines utilizing tools such as Informatica, Talend, Apache Spark, and Apache Hive

  • Build and maintain robust data architectures on cloud platforms including AWS, Azure Data Lake, and Public Cloud environments
  • Collaborate with cross-functional teams to gather requirements and translate them into scalable data solutions
  • Develop and implement data models using dimensional modeling techniques for data warehousing and analytics purposes
  • Manage data integration processes involving linked data, RESTful APIs, and other external sources
  • Ensure data quality, security, and compliance standards are met throughout the data lifecycle
  • Support the deployment of machine learning models through efficient data pipeline integration
  • Utilize SQL, Python, Java, Bash scripting, and Shell Scripting for automation and process enhancement
  • Contribute to agile development cycles by participating in sprint planning, reviews, and continuous improvement initiatives
  • Maintain documentation of data workflows and architecture diagrams for transparency and knowledge sharing

Requirements

  • Proven experience as a Data Engineer or similar role with a focus on big data systems and cloud-based architectures
  • Extensive knowledge of AWS services such as S3, Redshift, Glue; Azure Data Lake; Hadoop ecosystem components; and other public cloud offerings
  • Strong proficiency in SQL databases including Microsoft SQL Server, Oracle, and cloud databases
  • Hands-on experience with ETL/ELT development using tools like Informatica or Talend
  • Demonstrated expertise in data modeling techniques including dimensional modeling for enterprise data warehouses
  • Skilled in programming languages such as Python, Java, VBA, Bash scripting for automation tasks
  • Familiarity with business intelligence tools like Looker or similar platforms for reporting and analysis
  • Understanding of RESTful API integration for linked data access
  • Knowledge of analysis skills related to database design, query management, model training, and analytics methodologies
  • Experience working within Agile methodologies to deliver iterative value in fast-paced environments

If you are passionate about harnessing the power of big data through innovative engineering solutions and thrive in a collaborative setting focused on continuous improvement, we encourage you to apply today. Join us in shaping the future of data-driven insights across our organization.

Benefits & conditions

Pay: $65.00 - $85.00 per hour

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:43 min

The enduring legacy of the amazon S3 storage API

Chris Heilmann +3 · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

5:08 min

Automating data collection and managing crowdsourced training image sets

Kris Howard · LIVE

Videos

See all

Related articles

See all