Data Engineer

Inc. (dci)
New York, NY, United States
about 1 month ago
Apply on www.indeed.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience required
1 year minimum
Compensation
$175,000.0 - $260,000.0
Working hours
Regular working hours
Job source

Tech stack

Computer Programming Information Engineering Software Debugging Python (Programming Language) SQL Databases Apache Spark Pandas Pyspark Information Technology

Job description

  • Serve as a Data Engineer on a cross-functional team to help solve pressing problems
  • Integrate data, transform insights, and build applications for operational decisions
  • Setup transfers of data feeds from source systems into location
  • Write transformations and derive new datasets using Spark for distributed computation
  • Monitor build progress and debug build problems
  • Design applications that address operational questions
  • Execute requests for information (RFI’s) surrounding the platform’s data footprint
  • Work with product teams while debugging/understanding the code backing our platform

Requirements

· BS in Computer Science / Engineering (or related career field) preferred

· Strong Data Engineering background

· Experience with Spark, Pyspark, and/or Pandas

· Extensive experience coding/programming in:

  • Python
  • SQL, R or Similar

· Strong understanding of algorithms and data structures as they pertain to development

· Experience with Foundry is preferred

Required: Active Secret Security Clearance, * data engineering: 5 years (Preferred)

  • Python coding/programming: 5 years (Preferred)
  • building and managing data pipelines: 5 years (Preferred)
  • data transformation: 5 years (Preferred)
  • Foundry : 1 year (Preferred)
  • Spark and/or Pyspark: 3 years (Preferred)
  • Pandas: 3 years (Preferred)

Benefits & conditions

Pulled from the full job description Professional development assistance Tuition reimbursement Parental leave 401(k) Health insurance Retirement plan 401(k) matching, * 401(k)

  • 401(k) matching
  • Dental insurance
  • Flexible schedule
  • Flexible spending account
  • Health insurance
  • Health savings account
  • Life insurance
  • Paid time off
  • Parental leave
  • Professional development assistance
  • Relocation assistance
  • Retirement plan
  • Tuition reimbursement
  • Vision insurance

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:34 min

Capabilities of the Apache Spark processing engine

Ayon Roy · LIVE

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

2:03 min

Accelerating pandas dataframes using cudf module plugins

Ankit Patel Ankit Patel · World Congress 2024

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

6:58 min

Analyzing production code coverage data using pandas

Markus Harrer Markus Harrer · World Congress 2021

Videos

See all

Related articles

See all