Senior Data Engineer

Archer Daniels Midland Company
Decatur, IL, United States
5 days ago
Apply on www.agcareers.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Compensation
$111,592.0 - $121,592.0
Working hours
Regular working hours

Tech stack

Application Programming Interfaces (APIs) Agile Methodology Microsoft Azure Cloud Computing Continuous Integration Extract Transform Load (ETL) Data Transformation Data Systems Data Warehousing Database Connection Database Storage Structures Fault Tolerance
+18 more
Github Python (Programming Language) Mainframes Power BI DataOps SAP (Applications) Software Engineering SQL Databases Azure Data Factory Data Lakes Pyspark Information Technology Machine Learning Operations Data Delivery Code Restructuring Azure Synapse Analytics Data Pipelines Databricks

Job description

Senior Data Engineer, Archer Daniels Midland Company, Decatur, IL. Lead a delivery team to develop data pipelines, transform raw data into useful data systems, and optimize data delivery architecture. Design, build, refactor, and maintain data pipelinesusing Microsoft Azure, SQL, Azure Data Factory, Azure Synapse, Databricks, Python, and PySpark to meet business requirements for reporting, analysis, and data science. Teach, adhereto, and contribute to DataOps and MLOps standards and best practices to continuously improve data system performance. Design and integrate fault tolerance and enhancements intodata pipelines to improve quality and performance. Lead and conduct root cause analysis to solve problems, optimize data delivery, and reduce costs. Develop and maintain pricing modelsand tools to support business decisions. Collaborate with sales, marketing, and finance teams to align pricing strategies with business goals. Create solutions for Prospect-to-Cash,including Order Management and Sales Management. Create solutions for GBS to provide end-to-end solutions.

Requirements

Master’s degree or foreign equivalent degree in Computer Science, Data Science and Business Analytics, Software Engineering, Information Technology or a related field, and three (3) years of related work experience.

Must have three (3) years of experience with/in:

  • At least three (3) of the following cloud-based data systems, such as Microsoft Azure, SQL, Azure Data Factory, Azure Synapse, Databricks, Python, PySpark, SAP Datasphere, and/or Power BI;
  • Azure DevOps, GitHub, and CI/CD;
  • Database storage systems such as cloud, relational, mainframe, data lake, and data warehouse;
  • Building cloud ETL pipelines using code;
  • Building ETL platforms utilizing database connections, APIs, or file-based;
  • Data warehousing concepts and agile methodology;
  • Designing and coding data manipulations;
  • Applying processing techniques to extract value from large and disconnected datasets; and
  • Presenting conceptual and technical improvements.

100% telecommuting permitted.

Benefits & conditions

40 hrs/week, Mon-Fri, 8:30 a.m. - 5:30 p.m. Salary: $111,592 - $121,592/yr. Standard company benefits.

About the company

Diversity, equity, inclusion and belonging are cornerstones of ADM’s efforts to continue innovating, driving growth, and delivering outstanding performance. We are committed to attracting and retaining a diverse workforce and create welcoming, truly inclusive work environments - environments that enable every ADM colleague to feel comfortable on the job, make meaningful contributions to our success, and grow their career. We respect and value the unique backgrounds and experiences that each person can bring to ADM because we know that diversity of perspectives makes us better, together.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.agcareers.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:00 min

Separating dataset creation from low-level software implementation steps

Jan Zawadzki · World Congress 2022

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

6:36 min

Funding open source through GitHub Accelerator and Sponsors

Stormy Peters · World Congress 2023

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

3:14 min

Structuring career paths and localized data architectures

Ulrich Wurstbauer +1 · LIVE

Videos

See all

Related articles

See all