Enterprise Data Engineer Sr.
Role details
Job location
Tech stack
Job description
The Enterprise Data Engineer position on the Enterprise Data Solutions team is part of our Corporate Systems, Enterprise Architecture and Information Management department within Enterprise Technology. The role is responsible for establishing, expanding and optimizing our Enterprise Analytical Data Platform and associated data pipeline architecture, as well as optimizing data flow for cross-functional analytical teams.
The Data Engineer will work with our software developers, database architects, data analysts and data scientists on initiatives and will ensure the data delivery architecture is efficient and consistent. The Enterprise Data Engineer works in a self-directed fashion, concurrently supporting the data needs of multiple teams, systems, and products. This role understands complex source system data structures and implements models, patterns, and solutions in both 'big data' and traditional data warehouse platforms.
Principal Duties & Responsibilities:
· Consultative solution development in collaboration with business and IT stakeholders
· Assemble large, complex data sets that meet functional / non-functional business requirements
· Develop and maintain an optimized Enterprise Data Pipeline architecture
· Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
· Develop, maintain and support Data Solutions using both Spark and SQL
· Design, configures/code, test, debug and document software/solution according to Industry best practices
· Work with Enterprise Architects and Data Governance to help in data ingestion, staging, data quality monitoring
· Preparation of design/technical/functional/operational specification documentation
· Work with stakeholders and design teams to assist with data-related technical issues and support their data platform needs
· Develop platform strategies and patterns for Data Integration, Delivery, Partitioning and Access Control
· Identify, design, and implement internal process improvements, such as:
o Automating manual processes
o Optimizing data delivery
o Re-designing or aligning infrastructure for greater scalability
· Mentor and provide guidance on data solutions and patterns
Requirements
Bacherlor's degree in Computer Science, Information Systems or a related field or equivalent work experience.
Experience Requirements
· 5+ Years in IT - Software development and Engineering
· 5 Years in Big Data and Analytics (Spark/Pyspark)
· 5 Years' experience with Databricks or equivalent cloud data platform
· 5 Years' experience working with advanced Structured Query Language (SQL) and relational databases, query authoring as well as working familiarity with a variety of other database engines, including MS SQL Server, IBM DB2 and Oracle
· 5 Years' experience working with online cloud providers (e.g., Azure, AWS, Google)
· 3 years' experience using programming languages Python or Scala
· 3 Years working with unstructured datasets
· Strong project delivery and organizational skills
· Experience supporting and working with cross-functional teams in a dynamic environment
· The following personal skills are required:
o Demonstrated inter-personal and consultative skills to work with business and IT stakeholders
o Ability to explain, promote and evangelize about data solutions
Benefits & conditions
-
Medical with wellness incentives, dental, and vision
-
HSA with company match
-
Maternity and parental leave
-
Tuition reimbursement
-
Mentor program
-
401(k) with 6% match
