Lead Data Engineer
Role details
Job location
Tech stack
Job description
Primary/Must Have: ETL, ML OPS, AI-ML, Data warehousing, Python
Secondary/Good to Have: AWS
Detailed Job Description:
As part of the Network Analytics team, the Sr Data Engineer role is crucial in
executing advanced analytic techniques to enhance security and operational efficiency
of client enterprise networks. This position focuses on constructing robust ETL pipelines using Spark to ensure data availability and quality for analytics tasks. The role requires expertise in data engineering, leveraging AWS for scalable solutions and enhancing operational efficiency. The role also requires expertise in statistical analysis and machine learning, utilizing generative AI to build data-driven, automated systems that enhance operational efficiency.
Roles & Responsibilities:
Design, develop, and maintain scalable ETL pipelines to ensure data quality and availability
Implement monitoring and alerting solutions to ensure data pipeline reliability and performance
Develop and manage deployment pipelines to facilitate continuous integration and delivery of data engineering solutions
Implement data integration solutions to support analytics and reporting needs
Execute the complete analytics lifecycle for problem solving, including:
-
Algorithm traditionalization
-
Model validation
-
Model prototyping
-
Data exploration
-
Data grooming
Survey varied data sources for analytic relevance, including:
-
External sources accessed via API
-
Flat files
-
Relational databases
-
Distributed file systems
Interpret, synthesize and communicate results of analyses to effect action and changes within the organization
Collaborate across teams to integrate analytic products with existing production architecture, develop, execute, and evaluate courses of action, and socialize results
Requirements
Skills/Abilities and Knowledge
-
Ability to read, write, speak and understand English
-
Strong communication and presentation skills
-
Expertise in data engineering languages such as Scala (preferred) or Java, with proficiency in Python
-
Experience with BigData tools, particularly Spark
-
Proficiency in building and managing ETL pipelines
-
Expert-level quantitative analysis skills including interpretation of model results, consideration of causality, treatment of multicollinearity
-
The ability to work in compiled, high-performance languages (e.g., Scala, Java, C++)
-
Experience with relational databases
-
Strong understanding of relational databases and SQL, and familiarity with NoSQL databases
-
Broad experience and solid theoretical foundation on the modeling process using a
-
variety of algorithmic techniques, including Machine Learning, and Graph/Network Analytics
-
Data pre-processing, exploratory data analysis using a variety of techniques
-
Basic understanding of data architecture, data warehouse, and data marts
-
Demonstrated ability and desire to continually expand skill set, and learn from and teach others
Preferred Qualifications:
-
Experience with Linux-based operating systems
-
Knowledge of other relevant techniques such as text analysis and text mining
-
Bachelors degree (or relevant experience) in computer science, mathematics, statistics, physics, operations research, or other quantitatively-focused fields
-
Masters: 6+ Years Work Experience
-
Bachelors: 8+ Years Work Experience
We are committed to fostering a diverse, inclusive, and equitable workplace where individuals from all backgrounds feel valued and empowered to contribute their unique perspectives. We strongly encourage applications from candidates of all genders, races, ethnicities, abilities, and experiences to join our team and help us build a culture of belonging.
Benefits & conditions
National average IT Jobs average Colorado average
Salaries
The number of jobs in each salary range for all
