Data Engineer II
Role details
Job location
Tech stack
Job description
This position is responsible for designing, developing, and maintaining business data platforms that allow for insights into customer data. The Data Engineer will use SQL and Python skills to execute high-quality projects and assignments with a high degree of accuracy in a timely manner., * Design, code, test, correct and document programs and scripts using agreed standards and tools to achieve a well-engineered result.
- Ensures data quality and implements tools and frameworks for automating the identification of data quality issues.
- Profile data sources and develop ETL processes with knowledge of data modeling fundamentals, using both SQL, Python and supporting ETL/ELT tools.
- Assists management in creating estimates and proposals for clients.
- Plan effective data storage, security, sharing and publishing within the organization.
- Create and update documentation of process flows and business rules.
- Maintain current workflows up to date and process large amounts of data as needed.
Requirements
- Bachelor's degree in computer science, statistics, mathematics, or relevant discipline.
- 3-5 years of Data engineering and/or data warehousing experience.
- Experience in designing and developing ETL/ELT data pipelines.
- Proficiency in writing Advanced SQL queries.
- Proficiency with Python and different Python Libraries.
- Strong analytical skills with ability to understand and communicate the meaning of the data being presented.
- Excellent Mathematical/Statistical and Logic skills.
- Excellent Reasoning skills.
- Enthusiastic, highly motivated and ability to learn quickly.
- Experience with creating Databricks ETL Workflows.
- An understanding of Apache Spark.
PREFERRED QUALIFICATIONS
· 3-5 years of automotive operations experience, or similar business (quick-paced environment)
· Experience building Azure or AWS cloud data solutions and migrating from on-prem to cloud.
· Previous external client facing experience.
· Familiarity with administering MS SQL Server.
· Experience with GIT repositories.
· Experience with FTP and APIs
· Strong communication skills and a working knowledge of agile development, including DevOps concepts.
· Experience with data visualization tools (Tableau, PowerBI, etc.).
OTHER COMPETENCIES & CHARACTERISTICS
- Knowledge Retention - To provide the highest quality of work, analysts must be able to learn and memorize a large amount of information.
- Detail-Oriented - Analysts must be able to focus on the details to identify and isolate issues.
- Organizational Skills - Analysts must be organized and able to keep track of multiple projects at once.
- Flexibility - Analysts must be able to handle multiple tasks at once and follow changes in priority level.
- Speed - Must be able to work quickly and efficiently without sacrificing the quality of their work.
- Solutions-Oriented - Must be able to find workable solutions for any problems they might face.
LANGUAGE SKILLS
- Full Professional Proficiency in English is required.