Data Engineer

The Data Engineer will be responsible for designing, developing, and maintaining data pipelines, ensuring, Data Pipeline Development

• Design and develop ETL/ELT data pipelines
• Ensure data pipelines are scalable, reliable, and efficient, meeting the needs of the organization.

Data Management

• Implement and manage the data architecture, including Landing Zone, Bronze, Silver, and Gold layers within the Databricks environment.
• Develop processes for data cleaning, normalization, and enrichment to maintain high data quality standards.

Data Governance

• Integrate core data management with existing systems to ensure seamless data flow and consistency across platforms.
• Develop a comprehensive tagging system for core data, ensuring that all data assets are tagged appropriately for easy retrieval and association.

Performance Optimization

• Monitor and optimize the performance of data pipelines and processing workflows.
• Ensure data architecture supports different stages of the data lifecycle, from development to production, with appropriate access controls.
• Work closely with data scientists, data stewards, and other stakeholders to understand their data requirements and support their analytical needs.
• Provide technical support and guidance to other team members and departments as needed.

• Proven experience as a Data Engineer or in a similar role, with previous experience in the pharmaceutical industry being highly regarded.
• Hands-on experience with AWS, scripting and ETL pipelines
• Strong proficiency in programming languages such as Python
• Jira exp

Skills and Competencies

• Strong analytical and problem-solving skills.
• Excellent understanding of ETL/ELT processes and data pipeline development.
• Knowledge of metadata management and data quality processes.
• Proficiency in data modeling, data warehousing, and big data processing frameworks.
• Excellent communication and collaboration skills.
• Ability to learn new technologies quickly and develop new skills as needed.
• Capable of identifying the best approach forward for various data engineering challenges.
• Ability to work in a fast-paced, dynamic environment and manage multiple priorities.