Data Engineer
Role details
Job location
Tech stack
Job description
In this capacity, you will serve as a critical technical partner to our software developers, data analysts, and data scientists, ensuring an optimal and consistent data delivery architecture across all ongoing projects. We are seeking a professional who is comfortable supporting the diverse data requirements of multiple teams, systems, and products simultaneously., * Develop and operationalize scalable, reliable data pipelines to ensure seamless data flow.
- Assemble large, complex datasets designed to meet both functional and non-functional business requirements.
- Apply advanced techniques in database management, including logical data modeling, physical database optimization, and security protocols.
- Continuously improve the integrity of data pipelines by implementing comprehensive Data Quality checks.
- Partner with Data and Analytics experts to drive greater functionality and innovation within our data systems.
- Provide On-Call support to identify and unblock user issues based on system severity.
Requirements
Plan A Technologies is looking for a highly experienced and self-directed Data Engineer to spearhead the construction, optimization, and maintenance of our cloud-based distributed computing data pipelines. The successful candidate will be an expert "data wrangler" capable of understanding and optimizing complex data systems from the ground up., Please note: you must have at least 5+ years of experience in Engineering and Operationalizing Data Pipelines with large and complex datasets to be considered for this role., * 5+ years of experience in Engineering and Operationalizing Data Pipelines with large and complex datasets.
- 3+ years of professional experience working with unstructured data formats (JSON).
- Extensive experience in Advanced SQL and 3+ years of experience with Python/PySpark.
- 3+ years of experience transforming data using Databricks and working within the AWS ecosystem.
- Proven experience with GIT and CI/CD methodologies.
- Proficiency with Power BI or similar data visualization platforms.
- Excellent communication and collaboration skills.
- Strong analytical, organizational, and detail-oriented mindset.
- Excellent verbal and written English communication skills.
- Initiative and drive to do great things.