Cloud Data Engineer
Role details
Job location
Tech stack
Job description
As a Cloud Data Engineer, you will create and modify solutions for complex software problems. This includes coding, testing, debugging, documenting, and maintaining those solutions. You will participate in leading smaller engineering efforts and contribute to larger, enterprise-wide initiatives. The engineering teams partner across departments to achieve outcomes for customers., * Design and implement scalable and secure data storage solutions in the cloud.
- Develop and maintain robust data pipelines for the ingestion, transformation, and distribution of large datasets.
- Automate data processes, integrate third-party services, and utilize cloud tools to streamline data workflows.
- Ensure compliance with data governance and security policies, including data encryption and access controls.
- Monitor cloud data systems' performance, identify bottlenecks, and implement improvements.
- Conduct data quality checks to ensure data accuracy and integrity.
- Optimize data retrieval and develop APIs for data consumption by enterprise consumers.
- Provide technical expertise and support for data-related issues, including troubleshooting pipeline failures.
- Collaborate with IT and security teams to plan and execute disaster recovery strategies.
- Document data engineering processes, create data flow diagrams, and maintain metadata.
- Collaborate with architects, analysts, and other engineers to support data modeling, analysis, and reporting.
- Stay current with emerging cloud technologies and data engineering practices.
Requirements
Education: Bachelor's degree in Computer Science or other technical field, or equivalent work experience.
Experience: 3 to 5 years of engineering experience is required.
Technical Skills:
- Cloud Platforms: AWS (Kubernetes), Google Cloud, Microsoft Azure.
- Data Storage: Data lakes, data warehouses, cloud storage service (S3).
- Data Pipelines: Developing and maintaining data pipelines for ETL processes.
- Programming Languages: Python, Java, Scala.
- Data Management Systems: SQL, NoSQL, Hadoop, Postgres.
- Knowledge of Software Development Life Cycle (SDLC) practices.
- Understanding of data security best practices and compliance regulations.
- Ability to debug code and complex log files for troubleshooting and analysis.
Benefits & conditions
The pay rate for this position is between $60.00 and $65.00 per hour. The permanent salary for this position is approximately $100,000.00 per year.