Data Engineer - Remote / Telecommute

• Develop and optimize ETL/ELT pipelines for large-scale data processing.
• Utilize distributed data processing frameworks such as Apache Spark or Databricks.
• Manage workflow orchestration and scheduling using tools like Apache Airflow, Cloud Composer, or Azure Data Factory.
• Work with cloud-native data services including BigQuery, Snowflake, Postgress, and CloudSQL.
• Implement data quality, governance, and security best practices.
• Manage CI/CD pipelines and adhere to DevOps/DataOps practices.
• Perform troubleshooting and defect resolution using cloud data platforms.
• Utilize metadata driven frameworks for data engineering tasks., The Lead Software Engineer is a hands-on technical leader responsible for driving the full-stack design, development, and delivery of software solutions across one or more small, h…
• 17 days ago, The Lead Software Engineer is a hands-on technical leader responsible for driving the full-stack design, development, and delivery of software solutions across one or more small, h…
• 17 days ago +

• 5-8+ years of overall experience in data engineering, software engineering, or related roles.
• 3-5+ years of hands-on experience with at least one major cloud platform (GCP, AWS, or Azure), with 1-2+ years exposure to a second cloud environment.
• 4-6+ years of experience developing and optimizing ETL/ELT pipelines for large-scale data processing.
• 3-5+ years of strong programming experience in Python and SQL.
• 2-4+ years of experience with distributed data processing frameworks such as Apache Spark or Databricks.
• 2-4+ years of experience with workflow orchestration and scheduling tools (e.g., Apache Airflow, Cloud Composer, Azure Data Factory).
• 2-3+ years of experience working with cloud-native data services (e.g., BigQuery, Snowflake, Postgress, CloudSQL).
• 2-3+ years of experience implementing data quality, governance, and security best practices.
• 1-2+ years of experience with CI/CD pipelines and DevOps/DataOps practices.
• 3-5+ years of exposure to trouble shooting, defect resolution using cloud data platforms.
• 2-3+ years of exposure in using meta data driven frameworks.
• Strong verbal and written communication skills.
• Excellent communication and presentation skills.
• Proficiency in MS Office suite., * Experience with Scala or Java.
• Any Cloud Certification, preferably GCP.
• Working experience with Pyspark, Dataproc, Composer, and Cloud functions.
• Experience with optimization/automation using AI platforms.
• Experience with cloud cost optimization.

Skills:

• Data Engineering.
• Python.
• SQL.
• GCP.
• AWS.
• Azure.
• ETL/ELT.
• Apache Spark.
• Databricks.
• Apache Airflow.
• Cloud Composer.
• Azure Data Factory.
• BigQuery.
• Snowflake.
• Postgress.
• CloudSQL.
• CI/CD.
• DevOps.
• DataOps.

Qualification And Education:

• 5-8+ years of overall experience in data engineering or software engineering.