Data Engineer-DataBricks

We are looking for a skilled and driven Data Engineer with 3-5 years of hands-on expertise in data engineering and cloud data warehousing. As a Data Engineer at ISG, you will be responsible for building, optimizing, and maintaining our enterprise Databricks Lakehouse platform. This platform serves as the foundational engine for our data and AI initiatives, and you will play a key role in translating technical requirements into robust data pipelines and implementing modern engineering standards., * Build and maintain robust, enterprise-grade data pipelines on the Databricks Lakehouse platform to meet business intelligence and AI/ML demands.

• Develop reusable frameworks and apply performance optimization techniques to improve cloud compute efficiency and reduce runtime costs.
• Write and refine clean, efficient code for complex data processing using Python, SQL, Scala, or PySpark.
• Execute engineering best practices across the entire data lifecycle, spanning ingestion, transformation, and workflow orchestration.
• Implement end-to-end data pipelines utilizing modern patterns such as Delta Live Tables (DLT) and Apache Spark.
• Apply data governance frameworks, access controls, and data protection strategies locally using Unity Catalog.
• Support automated CI/CD practices and robust monitoring infrastructure for seamless, reliable deployment of data solutions.
• Collaborate closely with senior architects, analysts, and developers in an environment focused on engineering excellence.
• Work alongside cross-functional engineering units, InfoSec teams, and product owners to deliver high-quality data products.

• Education: Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related technical field.
• Experience: 3-5 years of dedicated experience in data engineering, with a strong focus on building cloud-based data solutions.

Technical Skills

• Proficiency with the Databricks ecosystem (Delta Lake, Unity Catalog, Delta Live Tables).
• Strong hands-on experience with Apache Spark (PySpark/Scala) and advanced SQL.
• Experience with cloud platforms (AWS, Azure, or Google Cloud Platform) and orchestration tools (Airflow, ADF, or Databricks Workflows).
• Familiarity with Version Control (Git) and CI/CD tools (GitHub Actions, Azure DevOps, or Jenkins).