Databricks Engineer - Hybrid

We are seeking a dynamic and highly skilled Databricks Engineer to join our data team. In this role, you will be at the forefront of designing, developing, and maintaining scalable big data solutions using Databricks and related technologies. Your expertise will enable us to harness the power of large datasets, optimize data workflows, and deliver actionable insights that drive strategic decision-making. This position offers an exciting opportunity to work with cutting-edge cloud platforms and innovative data tools in a fast-paced, collaborative environment., Design, develop, and optimize data pipelines leveraging Databricks Unified Analytics Platform to process large-scale datasets efficiently. Implement robust ETL workflows using Spark, Python, SQL, and shell scripting to ensure high-quality data ingestion and transformation. Collaborate with cross-functional teams to integrate diverse data sources such as AWS, Azure Data Lake, Hadoop, Oracle, Microsoft SQL Server, and Informatica into unified data warehouses. Develop and maintain scalable data models and database schemas aligned with best practices in database design for analytical applications. Utilize analytics tools like Looker for creating dashboards and visualizations that facilitate insightful business analysis. Conduct model training and analysis to support machine learning initiatives, ensuring data readiness and quality. Write RESTful APIs for seamless data access across platforms and applications while adhering to security standards. Support agile development processes by participating in sprint planning, code reviews, and continuous integration efforts. Maintain comprehensive documentation of data workflows, system configurations, and technical specifications to ensure operational transparency.

big data technologies, cloud platforms, AWS, Azure, Databricks platform, scalable analytics solutions, Spark, Unstructured format, Unity Catalog, Genie, Immuta, lambda, dynamodb, aws data sync, aws sftp, programming languages, Java, Python, Bash, Unix shell, Shell Scripting, VBA, automation, scripting, ETL processes, SQL query optimization, database design, Data Warehouses, Oracle, Microsoft SQL Server databases, big data solutions, data workflows, cutting-edge cloud platforms, data pipelines, ETL workflows, data ingestion, data transformation, 8+ Years of experience Proven experience working with big data technologies and cloud platforms like AWS or Azure. Hands-on experience with Databricks platform for developing scalable analytics solutions in a cloud environment. Experience working with Spark, Unstructured format, Unity Catalog, Genie, Immuta, lambda, dynamodb, aws data sync, aws sftp Strong proficiency in programming languages including Java, Python, Bash (Unix shell), Shell Scripting, and VBA for automation and scripting tasks. Extensive knowledge of ETL processes, SQL query optimization, database design principles for Data Warehouses, and familiarity with Oracle and Microsoft SQL Server databases.