Senior Data Engineer-Databricks

We are seeking a highly skilled and motivated Senior Data Engineer specializing in Databricks to join our dynamic data team. In this role, you will lead the design, development, and optimization of large-scale data systems, enabling advanced analytics and business intelligence solutions. Your expertise will drive innovative data management strategies across cloud platforms, ensuring seamless integration and high-performance processing of complex datasets. This position offers an exciting opportunity to work with cutting-edge big data technologies and contribute to impactful data-driven decision-making., * Design, develop, and maintain scalable ETL pipelines using Databricks, Spark, Python, and SQL to process vast amounts of structured and unstructured data.

• Collaborate with cross-functional teams to translate business requirements into robust data models and architectures aligned with best practices in dimensional modeling and data warehousing design.
• Optimize big data systems leveraging Hadoop, Azure Data Lake, AWS, and other cloud-based platforms to ensure high availability and performance.
• Implement data management integration strategies utilizing Informatica, Talend, RESTful APIs, and other tools to streamline data ingestion and transformation processes.
• Develop comprehensive documentation for data workflows, models, and pipelines while ensuring adherence to security standards and compliance regulations.
• Support model training, analysis activities, and business intelligence reporting using Looker, SQL databases such as Oracle or Microsoft SQL Server, and cloud databases.
• Participate in Agile development cycles to continuously improve data engineering solutions while maintaining flexibility for evolving project needs.

• Proven experience in software development with a strong background in data engineering within big data environments.
• Extensive hands-on expertise with Databricks platform, Spark (including Apache Hive), Hadoop ecosystems, and cloud services such as Azure Data Lake or AWS Public Cloud.
• Proficiency in programming languages including Python, Java, Bash (Unix shell), Shell Scripting, and VBA for automation and analysis tasks.
• Deep understanding of ETL pipeline development, SQL query management, database design (including dimensional modeling), and data warehousing concepts.
• Familiarity with business intelligence tools like Looker or similar platforms for visualization and reporting purposes.
• Knowledge of data modeling techniques for linked data, Oracle databases, SQL databases, cloud databases, and enterprise data warehouse architecture.
• Experience working with Big Data systems such as Hadoop or Spark for large-scale processing projects.
• Strong analytical skills combined with excellent communication abilities to translate complex technical concepts into clear insights.
• Ability to work effectively within an Agile environment while managing multiple priorities simultaneously.

3.23.2 out of 5 stars Remote $110,000 - $150,000 a year - Full-time, Pulled from the full job description

• Parental leave
• 401(k)
• Health insurance
• Retirement plan
• 401(k) matching
• Paid time off
• Vision insurance, * 401(k)
• 401(k) matching
• Dental insurance
• Health insurance
• Health savings account
• Life insurance
• Paid time off
• Parental leave
• Retirement plan
• Vision insurance