Principal Data Engineer

As a Principal Data Engineer, you will design, build, and maintain scalable data pipelines and infrastructure to support analytics, reporting, and data science initiatives. You will work closely with cross-functional teams to ensure data is accessible, reliable, and secure across the organization., Primary Job Duties and Responsibilities (Essential Job Functions)

1. Design and Develop Scalable Data Pipelines

• Design and implement scalable data ingestion and transformation frameworks using Azure services enabling structured, semi-structured, and unstructured data to be efficiently processed and integrated into enterprise data platforms
• Build and maintain robust ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
• Integrate data from diverse sources including on-premises systems, cloud storage, APIs, and streaming platforms.

2. Databricks Development and Optimization

• Develop and optimize notebooks and workflows in Azure Databricks using PySpark, SQL.
• Implement Delta Lake for efficient data storage, versioning, and ACID transactions.
• Leverage Databricks features such as Unity Catalog and job orchestration.

3. Data Modeling and Architecture

• Design and implement data models (star/snowflake schemas) for analytics and reporting.
• Collaborate with architects to define data lakehouse architecture and best practices.
• Hands-on experience implementing and optimizing data solutions using the Medallion Architecture (Bronze, Silver, Gold layers) for scalable and structured data processing

4. Data Quality and Governance

• Implement data validation, profiling, and cleansing routines.
• Ensure compliance with data governance policies, including data lineage and metadata management.

5. Performance Tuning and Monitoring

• Monitor and optimize performance of Spark jobs and data pipelines.
• Troubleshoot and resolve issues related to data latency, job failures, and resource utilization.

6. Collaboration and Stakeholder Engagement

• Work closely with data scientists, analysts, and business units to understand data requirements.
• Translate business needs into technical solutions that are scalable and maintainable.

7. Security and Compliance

• Implement role-based access control (RBAC), encryption, and secure data handling practices.
• Ensure compliance with industry regulations (e.g., NERC CIP, GDPR, HIPAA if applicable).

8. Documentation and Best Practices

• Maintain clear documentation of data flows, architecture, and operational procedures.
• Promote best practices in code versioning, testing, and CI/CD for data engineering.

• Bachelor's degree in information systems, Computer Science, or a related technical field; or equivalent work experience.
• 10 years of experience with advanced knowledge of data architecture, cloud platforms (especially Azure), and enterprise data solutions.
• Advanced understanding of data modeling, ETL/ELT processes, and performance tuning of enterprise-level applications.
• Expert-level knowledge of data-related technologies from architecture to administration, including design, development, optimization, and licensing.
• Proven experience working in the utility industry is required
• Soft Skills:
• Effective oral and written communication skills, with the ability to collaborate across teams and mentor junior engineers.
• Strong analytical and problem-solving abilities.
• Ability to prioritize and manage multiple tasks and projects concurrently.

MidAmerican Energy Company, a Midwest utility, provides regulated electric and natural gas service to more than 1.6 million customers in Illinois, Iowa, Nebraska and South Dakota. The company owns and operates a portfolio of power-generating assets, approximately 61% of which is wind generation.