Senior Manager - Data Engineering
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+16 more
Job description
Position Responsibilities: Design and architect scalable data platforms in the cloud, ensuring they meet both technical and business requirements. Lead the technical vision for cloud-based data solutions, aligning architecture and implementation with strategic goals. Implement and optimize star schema models within cloud data warehouses, focusing on performance and scalability. Regularly review data workloads to identify opportunities for operational and performance improvements. Manage capacity in cloud data warehouses by proactively adjusting resources based on usage patterns and demand. Monitor system performance and ensure optimal operation through tuning and resource management. Build high-performance data models by applying effective table distribution and indexing strategies. Implement and manage cloud-based data lakes, leveraging hierarchical namespaces and POSIX-compliant access control mechanisms to ensure secure and organized data storage. Design, develop, and optimize ETL/ELT pipelines for efficient data movement and transformation. Implement big data processing solutions to handle large-scale datasets, and automate workflows to support real-time data streaming and analytics. Integrate cloud-based analytics platforms with other services such as data lakes, orchestration tools, and data warehouses to create seamless data ecosystems. Manage and maintain cloud compute clusters, optimizing resource utilization for both performance and cost efficiency. Develop and maintain CI/CD pipelines to support continuous integration and deployment of data processes. Execute change tasks such as table partitioning using appropriate strategies to enhance query performance and manage data lifecycle effectively. Identify and resolve performance bottlenecks across the cloud data environment, ensuring systems run efficiently and reliably. Provide guidance and documentation to developers and end-users on best practices for cloud services, including maintenance and housekeeping. Address incidents and issues related to data migration from on-premises systems to the cloud, ensuring smooth transitions and minimal disruption. Uphold data quality, governance, and security by implementing industry best practices and compliance standards.
Requirements
Engineering Management, Engineering Management, Star Schema, POSIX Operating Systems, Data Storage, Big Data, People Management, Documentation, Data Modeling, Best Practices, Position Requirements: Masterβs degree (or foreign equivalent) in Information Systems, or a related field, PLUS two (2) years of experience in the job offered or a related position. Experience must include demonstrable knowledge of: SQL; Python; ETL/ELT data pipelines; partitioning tables; Star & Snowflake data models; CI/CD pipelines; AWS S3; Databricks; JIRA, and; Confluence. Required knowledge may be gained prior to or concurrently with Masterβs degree. Travel to unanticipated client locations throughout the U.S., approximately 30%, as required. May telecommute from any location within the U.S.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦