Senior Manager - Data Engineering

Tredence Inc.
San Jose, CA, United States
7 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Compensation
$180,398
Working hours
Regular working hours

Tech stack

Query Performance Amazon S3 Confluence JIRA Big Data Cloud Computing Cloud Database Information Systems Continuous Integration Information Engineering Extract Transform Load (ETL) Data Migration
+16 more
Data Warehousing Monitoring of Systems Python (Programming Language) Posix Cloud Services SQL Databases Workflow Management Systems Data Storage Technologies Snowflake Database Optimization Data Lakes Star Schema Data Management Stream Processing Data Pipelines Databricks

Job description

Position Responsibilities: Design and architect scalable data platforms in the cloud, ensuring they meet both technical and business requirements. Lead the technical vision for cloud-based data solutions, aligning architecture and implementation with strategic goals. Implement and optimize star schema models within cloud data warehouses, focusing on performance and scalability. Regularly review data workloads to identify opportunities for operational and performance improvements. Manage capacity in cloud data warehouses by proactively adjusting resources based on usage patterns and demand. Monitor system performance and ensure optimal operation through tuning and resource management. Build high-performance data models by applying effective table distribution and indexing strategies. Implement and manage cloud-based data lakes, leveraging hierarchical namespaces and POSIX-compliant access control mechanisms to ensure secure and organized data storage. Design, develop, and optimize ETL/ELT pipelines for efficient data movement and transformation. Implement big data processing solutions to handle large-scale datasets, and automate workflows to support real-time data streaming and analytics. Integrate cloud-based analytics platforms with other services such as data lakes, orchestration tools, and data warehouses to create seamless data ecosystems. Manage and maintain cloud compute clusters, optimizing resource utilization for both performance and cost efficiency. Develop and maintain CI/CD pipelines to support continuous integration and deployment of data processes. Execute change tasks such as table partitioning using appropriate strategies to enhance query performance and manage data lifecycle effectively. Identify and resolve performance bottlenecks across the cloud data environment, ensuring systems run efficiently and reliably. Provide guidance and documentation to developers and end-users on best practices for cloud services, including maintenance and housekeeping. Address incidents and issues related to data migration from on-premises systems to the cloud, ensuring smooth transitions and minimal disruption. Uphold data quality, governance, and security by implementing industry best practices and compliance standards.

Requirements

Engineering Management, Engineering Management, Star Schema, POSIX Operating Systems, Data Storage, Big Data, People Management, Documentation, Data Modeling, Best Practices, Position Requirements: Master’s degree (or foreign equivalent) in Information Systems, or a related field, PLUS two (2) years of experience in the job offered or a related position. Experience must include demonstrable knowledge of: SQL; Python; ETL/ELT data pipelines; partitioning tables; Star & Snowflake data models; CI/CD pipelines; AWS S3; Databricks; JIRA, and; Confluence. Required knowledge may be gained prior to or concurrently with Master’s degree. Travel to unanticipated client locations throughout the U.S., approximately 30%, as required. May telecommute from any location within the U.S.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…