Senior Data Engineer - Google Cloud Storage
Role details
Job location
Tech stack
Requirements
Looking for a Senior Data Engineer with Google Cloud Storage best practices and security experience., 8+ years of experience with a focus on building, designing, and maintaining software solutions.
Must have architected, built, and maintained modern cloud-based data lake environments
Experience with large-scale data organization, skilled in the practical use of efficient storage formats and optimization techniques, and can deliver highly usable datasets for business analytics teams.
Must have experience with Google Cloud Storage best practices and security.
Proven experience designing and implementing data lake architectures (e.g., Bronze/Silver/Gold or layered models) - creation of multi-layered architectures (such as the Bronze/Silver/Gold pattern) to manage raw, cleaned, and curated data for different consumption use cases. Experience in ETL/ELT pipelines to move and transform data across these layers.
Experience with designing and managing data storage in the cloud - Google Cloud Storage (GCS).
Experience should include structuring cloud storage with thoughtful bucket organization, standardized naming conventions, automated lifecycle management (archiving, deleting), and detailed access controls (security, permissions).
Experience with columnar file formats (Parquet, Avro, or ORC) for storing large datasets efficiently and for fast querying.
Direct experience with methods for compressing data within these formats for optimal storage and performance is required.
In-depth understanding of how to partition data (for example, by date/time, customer, or source), which is critical for performance and scalability in large data environments.
Experience running backfills, the process of loading or correcting historical data, efficiently and safely at scale.
Proven skill in designing data models and building curated datasets that are user-friendly, documented, and optimized for consumption by analysts and Business Intelligence (BI) tools.