Senior Lead Data Engineer

Brillio 2
Toledo, Spain
3 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience required
3 years minimum
Working hours
Regular working hours

Tech stack

Big Data Software as a Service Continuous Integration Data Validation Information Engineering Data Transformation Data Systems Data Warehousing Python (Programming Language) Power BI Data Streaming Unstructured Data
+9 more
Data Processing Sql Optimization Git Data Lakes Pyspark Data Analytics Data Delivery Data Pipelines Databricks

Job description

Senior Lead Data Engineer at Brillio 2.About the role This position involves developing and expanding Brillio’s data and analytics infrastructure.You will focus on creating, building, and managing efficient, dependable, and cost-effective data pipelines and platforms within a cloud environment.The role requires a hands?on approach to designing data systems from scratch.Key facts Location: Guadalajara, Jalisco, Mexico Engagement: EmployeeWhat you’ll do Design, build, and operate cloud?based data pipelines that are scalable, reliable, and performant.Create and maintain large, complex datasets that fulfill both functional and non?functional business needs.Implement database management best practices, including logical data modeling, physical database optimization, and data security.Develop and improve data quality checks to ensure the accuracy, integrity, and reliability of data pipelines.Collaborate with Data & Analytics experts to continuously enhance data platform capabilities and usability.Support various teams, systems, and products by providing consistent and optimized data delivery architectures.Participate in on?call support rotations and proactively resolve user issues based on severity and business impact.Contribute to improving operational processes, monitoring, and the reliability of data platforms.Requirements 5+ years of experience designing, building, and operating data pipelines for large and complex datasets.Strong experience with structured and unstructured data, including JSON.Extensive hands?on experience with advanced SQL for data transformation and analysis.3+ years of experience with Python and PySpark for large?scale data processing.Hands?on experience transforming and processing data using Databricks.3+ years of experience building data solutions on AWS.Experience using Git and CI/CD practices for data and platform development.Strong collaboration skills to support data engineers, analysts, data scientists, and software engineers.Self?directed, ownership?driven mindset with the ability to manage multiple concurrent priorities.Nice to have Experience with Power BI or similar business intelligence and visualization tools.Familiarity with modern data lake and data warehouse architectures.Experience implementing observability, monitoring, and alerting for data pipelines.Exposure to streaming data technologies or near real?time data processing.Experience in a fast?growing, platform?focused or SaaS environment.Skills & tools AthenaSNSSQSCloudWatchKinesisRedshiftAWS Data Engineering Advanced: Associate Data Engineer specialization#J-*****-Ljbffr

Requirements

reliability of data platforms.Requirements 5+ years of experience designing, building, and operating data pipelines for large and complex datasets.Strong experience with structured and unstructured data, including JSON.Extensive hands?on experience with advanced SQL for data transformation and analysis.3+ years of experience with Python and PySpark for large?scale data processing.Hands?on experience transforming and processing data using Databricks.3+ years of experience building data solutions on AWS.Experience using Git and CI/CD practices for data and platform development.Strong collaboration skills to support data engineers, analysts, data scientists, and software engineers.Self?directed, ownership?driven mindset with the ability to manage multiple concurrent priorities.Nice to have Experience with Power BI or similar business intelligence and visualization tools.Familiarity with modern data lake and data warehouse architectures.Experience implementing observability, monitoring, and alerting for data pipelines.Exposure to streaming data technologies or near real?time data processing.Experience in a fast?growing, platform?focused or SaaS environment.Skills & tools AthenaSNSSQSCloudWatchKinesisRedshiftAWS Data Engineering Advanced: Associate Data Engineer specialization#J-*****-Ljbffr

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.buscojobs.com.es

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer · Coffee With Developers

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

Videos

See all

Related articles

See all