Sr. Data Engineer
Role details
Job location
Tech stack
Job description
- Help design, develop, and optimize scalable data pipelines and data solutions using Databricks, PySpark, SQL, and Python.
- Help build and maintain reliable data models and ETL/ELT processes that support analytics, business intelligence, machine learning and AI.
- Help establish and maintain data engineering standards, governance, security, and best practices across the Azure and Databricks platforms.
- Evaluate new Databricks capabilities and recommend adoption strategies, standards, and implementation patterns.
- Monitor, troubleshoot, and optimize data pipelines and platform performance for reliability, scalability, and cost efficiency.
- Partner with cross-functional teams to solve complex data challenges and provide technical guidance on data architecture and engineering best practices.
- Mentor junior engineers and contribute to a culture of technical excellence and continuous improvement.
These duties must be performed with or without reasonable accommodation. We know experience comes in many forms and that many skills are transferable. If your experience is close to what we're looking for, consider applying. Diversity has made us the entrepreneurial and innovative company that we are today.
Requirements
- 5+ years of experience in Data Engineering, Data Platform Engineering, or a related field.
- Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Azure Data Lake Storage, and Delta Lake.
- Deep understanding of distributed data processing and large-scale data architectures.
- Strong knowledge of data engineering principles, including data modeling, ETL and ELT design patterns, Data quality and observability, Metadata management, Data lifecycle management, Master Data Management concepts.
- Strong understanding of modern table formats and storage architecture.
- Experience with performance optimization techniques.
- Experience implementing CI/CD and infrastructure automation using tools.
- Excellent problem-solving, analytical, and communication skills.
- Ability to operate independently while leading initiatives in a fast-paced, agile environment.
- Passion for building scalable, maintainable, and well-governed data platforms., * Experience with Unity Catalog, Databricks Workflows, Delta Live Tables / Spark Declarative Pipelines, Structured Streaming, Kafka or Event Hubs.
- Experience implementing data governance frameworks and cloud security best practices.
- Experience with RBAC, ABAC, data classification, auditing, and compliance controls.
- Experience supporting self-service analytics platforms and integrating Databricks with Power BI or other analytics tools.
- Experience evaluating and adopting emerging cloud and data platform technologies.
- Experience creating engineering standards, reference architectures, and operational best practices for enterprise data platforms.
- Experience building and delivering internal training or enablement programs for technical platforms.
Benefits & conditions
Pulled from the full job description
- Tuition reimbursement
- Pet insurance
- AD&D insurance
- Health insurance
- 401(k) matching
- Paid time off
- Vision insurance, * Medical, Dental and Vision insurance for you and your family
- Relax and recharge with Paid Time Off (PTO)
- 6 company-observed paid holidays, plus 3 paid floating holidays
- 401k (after 90 days) plus employer match up to 4%
- Pet Insurance for your furry family members
- Wellness perks including onsite fitness equipment at both locations, EAP, and access to the Headspace App
- We invest in your future through Tuition Reimbursement
- Save on taxes with Flexible Spending Accounts
- Peace of mind with Life and AD&D Insurance
- Protect yourself with company-paid Long-Term Disability and voluntary Short-Term Disability
Concora Credit provides equal employment opportunities to all Team Members and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. Employment-based visa sponsorship is not available for this role. Concora Credit is an equal opportunity employer (EEO).