Sr Data Engineer

Corporate Brokers, LLC
Dresher, PA, United States
about 1 month ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours

Tech stack

SAP Sybase Adaptive Server Enterprise Artificial Intelligence Business Analytics Applications Data Analysis Static Program Analysis Code Generation Code Review Databases Continuous Integration Data Architecture Data Integration Data Integrity
+25 more
Extract Transform Load (ETL) Data Mart Data Warehousing Relational Databases Enterprise Information Management Entity Relationship Models Fault Tolerance Liquibase Meta-Data Management MySQL Oracle (Applications) Oracle SQL Developer Power BI SQL Server Analysis Services Data Processing Data Storage Technologies Data Ingestion Change Data Capture Pandas Build Management Pyspark Star Schema Data Management Database Replication Physical Data Models

Job description

As a Senior Data Engineer, you will enable data-driven decision-making by collecting, transforming, and visualizing data. You will design, build, maintain, and troubleshoot data processing systems with an emphasis on security, reliability, fault tolerance, scalability, data integrity, and efficiency. You will be part of the Enterprise Information Management (EIM) team., * Responsible for protecting, securing, and properly handling all confidential data to ensure against unauthorized access, improper transmission, and/or unapproved disclosure of information that could result in harm to the organization or its clients.

  • Demonstrate professionalism, quality, integrity, and a commitment to organizational values in day-to-day activities.
  • Utilize Artificial Intelligence (AI) tools to accelerate ETL development and data analysis.
  • Design, build, and maintain data structures and data processing systems, including tools, infrastructure, frameworks, and services.
  • Assist with EIM project planning and definition.
  • Implement, support, and troubleshoot deployments.

Data Modeling

  • Create conceptual, logical, and physical data models.
  • Develop Entity Relationship Diagrams (ERDs) and dimensional models.

Data Integration Services

  • Design and build data ingestion tools, metrics, alerts, notifications, and metadata management solutions.
  • Design and build data replication services and Change Data Capture (CDC) processes to incrementally update data warehouses and data marts from systems of record.

ETL Development

  • Contribute to standards and best practices for implementing ETL tools in support of the EIM vision and reference architecture.
  • Build and maintain ETL solutions and supporting platforms.
  • Participate in creating long-term ETL application support models.

Data Warehouse & Data Marts

  • Develop technical standards and specifications addressing ETL performance, security, orchestration, and data warehouse management.
  • Organize metadata and define processes for loading data into and extracting data from data warehouses and data marts.

Leadership & Support

  • Proactively mentor junior team members.
  • Participate in an on-call rotation supporting production issues and outages.
  • Assist with other tasks and projects as assigned.

Requirements

  • 5+ years of relevant experience in data/software development, including data warehousing and ETL tools, techniques, and technologies.
  • Bachelor’s degree or equivalent work experience.
  • Excellent analytical skills.
  • Strong verbal and written communication skills, including active listening, asking effective questions, clarifying requirements, and producing clear documentation.
  • Strong knowledge of Oracle PL/SQL or other database programming languages.
  • Solid understanding of data modeling techniques.
  • Strong knowledge of Data Warehousing concepts and ETL design/tools.
  • Experience with relational databases such as Oracle, Sybase ASE, and MySQL.
  • Working knowledge of SQL Server Analysis Services (SSAS).
  • Experience utilizing AI agents and AI-assisted development tools for code generation, code analysis, and data analysis.
  • Experience with Python for data manipulation and analysis, including Pandas and PySpark.
  • Proven experience diagnosing and resolving performance issues.
  • Strong understanding of data storage, partitioning, and retention strategies.
  • Strong understanding of dimensional data modeling and star schema design.
  • Ability to conduct code reviews and provide constructive, actionable feedback.

Preferred Skills and Experience

  • Experience with data modeling tools such as IDERA’s Embarcadero ER/Studio Data Architect.
  • Experience with Power BI or similar reporting and analytics platforms.
  • Experience with CI/CD processes for database deployments using Liquibase or similar tools.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on corporatebrokers.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:03 min

Accelerating pandas dataframes using cudf module plugins

Ankit Patel Ankit Patel · WWC 2024

2:18 min

Scaling MySQL databases for massive user growth

Johannes Nicolai Johannes Nicolai +1 · LIVE

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

Videos

See all

Related articles

See all