Data Engineer
Role details
Job location
Tech stack
Job description
As a Junior Data Engineer , you will be a vital part of the team responsible for the lifeblood of our onboarding process: client data. You will help bridge the gap between legacy client systems and the Artlogic ecosystem by building and executing robust ETL (Extract, Transform, Load) pipelines.
This role is ideal for a developer early in their career who loves the "puzzle" aspect of data. You will work within the Data Migration team to dismantle complex external data structures (Relational data, FileMaker), reassemble them into our sophisticated data model, and contribute to the internal Python/Ruby libraries that make our migrations faster and smarter. You will be mentored by senior engineers and work closely with Project Managers to ensure a seamless transition for our clients. Key Responsibilities ETL Development & Execution
- Execute End-to-End Migrations: Learn to navigate the full migration lifecycle, moving data from diverse client environments into Artlogic.
- Extraction: Use scripts and tools to pull data from external platforms (e.g., FileMaker, legacy SQL databases).
- Transformation: Write clean, maintainable code to map, clean, and transform "messy" legacy data to fit the Artlogic schema.
- QA & Validation: Perform rigorous data validation and unit testing to ensure 100% accuracy and data integrity before go-live.
Technical Growth & Tooling
- Code Contribution: Assist in developing and maintaining our internal migration frameworks and R&D tools.
- Pipeline Optimization: Identify repetitive tasks in your daily workflow and work with senior engineers to automate them.
- Systems Analysis: Research unknown database structures to determine the most efficient extraction methods.
Collaboration & Project Delivery
- Cross-functional Teamwork: Partner with Client Liaison Project Managers to understand client-specific data nuances.
- Documentation: Document migration mappings and technical edge cases to ensure knowledge is shared across the team.
- Iterative Delivery: Adapt migration scripts quickly as project requirements evolve during the onboarding phase.
AI & Data
- AI Recommendation Verification and Data Mapping: Review and verify AI (Claude) recommendations for column mappings during cloud data migration, including confirming that columns fit perfectly or identifying where data requires manipulation or concatenation.
- Data Transformation and Cloud Push: Perform final data transformation steps using basic Python, Pandas, and Reax code to prepare and push relational data from Excel into the cloud environment.
Key Attributes
- Data Integrity: Success is measured by the accuracy of migrated data and the absence of "broken" records post-launch.
- Technical Progression: Demonstrates a growing ability to handle increasingly complex data structures with less supervision.
- Code Quality: Contributions to internal libraries follow team standards and improve overall migration speed.
- Reliability: Consistently meets project milestones and keeps stakeholders informed of technical blockers.
Requirements
- Coding Foundations: Proficiency in at least one object-oriented language- Python is preferred, but we also value strong logic in C#.
- SQL & Data Logic: Solid understanding of relational databases (Joins, Keys, Indexes) and how to manipulate data structures.
- Problem-Solving Mindset: You enjoy the "detective work" of figuring out how a legacy system was built and how to extract its value.
- Detail Oriented: An obsession with data accuracy; you catch the "edge case" before it becomes a bug.
- Communication: Ability to explain technical data hurdles to non-technical Project Managers.
Desirable
- The "Data Stack": Experience with ETL tools, pandas (Python), or regular expressions (Regex).
- API Knowledge: Familiarity with REST APIs and how to handle JSON/XML data.
- Niche Systems: Previous exposure to FileMaker (Claris) or NoSQL environments.
- Web Basics: A high-level understanding of how back-end data interacts with front-end UIs.