AI Engineer
Role details
Job location
Tech stack
Job description
By joining Core AI as an AI Engineer, you'll play a pivotal role in large projects, bringing your engineering and machine learning expertise to build the agentic intelligence, quality and models that our solution squads rely on., * Build and maintain the agent harness - the core runtime that orchestrates LLM calls, tool use, context construction, memory and guardrails - and make it reliable in production.
- Own the evaluation stack: design eval pipelines, golden datasets, LLM-as-judge and human-eval workflows; track agent performance, regressions and failure modes, and turn production failures into systematic improvements.
- Do prompt and behaviour engineering and retrieval/RAG at scale, defining what "good" looks like for agents completing complex accounting workflows end to end.
- Host and improve our own small models where it creates leverage: serving and inference of open-source LLMs, help fine-tuning and post-training (SFT / DPO / RL), and document-extraction models - for performance, cost and sovereignty.
- Work closely with Product teams and domain experts (accountants) so that fulfilling real user needs stays at the center of what we build.
What you can expect
Within one month:
- You'll learn everything about our company, our teams and our vision during the first onboarding week.
- You'll get familiar with our stack and AI tooling (agent harness, evaluation tooling, model providers, MCP), and deliver a few small contributions that give you a concrete taste of our tools and processes.
- You'll be given time to meet your future stakeholders and gain a deep knowledge of our product and operations.
Within 3 months:
- You'll be in charge of items in our roadmap, defining and prioritizing your tasks autonomously.
- You'll be comfortable with our technical stack (Python, agentic frameworks, evaluation tooling, model serving and AWS).
- You'll contribute to larger cross-team projects.
Within 6 months:
- You'll proactively contribute to the team's roadmap.
- You'll work with engineers and data practitioners on improving our stack, AI platform and evaluation practices.
- You'll share your learnings and best practices within the team.
And beyond - the team will keep growing with the company, which means:
- Opportunities to recruit and mentor new team members
- Increased accountability in project leadership
- Responsibilities to design and implement new processes, tools and best practices so your team works even more efficiently
Good to know: across our Tech organisation, engineers can take part in cross-cutting specialist groups ("fellowships") on topics such as evaluation, retrieval, prompt and behaviour design or model serving - a great way to share practices beyond your own squad.
Requirements
Do you have experience in Python?, Are you looking to have an impact on the daily life of millions of entrepreneurs in France (and tomorrow in Europe)? Are you looking for a work environment that values trust, proactivity, and autonomy? Are our Engineering principles aligned with your vision? Then Pennylane is the right place for you!, * Have 5-8 years of experience and are very strong in Python, with hands-on experience building LLM and agentic systems at scale in production - prompting, tool use, context construction, RAG, and handling failure, state and reliability (not just calling a model API).
- Treat evaluation as a first-class discipline: golden datasets, LLM-as-judge, human eval, A/B testing, and measuring agent quality, regressions and edge cases.
- Have a good grasp of applied LLM/ML and AI infrastructure (model serving, vector databases, cost and latency).
- Have exposure to model fine-tuning and post-training (SFT, DPO, RL).
- Have a balanced blend of technical, business and product skills, communicate well (including with non-technical domain experts), and are fluent in English (French is not mandatory)., * To speak English (levels are assessed and appreciated according to the department you're applying to)
- To be energized by an ever-shifting work environment
- To be highly collaborative (within your team and with other stakeholders)
- To be sufficiently experienced to prioritize business-led actions in your day-to-day activity
Benefits & conditions
- A case study interview to discuss a topic closely related to one of our priorities (75 min)
- A past-project interview to hear about your experience (60 min)
- An interview with our Tech & Product leaders to discuss our company culture (60 min)
What we do to make your work life easier
- Wherever you are based, you'll get 25 vacation days paid by Pennylane
- You'll have a competitive compensation package
- You'll get company shares to enjoy a piece of the success story you're building with us
- You'll have a budget to turn your home into a more comfortable workspace, as well as a monthly allowance to work from a coworking space whenever you feel like it
- Through our partner Gymlib, you'll have access to 8,000 fitness spaces in Europe and more than 300 wellness activities
- You'll have access to Busuu to perfect your English or your French
- You'll get the latest Apple equipment
- Depending on the team and the requirements of the position, you'll be able to work remotely from your country of residence, as long as it is in Europe and within a maximum time difference of two hours from the CET time zone
- We regularly come together for company events such as Tech Days (which bring remote Pennylaners together every 2 months) and our annual company seminar
If you are based in France, you'll have a French contract following French regulation, on top of the additional perks: 6 to 12 RTT, 5 weeks of PTO, lunch credits (Swile), Alan Blue healthcare cover, and regular events in cities where Pennylaners are mostly present (Lyon, Bordeaux, Nantes…).
We're working on providing those last advantages to people based outside of France as well, though it can be more complex depending on the country., Recruitment scam attempts are on the rise. We invite you to remain vigilant during your exchanges.