AI Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
Job description
By joining Core AI as an AI Engineer, youâll play a pivotal role in large projects, bringing your engineering and machine learning expertise to build the agentic intelligence, quality and models that our solution squads rely on., * Build and maintain the agent harness - the core runtime that orchestrates LLM calls, tool use, context construction, memory and guardrails - and make it reliable in production.
- Own the evaluation stack: design eval pipelines, golden datasets, LLM-as-judge and human-eval workflows; track agent performance, regressions and failure modes, and turn production failures into systematic improvements.
- Do prompt and behaviour engineering and retrieval/RAG at scale, defining what âgoodâ looks like for agents completing complex accounting workflows end to end.
- Host and improve our own small models where it creates leverage: serving and inference of open-source LLMs, help fine-tuning and post-training (SFT / DPO / RL), and document-extraction models - for performance, cost and sovereignty.
- Work closely with Product teams and domain experts (accountants) so that fulfilling real user needs stays at the center of what we build.
What you can expect
Within one month:
- Youâll learn everything about our company, our teams and our vision during the first onboarding week.
- Youâll get familiar with our stack and AI tooling (agent harness, evaluation tooling, model providers, MCP), and deliver a few small contributions that give you a concrete taste of our tools and processes.
- Youâll be given time to meet your future stakeholders and gain a deep knowledge of our product and operations.
Within 3 months:
- Youâll be in charge of items in our roadmap, defining and prioritizing your tasks autonomously.
- Youâll be comfortable with our technical stack (Python, agentic frameworks, evaluation tooling, model serving and AWS).
- Youâll contribute to larger cross-team projects.
Within 6 months:
- Youâll proactively contribute to the teamâs roadmap.
- Youâll work with engineers and data practitioners on improving our stack, AI platform and evaluation practices.
- Youâll share your learnings and best practices within the team.
And beyond - the team will keep growing with the company, which means:
- Opportunities to recruit and mentor new team members
- Increased accountability in project leadership
- Responsibilities to design and implement new processes, tools and best practices so your team works even more efficiently
Good to know: across our Tech organisation, engineers can take part in cross-cutting specialist groups (âfellowshipsâ) on topics such as evaluation, retrieval, prompt and behaviour design or model serving - a great way to share practices beyond your own squad.
Requirements
Do you have experience in Python?, Are you looking to have an impact on the daily life of millions of entrepreneurs in France (and tomorrow in Europe)? Are you looking for a work environment that values trust, proactivity, and autonomy? Are our Engineering principles aligned with your vision? Then Pennylane is the right place for you!, * Have 5-8 years of experience and are very strong in Python, with hands-on experience building LLM and agentic systems at scale in production - prompting, tool use, context construction, RAG, and handling failure, state and reliability (not just calling a model API).
- Treat evaluation as a first-class discipline: golden datasets, LLM-as-judge, human eval, A/B testing, and measuring agent quality, regressions and edge cases.
- Have a good grasp of applied LLM/ML and AI infrastructure (model serving, vector databases, cost and latency).
- Have exposure to model fine-tuning and post-training (SFT, DPO, RL).
- Have a balanced blend of technical, business and product skills, communicate well (including with non-technical domain experts), and are fluent in English (French is not mandatory)., * To speak English (levels are assessed and appreciated according to the department youâre applying to)
- To be energized by an ever-shifting work environment
- To be highly collaborative (within your team and with other stakeholders)
- To be sufficiently experienced to prioritize business-led actions in your day-to-day activity
Benefits & conditions
- A case study interview to discuss a topic closely related to one of our priorities (75 min)
- A past-project interview to hear about your experience (60 min)
- An interview with our Tech & Product leaders to discuss our company culture (60 min)
What we do to make your work life easier
- Wherever you are based, youâll get 25 vacation days paid by Pennylane
- Youâll have a competitive compensation package
- Youâll get company shares to enjoy a piece of the success story youâre building with us
- Youâll have a budget to turn your home into a more comfortable workspace, as well as a monthly allowance to work from a coworking space whenever you feel like it
- Through our partner Gymlib, youâll have access to 8,000 fitness spaces in Europe and more than 300 wellness activities
- Youâll have access to Busuu to perfect your English or your French
- Youâll get the latest Apple equipment
- Depending on the team and the requirements of the position, youâll be able to work remotely from your country of residence, as long as it is in Europe and within a maximum time difference of two hours from the CET time zone
- We regularly come together for company events such as Tech Days (which bring remote Pennylaners together every 2 months) and our annual company seminar
If you are based in France, youâll have a French contract following French regulation, on top of the additional perks: 6 to 12 RTT, 5 weeks of PTO, lunch credits (Swile), Alan Blue healthcare cover, and regular events in cities where Pennylaners are mostly present (Lyon, Bordeaux, NantesâŚ).
Weâre working on providing those last advantages to people based outside of France as well, though it can be more complex depending on the country., Recruitment scam attempts are on the rise. We invite you to remain vigilant during your exchanges.
About the company
Pennylane is one of the fastest growing Fintechs in France (and soon in Europe!). In 5 years of existence, weâve managed to:
- Make ourselves known as a groundbreaking accounting and financial software for small businesses and their accountants
- Raise a total of âŹ400 million, including from Sequoia - the famous Silicon Valley fund that invested early in companies like Google, Facebook, Airbnb, Stripe and Paypal
- Grow from 7 cofounders to 1,000 happy Pennylaners, and earn a place among the greatest companies to work for in France (and remotely), with a 4.6/5 rating on Glassdoor
- Build an international environment with more than 25 nationalities and a strong remote-friendly culture, where 30% of employees already work from all parts of Europe
- Earn the trust of thousands of customers and accounting firms, with outstanding ratings
- Reach more than 1,000,000 small and medium-sized enterprises (SMEs) and over 6,000 accounting firms using Pennylane in France, AI is reshaping how entrepreneurs and accountants work - and a growing share of the traffic to our product will increasingly come from agents, not only from humans. The Core AI team sits at the heart of our AI strategy: while our solution squads (Copilot, Autopilot, AI Studio) build customer-facing AI features, Core AI builds the shared foundations they all rely on, giving Pennylane a durable technical edge - including on cost and sovereignty.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on indeed.comGood distractions
Talks and stories from around this role â technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
What Are Large Language Models?
Navigating the AI Shift
Best Companies to Work For in Paris: Top 25 Companies in 2023Â
MLOps And AI Driven Development