Senior Software Engineer, ML Platform | Parafin

Carnaby Fox
San Francisco, CA, United States
2 months ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
$230,000.0
Working hours
Shift work
Job source

Tech stack

Airflow Amazon Web Services Batch Processing Big Data Software Quality Data Infrastructure Programming Tools Python (Programming Language) Machine Learning Software Architecture Standard Sql Azure Machine Learning
+14 more
Software Engineering SQL Databases Data Streaming Feature Engineering System Availability Delivery Pipeline Apache Spark Model Validation Pyspark Low Latency Apache Kafka Machine Learning Operations Data Pipelines Databricks

Job description

Parafin is seeking a Senior Software Engineer, ML Platform to own and scale the infrastructure powering machine learning-driven underwriting and financial products. This is a unique opportunity to build a critical ML platform from the ground up, enabling Data Scientists to efficiently develop, deploy, monitor, and scale production ML models that directly impact small businesses across partner ecosystems including Amazon, DoorDash, Walmart, and TikTok.

What You’ll Do

  • Architect and build the next generation ML Platform supporting experimentation, training, deployment, inference, monitoring, and retraining.
  • Transform data science workflows into production-grade software by building reusable libraries, pipelines, frameworks, SDKs, CLIs, templates, and developer tooling.
  • Design and scale real-time inference infrastructure, ensuring low latency, reliability, and high availability.
  • Expand and optimize large-scale batch inference systems, focusing on scheduling, observability, cost optimization, rollback capabilities, and operational excellence.
  • Own and evolve the feature store ecosystem, including offline and online feature management, point-in-time correctness, high-throughput access patterns, and feature governance.
  • Drive platform reliability through monitoring, alerting, dashboards, incident response, model performance tracking, drift detection, data quality validation, and infrastructure observability.
  • Partner closely with Data Science, Infrastructure, and Product teams to support underwriting systems, define model interfaces, establish SLAs, and implement production safeguards.

Requirements

Do you have experience in Statistics?, 5+ years of Software Engineering experience, including ML Platform, MLOps, Data Infrastructure, or Machine Learning Engineering environments.

Strong software engineering fundamentals with expertise in:

  • Python
  • SQL
  • Software architecture & design
  • Testing and code quality

Hands-on experience with:

  • Spark / PySpark
  • Databricks
  • MLflow
  • Airflow (or equivalent orchestration tools)
  • AWS Cloud Services

Experience building:

  • Feature Stores
  • Model Registries
  • ML Deployment Pipelines
  • Real-Time Inference Systems
  • Large-Scale Data Processing Platforms
  • Batch and Streaming Data Architectures

Strong understanding of:

  • ML lifecycle management
  • Model evaluation & validation
  • Feature engineering
  • Drift monitoring
  • Experiment tracking
  • Production ML best practices
  • Probability & Statistics fundamentals

Proven ability to build scalable platforms that enable Data Scientists and ML Engineers to operate efficiently at scale.

Preferred Qualifications

Experience with:

  • Kafka or Kinesis
  • Feast, Tecton, or similar Feature Store technologies
  • Databricks Model Serving
  • Low-latency model serving architectures
  • A/B testing platforms
  • Shadow deployments
  • Canary releases
  • Automated rollback frameworks

Domain expertise in:

  • FinTech
  • Lending
  • Credit Risk
  • Underwriting
  • Risk Modeling
  • Financial Services

Previous experience in startup or high-growth environments where you’ve built systems from the ground up and influenced technical strategy.

Ideal Candidate

We’re looking for an engineer who thinks beyond model development and focuses on building the infrastructure that makes ML teams successful. The ideal candidate has built reusable, scalable, software-engineering-grade platforms that empower data scientists to ship models safely and efficiently into production.

You’ll thrive in this role if you enjoy solving platform-scale challenges, influencing architectural direction, working in fast-paced environments, and taking ownership of mission-critical systems. Experience supporting high-volume ML workloads and enabling cross-functional teams is highly valued., * “Do you have experience building and scaling ML platforms/MLOps systems, including model deployment, feature pipelines, real-time/batch processing, and technologies such as Python, Spark/PySpark, AWS, Databricks, MLflow, and Airflow?” (Yes/No)

  • Current Compensation ?

Expected Compensation?

  • Do you possess all the required skills ;

Skills: Python, SQL, Spark/PySpark, AWS, Databricks, MLflow, Airflow, Feature Stores, Real-Time & Batch Pipelines, MLOps/ML Platforms. (Yes/No)

Experience:

  • Software Engineering: 5 years (Required)
  • ML Platform & MLOps Infrastructure, Training, Deployment : 5 years (Required)

Work Location: Hybrid remote in San Francisco, CA 94114

Benefits & conditions

Pulled from the full job description

  • Relocation assistance
  • Visa sponsorship
  • Flexible schedule, * Flexible schedule

About the company

Direct impact on financial products serving millions of users through leading technology platforms

Opportunity to shape company-wide ML architecture and infrastructure strategy

Backed by top-tier investors with over $194M raised

High-growth environment with significant investment in platform modernization and scalability

Competitive compensation, equity participation, and visa sponsorship support

Work alongside exceptional engineers, data scientists, and infrastructure leaders solving complex real-world challenges.

Tech Stack

Python SQL Spark PySpark Databricks MLflow Airflow AWS Snowflake Kafka Kinesis Feature Stores Real-Time Inference MLOps Platforms

If you’re passionate about building world-class ML infrastructure and enabling machine learning at scale, we’d love to hear from you.

Hiring #SeniorSoftwareEngineer #MLPlatform #MLOps #MachineLearningEngineering #Python #Spark #PySpark #Databricks #MLflow #AWS #Kafka #FeatureStore #DataEngineering #FinTech #InfrastructureEngineering #SoftwareEngineering #AIJobs #TechHiring #SanFranciscoJobs #MachineLearningInfrastructure #DistributedSystems #PlatformEngineering #HiringNow

Pay: From $230,000.00 per year

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

4:18 min

Prioritizing communication and structural awareness over strict tool mastery

Liam Hurrel +1 · WWC 2021

Videos

See all

Related articles

See all