> Markdown version of [/jobs/ext/2465381-staff-r-d-data-engineer-hardware-systems](https://www.wearedevelopers.com/jobs/ext/2465381-staff-r-d-data-engineer-hardware-systems). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Staff R&D Data Engineer, Hardware Systems - **Company:** Lydian Partners, LLC - **Location:** Boston, MA, United States - **Experience:** Expert - **Salary:** $160,000.0 - $210,000.0 - **Contract:** Permanent contract - **Skills:** Airflow, Data Analysis, Databases, Data Validation, Data Infrastructure, Data Visualization, Database Schema, Database Testing, Software Debugging, Experimental Data, Information Lifecycle Management, Python (Programming Language), LabVIEW, Regression Analysis, NumPy, Query Optimization, Software Systems, SQL Databases, JMP (Statistical Software), Data Storage Technologies, Snowflake, Pandas, Information Technology - **Published:** August 4, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=fcb0c8403083dc0a ## About the Role * BS/MS in engineering, physical science, computer science, data science, or related field * 10+ years experience with data in a hardware, lab, or physical-science R&D environment (software-only experience does not qualify) * Demonstrated strength across the full data lifecycle: acquisition, infrastructure, pipelines, visualization, and statistical analysis * Proven track record establishing or improving data storage and analysis infrastructure * Expert level SQL and Python proficiency (pandas/NumPy, query optimization, schema design) * Experience handling time-series and sensor data; alignment, gap-filling, and reconciling schema drift across instrument runs and batches * Statistical/regression analysis on experimental data, with the ability to turn it into clear visualizations and dashboards * Proficiency with at least one DAQ platform (LabVIEW, DAQ Factory, or Ignition) * Fluency using AI coding assistants to scaffold and debug pipelines and SQL, with judgment to validate their output * Strong communication skills; able to work independently as the team's senior-most data professional, * Background in chemical/materials/mechanical engineering * Modern data-stack tools (Airflow, Dagster, dbt, Snowflake/DuckDB/Iceberg) * Automated data testing (Great Expectations, dbt tests) * Experience with Notion, Nominal, or JMP * Additional DAQ platform familiarity * Basic IT and networking skills ## Description Lydian is looking for an R&D Data Engineer to build and own the data backbone of our technical organization. As we scale from pilot operations toward commercial production, our teams are generating a growing volume of experimental, manufacturing, and quality data. The ability to store, connect, and analyze that data well has become central to how quickly we can learn and improve. In this role you will develop our data infrastructure, build the analysis and visualization tools our engineers and scientists rely on, and carry out statistical analyses in support of the entire technical team., This is a data generalist role at the intersection of hardware and software. You should be equally at home designing a database schema, writing Python to wrangle time-series data streaming off lab instruments, building a clear plot or dashboard, and talking through an analysis with a process engineer. You will report to the VP of R&D and work closely with the full R&D team as well as our mechanical and process engineering groups., * Design and maintain data infrastructure. Establish structured, systematic data storage across our tools (currently Nominal and Notion, plus others as needs evolve). Ensure experimental run data (time series), run metadata, and manufacturing and QC data are integrated, consistently formatted, and easy to sort, filter, and query for analysis * Build analysis and visualization tools. Create the data views, plots, and statistical tools that help the team analyze data faster and better. For example, this will include standardizing and automating run summary statistics and overall performance metrics across R&D and pilot experiments * Enable laboratory data acquisition. Program and maintain data-acquisition software (e.g., DAQ Factory, Ignition) so that R&D and pilot test equipment stream reliably into our centralized databases. Build in data validation checks that quality-control data at the point of acquisition or entry * Run cross-functional analyses. Perform statistical and regression analyses that connect manufacturing and QC data with experimental run data to surface insights across functions * Partner across the technical team. Work with scientists and engineers to understand their data needs and translate them into reliable tools and workflows, and help maintain the IT infrastructure of servers and software systems that house our data ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Advanced Typing in TypeScript](https://www.wearedevelopers.com/videos/496-advanced-typing-in-typescript) - [Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.](https://www.wearedevelopers.com/videos/562-vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast) - [Data Science on Software Data](https://www.wearedevelopers.com/videos/162-data-science-on-software-data) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Python Data Visualization @ Deepnote (w/ PyViz overview)](https://www.wearedevelopers.com/videos/113-python-data-visualization-deepnote-w-pyviz-overview) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)