Linux Kernel Engineer

We are seeking a Principal / Senior level engineer with deep Linux kernel expertise to own testing, validation, and quality strategy for the custom Linux kernel that powers our global cloud and CDN platform. This is not a traditional QA automation role. We need someone who can operate at the kernel level: investigating difficult low-level issues, debugging kernel behavior, and identifying root causes of performance, stability, and security problems across drivers, memory management, networking, storage, and virtualization. You will design and build the testing infrastructure needed to validate kernel behavior at scale, from single-machine driver and filesystem tests to complex multi-machine, distributed test scenarios., * Own testing, validation, and quality strategy for our custom Linux kernel

• Lead the annual kernel upgrade lifecycle: adopting new Linux Long-Term Support (LTS) kernels, rebasing our custom modifications, porting internal features, executing validation testing, running pilot deployments, resolving discovered issues, and rolling out broadly
• Investigate and debug complex kernel-level issues, identifying root causes of performance, stability, and security problems across drivers, memory management, networking, storage, and virtualization subsystems
• Design tests and build regression suites - for example, when a driver has corrupted a filesystem under certain conditions, investigate the root cause, reproduce the failure, and design a permanent regression test to ensure future kernels don’t reintroduce the issue
• Architect and build automated testing infrastructure for single-machine tests (driver, storage, filesystem, and basic kernel functionality validation) and multi-machine, distributed tests (load balancing, networking, CDN traffic flows, distributed systems behavior, compute platform workloads)
• Build tooling to provision test machines, configure environments, run distributed tests, capture results, tear down environments, and return hardware to the test pool
• Set quality standards, influence architecture decisions, and mentor engineers as a technical leader on the team, * Linux Kernel Fundamentals: Scheduling, memory management concepts, device drivers, and the networking stack
• Troubleshooting Stories: Your most difficult kernel bug, a production outage investigation, a performance bottleneck root-cause analysis, or a driver issue investigation
• Automation & Test Design: How you would design a distributed test framework, validate a new kernel release, or build regression testing for a hardware driver issue
• Programming: C experience, Python/Shell scripting, and any exposure to Perl

Pay: $175,000.00 - $300,000.00 per year

• Linux Kernel Internals: Deep, hands-on experience with kernel internals, including KVM, virtio, memory management, cgroups, the TCP/IP stack, Ethernet drivers, file systems, and block I/O
• C Programming: Significant experience reading, understanding, debugging, and potentially modifying kernel code - not simply reading it
• Linux Debugging Tools: Proficiency with tools such as crash, perf, ftrace, eBPF, kernel logs, and tracing tools
• Experience: 10-15+ years in Linux kernel engineering, systems software development, infrastructure/platform engineering, kernel validation and testing, or performance engineering, * Python: Preferred language for building automation and test tooling
• Shell Scripting: Comfortable scripting in Linux environments
• Perl: Important due to our existing codebase. Candidates strong in Python and Shell who are willing to learn Perl are welcome to apply

NICE TO HAVE:

• CI/CD experience (Jenkins, GitLab CI, or similar build systems) - note that this is not primarily a CI pipeline role, * A proactive self-starter who identifies issues and proposes solutions rather than waiting for assignments
• Deep technical curiosity and genuine enjoyment of diving into hard, ambiguous problems at the kernel level
• Comfort working with high autonomy in a fully remote environment
• Demonstrated technical leadership: setting standards, influencing architecture, and mentoring other engineers

(part of Express Employment Professionals) 3.43.4 out of 5 stars United States Remote $175,000 - $300,000 a year - Full-time