Security Engineer

We are looking for a strong security engineer to join our team at Vals AI. You will be a generalist, handling tasks on both product and corporate security alike. The role will require a high degree of ownership, autonomy, and responsibility, and will provide the opportunity to work on the cutting edge of the AI industry.

We work with all the major foundation model labs, some of the largest financial institutions, and hospital systems in the world. Our work has been featured by the Wall Street Journal, Washington Post, and Bloomberg.

We are building the standard for evaluating the ability of LLMs to perform real-world tasks. You secure the systems that make this possible.

What You’ll Do

Overall

• Help set security standards and practices across the organization
• Conduct internal penetration testing of our product and systems, and work with external penetration testing firms.
• Lead incident response activities and investigations into how incidents occurred

Product security

• Partner with engineers to embed security into the SDLC, including threat modeling new features, reviewing designs, and providing guidance during development
• Establish logging, monitoring, and detection capabilities to catch suspicious activity across both corporate and production environments
• Secure cloud infrastructure (AWS), including IAM, security groups, and network isolation and controls

Corporate Security

• Secure our physical device fleet, establishing hardened baselines to deploy via MDM
• Deploy and tune email security tooling to defend against phising attacks and email spoofing.
• Secure corporate network infrastructure, including office networks and VPN/Tailscale

We expect approximately 70% / 30% split between product and corporate security (depending on the needs of the business at any given time).

• 2+ Years of Experience: Counting only full-time professional experience
• Network and Application Security: Experience in designing secure networks, systems, and application architectures
• Cloud Security: Knowledge of cloud security best practices, especially relating to AWS services.
• Monitoring and Prevention: Expertise in anti-malware software, intrusion detection, firewalls, and content filtering
• Risk Assessment: Knowledge of risk assessment tools, technologies, and methods. We are looking for the ability to accurately predict and model likely threats.
• IT Security: Including MDMs, EDR systems, DNS filtering, corporate networking, and other security tools.
• Programming Experience: You should feel comfortable writing, reviewing, and testing code.
• Location: We are an in-person team based in San Francisco. We will support your relocation or transportation as needed.

Nice-to-Haves

• Penetration Testing: Prior experience with red-teaming software applications or participating in bug bounties.
• Python experience: Python experience will be critical for application-layer security.
• Familiarity with the LLMs: It is a bonus if you are already familiar with the industry.
• Startup Background: Previous experience working at a startup, or starting your own company

Pulled from the full job description

• 401(k)
• Health insurance
• Dental insurance
• Unlimited paid time off
• Snacks provided, * Highly competitive salary and meaningful ownership. Excellence is well rewarded.
• Relocation and transportation support
• Health/dental insurance coverage
• Lunch and dinner provided, free snacks/coffee/drinks
• 401K plan
• Unlimited PTO

Founding team: The core methodology behind this platform comes from NLP evaluation research we had done at Stanford. We raised a $5M seed from some of the top institutional and angel investors in the valley. Our team has prior work experience at NVIDIA, Meta, Microsoft, Palantir and HRT. Collectively, we have over 300 citations in our published work. Our early team include Stanford PhDs, ex-Jane Street quants, and the first designer at Snorkel.

Tech stack: We use Python for most things at Vals. Our platform is built on Django, with a React frontend. All of the infra is on AWS using CDK for IaC.

What We’re Looking For

• Learning velocity: The role encompasses a wide variety of tasks. Rather than expecting you to be an expert on Day 1, we are looking for someone who can learn new skills and technologies extremely quickly.
• Ownership: Working in a small, talent-dense team, we expect everyone to show initiative to build where it’s needed, not where it’s asked. We strive for autonomy over consensus. This is especially true for this role.
• Intensity: The LLM landscape is constantly changing. Foundation model labs are continuously pushing the frontier. The unicorn companies that will emerge from this technology shift are being built now. Those that win will have an incredibly high speed of execution.
• Solution-oriented mindset: We’re looking for people who see opportunities to craft solutions at each juncture, not those who pass hard problems to others or admit defeat.

Further Reading:

• Hugging Face blog on evaluation
• Anthropic’s blog on challenges in evaluation
• New York Times article on issues in benchmarking
• Stanford HAI report showing hallucinations in legal tech tools