Member of Technical Staff

About the Role:We are always looking to bring on extremely high performing engineers and scientists. There is a lot to build. If you are quick to learn, build, and ship you would thrive in our development environment.This role is intended for those interested in work that might be categorized as “full stack engineer” or “research scientist.” You will contribute to the creation of our public benchmarks and the underlying platform that enables it. We support startups, enterprises and research labs in measuring model performance. Our biggest bottleneck is the speed at which we can build.We are building the standard for evaluating the ability of LLMs to perform real-world tasks. You will contribute directly to that effort in this role.Requirements:A tenacity to iterate and develop quickly.A significant portfolio of prior work (including work at jobs, but especially side projects).Strong experience with Python, especially in production settings. For front-end development, experience with React.Experience working in teams. This includes working in development sprints, knowledge of best practices in working with Git, reviewing pull requests.Strong communication skills. You can provide input to others and equally receive/integrate feedback.We are an in-person team, based in San Francisco. We will support your relocation or transportation as needed.Nice to haves:NLP research experience with papers published in reputable journals.Experience working with Django or other Python-based servers (e.g. Flask).Interest and familiarity with LLM infrastructure.Experience working at other early-stage start-ups or your own company.What we offer:Highly competitive salary and meaningful ownership. Excellence is well rewarded.Relocation and transportation support.Health/dental insurance coverage.Lunch and dinner provided, free snacks/coffee/drinks.Unlimited PTO.About us:Founding team: The core methodology behind this platform comes from NLP evaluation research we had done at Stanford. We raised a 5M seed from some of the top institutional and angel investors in the valley. Our team has prior work experience at NVIDIA, Meta, Microsoft, Palantir and HRT. Collectively, we have over 300 citations in our published work.Tech stack: Our frontend is built in React with TSX. We use Django as our back-end framework. All of the infra is on AWS.What we’re looking for:Intelligence is more important than a good-looking resume. Industry experience and pedigree valuable only insofar as it is a proxy for talent itself.Ownership to create products. We don’t have the scale or time to actively “manage” every project or task. Working in a small, talent-dense team, we expect everyone to show initiative to build where it’s needed, not where it’s asked. We strive for autonomy over tensity. The LLM landscape is constantly changing. Foundation model labs are continuously pushing the frontier, enterprises are seeing massive pressure to adopt technology, startups are hungry to chase the white space. The unicorn companies that will emerge from this technology shift are being built now. Those that win will have an incredibly high speed of execution.See solutions not problems. We’re not looking for people that pass hard problems to others or admit defeat, but instead only see the opportunity to craft solutions at each juncture.Further Reading:Hugging Face blog on evaluationAnthropic’s blog on challenges in evaluationNew York Times article on issues in benchmarkingStanford HAI report showing hallucinations in legal tech toolsReferral BonusKnow someone who would be a good fit? Connect them with . If we hire them and they stay on for 90 days you’ll get a $10,000 referral bonus and Vals AI merch! #J-18808-Ljbffr

Created: 2025-09-17