Senior Engineer I, Inference Service

Join to apply for the Senior Engineer I, Inference Service role at DigitalOcean3 days ago Be among the first 25 applicantsJoin to apply for the Senior Engineer I, Inference Service role at DigitalOceanDive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world.We’re seeking an experienced Software Engineer to drive the design, optimization and scaling of our inference systems. In this role, you will build systems for optimized inference serving of well known open source / open weights models, develop novel techniques for serving custom models and scale the platform to handle millions of users across the globe.As a Senior Software Engineer at DigitalOcean, you will join a dynamic team dedicated to revolutionizing cloud computing and AI.What You’ll DoDesign and implement distributed inference platform for serving large language modelsOptimize runtime and infrastructure layers of the inference stack for best model performanceBuilding native cross platform inference support across NVIDIA and AMD GPUs for a variety of model architecturesContribute to open source inference engines to make them perform better on DigitalOcean cloud.Building tooling and observability to monitor system health, and build auto tuning capabilitiesBuilding benchmarking frameworks to test model serving performance to guide system and infrastructure tuning effortsMentor engineers on inference systems, GPU infrastructure and distributed inference best practicesRequired QualificationsWhat You’ll Add to DigitalOcean:Bachelor’s or Master’s in Computer Science, Electrical Engineering, or related fieldHave experience building distributed systems with Kubernetes, gRPC, golang, pythonHave experience with GPU programming with CUDA, ROCmHave experience with L3-L7 network protocols, block storage, object storageHave experience with building multi-tenant systems - identity management, tenant isolation etcProven track record defining and achieving performance KPIs (latency, throughput, cost)Preferred QualificationsExperience with one or more inference engines: vLLM, SGLang, Modular etcFamiliarity with one or more distributed inference serving frameworks - llm-d, NVIDIA dynamo, Ray Serve etcKnowledge of distributed inference optimization techniques - tensor/data parallelism, KV cache optimizations, smart routing etcFamiliarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization etc)Experience with GPU Interconnect technologies like NVlink, XGMI, RoCEOpen source contributions to inference libraries, frameworks, model kernels etcWhy You’ll Like Working For DigitalOceanWe innovate with purpose. You’ll be a part of a cutting-edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.We prioritize career development. At DO, you’ll do the best work of your career. You will work with some of the smartest and most interesting people in the industry. We are a high-performance organization that will always challenge you to think big. Our organizational development team will provide you with resources to ensure you keep growing. We provide employees with reimbursement for relevant conferences, training, and education. All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.We care about your well-being. Regardless of your location, we will provide you with a competitive array of benefits to support you from our Employee Assistance Program to Local Employee Meetups to flexible time off policy, to name a few. While the philosophy around our benefits is the same worldwide, specific benefits may vary based on local regulations and preferences.We reward our employees. The salary range for this position is between $114,200 - $142,700 based on market data, relevant years of experience, and skills. You may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance. We also provide equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.We value diversity and inclusion. We are an equal-opportunity employer, and recognize that diversity of thought and background builds stronger teams and products to serve our customers. We approach diversity and inclusion seriously and thoughtfully. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.This is a remote roleSeniority levelSeniority levelMid-Senior levelEmployment typeEmployment typeFull-timeJob functionJob functionSupply Chain, Information Technology, and EngineeringIndustriesInternet PublishingReferrals increase your chances of interviewing at DigitalOcean by 2xSign in to set job alerts for “Senior Engineer” roles.Boston, MA $150,000.00-$175,000.00 3 months agoBoston, MA $100,000.00-$110,000.00 2 weeks agoBoston, MA $150,000.00-$180,000.00 3 days agoBoston, MA $140,000.00-$200,000.00 2 days agoSoftware Developer – Full stack/back-endBoston, MA $190,000.00-$210,000.00 3 weeks agoBoston, MA $140,000.00-$170,000.00 6 days agoSoftware Engineer, AI Studio - Boston, USABoston, MA $140,000.00-$200,000.00 2 days agoBoston, MA $120,000.00-$240,000.00 1 month agoSenior Software Engineer, Backend - FintechBoston, MA $175,000.00-$200,000.00 3 hours agoBedford, MA $100,000.00-$120,000.00 1 week agoWe’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr

Created: 2025-09-17