Staff Applied Scientist - Observability Data Platform
Datadog - Denver, CO
Apply NowJob Description
OverviewStaff Applied Scientist - Observability Data PlatformLocations: Boston, Massachusetts, USA; Denver, Colorado, USA; New York, New York, USAThe Observability Data Platform (ODP) powers the core of Datadog’s telemetry systems, handling exabytes of multimodal observability data. As AI agents become first-class consumers of telemetry, ODP is evolving to meet their demands by scaling with data growth, exposing new query mechanisms, rethinking how telemetry is stored, transformed, and served, and enforcing guardrails that ensure security and reliability.Our team is building an intelligent control plane for production systems. This involves moving beyond passive monitoring to create a platform where AI agents can safely and effectively take action in live environments, integrating techniques from symbolic reasoning, formal methods, and generative AI. We are looking for an experienced Staff Applied Scientist with a background spanning systems engineering, AI, and formal reasoning to apply these skills to reliable systems.You will join the team behind Datadog’s most ambitious projects: evolving observability infrastructure for stochastic, self-improving systems.Datadog values collaboration and balance. We operate as a hybrid workplace to support work-life harmony.What You’ll DoDesign and prototype intelligent systems for AI-native observability, including cost-aware agent orchestration, adaptive query execution, and self-optimizing system components.Lead efforts to apply reinforcement learning, search, or hybrid approaches to infrastructure-level decision-making, such as autoscaling, scheduling, or load shaping.Collaborate with AI researchers and platform engineers to design experimentation loops and verifiers that guide LLM outputs using runtime metrics and formal models.Explore emerging paradigms like AI compilers, “programming after code,” and runtime-aware prompt engineering to inform Datadog’s infrastructure and product design.Help define the direction of BitsEvolve - Datadog’s optimization agent that uses LLMs and evolutionary search to discover code improvements, optimize GPU kernels, and tune configurations to improve performance.Partner with product teams and platform stakeholders to ensure scientific advances translate into measurable improvements in cost, performance, and observability depth.Who You AreYou have a BS/MS/PhD in a scientific field or equivalent experienceYou have 8+ years of experience in systems engineering, database internals, or infrastructure research, including hands-on experience in a production environmentYou have a strong software engineering foundation, ideally in C++, Rust, Go, or Python, and are comfortable writing performant, maintainable codeYou have deep expertise in at least one of the following areas: query optimization, data center scheduling, compiler design, reinforcement learning, or distributed systems designYou have experience applying search, planning, or learning techniques to solve real-world optimization problemsYou are excited by systems that learn, adapt, and improve over time using feedback from runtime metrics and human-defined objectivesYou are hypothesis-driven and enjoy designing experiments and evaluation loops, whether through simulations, benchmarks, or live systemsYou thrive in ambiguity, enjoy reading papers and building prototypes, and want to help shape the future of infrastructure in the AI eraYou enjoy collaborating across research, engineering, and product to bring scientific insights to practical outcomesDatadog values people from all walks of life. We understand not everyone will meet all the above qualifications on day one. If you’re passionate about technology and want to grow your skills, we encourage you to apply.Benefits and GrowthGet to build tools for software engineers, just like yourself. And use the tools we build to accelerate our development.Have a lot of influence on product direction and impact on the business.Work with skilled, knowledgeable, and kind teammates who are happy to teach and learnBenefits and Growth listed above may vary based on the country of your employment and the nature of your employment with Datadog.Datadog offers a competitive salary and equity package, and may include variable compensation. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, Datadog offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, dental, parental planning, and mental health benefits, a 401(k) plan and match, paid time off, fitness reimbursements, and a discounted employee stock purchase plan.The reasonably estimated yearly salary for this role at Datadog is:$234,000 — $300,000 USDDatadog (NASDAQ: DDOG) is a global SaaS business, delivering a rare combination of growth and profitability. We are on a mission to break down silos and solve complexity in the cloud age by enabling digital transformation, cloud migration, and infrastructure monitoring of our customers’ entire technology stacks. Built by engineers, for engineers, Datadog is used by organizations of all sizes across a wide range of industries. We champion professional development, diversity of thought, innovation, and work excellence to empower continuous growth. Join the pack and become part of a collaborative, pragmatic, and thoughtful people-first community where we solve tough problems, take smart risks, and celebrate one another. Learn more about #DatadogLife on Instagram, LinkedIn, and Datadog Learning Center.Equal Opportunity at Datadog: Datadog is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and other characteristics protected by law.Datadog endeavors to make our Careers Page accessible to all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please contact us here.For any questions about this posting or about applying, please contact the Datadog recruitment team.Equal opportunity statements and privacy notices apply to all applicants and employees in accordance with applicable law. #J-18808-Ljbffr
Created: 2025-09-27