Manager - Model Evaluation

In this role, you will: - Team Leadership & Execution: Lead, mentor, and scale a high-performing team of Data Scientists, ML Validation Engineers, and Software Engineers while driving roadmaps, sprint execution, resource allocation, and high-throughput model releases with rigorous safety guardrails. Culture of Rigor: Foster a culture of statistical excellence, healthy skepticism, proactive risk tracking, and data-driven decision-making.

- Validation Strategy & Methodologies: Define and execute end-to-end validation strategies across offline evaluation, open/closed-loop simulation, and shadow-mode fleet benchmarking to ensure robust behavioral model performance. Statistical uncertainties, and regressions into clear, data-driven recommendations for release gating and executive leadership.

- Metrics, Release Gating & Rigor: Oversee metric development and standardization with System Safety and Autonomy teams, establishing quantitative go/no-go release criteria for Behavioral Planner and Prediction ML models while fostering statistical rigor and proactive risk management.

- Cross-Functional & Infrastructure Partnership: Partner closely with Planner, Prediction, MLOps, and Developer Efficiency teams to translate behavioral requirements into measurable validation targets, streamline dataset and evaluation pipelines, and optimize runtime and compute costs.

- Executive Communication & Decision-Making: Translate complex model performance trade-offs, statistical uncertainty, regressions, and safety risks into clear, data-driven recommendations for release decisions and executive leadership.

Qualifications: - Experience: Masters or PhD in CS, Robotics, Applied Statistics or a related field and 3+ years of direct engineering management experience leading Data Science, Machine Learning, or V&V engineering teams, alongside 7+ years of technical experience in robotics, autonomous systems, or AI/ML.

- Domain Knowledge: Strong background in ML model validation, behavioral evaluation frameworks, system-level performance benchmarking, and statistics.

- Software & Systems Literacy: Strong technical foundation in Python and modern data/ML platforms, with exposure to or conceptual literacy in large-scale production codebases (C++ or distributed systems). Proven ability to partner with systems software engineers, review technical architecture, and understand compute/performance trade-offs, Track record of leading teams evaluating complex robotic systems

- Technical Depth: Proven familiarity with modern C++/Python ML environments, simulation frameworks, high-throughput ML evaluation pipelines.

- Cross-Functional Leadership: Demonstrated ability to navigate complex organizational trade offs between release velocity, compute cost, and safety rigor.

Bonus Qualification: - Experience with autonomous vehicles, robotics, or other safety-critical systems.