About the role

Product Manager on Evaluators team managing dataset annotation and evaluator development pipeline for EdTech products. Collaborating with data scientists and educators, focusing on AI evaluation in educational tools.

Responsibilities

Own the annotation-to-evaluator pipeline: Manage the end-to-end process from identifying pedagogical constructs through dataset creation, model training, and evaluator validation; balance accuracy requirements with delivery timelines
Shape evaluator roadmap and prioritization: Conduct discovery with EdTech developers building literacy and content generation tools to understand which evaluators deliver the most value; provide input on prioritization based on impact, feasibility, and strategic alignment
Make strategic tradeoffs between accuracy and velocity: Partner with data scientists to set appropriate accuracy thresholds for different evaluator types; decide when to ship, iterate, or restart based on performance against baselines
Translate learning science into product specifications: Work with learning scientists to convert pedagogical frameworks for literacy instruction into clear, measurable evaluation criteria; write detailed specs that guide annotation and model development
Optimize and scale the annotation workflow: Identify opportunities to improve annotator efficiency, reduce ambiguity in guidelines, and increase inter-rater reliability; document learnings to enable expansion into new subject areas

3-5 years of product management experience: A track record of shipping products, with direct experience working on AI/ML products, data annotation pipelines, or model evaluation workflows
Strong technical foundation in AI/ML - Comfort discussing annotation methodologies, model performance metrics, accuracy thresholds, inter-rater reliability, and evaluation frameworks with data scientists; ability to understand technical tradeoffs and make informed product decisions
Exceptional communication and storytelling skills - Ability to translate complex technical and pedagogical concepts into clear narratives for different audiences; skilled at building alignment across diverse stakeholders and making compelling cases for prioritization decisions
Self-directed execution mindset - Thrives in ambiguous, frontier environments where playbooks don't exist; takes initiative to identify problems, propose solutions, and drive outcomes; comfortable making decisions with incomplete information and learning through iteration
Customer-centric approach - Passion for understanding developer needs through direct engagement and feedback loops; commitment to building products that solve real problems; driven by mission-oriented work in education.

Provides a generous employer match on employee 401(k) contributions to support planning for the future.
Paid time off to volunteer at an organization of your choice.
Funding for select family-forming benefits.
Relocation support for employees who need assistance moving