Principal Product Manager - Copilot Extensibility DevX

What you'll do

• Set product vision, investment priorities, quality models, regression protections, and release criteria for plugin evaluation.
• Integrate evaluation into developer workflows and connect results with production telemetry, adoption, customer, support, and business-value signals.
• Define evaluation methods for AI outputs and agent workflows; lead cross-organizational product and technical decisions.

What they ask for

• Bachelor's Degree and 8+ years of experience in product/service/program management or software development, or equivalent experience.
• Experience with AI or LLM evaluation, including methods such as LLM-as-judge, human-rating calibration, or multi-step agent/tool-use evaluation.
• Experience defining product strategy and leading developer platforms, extensibility systems, AI products, or evaluation capabilities.
• Experience with data-driven product management and partnering with engineering on technical platforms, APIs, SDKs, or production services.

Nice to have

• Hands-on software development or equivalent experience building prototypes and integrations using APIs, SDKs, or AI-assisted development tools.
• Experience building products for technical audiences and influencing senior stakeholders across organizations.

Pay

USD $142,800 - $274,800 per year (base salary); USD $188,000 - $304,200 per year (base salary)

Tools

CLI, IDE, CI/CD, APIs, SDKs, LLM-as-judge

AI focus

Develops evaluation and quality-measurement capabilities for Copilot plugins, including Responsible AI, nondeterministic outputs, agent and tool-use flows, and continuous improvement.