DevOps Pilot is an AI assistant that lives inside Azure DevOps. Instead of hopping across five tabs to find where something was decided, you ask a question and get one streamed answer, grounded in the project’s own work items, pull requests, and wiki.
The Problem
Context in a DevOps project is scattered. The reasoning behind a change sits in a pull request, the requirement in a work item, the how-to in a wiki page. Answering “why was this built this way?” means opening five tabs and stitching the story together by hand.
Architecture
A RAG pipeline unifies three source types — work items, pull requests, and wiki pages — into a single retrievable index of 1,536-dimension embeddings. A query pulls the relevant chunks across all three, and the answer streams back grounded in them, so one question replaces the five-tab hunt.
The backend is FastAPI with server-sent events, deployed on Azure Container Apps. Two Azure OpenAI models do the work: GPT-4.1-mini for generation and text-embedding-3-small for retrieval.
Beyond Q&A
Two auxiliary features ride on the same stack: automated pull-request description drafting, and wiki page translation.
Cost Visibility
Every AI call is metered with tiktoken-based token tracking, powering a dashboard that shows per-call cost across all three features. Cost is a first-class metric, not an afterthought.