Microsoft AI-500: Designing and Implementing Multi-Agent AI Solutions

Last checked October 22, 2026 against the official Microsoft exam page.

1. Question 1Develop multi-agent solutions in Azure

   An agent's answers quietly degrade over several weeks after a provider updates the underlying model. The team wants automatic detection of quality regression. Which practice is most appropriate?

   1. AScheduled automated evaluations on a fixed test set with alerts when scores fall below a threshold
   2. BTurning off scheduled evaluations and dashboards to avoid alert noise during rollouts
   3. CReviewing user complaints once they reach the support desk ticket queue by hand
   4. DRetraining the agent every hour on fresh production logs without a regression test suite
   Show answer and explanationHide answer

   Correct answer: A

   Scheduled evaluations on a fixed test set detect quality changes early, and threshold alerts turn the signal into action. Waiting for complaints, disabling evaluation and retraining without tests do not detect regression reliably.

2. Question 2Secure, govern, and deploy multi-agent solutions

   An agent needs to call a backend API on behalf of the signed-in user, without storing the user's password. Which authentication flow is designed for this?

   1. ASharing the user's password with the agent at setup
   2. BAnonymous calls with the API key in the URL
   3. CBasic authentication with a shared administrator account
   4. DOAuth 2.0 on-behalf-of flow
   Show answer and explanationHide answer

   Correct answer: D

   The on-behalf-of flow exchanges the user's token for a token scoped to the downstream API, so the user's password is never stored. Shared passwords, URL-embedded keys and shared administrator accounts each break these requirements.

3. Question 3Evaluate, optimize, and monitor multi-agent solutions

   A team wants to evaluate open-ended answers from its agent at scale, where exact-match scoring is not useful. Which evaluation method is commonly used?

   1. ALLM-as-a-judge scoring against a rubric, checked by human spot reviews
   2. BCounting the number of characters in each answer after trimming
   3. CChecking whether the answer contains the word the as a keyword match across responses
   4. DMeasuring the server's CPU and memory usage during inference batches per node
   Show answer and explanationHide answer

   Correct answer: A

   A judge model scores answers against a rubric, and human spot checks calibrate the judge. Character counts, single-word checks and CPU metrics do not measure answer quality.

4. Question 4Architect multi-agent solutions

   A claims-processing system must route each claim through intake, fraud review and payout agents. Each step depends on the previous step's approved output, and no two steps can run at once. Which orchestration pattern fits best?

   1. AParallel fan-out to every agent at once with a majority vote on output
   2. BSequential orchestration with a handoff of the approved output between agents
   3. CPeer-to-peer broadcast where each agent chooses its own execution order
   4. DHub-and-spoke dispatch from a central coordinator with no shared state
   Show answer and explanationHide answer

   Correct answer: B

   Sequential orchestration passes each approved output to the next agent, which matches dependent, non-concurrent steps. Parallel fan-out would run dependent steps before inputs exist, peer-to-peer broadcast gives no fixed order, and hub-and-spoke without shared state loses the handoff context.

5. Question 5Develop multi-agent solutions in Azure

   A retrieval-augmented agent returns answers that are close in topic but miss the exact policy clause the user needs. The team suspects the chunking strategy. Which change is most likely to improve retrieval precision?

   1. AChunk documents by logical sections such as clauses, with overlap and metadata for filtering
   2. BRemove document metadata such as clause identifiers, effective dates and owner tags so the index stays smaller
   3. CSplit every sentence into a separate chunk with no surrounding context or clause reference and index it alone
   4. DUse one chunk per entire document regardless of length, with the full text embedded as a single vector per file
   Show answer and explanationHide answer

   Correct answer: A

   Section-aware chunks with overlap and metadata let retrieval match the precise clause and filter by document attributes. Whole-document chunks dilute relevance, sentence-level chunks lose context, and removing metadata reduces filtering precision.