CertArea
All ISACA exams

Free Advanced in AI Audit (AAIA) practice test: 30 questions with answers

30 original questions for Advanced in AI Audit (AAIA), drawn from every topic area. Answer each one to see whether you got it right and why. No sign-up.

Free questions
30
Full bank
112 questions

Advanced in AI Audit (AAIA) exam guide: topics, cost and format

All 30 Advanced in AI Audit (AAIA) questions with answers

The same questions as the test above, grouped by exam topic. Open any question to see the answer and explanation.

AI Operations (10 questions)

  1. Question 1

    An AI model's accuracy has declined over the past three months, but the model has not been changed. What cause should an auditor investigate first?

    1. AData drift, where production data no longer matches the data the model was trained on
    2. BA change in the colour scheme of the application, which affects how users read the outputs
    3. CAn increase in the number of user logins, which slows the model's response time each day
    4. DA change in the server's time zone, which shifts the timestamps in the model's logs
    Show answer and explanation
    Correct answer: B

    Data drift, where production inputs change from the training data, is a common cause of declining accuracy without model changes. Interface colours, login counts and time zones do not change model accuracy.

  2. Question 2

    Which control best protects an AI system's training data from unauthorised change?

    1. AA shared folder that all data scientists can edit at any time without any record of changes
    2. BA spreadsheet of data changes, kept on one analyst's laptop and updated when they remember
    3. CVersioned datasets with access controls, change approval and an audit trail of each modification
    4. DStoring the training data in a public repository, so that any change is visible to everyone
    Show answer and explanation
    Correct answer: C

    Versioned datasets with access controls, change approval and audit trails protect data integrity and make changes traceable. Shared editable folders, laptop spreadsheets and public repositories all allow unrecorded or unauthorised change.

  3. Question 3

    A model is retrained every month with new data, but nobody checks whether the new version performs worse. What is the main audit concern?

    1. AThe model is being trained too often, which wastes computing resources on each cycle
    2. BThe training data is stored in the wrong region, which breaks the data residency rule
    3. CRetraining without validation may deploy a model that performs worse, and this goes unnoticed
    4. DThe model version number changes each month, which confuses the users of the system
    Show answer and explanation
    Correct answer: C

    Retraining without validation can deploy a weaker model unnoticed, which is the main concern. Training frequency, data region and version numbers are secondary issues.

  4. Question 4

    Which evidence best shows that an AI system's outputs are monitored in production?

    1. AAn annual review of the model's code, which is carried out by the development team once a year
    2. BA list of the features the vendor promised to include in a future release of the product
    3. CA statement in the project plan that monitoring will be added to the system at a later date
    4. DMonitoring dashboards and alert records, showing thresholds, breaches and the responses taken
    Show answer and explanation
    Correct answer: D

    Monitoring dashboards and alert records with thresholds, breaches and responses are direct evidence of production monitoring. Future plans, promised features and code reviews do not show monitoring is happening.

  5. Question 5

    A generative AI assistant sometimes produces confident but false statements. Which operational control best reduces the risk to customers?

    1. AIncreasing the response length, so that the assistant has more space to explain each answer fully
    2. BAllowing the assistant to answer any question, with no limits on the topics it may address
    3. CRemoving all source links from the answers, so that users focus on the content of each reply
    4. DGrounding answers in verified sources, with citations and human review for low-confidence answers
    Show answer and explanation
    Correct answer: D

    Grounding answers in verified sources with citations, and human review for low-confidence answers, reduces false statements reaching customers. Longer answers, removed citations and unlimited topics increase the risk.

  6. Question 6

    An auditor reviews a chatbot that accepts free-text input from customers. Which risk is most important to test?

    1. ASpelling errors in customer messages, which may reduce the quality of the chatbot's responses
    2. BPrompt injection, where crafted input tries to override the system's instructions or expose data
    3. CLong messages from customers, which increase the response time of the chatbot for each user
    4. DCustomers using emojis, which may not be displayed correctly in some browsers used by the team
    Show answer and explanation
    Correct answer: B

    Prompt injection, where crafted input overrides instructions or exposes data, is a serious security risk for AI chatbots and must be tested. Spelling, message length and emojis are usability issues rather than the most significant risk.

  7. Question 7

    Which practice best supports the traceability of an AI system's decisions for an audit?

    1. ALogging each input, model version, output and the time of each decision in a protected store
    2. BRecording only the final decision, without the input data or the model version that produced it
    3. CStoring the logs on the developer's laptop, so that they are available when the developer is present
    4. DDeleting the logs after each decision, so that the system stays fast and uses less storage
    Show answer and explanation
    Correct answer: A

    Logging the input, model version, output and time for each decision, in a protected store, gives a complete and verifiable trail. Deleting logs, keeping only final decisions and storing logs on a laptop each break traceability.

  8. Question 8

    A business team wants to deploy an AI model quickly, but the model has not been tested on data from the groups it will serve. What should the auditor recommend?

    1. ATest the model on representative data for each affected group before deployment, and record the results
    2. BRemove the affected groups from the scope of the model, so that no testing is needed for them
    3. CDeploy the model to one user group only, and assume the results apply to every other group
    4. DDeploy the model now, and test on the affected groups only after users complain
    Show answer and explanation
    Correct answer: A

    Testing on representative data for each affected group before deployment, with recorded results, identifies performance gaps before they harm people. Waiting for complaints, removing groups and extrapolating from one group ignore that risk.

  9. Question 9

    Which activity should the AI system owner perform when a model is retired from production?

    1. ADelete all of the model's records immediately, so that the system is not confused by old data
    2. BKeep the model running in parallel forever, in case someone asks about its outputs in the future
    3. CArchive the model, its data lineage and its decision logs under the retention policy, with access controlled
    4. DPublish the model's code to a public repository, so that other organisations can use it freely
    Show answer and explanation
    Correct answer: C

    Archiving the model, its lineage and logs under the retention policy preserves what is needed for audit and legal purposes, with access controlled. Deleting records immediately, running the model forever and publishing it all fail the requirement.

  10. Question 10

    Which AI risk is most closely associated with training data that includes personal data used without a lawful basis?

    1. AModel overfitting, where the model memorises the training examples rather than generalising
    2. BSlow inference, where each prediction takes longer than the business expects to receive it
    3. CLower GPU usage, which reduces the cost of training but does not change the data's legal status
    4. DPrivacy and data protection breach from unlawful processing of personal data
    Show answer and explanation
    Correct answer: D

    Using personal data without a lawful basis creates a privacy and data protection breach, which is a legal and governance risk. Overfitting, inference speed and GPU usage are technical characteristics that do not change the legal status of the data.

AI Governance and Risk (10 questions)

  1. Question 11

    An internal audit team is reviewing a bank's new credit-scoring model. Which artefact should the auditor check first to confirm accountability for the model?

    1. AThe model's performance dashboard, which shows accuracy by month for the last year
    2. BA documented model owner and approval record, showing who accepted the model's risk
    3. CThe vendor's marketing brochure, which describes the model's intended business benefits
    4. DThe cloud region where the model is hosted, which determines the data residency rules
    Show answer and explanation
    Correct answer: B

    A documented owner and an approval record show who is accountable for the model's risk, which is the first governance question. Dashboards, brochures and hosting location are evidence about the model, not about accountability.

  2. Question 12

    An organisation wants a structured framework to manage risks from its AI systems across their life cycle. Which framework is designed for this purpose?

    1. AA general-purpose financial controls framework that covers only the accounting processes
    2. BThe NIST AI Risk Management Framework, organised around govern, map, measure and manage
    3. CAn accessibility standard for web interfaces, which governs how content is presented to users
    4. DA network hardening guide, which describes configuration baselines for routers and switches
    Show answer and explanation
    Correct answer: B

    The NIST AI Risk Management Framework organises AI risk management around govern, map, measure and manage across the life cycle. Financial controls, accessibility standards and hardening guides do not address AI-specific risk.

  3. Question 13

    Which control best ensures that an AI system used for hiring decisions is reviewed for unfair outcomes before deployment?

    1. AA user agreement that asks applicants to accept the system's decisions without questioning them
    2. BA bias assessment with defined fairness metrics, approved before go-live
    3. CA scheduled firewall review, which checks the network controls protecting the hiring data
    4. DA logo check on the vendor's website, which confirms that the vendor has a recognised brand
    Show answer and explanation
    Correct answer: B

    A bias assessment with defined fairness metrics, approved before go-live, tests for unfair outcomes at the point where they can be prevented. User agreements, firewall reviews and logo checks do not test fairness.

  4. Question 14

    An auditor finds that the AI governance committee meets only when a problem occurs. What is the most significant risk this creates?

    1. AThe committee's chair is not a technical expert, which slows the review of each model
    2. BThe committee uses more meeting rooms than it needs, which adds cost to the department budget
    3. CRisks may go unmanaged between incidents, and new models may be deployed without oversight
    4. DThe committee's minutes are longer than necessary, which increases the cost of the review
    Show answer and explanation
    Correct answer: A

    Governance that meets only after problems leaves risks unmanaged between incidents, and new models can reach production without review. Minutes length, room use and the chair's expertise are secondary issues.

  5. Question 15

    Which control best addresses the risk of an AI system whose decisions cannot be explained to the customers they affect?

    1. AA policy that bans the use of AI for every decision that affects customers in any way
    2. BA longer terms-of-service document that describes the system in general terms to all customers
    3. CA documented explainability approach for the decision type, with explanations available to affected customers
    4. DA higher model accuracy target, which the team assumes will make explanations unnecessary
    Show answer and explanation
    Correct answer: C

    A documented explainability approach, with explanations available to affected customers, addresses the transparency and accountability need. Longer terms, blanket bans and higher accuracy targets do not provide explanations.

  6. Question 16

    Which element is most important to include in an AI use policy for an organisation that buys AI services from third parties?

    1. ARequirements for vendor due diligence, data handling, monitoring and the right to audit
    2. BThe vendor's list of product features, copied from its public website for reference
    3. CThe names of the employees who use the vendor's products each day in the office
    4. DThe colour of the vendor's branding, which should match the company's own style guide
    Show answer and explanation
    Correct answer: B

    Vendor due diligence, data handling, monitoring and audit rights make third-party AI risk manageable. Feature lists, employee names and branding do not govern the risk.

  7. Question 17

    A regulator asks an organisation to show that its high-risk AI systems are inventoried and classified by risk. Which artefact best answers this?

    1. AA current AI system inventory that records each system's purpose, owner, data and risk class
    2. BA diagram of the office network, showing the placement of the servers and access points
    3. CA list of employees who have completed general cybersecurity awareness training this year
    4. DA copy of the annual financial statements, which describes the company's assets and liabilities
    Show answer and explanation
    Correct answer: A

    A current AI inventory with purpose, owner, data and risk classification is the artefact that answers the regulator. Training lists, network diagrams and financial statements do not classify AI systems by risk.

  8. Question 18

    An organisation is deciding how to classify an AI system that screens job applicants. Which factor most directly influences the risk classification?

    1. AThe number of developers who worked on the system during its build phase
    2. BThe impact of the system's decisions on individuals' rights and opportunities
    3. CThe colour scheme of the user interface that the recruiters see each working day
    4. DThe programming language used to write the code of the system's interface
    Show answer and explanation
    Correct answer: C

    The impact of decisions on individuals' rights and opportunities drives risk classification, and screening job applicants clearly affects them. Developer count, programming language and interface colours do not determine the impact.

  9. Question 19

    Which statement about model risk management is most accurate?

    1. AModel risk spans the full life cycle, including data, design, validation, deployment and monitoring
    2. BModel risk is the same as cybersecurity risk, so one control covers both without a separate review
    3. CModel risk exists only in the development stage, and models in production need no review
    4. DModel risk applies only to financial models, and AI systems outside finance are not in scope
    Show answer and explanation
    Correct answer: A

    Model risk spans the full life cycle, from data and design through validation, deployment and monitoring. Limiting it to development, equating it with cybersecurity or restricting it to finance is incorrect.

  10. Question 20

    Which two controls help an organisation manage the risk of an AI vendor changing a model without notice? (Choose two.)

    1. AContract terms that require notice of material model changes, with a right to test before adoption
    2. BA verbal assurance from the vendor's account manager that changes will not affect the organisation
    3. CPinning the model version where the vendor supports it, and tracking each version used in production
    4. DAllowing the vendor to update the model automatically in every environment, including production
    5. ERemoving all testing for vendor models, since the vendor is responsible for the quality of its product
    Show answer and explanation
    Correct answer: A, C

    Contract terms requiring notice and the right to test, together with pinned and tracked model versions, let the organisation control what changes reach production. Verbal assurances, automatic updates to production and removing testing all leave the risk unmanaged.

AI Auditing Tools and Techniques (10 questions)

  1. Question 21

    Which tool best supports an auditor's analysis of a complete population of AI system transactions?

    1. AA walk-through of one transaction, observed by the auditor while the system processes it
    2. BManual sampling of ten transactions chosen at random from the population, with no analysis
    3. CComputer-assisted audit techniques (CAATs) and data analytics run over the full population
    4. DReading the policy document only, without access to the transaction data or the system logs
    Show answer and explanation
    Correct answer: C

    CAATs and data analytics run over the full population, which lets the auditor test every record for exceptions and patterns. Ten random samples, a single walk-through and policy reading alone cannot test the full population.

  2. Question 22

    An auditor wants strong evidence that a model's approval control operated throughout the audit period. Which evidence is strongest?

    1. AA statement from the model owner, who confirms that the control operated as intended
    2. BThe vendor's marketing claim that every model in production is approved before release
    3. CA policy document dated at the start of the audit period, which describes the approval control
    4. DSamples of approval records across the full audit period, with the approvers and dates shown
    Show answer and explanation
    Correct answer: D

    Samples of approval records across the full period, showing approvers and dates, provide direct evidence that the control operated over time. Owner statements, policy documents and marketing claims are weaker or indirect evidence.

  3. Question 23

    Which audit technique verifies the design of a control by tracing one transaction through the process from start to finish?

    1. ASampling statistics, which estimate the error rate for the whole population from a sample
    2. BRatio analysis, which compares figures from one period with those from another period
    3. CBenchmarking, which compares the organisation's results with those of similar organisations
    4. DA walkthrough, which traces one transaction through the process to confirm the design
    Show answer and explanation
    Correct answer: D

    A walkthrough traces one transaction end to end to confirm that the control is designed as described. Sampling, ratio analysis and benchmarking test populations or compare results rather than confirming design.

  4. Question 24

    Which technique involves selecting a representative subset of transactions for testing?

    1. ABenchmarking, which compares the organisation's results with those of comparable organisations
    2. BAudit sampling, which selects a representative subset to test and draws a conclusion about the whole
    3. CWalkthrough, which traces a single transaction through the process from start to finish
    4. DInquiry, which asks staff how a process works without testing any transaction at all
    Show answer and explanation
    Correct answer: B

    Audit sampling selects a representative subset and draws a conclusion about the population. Benchmarking, walkthroughs and inquiry do not select a representative set of transactions for testing.

  5. Question 25

    Which technique recalculates an AI system's output independently, to check whether the system produced the correct result?

    1. AInterview, which asks the system owner to describe how the output is produced
    2. BReperformance, which independently recalculates the output and compares the results
    3. CObservation, which watches a user operate the system without checking any output
    4. DInquiry only, which asks the vendor to confirm that the output is correct each time
    Show answer and explanation
    Correct answer: B

    Reperformance independently recalculates the output and compares the results, which tests correctness directly. Interviews, observation and vendor inquiry do not verify the output.

  6. Question 26

    Which tool is most useful for analysing log data from an AI system at scale?

    1. APaper checklists, which are completed by hand after each review visit to the team
    2. BData analytics or SQL-based queries over the logs, to find patterns and exceptions
    3. CEmail threads with the system owner, which record the questions asked during the review
    4. DHand-written notes, which capture the auditor's observations during each interview
    Show answer and explanation
    Correct answer: B

    Data analytics or SQL queries can process large log volumes to find patterns and exceptions. Paper checklists, notes and email threads cannot analyse data at scale.

  7. Question 27

    An auditor wants to assess whether an AI system's explanations are reliable. Which method is appropriate?

    1. ACounting the number of words in each explanation, since longer explanations are more complete
    2. BAsking the developer only, since the developer built the explanation feature
    3. CTesting explanation consistency and fidelity against the model's behaviour on sample cases
    4. DReading the vendor's marketing copy, which describes how the explanations are generated
    Show answer and explanation
    Correct answer: C

    Testing whether explanations are consistent and faithful to the model's behaviour on sample cases shows whether they are reliable. Marketing copy, developer self-assessment and word counts do not test reliability.

  8. Question 28

    Which evidence is relevant when testing an AI system's security controls?

    1. AThe office floor plans, which show where the servers are physically located
    2. BPenetration test results and vulnerability scans for the AI infrastructure
    3. CThe holiday rota of the security team, which shows when staff are available
    4. DThe vendor's price list, which shows the cost of the licences for the system
    Show answer and explanation
    Correct answer: B

    Penetration test results and vulnerability scans for the AI infrastructure are direct evidence of security control performance. Floor plans, rotas and price lists do not test security controls.

  9. Question 29

    Which approach best supports continuous auditing of an AI system?

    1. AAnnual manual sampling, carried out once a year by the audit team on a fixed date
    2. BWaiting for complaints from customers, which are then investigated by the audit team
    3. CPrinted reports, distributed to the audit committee at the end of each financial year
    4. DAutomated control tests and dashboards that run on a schedule and flag exceptions
    Show answer and explanation
    Correct answer: D

    Automated control tests and dashboards that run on a schedule provide ongoing assurance and flag exceptions when they arise. Annual sampling, complaints and printed reports are periodic or reactive.

  10. Question 30

    Which factor most affects how often an AI system should be audited?

    1. AThe colour of the template used for the audit report that is sent to the board each year
    2. BThe system's risk rating and the rate at which the model or its data changes
    3. CThe number of auditors who are on holiday during the planned audit period in the summer
    4. DThe office size of the team that operates the system on a day-to-day basis in the building
    Show answer and explanation
    Correct answer: B

    Audit frequency should reflect the system's risk rating and how quickly the model or its data changes. Report colours, office size and holiday schedules do not determine the risk of the system.