Domain 6: AI Security Architecture

Microsoft SC-500 Security Copilot · 55 questions

• An organization is designing a Zero Trust architecture for an internal AI assistant built on Azure OpenAI Service. The AI assistant processes confidential business data. Which Zero Trust principle is most critical to apply when a user request reaches the AI inference layer?
• A company is preparing to deploy a large language model-powered customer service bot and wants to conduct AI red-teaming before production launch. The security team is using PyRIT (Python Risk Identification Toolkit for generative AI) developed by Microsoft. What is the primary purpose of using PyRIT in this context?
• A financial institution is implementing Responsible AI governance across its enterprise AI deployments. The institution needs to ensure that AI models producing credit decisions do not exhibit unfair bias against protected demographic groups. Which Microsoft tool provides fairness assessment capabilities for AI models?
• An enterprise is establishing an AI Trust, Risk, and Security Management (AI TRiSM) program. As part of this program, the security team needs to implement continuous monitoring of AI model outputs for concept drift and anomalous behavior changes post-deployment. Which Azure service provides model monitoring capabilities for production AI models?
• An organization is developing an AI security architecture review process. When evaluating a new AI-powered application, the security team needs to assess threats specific to AI systems, such as data poisoning, model inversion, and adversarial examples. Which threat modeling methodology is most applicable to AI systems?
• A security architect is designing secure model deployment pipelines for an organization that fine-tunes foundation models using proprietary training data. The architect is concerned about supply chain attacks where a malicious actor injects backdoors into pre-trained model weights downloaded from public repositories. Which control mitigates this risk?
• An organization's AI governance committee needs to implement a structured review process before any AI system can be deployed in production. The review must assess legal and regulatory compliance, data privacy, security risks, and ethical implications. Which Microsoft framework provides guidance for structuring this governance process?
• A security team is tasked with implementing data poisoning defenses for an AI model used in fraud detection that is retrained weekly on new transaction data. Which control most directly prevents poisoned data from corrupting the model during retraining?
• A company is establishing an AI red team program and needs to define the scope of a red team exercise against their customer-facing Azure OpenAI-powered chatbot. Which elements must the scope document include to ensure a safe and legally authorized test?
• An organization's security architecture team is designing a secure AI inference infrastructure where the AI model inference API must only be accessible to a specific set of microservices within a Kubernetes cluster, and not to any other workloads. Which Zero Trust network control pattern achieves this microsegmentation?
• A company is implementing a Secure AI Development Lifecycle (SDL) for its AI products. At which phase of the SDL should threat modeling for AI-specific threats (such as prompt injection, training data poisoning, and model inversion) be performed?
• An enterprise AI governance team wants to implement a model versioning and approval process to ensure that only vetted AI models are deployed to production. Before a model is promoted, it must pass security scans, bias evaluations, and compliance checks. Which Azure service best supports implementing this approval gate in an MLOps pipeline?
• A company's AI security team is evaluating the risk of adversarial examples—inputs crafted to fool an image classification AI model used for security camera analysis. The team wants to test the model's robustness to adversarial perturbations. Which approach assesses this robustness?
• A multinational corporation is deploying AI models and must comply with the EU AI Act. The organization's AI security architect needs to classify the risk level of an AI system used for resume screening in the hiring process. Which EU AI Act risk category applies, and what are the resulting compliance obligations?
• A company purchases a third-party AI model API service to power an internal HR tool. The security team needs to assess the risk of using this external AI service. Which element is most critical to evaluate in the vendor security review for an AI API service?
• An AI security architect is implementing a privacy-preserving ML pipeline for a healthcare AI model that must learn from patient data across multiple hospitals without the data leaving each hospital's premises. Which privacy-preserving technique achieves this?
• A company has experienced a security incident where an AI model in production was found to be generating harmful outputs after what appears to be a model update. The incident response team needs to quickly roll back to the previously known-good model version. Which Azure Machine Learning feature supports this rollback?
• A company is deploying a Retrieval Augmented Generation (RAG) AI system that retrieves documents from Azure AI Search and uses Azure OpenAI to generate responses. The security team needs to implement defense-in-depth for this architecture. Which combination of controls provides comprehensive protection across the retrieval and generation layers?
• An organization is using Microsoft Counterfit to assess the security of an image recognition AI model before production deployment. The security team wants to evaluate the model's resilience to black-box adversarial attacks—where the attacker can only query the model and observe outputs, without access to model weights or architecture. Which Counterfit attack configuration represents a black-box attack?
• An enterprise AI governance committee requires that all AI models deployed in production have explainability reports generated that explain why the model made specific high-stakes decisions (e.g., loan denial, hiring rejection). Which Azure Machine Learning capability generates these explanations?
• A company is designing an AI security architecture for a multi-tenant SaaS application where different enterprise customers' data must be strictly isolated. The application uses a shared Azure OpenAI deployment. Which architectural pattern ensures tenant data isolation at the AI layer?
• A security architect is designing the identity layer for an AI system that must serve as a trusted reference for other downstream systems' security decisions. The AI system attests to its configuration state (model version, content filter settings, and security controls enabled) before downstream systems consume its outputs. Which Microsoft technology enables cryptographic attestation of a system's security configuration?
• An organization wants to implement a centralized AI API Gateway pattern to govern all AI API traffic flowing from applications to Azure OpenAI Service. The gateway must enforce rate limiting per application, centralize logging of all AI interactions, and apply content safety screening before requests reach the model. Which Azure service is best suited to implement this AI API Gateway?
• An enterprise has deployed multiple AI systems and is building an AI governance framework aligned with Microsoft's Responsible AI principles. The security team needs to define the process for reporting AI safety incidents—cases where an AI system produces harmful or unexpected outputs that affect users. Which element of the governance framework should specify the incident classification, escalation path, and post-incident review process for AI safety incidents?
• A security architect needs to implement transparency controls for an AI system that makes automated credit decisions. Regulators require that the organization be able to explain to any affected individual why they received a specific credit decision from the AI system, in plain language. Which combination of Azure services supports meeting this regulatory transparency requirement?
• An organization's AI security team wants to implement cryptographic watermarking of AI-generated content so that any text produced by their Azure OpenAI deployment can be traced back to their system if it appears in external channels without authorization. Which approach is most technically sound for implementing AI output watermarking?
• An organization is implementing a Zero Trust architecture for their AI platform on Azure. An application uses a chain of AI services where each service calls the next. The security team needs to ensure each service-to-service hop is authenticated. What is the correct approach?
• A company is conducting a red-team exercise against their internal LLM-based customer service chatbot. The red team successfully exfiltrates the system prompt by asking the model to 'repeat all previous instructions.' This represents which category of AI attack?
• An organization's AI governance framework requires implementing AI TRiSM (Trust, Risk, and Security Management). A security architect must identify which control specifically addresses the risk that an AI model's decisions cannot be explained or audited. Which AI TRiSM pillar addresses this risk?
• A security architect is designing a defense-in-depth strategy for an Azure AI solution that processes healthcare records. The architect needs to identify which control most directly addresses the risk that a malicious insider with Azure subscription Owner role could extract training data.
• A security team is implementing responsible AI governance controls for a financial services organization deploying predictive AI models. The governance framework requires a human review process for all AI-driven decisions that exceed a certain financial impact threshold. Which Azure capability best supports implementing this human-in-the-loop control?
• An organization is performing an AI security risk assessment. A security architect identifies that their RAG system could be manipulated through malicious content injected into documents that are indexed in the knowledge base. What type of attack is this, and which control should be implemented?
• A security architect is reviewing the network architecture for an Azure AI solution. The solution includes an Azure OpenAI private endpoint in a spoke VNet, APIM in a hub VNet, and on-premises clients connected via ExpressRoute. The security team requires that DNS resolution for the Azure OpenAI private endpoint works correctly for on-premises clients. What DNS configuration is required?
• A financial services company is building an AI-powered loan approval system. The AI governance team requires a formal AI risk tiering process before deployment. According to responsible AI principles and EU AI Act alignment, what risk tier would an automated loan approval AI system typically fall under?
• A company is evaluating AI security architecture for a new generative AI application. The security architect needs to implement a secure AI gateway pattern. Which architectural component sits between client applications and AI model endpoints to provide centralized security enforcement including rate limiting, authentication, content filtering, and logging?
• A security team is designing a red team exercise for an enterprise AI chatbot. The red team lead asks which category of adversarial testing specifically evaluates whether the AI model produces harmful, biased, or unsafe outputs regardless of whether the prompt was adversarial. What is this testing approach called?
• A security architect is implementing the OWASP Top 10 for LLM Applications controls for an enterprise AI application. The application is at risk of 'LLM02: Insecure Output Handling' which could lead to XSS or SQL injection via AI-generated content. Which control addresses this risk?
• An organization's AI governance policy requires that all AI models deployed in production must have documented model cards that include fairness metrics, known limitations, and intended use cases. Where in Microsoft's AI platform should these model cards be stored and managed to be discoverable by the security and compliance teams?
• A security engineer is designing the authentication architecture for a multi-tenant SaaS AI application built on Azure. Each customer tenant must be isolated and their AI interactions must not be visible to other tenants. Which Azure architecture pattern best achieves tenant isolation for the AI components?
• A security architect is implementing controls to protect against training data poisoning attacks on a machine learning pipeline in Azure. The attacker model assumes that adversaries could inject malicious records into the data lake used for training. Which combination of controls provides the most comprehensive defense?
• An enterprise AI platform team needs to implement a secure software supply chain for their AI models. The team wants to ensure that model artifacts (trained model files, ONNX exports) have not been tampered with between training and deployment. Which Azure capability supports model artifact integrity verification?
• A security engineer is reviewing the security posture of an Azure AI Foundry deployment. The Responsible AI (RAI) evaluation results show that the deployed model occasionally produces responses with high 'Groundedness' violation scores when processing certain user queries. What does this indicate about the model's behavior?
• A security engineer is implementing secure AI development practices using Azure DevOps. The organization requires that AI model training code must be scanned for hardcoded credentials and sensitive data before merging to the main branch. Which Azure DevOps feature provides this capability natively?
• A security architect is reviewing the network security design for an Azure OpenAI deployment. The organization has an on-premises proxy server that all internet traffic must pass through for inspection. The AI application running on Azure App Service needs to call Azure OpenAI via the private endpoint. How should the network route be configured?
• A compliance officer asks a security engineer about the Microsoft model for shared responsibility for AI security. The engineer needs to explain what security responsibilities remain with the customer when using Azure OpenAI as a PaaS service. Which of the following is a customer responsibility when using Azure OpenAI?
• A security architect is designing a Zero Trust architecture for an AI pipeline that processes sensitive healthcare data. The AI models must be trained in a network-isolated environment where no internet access is permitted. Which Azure Machine Learning network configuration provides a fully isolated training environment with no public internet connectivity?
• A security team is implementing the MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) framework to categorize AI-specific threats to their Azure AI platform. Which MITRE ATLAS tactic corresponds to an attacker querying a deployed model with many inputs to understand its decision boundary and reconstruct proprietary training data?
• A security engineer is implementing Microsoft's Secure Future Initiative (SFI) principles for an AI platform. The SFI principle of 'Secure by Default' requires that all security controls are enabled automatically without customer action. For Azure OpenAI, which control is an example of a 'Secure by Default' implementation?
• A financial services firm is implementing AI governance for an automated trading signal generation system. The governance board requires that all AI-generated trading signals above a certain risk threshold must be reviewed by a licensed trader before execution. This is an example of which responsible AI principle?
• A security architect needs to design a solution for detecting and preventing model theft (model extraction attacks) against an Azure OpenAI deployment used in a commercial SaaS product. Which combination of controls provides the most effective defense?
• A security engineer at a company that built a custom AI application needs to test whether their application is vulnerable to prompt injection through uploaded documents. The engineer wants to use Microsoft's open-source AI red teaming tool. Which Microsoft tool provides automated red teaming capabilities for generative AI applications?
• A security team is designing the logging strategy for an AI application built on Azure OpenAI. The application processes sensitive legal documents. The team needs to balance security visibility (logging prompt content for forensics) against privacy (not logging PII in legal documents). What approach achieves this balance?
• A security engineer is implementing threat modeling for a new Azure AI Foundry application using the STRIDE methodology. The engineering team is concerned that an attacker could submit specially crafted inputs that cause the AI model to produce outputs bypassing the organization's safety guidelines. Which STRIDE category does this threat fall under?
• A security engineer is implementing Microsoft Defender for Cloud's governance capabilities to track remediation of AI security recommendations. The CISO wants to assign owners to each AI-related recommendation and track remediation within a 30-day SLA. Which Defender for Cloud feature provides this governance workflow?
• A company's AI security team is implementing the NIST AI Risk Management Framework (AI RMF) for their Azure AI platform. The team needs to identify which AI RMF function maps to the activity of continuously monitoring deployed AI model performance for unexpected behavior changes. Which AI RMF function does this activity belong to?