Managing AI

CPMAI · 16 questions

• Meridian's data science team has finished building a predictive-maintenance model that forecasts engine bearing wear from sensor data, and wants to move straight to production. The program manager insists on a formal quality assurance step first. What should that QA step center on?
• Meridian's predictive-maintenance model scores 98% accuracy on the training data used to build it, but when engineers apply it to sensor readings from aircraft it has never analyzed, its false-alarm rate spikes and it misses real bearing failures. What is the most likely explanation, and what should the PM do?
• A simple rules-based scoring model Meridian built to flag suspicious frequent-flyer mileage redemptions performs poorly on both the training data and new transactions, catching only the most obvious fraud cases. What does this pattern indicate, and what is the appropriate response?
• Meridian's cargo and passenger demand-forecasting model achieves strong statistical accuracy (low mean absolute error) in testing, but the finance team reports that pricing and capacity decisions based on its forecasts have not improved route profitability. What should the PM conclude?
• Meridian's computer-vision system for detecting foreign-object debris on the ramp reports 96% precision and 91% recall in lab testing. Operations leadership instead wants to know whether the system is reducing ramp safety incidents. What does this situation illustrate about model evaluation?
• After three months in production, evaluation data shows Meridian's customer-service virtual assistant correctly resolves rebooking requests but frequently mishandles baggage-status questions, escalating them to human agents far more often than necessary. What is the appropriate CPMAI-aligned response?
• Before deploying the loyalty-program fraud-detection model to production, Meridian's PM wants a quality assurance checklist that goes beyond a single accuracy number. Which combination best reflects a thorough QA process?
• Meridian's data science team has a modest, limited dataset of past crew and gate scheduling decisions to train an optimization model. The PM asks how the team should validate the model given this data constraint. What is the appropriate validation approach?
• Meridian's generative-AI knowledge assistant for maintenance manuals has completed fine-tuning and passed evaluation. The PM now needs to plan its move into production use by dispatch staff. What does the transition from training to inference primarily involve?
• Meridian's predictive-maintenance model has cleared evaluation and is ready to move toward production. The PM is developing an operationalization strategy. Which set of concerns most appropriately belongs in that strategy?
• Meridian is deciding where to deploy its computer-vision ramp-safety monitoring system, which must process live camera feeds with minimal delay to flag foreign-object debris before ground crew are exposed to danger, and which touches operational data Meridian is reluctant to send off-site. What deployment approach best fits these requirements?
• Traffic to Meridian's customer-service virtual assistant spikes sharply during weather disruptions, when rebooking volume can jump tenfold within an hour, then falls back to normal levels once flights resume. What deployment approach best matches this workload pattern?
• Meridian's PM is evaluating cloud-based machine learning services to host the cargo and passenger demand-forecasting model. Several vendor offerings are available. What is the most appropriate basis for selecting among them?
• Meridian's loyalty-program fraud-detection model is now live in production, continuously scoring mileage-redemption transactions. What is the PM's responsibility regarding data lifecycle management once the model is deployed?
• Six months after deployment, Meridian's data science team wants to release an improved version of the predictive-maintenance model that uses newly available sensor types. What should the PM ensure is in place before this new version replaces the current production model?
• Meridian recently upgraded the cameras used by its ramp-safety computer-vision system to a higher-resolution model. Detection performance has since quietly declined, and dispatch is now questioning whether the alerts can still be trusted. What should the PM's response be, framed within Managing AI production practices?