Meridian's generative-AI knowledge assistant for maintenance manuals has completed fine-tuning and passed evaluation. The PM now needs to plan its move into production use by dispatch staff. What does the transition from training to inference primarily involve?
Select an answer to reveal the explanation.
Short Explanation
Training is the model in study-hall mode, cramming on a mountain of examples. Inference is exam day, answering one question at a time under a clock. Different modes need different setups — training wants throughput, inference wants speed on a single response.
Full Explanation
The transition from training to inference is a core operationalization concept in CPMAI's Managing AI domain: it marks the shift from a learning phase, where the model consumes large batches of data to adjust its parameters, to a serving phase, where the already-trained model responds to individual real-time queries — here, dispatch staff asking about maintenance procedures. This shift typically demands different infrastructure: training environments are optimized for throughput over large batches, while inference environments are optimized for low latency and reliable availability under live user load. Continuously retraining on every live query conflates inference with training and is neither how a fine-tuned model is normally operated nor practical for real-time response times. Discarding the trained model in favor of a rules-based system abandons the investment already made in fine-tuning and evaluation without justification. Assuming training and inference infrastructure are interchangeable ignores the very real operational and cost differences between the two phases, which is precisely why operationalization planning is called out as its own CPMAI task.