A community-choir attendance model looks excellent because the same holiday-concert weekends were used both to fit it and to announce its score. The director wants a number that estimates how it will do on future weekends it has not seen. What should the team do?

A community-choir attendance model that looks excellent because the same holiday-concert weekends were used both to fit it and to announce its score needs honest held-out evaluation. Holding out a test slice—and a validation slice for model choices—that the fitting step never saw is correct because the published number must estimate performance on future weekends, not replay weekends already memorized during training. Reusing holiday weekends for the published score keeps the trophy high but does not measure generalization to next season. Deleting rows only to shrink the file and then scoring the same training rows again fails both purposes of a split: size reduction is not validation design, and rescoring train data is not a future-weekend estimate. Skipping splits because attendance already lives in a spreadsheet confuses storage format with evaluation design; spreadsheets still need train, validation, and test discipline. Inflated in-sample scores are a classic trap when directors want a flattering number. Exam caveat: if train and report share the same rows, treat the score as biased upward until a sealed holdout exists. Operational check: time-based split by weekend, fit only on earlier weekends, tune on validation weekends, and publish the test-weekend metric as the official number the director sees.