A pipeline's Copy activity pulls conservation sensor telemetry from a third-party gallery-monitoring API that occasionally throttles requests and returns a transient error for a few seconds before recovering. Right now, any throttling event causes the whole nightly pipeline to fail and require a manual rerun. What should the data engineer configure on the Copy activity to handle this gracefully?
Select an answer to reveal the explanation.
Short Explanation
A few seconds of throttling shouldn't cost anyone a manual rerun at 6 a.m. Give the Copy activity a retry count and a retry interval, and it just waits a beat and tries again on its own — no human needed for something this transient.
Full Explanation
Fabric pipeline activities expose retry settings, a count and an interval between attempts, that let a transient failure like brief API throttling resolve itself automatically instead of failing the entire run. Configuring a small retry count with a short wait between attempts matches the described failure pattern precisely: a few seconds of throttling followed by recovery. Removing the Copy activity and pulling data by hand abandons automation altogether and reintroduces the manual burden the pipeline exists to eliminate. Lowering the overall pipeline timeout does the opposite of what's needed — it makes the pipeline fail faster on exactly the kind of transient hiccup that would otherwise recover with a short wait. Splitting one Copy activity into ten parallel copies would likely make the throttling worse, not better, since simultaneous requests against a rate-limited API tend to trigger more throttling, not less, and it does nothing about the underlying transient-error handling. The caveat: retries are appropriate for transient, self-resolving errors; a persistent outage on the source API still needs a failure branch and an alert, not an endless retry loop. A concrete check: review the activity's run details after a throttling night and confirm it shows one or two retry attempts followed by a successful completion, rather than an outright failure.