An analytics team proposes writing a scraper to collect daily prices from a supplier's public web pages. The supplier also offers a documented API to registered partners, and the company is a registered partner. What should the DBA recommend before any scraper is built?
Select an answer to reveal the explanation.
Short Explanation
Scraping is like reading prices off a shop window through binoculars: it works until they rearrange the display. If the shop hands you a printed price list, the API, take it. And always check the terms of use, because "it's public" doesn't mean "you're allowed to harvest it."
Full Explanation
Scraping extracts data from web pages by parsing their HTML. It is a legitimate acquisition technique when no other source exists, but it has well-known drawbacks: it depends on page layout, so any redesign, renamed element or added script can silently break extraction or return wrong values; it can violate a site's terms of use; and heavy request volumes can be treated as abuse and blocked. A documented API provides structured data with a stable contract, versioning, authentication and agreed usage limits, which makes it the more reliable and defensible source when one is available. Recommending the API and confirming the terms of use is therefore the sound first step. Scraping as often as possible despite throttling disregards the provider's limits and risks the company's partner relationship, while increasing the chance of being blocked. Storing only raw HTML defers parsing but leaves analysts without usable fields and multiplies storage, so it does not address reliability. Treating public pages as stable is the core misconception; layouts change frequently and without notice. Exam caveat: if scraping is the only option, build validation that detects layout changes, such as row counts and value ranges, rather than trusting output blindly. Operational check: obtain API credentials, compare one day of API prices with the website, and document the source and its usage terms in the data dictionary.