Scraping & APIs
===============

`BBC. Editorial Guidelines sec. 11: Data etc. <https://www.bbc.co.uk/editorialguidelines/guidelines/statistics-data-polls-surveys>`_

`Belanger, Ashley. Arstechnica, 2025. <https://arstechnica.com/tech-policy/2025/01/ai-haters-build-tarpits-to-trap-and-trick-ai-scrapers-that-ignore-robots-txt/>`_

Bradshaw, Paul. *Scraping for Journalists*. Leanpub, 2017

`Bradshaw, Paul. What Data Journalists Need to Know About APIs. GIJN, 2022 <https://gijn.org/2022/12/06/what-data-journalists-need-to-know-about-application-programming-interfaces-apis/>`_

`Carter, Laura. There is always an element of judgement. Datakind UK <https://medium.com/datakinduk/there-is-always-an-element-of-judgement-46e8dc3838c>`_

`Google. Robots.txt files overview <https://developers.google.com/search/docs/advanced/robots/intro>`_

Gold, Zachardy & Latonero, Mark. Robots Welcome? Ethical and Legal Considerations for Web Crawling and Scraping. *Washington Journal of
Law, Technology & Arts*. 13/3, 2018, p. 275.
`Library resource <https://librarysearch.cardiff.ac.uk/permalink/f/1tfrs8a/44CAR_ALMA5186951260002420>`_

`Golumbia, David. Fair Game: commonly used by researchers and journalists, data scraping is an underacknowledged privacy concern. 2022 <https://reallifemag.com/fair-game/>`_

`Harlow, Max. Fetch and enrich data with APIs. <https://github.com/maxharlow/tutorials/tree/master/fetch-and-enrich-data-with-apis>`_

Heydt, Michael. Scraping. Code of conduct. *Python web scraping
cookbook*, Packt, 2018. `Library resource <https://librarysearch.cardiff.ac.uk/permalink/f/1tfrs8a/44CAR_ALMA51156094260002420>`_

Jarmul, Katherine & Lawson, Richard. *Python web scraping: fetching data
from the web*. Packt, 2nd ed., 2017
`Library resource <https://librarysearch.cardiff.ac.uk/permalink/f/1tfrs8a/44CAR_ALMA51125405060002420>`_

Kouzis-Loukas, Dimitrios. *Learning Scrapy: learn the art of efficient
web scraping and crawling with Python*. Packt, 2016
`Library resource <https://learning.oreilly.com/library/view/learning-scrapy/9781784399788/ch03s02.html>`_

`McCarthy, Kieran. Web scraping for me, but not for thee. 2023. <https://blog.ericgoldman.org/archives/2023/08/web-scraping-for-me-but-not-for-thee-guest-blog-post.htm>`_

Mitchell, Ryan. *Web scraping with Python*. O'Reilly, 2nd ed., 2018
`Library resource <https://librarysearch.cardiff.ac.uk/permalink/f/3go6c4/44CAR_ALMA21109771520002420>`_

Mitchell, Ryan. Legalities & ethics of web scraping (p. 265-79); Note on ethics (p. 217-18). *Web scraping with Python*. O'Reilly, 2nd ed., 2018
`Library resource <https://librarysearch.cardiff.ac.uk/permalink/f/1tfrs8a/44CAR_ALMA51125433210002420>`_

`Ni, Daniel. Five Tips for web scraping without getting booted.
2019 <https://www.scraperapi.com/blog/5-tips-for-web-scraping/>`_

`ONS. Guide to bot development <https://developer.ons.gov.uk/bots/>`_

`ONS. Fair use policy <https://www.ons.gov.uk/help/fairusepolicy>`_

`ONS. Web scraping policy <https://www.ons.gov.uk/aboutus/transparencyandgovernance/datastrategy/datapolicies/webscrapingpolicy>`_

`Parkrun. Anti-scraping policy <https://docs.google.com/document/d/1wT5QbYBWihsNP919wzpKi5m2nEAO4ZajxERAYTui3uA/edit?tab=t.0#heading=h.96r4cksn1gos>`_

`Schacht, Kira. A web scraping toolkit for journalists,
2019 <https://journocode.com/en/tutorials/a-web-scraping-toolkit-for-journalists>`_

`Scrapinghub. Web Scraping Best Practices Guide <https://scrapinghub.com/guides/web-scraping-best-practices>`_

`Sellars, Andrew. Twenty years of web scraping and the Computer Fraud
and Abuse Act. Boston University Journal of Science & Technology Law.
24, 2018 <https://scholarship.law.bu.edu/faculty_scholarship/465>`_

`Shiab, Nael. On the ethics of web scraping. GIJN,
2015 <https://gijn.org/2015/08/12/on-the-ethics-of-web-scraping-and-data-journalism/>`_

`Shiab, Nael, Web scraping. A journalist’s guide, GIJN,
2015 <https://gijn.org/2015/08/11/web-scraping-a-journalists-guide/>`_

`Smith, Madolyn. APIs for journalism.
Datajournalism.com <https://datajournalism.com/read/newsletters/apis-for-journalism>`_

Sweigart, Al. "Web scraping", *Automate the boring stuff*, No starch press, 2nd ed, 2020, p. 267-300.

`The Markup. Why web scraping is vital to democracy.
2020 <https://gijn.org/2020/12/17/why-web-scraping-is-vital-to-democracy/>`_

`Velotio. Scraping guidelines & best practices <https://www.velotio.com/engineering-blog/web-scraping-introduction-best-practices-caveats>`_

`Whittaker, Zack. eb scraping is legal, US appeals court reaffirms, 2022 <https://techcrunch.com/2022/04/18/web-scraping-legal-court/>`_