Fandom scraper and Corrie faces

I had a few hours on a train and was bored of example datasets that did not interest me. Wikipedia is usually where I go for data, via DBpedia, but Wikipedia has rules about what is worthy of notice. I wondered where you would find the stuff that is not. Fandom is a pile of wikis for people who care a lot about obscure things.

Infoboxes still have structure, but there is no SPARQL, and the API tools I tried did not play nicely. I wrote a fairly dirty Python script: take a category, walk the subcategories, pull the top of each page, save the infobox as JSON. Default target was Coronation Street characters. You can point it at another fandom by changing a variable. I said in the blog post I would not maintain it, because scraping is a pain. The repo still gets stars.

Around the same time I pointed a face detector at a Corrie clip, just to see. The 2013 posts are the same habit earlier: text-mining Coronation Street in R, then mapping telenovela casts. Fandom was what I reached for once Wikipedia’s structured layer was not enough.