This doesn’t tell you how many people of that profession there are, it tells you how many people of that profession are in Wikipedia, with structured data on both the persons birthplace and profession. It might tell us as much about how Wikipedia is being used as how many people Read more…
I wanted to come up with a quick method that I could reuse where I could ask Wikipedia a quick question, get some numbers and create a heat map. I still haven’t come up with the quick process, and when I mess with data I tend to stick with what Read more…
I’ve been informed by Davide Taibi that the LAK dataset has been updated, this update includes some paper text that I reported on, but also has lots more data. As described by Davide: This version includes papers from: – EDM conferences (2008-2014) – LAK conferences (2011-2014, 2014 only abstract since Read more…
For some work I plan on doing for the LACE tech focus blog I wanted to get some information off a webpage and in to an R dataframe. It turns out this is a three line solution (1 line if you throw your URL straight to the function paramaters and Read more…
Quite often I have a dataframe in R with lots of columns with one of the columns being the year. I then want to pick another column and make averages for each year and plot it to show how things have changed over time. I know this is really easy Read more…
Recently I’ve been playing with an R wrapper for a machine language library called Mallet to generate lists of topics from a series of text documents. The technique is called Topic Modelling and I have gotten to grips with it from Ben Marwick‘s readings of archaeology papers which has some Read more…
As of late I’ve been playing with lots of techniques to churn through a corpus of information related to me that I have scraped from the web. Many of these techniques pull back topics, phrases, documents and tell me that they have been identified as important; it is then left Read more…
Something I had been meaning to do for a long time was write a quick script to scrape Reddit comments. A chap has beaten me to it and you can find the code here: https://github.com/ctaggart878/redditscraper. During lunch today I had a little play with it the script (and I mean quick!). Read more…
Day 1 was mostly thinking and talking to Mark. Day two was just getting R to talk to various APIs, MySQL and do some basic mining on the data. I’m a bit busy to type it up now, but I did a video for Mark:
Bah Humbug. I feel very annoyed at everything. Firstly I’m finding R a difficult language to penetrate. I’m getting there, its just hard. There are lots of resources and people willing to help you get to grips with the tool, but getting to grips with the tool isn’t the problem., Read more…
