Using Facepager to find comments on Facebook page posts

By | June 17, 2014

If you want to grab comments from one post, the Graph API Explorer might be easier.

I’ve been trying to find ways to poke the Facebook graph that will be easy for people who find working with the API directly difficult and currently I am using a tool called Facepager to extract data. After a few minutes working with the program it seems easy enough to get it to poke the Facebook Graph for the comments and then store them in a SQLite database. I’ll be looking to work with the results in R… but one thing at a time!

1) After downloading and extracting Facepager for your system you should see a screen like the one in the image. Click the new database button and give it a filename. This will be your SQLite database, I’ll be using this in R in other posts, but for now you can just reload it it in Facepager to do stuff.

2)Then you want to click add node. The node name is the name of the Facebook page you want to explore. For example if you want to get all the comments from posts on the Minecraft page at: https://www.facebook.com/minecraft then the node name is minecraft.

3)Select what you are after in the ‘Resources’ Tab, I went with <page>/posts.

4)You need an access token to get any data out of Facebook, press the login to Facebook button and log in.

5)Press fetch data.

Easy, now you have all the results viewable in Facepager, each post only shows the first 25 comments though and I want them all. To find them all I just clicked each individual node and changed the resource to post/comments. A video on how I did it:

 Code

I have been asked in the comments to share my R code. I’m not sure I still have the script in the state it was in during the video, but I did find my finished thing. This script takes all the comments I have scraped from Facebook (from political party pages) and then it finds repeated phrases. If you would like to see what I did with it then you can read this post about it here. Hope it helps.

options(mc.cores=1)

#import packages

install.packages("tm")
install.packages("RWeka")
install.package("slam")

library(tm)
library("RWeka")
library("slam")

#import csv
mydata = read.csv("5partycomments", sep = ";")  # read csv file 

#prepare text
corpus <- Corpus(VectorSource(mydata$message)) # create corpus object

corpus <- tm_map(corpus, mc.cores=1, removePunctuation)
corpus <- tm_map(corpus, removeNumbers, mc.cores=1)
corpus <- tm_map(corpus, removeWords, stopwords("english"), mc.cores=1)

# convert all text to lower case
corpus <- tm_map(corpus, tolower, mc.cores=1)

#proplem with to lower means we need to make it type of plain text document again
corpus <- tm_map(corpus, PlainTextDocument)

#make the term document matrix
tdm <- TermDocumentMatrix(corpus)

#find the frequent terms
findFreqTerms(tdm, lowfreq = 500)

#tokenizer for tdm with ngrams
BigramTokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 6, max = 6))
tdm <- TermDocumentMatrix(corpus, control = list(tokenize = BigramTokenizer))
findFreqTerms(tdm, lowfreq = 15)

#create dataframe and order by most used
rollup <- rollup(tdm, 2, na.rm=TRUE, FUN = sum)
mydata.df <- as.data.frame(inspect(rollup))
colnames(mydata.df) <- c("count")
mydata.df$ngram <- rownames(mydata.df) 
newdata <- mydata.df[order(-count),] 
newdata<-newdata[order(newdata$count, decreasing=TRUE), ]

 

54 thoughts on “Using Facepager to find comments on Facebook page posts

  1. Pingback: The recurring phrases of BNP members. | David Sherlock

  2. Patrícia Rossini

    So I need to fetch data from pages with thousands of comments per post. Will Facepager be adequated for that?
    Thanks!

    Reply
    1. David Sherlock Post author

      it has been a while since I’ve used it but I think it depends on how a many pages you want to mine. You might be best just using the facebook graph api. Facepager will help you get started understand the calls so it still might be worth taking a look.

      Reply
      1. addien

        Hi david. Can you help me how to design tools to mencrape facebook data just by entering certain keywords then the result is displayed in table form. In the table there are names, addresses, contacts, emails. And then the data can be stored in exel format. Please email me at joey_nemo84@yahoo.com

        Reply
  3. B Kelly

    I just want to find how many people have an open profile on Facebook i.e.. where anyone can read their details. Could I use face pager for this?

    Reply
      1. nizam

        i wanted to get the data like Pakistani students studying in CHina Beijing, can you guide me how to do that using facepager ???

        thanks

        Reply
  4. nizam

    I tried to log in, but i cant see login page of facebook after clicking log in to facebook, an empty form is open, and i get this error “Network error (99): Connection timed out” . can anyone tell me how to solve this problem ?

    Reply
  5. Mario

    do you mind to share you R code that you show in to the video?
    good job man!

    Reply
    1. David Sherlock Post author

      Sure, I’m on my phone at the moment. I will look for the code when I’m on my machine tomorrow

      Reply
      1. Mario

        awesome! And thank you for the quick turnaround. I really enjoyed the video.

        Reply
        1. David Sherlock Post author

          I’ve updated the post with some code I wrote in R that uses the a CSV file generated by facepager. If your doing anything interesting and want help/collaboration on a project let me know. hth.

          Reply
          1. Mario

            awesome & thank you. Count on it.
            Thanks again.

            ;mE

      2. nizam

        can you help me explain how to get login and mine data from facebook, like i want to collect data of people added with me on facebook…
        when i try to log in i dnt get log in page… looking forward to your reply..

        Reply
        1. David Sherlock Post author

          Im not sure what you mean you don’t get the log in page, When you press the button? Can you screen record what you have done so I can see if I can replicate the problem. If it is a software error you might want to contact Till Keyling, it might be best through the github repository for the app but he has been very good replying to me on Twitter at @tkeyling . The other author Jakob Jünger sometimes leaves comments on my blog, but I do not have hos contact details.

          I think to start it would be good to exactly identify the problem, is it when you press the log in to facebook button?

          Reply
          1. nizam

            Yes exactly, when i follow all the steps, and click on the Log in Facebook button i just get an empty window, and some errors are shown like ” SSL certificate error ignored: (Warning: Your connection might be insecure!)
            Network error (2): Connection closed” ,
            i have a screenshot but i dont know how to send you , as there is no option for attaching photos here.

  6. Mario

    Thx for the source – I am getting an error on Error in order(-count) : object ‘count’ not found

    Reply
    1. David Sherlock Post author

      I’ll look in to it for you. Might not be until mid week tho

      Reply
      1. David Sherlock Post author

        ok, I cant see what I was trying to do there. I think I was trying to create a dataframe of all phrases and the number of times they were said. I’m trying to order it by the number of times they have been said.

        Delete this line:

        newdata <- mydata.df[order(-count),]

        and change the last line to:

        newdata<-mydata.df[order(mydata.df$count, decreasing=TRUE), ]

        Reply
  7. Pei

    Thx David! You video is so helpful! There’s one more question, can I extract my friends’ action on FB, like post, comments, like? Or I could only extract my friends’ action on my homepage? Trapped with this problem for weeks, hope to hear your suggestions.

    Thanks again!

    Reply
  8. Pei

    Hey David, thanks for your nice tutorial! I just start with Facepager and face some problem. I test with my own FB account following you instruction. But get nothing.
    If I write “me” in Add Nodes, then it returns “empty” in Object Type and “fetched (200)” in Query Status.
    If I past my user_id in Add Nodes, then it returns “data” in Object Type and “error (404)” in Query Status. If I click into error (404), it gives me following message:
    {
    “error”: {
    “message”: “(#803) Some of the aliases you requested do not exist: me_x”,
    “code”: “803”,
    “type”: “OAuthException”
    }
    }

    I’m not sure what’s the problem and how to fix it. Grateful if you could help me with it.
    Many many thx!

    Reply
  9. Maissa

    Hello,

    I have succefully installed the Facepager and I intend to use the fetched Data to refine it in hortonworks platform. Any help please !

    Thanks a lot,

    Reply
    1. David Sherlock Post author

      Hi, What do you need help with? Hortonworks?

      Reply
  10. Stephen Smith

    I am using face pager to pull data help my friend get insights about her customers. I have pulled a list of Facebook users who are attending an event that her business is hosting. Any idea how to pull more Facebook information about the list of users I have? I have their names and Object IDs. Thanks

    Reply
  11. Somantri

    Thanx for the application… I have an empty message after fetching data. how to fix it?

    Reply
  12. Mario

    try to with the minecraft example as shown in the video if it does work than there’s something wrong (permission/privacy…) with the settings you’re inputing.

    Reply
  13. Pingback: Storing Facebook comments from post for analysis - David Sherlock

  14. Ali

    Hi, I downloaded the software and I created a new nod but when I click the login to Facebook to get the access token it opens a blank white window and nothing happens, in the status log it shows the following:

    2015-10-18 22:19:35.690000 Network error (99): Connection timed out

    Any suggestions how can I fix that,

    thanks for the video example

    Reply
  15. mariya khan

    i am from pakistan peshawar so my project is to extract data from facebook about terrorism its effect on education, children ,Socity ,Ecnomy so i used facepager but i am confused how i save or use for implementation.

    Reply
  16. Maud

    Hello David,
    I didn’t understand if it works for more than 25 posts ? Is it possible to indicate from which date we would to like collect data ?

    Thanks a lot in advance !
    And thanks for the video !

    Maud

    Reply
  17. Erin

    Hi, dear
    I try many times, but there is no comments showing up all the time? IS there any problem with my computer?
    Thanks for you helpping

    Reply
  18. Rosa

    Hello,
    I’ve tried many times, but I always get error 404. Any help, please?

    Reply
  19. Sumit Rawal

    Whenever I am clicking login to facebook on Facepager it is showing HOST NOT FOUND ERROR

    Can you please help. Because of this I am still unable to extract data

    Reply
  20. awhy oghy

    Hi, I’ve tried the software, and it worked just fine. But unfortunately I can only get 50 comments. Is there any way i can get ALL the comments and REPLY, LIKES and COMMENTS for the comments as well. PLUS how to get non-english comments automatically translated into English? Please, I need this for my project. Thanks

    Reply
  21. Sourire

    Hi,
    I tried to use Facepager to extra post and comments from my Facebook group but I didn’t get it work.
    I only got it on my Facebook page. I cannot find any documentation on Facebook group. I could only extract the group members name.
    Many thanks in advance for any tips.
    Best Regards,
    Véronqiue

    Reply
  22. Nina

    Hi David,
    thank you for your great video. It was a great explanation! 😀 Does FP also have the possibility to download member names of a group and the name of the person who added them to the group?
    This would be a great tool to work with 🙂
    Maybe you know someone who can point me into the right direction
    Thanks a lot 🙂
    Nina

    Reply
  23. Michael dela Fuente

    Hi! I was trying out Facepager to get data for my research and was getting an error(404) and when I check I have this “Cannot query users by their username”. I tried fetching data from “minecraft” and was able to get some data. Does this mean that we can only get data to publicly available FB pages? I tried it with my own FB account and got the same error.

    Reply
  24. Jakkarin

    {
    “error”: {
    “message”: “(#803) Cannot query users by their username (virin.kumlertluck)”,
    “code”: “803”,
    “type”: “OAuthException”,
    “fbtrace_id”: “Gk07mI08Nv5”
    }
    }

    I got this I can’t fetch any data except from minecraft

    Reply
  25. rasha deib

    hi
    I need to fetch data from pages with share counter per post. Will Facepager be adequated for that?
    Thanks alot !

    Reply
  26. semeh

    dear Sir

    Thank you very much for the solution however I have some problems when getteing access using the token. When I try to connect to facebook, the connexion is not considered by facepager and my token is not taken.

    Reply
  27. ari

    I want to scrape comment from user post, can we do this, because it show error not working.

    Reply
  28. sophia

    is there another way to get the access token apart from the login button from the facepager? because when i tried to get the login from facepager it gives me an empty page

    Reply
  29. sophia

    is there another way of getting the token because i have tried login using the login button from the facepager it gave me empty window

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *