Press "Enter" to skip to content

Using Facepager to find comments on Facebook page posts

If you want to grab comments from one post, the Graph API Explorer might be easier.

I’ve been trying to find ways to poke the Facebook graph that will be easy for people who find working with the API directly difficult and currently I am using a tool called Facepager to extract data. After a few minutes working with the program it seems easy enough to get it to poke the Facebook Graph for the comments and then store them in a SQLite database. I’ll be looking to work with the results in R… but one thing at a time!

1) After downloading and extracting Facepager for your system you should see a screen like the one in the image. Click the new database button and give it a filename. This will be your SQLite database, I’ll be using this in R in other posts, but for now you can just reload it it in Facepager to do stuff.

2)Then you want to click add node. The node name is the name of the Facebook page you want to explore. For example if you want to get all the comments from posts on the Minecraft page at: https://www.facebook.com/minecraft then the node name is minecraft.

3)Select what you are after in the ‘Resources’ Tab, I went with <page>/posts.

4)You need an access token to get any data out of Facebook, press the login to Facebook button and log in.

5)Press fetch data.

Easy, now you have all the results viewable in Facepager, each post only shows the first 25 comments though and I want them all. To find them all I just clicked each individual node and changed the resource to post/comments. A video on how I did it:

 Code

I have been asked in the comments to share my R code. I’m not sure I still have the script in the state it was in during the video, but I did find my finished thing. This script takes all the comments I have scraped from Facebook (from political party pages) and then it finds repeated phrases. If you would like to see what I did with it then you can read this post about it here. Hope it helps.

options(mc.cores=1)

#import packages

install.packages("tm")
install.packages("RWeka")
install.package("slam")

library(tm)
library("RWeka")
library("slam")

#import csv
mydata = read.csv("5partycomments", sep = ";")  # read csv file 

#prepare text
corpus <- Corpus(VectorSource(mydata$message)) # create corpus object

corpus <- tm_map(corpus, mc.cores=1, removePunctuation)
corpus <- tm_map(corpus, removeNumbers, mc.cores=1)
corpus <- tm_map(corpus, removeWords, stopwords("english"), mc.cores=1)

# convert all text to lower case
corpus <- tm_map(corpus, tolower, mc.cores=1)

#proplem with to lower means we need to make it type of plain text document again
corpus <- tm_map(corpus, PlainTextDocument)

#make the term document matrix
tdm <- TermDocumentMatrix(corpus)

#find the frequent terms
findFreqTerms(tdm, lowfreq = 500)

#tokenizer for tdm with ngrams
BigramTokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 6, max = 6))
tdm <- TermDocumentMatrix(corpus, control = list(tokenize = BigramTokenizer))
findFreqTerms(tdm, lowfreq = 15)

#create dataframe and order by most used
rollup <- rollup(tdm, 2, na.rm=TRUE, FUN = sum)
mydata.df <- as.data.frame(inspect(rollup))
colnames(mydata.df) <- c("count")
mydata.df$ngram <- rownames(mydata.df) 
newdata <- mydata.df[order(-count),] 
newdata<-newdata[order(newdata$count, decreasing=TRUE), ]

 

60 Comments

  1. […] I wanted to find out what people were saying on the facebook pages of extreme political parties. At this stage I wasn’t bothered so much about what the party was saying, but what the comments on the posts where saying. The task was to find n-grams in a CSV file and  I decided to do it in R. Originally the CSV was created from comments on 10 pages of BNP Facebook posts, I generated the CSV quite quickly using an application called FacePager, it was very easy to do and if you are interested you can find instructions on this post here. […]

  2. Patrícia Rossini Patrícia Rossini

    So I need to fetch data from pages with thousands of comments per post. Will Facepager be adequated for that?
    Thanks!

    • it has been a while since I’ve used it but I think it depends on how a many pages you want to mine. You might be best just using the facebook graph api. Facepager will help you get started understand the calls so it still might be worth taking a look.

      • Hi david. Can you help me how to design tools to mencrape facebook data just by entering certain keywords then the result is displayed in table form. In the table there are names, addresses, contacts, emails. And then the data can be stored in exel format. Please email me at joey_nemo84@yahoo.com

    • Jakob Jünger Jakob Jünger

      Yes, it will 🙂

  3. B Kelly B Kelly

    I just want to find how many people have an open profile on Facebook i.e.. where anyone can read their details. Could I use face pager for this?

    • B Kelly B Kelly

      I’m doing this as a project for college.

      • nizam nizam

        i wanted to get the data like Pakistani students studying in CHina Beijing, can you guide me how to do that using facepager ???

        thanks

  4. nizam nizam

    I tried to log in, but i cant see login page of facebook after clicking log in to facebook, an empty form is open, and i get this error “Network error (99): Connection timed out” . can anyone tell me how to solve this problem ?

    • Cracken Cracken

      Seems your internet connection is down

  5. Mario Mario

    do you mind to share you R code that you show in to the video?
    good job man!

    • David Sherlock David Sherlock

      Sure, I’m on my phone at the moment. I will look for the code when I’m on my machine tomorrow

      • Mario Mario

        awesome! And thank you for the quick turnaround. I really enjoyed the video.

        • David Sherlock David Sherlock

          I’ve updated the post with some code I wrote in R that uses the a CSV file generated by facepager. If your doing anything interesting and want help/collaboration on a project let me know. hth.

          • Mario Mario

            awesome & thank you. Count on it.
            Thanks again.

            ;mE

      • nizam nizam

        can you help me explain how to get login and mine data from facebook, like i want to collect data of people added with me on facebook…
        when i try to log in i dnt get log in page… looking forward to your reply..

        • David Sherlock David Sherlock

          Im not sure what you mean you don’t get the log in page, When you press the button? Can you screen record what you have done so I can see if I can replicate the problem. If it is a software error you might want to contact Till Keyling, it might be best through the github repository for the app but he has been very good replying to me on Twitter at @tkeyling . The other author Jakob Jünger sometimes leaves comments on my blog, but I do not have hos contact details.

          I think to start it would be good to exactly identify the problem, is it when you press the log in to facebook button?

          • nizam nizam

            Yes exactly, when i follow all the steps, and click on the Log in Facebook button i just get an empty window, and some errors are shown like ” SSL certificate error ignored: (Warning: Your connection might be insecure!)
            Network error (2): Connection closed” ,
            i have a screenshot but i dont know how to send you , as there is no option for attaching photos here.

  6. Ali Ali

    How to download the software ?

  7. Mario Mario

    Thx for the source – I am getting an error on Error in order(-count) : object ‘count’ not found

    • David Sherlock David Sherlock

      I’ll look in to it for you. Might not be until mid week tho

      • David Sherlock David Sherlock

        ok, I cant see what I was trying to do there. I think I was trying to create a dataframe of all phrases and the number of times they were said. I’m trying to order it by the number of times they have been said.

        Delete this line:

        newdata <- mydata.df[order(-count),]

        and change the last line to:

        newdata<-mydata.df[order(mydata.df$count, decreasing=TRUE), ]

  8. Pei Pei

    Thx David! You video is so helpful! There’s one more question, can I extract my friends’ action on FB, like post, comments, like? Or I could only extract my friends’ action on my homepage? Trapped with this problem for weeks, hope to hear your suggestions.

    Thanks again!

  9. Pei Pei

    Hey David, thanks for your nice tutorial! I just start with Facepager and face some problem. I test with my own FB account following you instruction. But get nothing.
    If I write “me” in Add Nodes, then it returns “empty” in Object Type and “fetched (200)” in Query Status.
    If I past my user_id in Add Nodes, then it returns “data” in Object Type and “error (404)” in Query Status. If I click into error (404), it gives me following message:
    {
    “error”: {
    “message”: “(#803) Some of the aliases you requested do not exist: me_x”,
    “code”: “803”,
    “type”: “OAuthException”
    }
    }

    I’m not sure what’s the problem and how to fix it. Grateful if you could help me with it.
    Many many thx!

  10. Maissa Maissa

    Hello,

    I have succefully installed the Facepager and I intend to use the fetched Data to refine it in hortonworks platform. Any help please !

    Thanks a lot,

    • David Sherlock David Sherlock

      Hi, What do you need help with? Hortonworks?

  11. Stephen Smith Stephen Smith

    I am using face pager to pull data help my friend get insights about her customers. I have pulled a list of Facebook users who are attending an event that her business is hosting. Any idea how to pull more Facebook information about the list of users I have? I have their names and Object IDs. Thanks

  12. Somantri Somantri

    I still have some error (Query Status) to fetch data from facebook

  13. Somantri Somantri

    Thanx for the application… I have an empty message after fetching data. how to fix it?

  14. Mario Mario

    try to with the minecraft example as shown in the video if it does work than there’s something wrong (permission/privacy…) with the settings you’re inputing.

  15. Ali Ali

    Hi, I downloaded the software and I created a new nod but when I click the login to Facebook to get the access token it opens a blank white window and nothing happens, in the status log it shows the following:

    2015-10-18 22:19:35.690000 Network error (99): Connection timed out

    Any suggestions how can I fix that,

    thanks for the video example

  16. Yanis Giannopoulos Yanis Giannopoulos

    Helo

    How to query a group I am member of please?

  17. mariya khan mariya khan

    hi sir i need ur help in my project

  18. mariya khan mariya khan

    i am from pakistan peshawar so my project is to extract data from facebook about terrorism its effect on education, children ,Socity ,Ecnomy so i used facepager but i am confused how i save or use for implementation.

  19. Maud Maud

    Hello David,
    I didn’t understand if it works for more than 25 posts ? Is it possible to indicate from which date we would to like collect data ?

    Thanks a lot in advance !
    And thanks for the video !

    Maud

  20. Erin Erin

    Hi, dear
    I try many times, but there is no comments showing up all the time? IS there any problem with my computer?
    Thanks for you helpping

  21. Rosa Rosa

    Hello,
    I’ve tried many times, but I always get error 404. Any help, please?

  22. Sumit Rawal Sumit Rawal

    Whenever I am clicking login to facebook on Facepager it is showing HOST NOT FOUND ERROR

    Can you please help. Because of this I am still unable to extract data

  23. awhy oghy awhy oghy

    Hi, I’ve tried the software, and it worked just fine. But unfortunately I can only get 50 comments. Is there any way i can get ALL the comments and REPLY, LIKES and COMMENTS for the comments as well. PLUS how to get non-english comments automatically translated into English? Please, I need this for my project. Thanks

  24. Sourire Sourire

    Hi,
    I tried to use Facepager to extra post and comments from my Facebook group but I didn’t get it work.
    I only got it on my Facebook page. I cannot find any documentation on Facebook group. I could only extract the group members name.
    Many thanks in advance for any tips.
    Best Regards,
    Véronqiue

  25. Nina Nina

    Hi David,
    thank you for your great video. It was a great explanation! 😀 Does FP also have the possibility to download member names of a group and the name of the person who added them to the group?
    This would be a great tool to work with 🙂
    Maybe you know someone who can point me into the right direction
    Thanks a lot 🙂
    Nina

  26. Michael dela Fuente Michael dela Fuente

    Hi! I was trying out Facepager to get data for my research and was getting an error(404) and when I check I have this “Cannot query users by their username”. I tried fetching data from “minecraft” and was able to get some data. Does this mean that we can only get data to publicly available FB pages? I tried it with my own FB account and got the same error.

  27. {
    “error”: {
    “message”: “(#803) Cannot query users by their username (virin.kumlertluck)”,
    “code”: “803”,
    “type”: “OAuthException”,
    “fbtrace_id”: “Gk07mI08Nv5”
    }
    }

    I got this I can’t fetch any data except from minecraft

  28. hi
    I need to fetch data from pages with share counter per post. Will Facepager be adequated for that?
    Thanks alot !

  29. semeh semeh

    dear Sir

    Thank you very much for the solution however I have some problems when getteing access using the token. When I try to connect to facebook, the connexion is not considered by facepager and my token is not taken.

  30. ari ari

    I want to scrape comment from user post, can we do this, because it show error not working.

  31. sophia sophia

    is there another way to get the access token apart from the login button from the facepager? because when i tried to get the login from facepager it gives me an empty page

  32. is there another way of getting the token because i have tried login using the login button from the facepager it gave me empty window

  33. nishant nishant

    how to find download link

  34. Tahmina Tahmina

    I am trying to follow instructions from the video to extract data from a fb group that I am a member, however facepager just freezes when I add the id and try to click ok. nothing happens from there onwards.

  35. Anderson O Muniz Anderson O Muniz

    it doesnt connect with fb button. how to solve it ?

  36. Amy Amy

    Can this work in a closed facebook gruop?

  37. Engy Engy

    Hello, David!

    Thank you very much for helping us to figure this out. I have learned how to collect data from FB pages. I need to know how to do the same with groups. How can I collect posts and comments from any group? Can you walk me through the process? I would be very grateful.

    One more thing, since FB has changed its privacy policy, I no longer can get the name of the commentator which is a terrible thing for my research. Is there any other way/program/app to overcome this problem? I can not use the API Graph because I don’t have the access token of the page since it is not mine. Any clue?

    Thank a lot in advance.

  38. Noufan PMC Noufan PMC

    Hello,
    Thanks for this helpful tutorial, But I’m getting error when I ran the R code,
    Can you please help me out on this too, This is the error I’m getting

    corpus <- tm_map(corpus, removeWords, stopwords("english"), mc.cores=1)

    Error in FUN(content(x), …) : unused argument (mc.cores = 1)

    Thank you

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

css.php