Using Facepager to find comments on Facebook page posts

If you want to grab comments from one post, the Graph API Explorer might be easier.

I’ve been trying to find ways to poke the Facebook graph that will be easy for people who find working with the API directly difficult and currently I am using a tool called Facepager to extract data. After a few minutes working with the program it seems easy enough to get it to poke the Facebook Graph for the comments and then store them in a SQLite database. I’ll be looking to work with the results in R… but one thing at a time!

1) After downloading and extracting Facepager for your system you should see a screen like the one in the image. Click the new database button and give it a filename. This will be your SQLite database, I’ll be using this in R in other posts, but for now you can just reload it it in Facepager to do stuff.

2)Then you want to click add node. The node name is the name of the Facebook page you want to explore. For example if you want to get all the comments from posts on the Minecraft page at: https://www.facebook.com/minecraft then the node name is minecraft.

3)Select what you are after in the ‘Resources’ Tab, I went with <page>/posts.

4)You need an access token to get any data out of Facebook, press the login to Facebook button and log in.

5)Press fetch data.

Easy, now you have all the results viewable in Facepager, each post only shows the first 25 comments though and I want them all. To find them all I just clicked each individual node and changed the resource to post/comments. A video on how I did it:

 Code

I have been asked in the comments to share my R code. I’m not sure I still have the script in the state it was in during the video, but I did find my finished thing. This script takes all the comments I have scraped from Facebook (from political party pages) and then it finds repeated phrases. If you would like to see what I did with it then you can read this post about it here. Hope it helps.

options(mc.cores=1)

#import packages

install.packages("tm")
install.packages("RWeka")
install.package("slam")

library(tm)
library("RWeka")
library("slam")

#import csv
mydata = read.csv("5partycomments", sep = ";")  # read csv file 

#prepare text
corpus <- Corpus(VectorSource(mydata$message)) # create corpus object

corpus <- tm_map(corpus, mc.cores=1, removePunctuation)
corpus <- tm_map(corpus, removeNumbers, mc.cores=1)
corpus <- tm_map(corpus, removeWords, stopwords("english"), mc.cores=1)

# convert all text to lower case
corpus <- tm_map(corpus, tolower, mc.cores=1)

#proplem with to lower means we need to make it type of plain text document again
corpus <- tm_map(corpus, PlainTextDocument)

#make the term document matrix
tdm <- TermDocumentMatrix(corpus)

#find the frequent terms
findFreqTerms(tdm, lowfreq = 500)

#tokenizer for tdm with ngrams
BigramTokenizer <- function(x) NGramTokenizer(x, Weka_control(min = 6, max = 6))
tdm <- TermDocumentMatrix(corpus, control = list(tokenize = BigramTokenizer))
findFreqTerms(tdm, lowfreq = 15)

#create dataframe and order by most used
rollup <- rollup(tdm, 2, na.rm=TRUE, FUN = sum)
mydata.df <- as.data.frame(inspect(rollup))
colnames(mydata.df) <- c("count")
mydata.df$ngram <- rownames(mydata.df) 
newdata <- mydata.df[order(-count),] 
newdata<-newdata[order(newdata$count, decreasing=TRUE), ]

 

68 Replies to “Using Facepager to find comments on Facebook page posts”

  1. So I need to fetch data from pages with thousands of comments per post. Will Facepager be adequated for that?
    Thanks!

    1. it has been a while since I’ve used it but I think it depends on how a many pages you want to mine. You might be best just using the facebook graph api. Facepager will help you get started understand the calls so it still might be worth taking a look.

      1. Hi david. Can you help me how to design tools to mencrape facebook data just by entering certain keywords then the result is displayed in table form. In the table there are names, addresses, contacts, emails. And then the data can be stored in exel format. Please email me at joey_nemo84@yahoo.com

  2. I just want to find how many people have an open profile on Facebook i.e.. where anyone can read their details. Could I use face pager for this?

      1. i wanted to get the data like Pakistani students studying in CHina Beijing, can you guide me how to do that using facepager ???

        thanks

  3. I tried to log in, but i cant see login page of facebook after clicking log in to facebook, an empty form is open, and i get this error “Network error (99): Connection timed out” . can anyone tell me how to solve this problem ?

    1. Sure, I’m on my phone at the moment. I will look for the code when I’m on my machine tomorrow

        1. I’ve updated the post with some code I wrote in R that uses the a CSV file generated by facepager. If your doing anything interesting and want help/collaboration on a project let me know. hth.

      1. can you help me explain how to get login and mine data from facebook, like i want to collect data of people added with me on facebook…
        when i try to log in i dnt get log in page… looking forward to your reply..

        1. Im not sure what you mean you don’t get the log in page, When you press the button? Can you screen record what you have done so I can see if I can replicate the problem. If it is a software error you might want to contact Till Keyling, it might be best through the github repository for the app but he has been very good replying to me on Twitter at @tkeyling . The other author Jakob Jünger sometimes leaves comments on my blog, but I do not have hos contact details.

          I think to start it would be good to exactly identify the problem, is it when you press the log in to facebook button?

          1. Yes exactly, when i follow all the steps, and click on the Log in Facebook button i just get an empty window, and some errors are shown like ” SSL certificate error ignored: (Warning: Your connection might be insecure!)
            Network error (2): Connection closed” ,
            i have a screenshot but i dont know how to send you , as there is no option for attaching photos here.

      1. ok, I cant see what I was trying to do there. I think I was trying to create a dataframe of all phrases and the number of times they were said. I’m trying to order it by the number of times they have been said.

        Delete this line:

        newdata <- mydata.df[order(-count),]

        and change the last line to:

        newdata<-mydata.df[order(mydata.df$count, decreasing=TRUE), ]

  4. Thx David! You video is so helpful! There’s one more question, can I extract my friends’ action on FB, like post, comments, like? Or I could only extract my friends’ action on my homepage? Trapped with this problem for weeks, hope to hear your suggestions.

    Thanks again!

  5. Hey David, thanks for your nice tutorial! I just start with Facepager and face some problem. I test with my own FB account following you instruction. But get nothing.
    If I write “me” in Add Nodes, then it returns “empty” in Object Type and “fetched (200)” in Query Status.
    If I past my user_id in Add Nodes, then it returns “data” in Object Type and “error (404)” in Query Status. If I click into error (404), it gives me following message:
    {
    “error”: {
    “message”: “(#803) Some of the aliases you requested do not exist: me_x”,
    “code”: “803”,
    “type”: “OAuthException”
    }
    }

    I’m not sure what’s the problem and how to fix it. Grateful if you could help me with it.
    Many many thx!

  6. Hello,

    I have succefully installed the Facepager and I intend to use the fetched Data to refine it in hortonworks platform. Any help please !

    Thanks a lot,

  7. I am using face pager to pull data help my friend get insights about her customers. I have pulled a list of Facebook users who are attending an event that her business is hosting. Any idea how to pull more Facebook information about the list of users I have? I have their names and Object IDs. Thanks

  8. try to with the minecraft example as shown in the video if it does work than there’s something wrong (permission/privacy…) with the settings you’re inputing.

  9. Hi, I downloaded the software and I created a new nod but when I click the login to Facebook to get the access token it opens a blank white window and nothing happens, in the status log it shows the following:

    2015-10-18 22:19:35.690000 Network error (99): Connection timed out

    Any suggestions how can I fix that,

    thanks for the video example

  10. i am from pakistan peshawar so my project is to extract data from facebook about terrorism its effect on education, children ,Socity ,Ecnomy so i used facepager but i am confused how i save or use for implementation.

  11. Hello David,
    I didn’t understand if it works for more than 25 posts ? Is it possible to indicate from which date we would to like collect data ?

    Thanks a lot in advance !
    And thanks for the video !

    Maud

  12. Hi, dear
    I try many times, but there is no comments showing up all the time? IS there any problem with my computer?
    Thanks for you helpping

  13. Whenever I am clicking login to facebook on Facepager it is showing HOST NOT FOUND ERROR

    Can you please help. Because of this I am still unable to extract data

  14. Hi, I’ve tried the software, and it worked just fine. But unfortunately I can only get 50 comments. Is there any way i can get ALL the comments and REPLY, LIKES and COMMENTS for the comments as well. PLUS how to get non-english comments automatically translated into English? Please, I need this for my project. Thanks

  15. Hi,
    I tried to use Facepager to extra post and comments from my Facebook group but I didn’t get it work.
    I only got it on my Facebook page. I cannot find any documentation on Facebook group. I could only extract the group members name.
    Many thanks in advance for any tips.
    Best Regards,
    Véronqiue

  16. Hi David,
    thank you for your great video. It was a great explanation! 😀 Does FP also have the possibility to download member names of a group and the name of the person who added them to the group?
    This would be a great tool to work with 🙂
    Maybe you know someone who can point me into the right direction
    Thanks a lot 🙂
    Nina

  17. Hi! I was trying out Facepager to get data for my research and was getting an error(404) and when I check I have this “Cannot query users by their username”. I tried fetching data from “minecraft” and was able to get some data. Does this mean that we can only get data to publicly available FB pages? I tried it with my own FB account and got the same error.

  18. {
    “error”: {
    “message”: “(#803) Cannot query users by their username (virin.kumlertluck)”,
    “code”: “803”,
    “type”: “OAuthException”,
    “fbtrace_id”: “Gk07mI08Nv5”
    }
    }

    I got this I can’t fetch any data except from minecraft

  19. dear Sir

    Thank you very much for the solution however I have some problems when getteing access using the token. When I try to connect to facebook, the connexion is not considered by facepager and my token is not taken.

  20. is there another way to get the access token apart from the login button from the facepager? because when i tried to get the login from facepager it gives me an empty page

  21. I am trying to follow instructions from the video to extract data from a fb group that I am a member, however facepager just freezes when I add the id and try to click ok. nothing happens from there onwards.

  22. Hello, David!

    Thank you very much for helping us to figure this out. I have learned how to collect data from FB pages. I need to know how to do the same with groups. How can I collect posts and comments from any group? Can you walk me through the process? I would be very grateful.

    One more thing, since FB has changed its privacy policy, I no longer can get the name of the commentator which is a terrible thing for my research. Is there any other way/program/app to overcome this problem? I can not use the API Graph because I don’t have the access token of the page since it is not mine. Any clue?

    Thank a lot in advance.

  23. Hello,
    Thanks for this helpful tutorial, But I’m getting error when I ran the R code,
    Can you please help me out on this too, This is the error I’m getting

    corpus <- tm_map(corpus, removeWords, stopwords("english"), mc.cores=1)

    Error in FUN(content(x), …) : unused argument (mc.cores = 1)

    Thank you

  24. Hey, The section name for me is totally blank. Is it possible to fetch names on mac, or is this a glitch? (I’ve tried from.name and name.)

  25. Is there a way to get the user ids of all users who have engaged with posts on a FB page? We basically want to analyse user data to create user personas for our clients. Further on, is there a way to pull user related data for these users (demographic data, interests etc).?

  26. Hi, I want to extract comments to may posts of my facebook page, but Facepager doesn’t give the name or fbk id, or other informations about the commentator. I need these informations, because I want to run a contest and extract the winner. If there are many comments I will need to collect their name manually, or there is another possibility?
    Thank you!

  27. Hello, David!

    Thank you very much for this detailed explanation and for the video.

    I would really appreciate it if you updated the Facepager tutorials and make another video about extracting comments from groups.

    Thank you very much in advance.

  28. Trying to get data from one of my groups but getting this error.

    Error ‘error (400)’ for 140312773220685 with message (#10) To use ‘Groups API’, your use of this endpoint must be reviewed and approved by Facebook. To submit this ‘Groups API’ feature for review please read our documentation on reviewable features: https://developers.facebook.com/docs/apps/review..

  29. Great stuff you have here David.
    Can I ask for some pointers if I want to extract data from Facebook Marketplace.
    For starters, Facepager does not recognise “marketplace” as an Object_ID, hence would appreciate if you can help identify this.
    Basically I want to extract (Browse) posts under a specific category (say “Garden”) of properties for sale found in a specific Location (or Locations), the price, description and if possible the url of the advert. Grateful if you can provide the Resource, Parameters and Column names to use in Facepager. Looking forward to your help. Much thanks in advance.

  30. Hi David,

    First, let me thank you for everything its being really useful.
    David, I was having issues capturing all the comments in a single post, I was only being able to capture the first 20 or so comments. Now I am having the issue I am only been able to capture the last 3 o 4 posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.