User lookup on Twitter API from R results in error (403) -


using twitter api , twitter-package, trying retrieve user objects long list of names (between 50.000 , 100.000).

i keep getting following error:

error in twinterfaceobj$doapicall(paste("users", "lookup", sep = "/"),  :    client error: (403) forbidden 

the error code supposedly hints @ "update limits". rate limit on user lookups 180 , lookups performed in batches of 100 user names. therefore 18.000 users shouldn't problem. reducing number 6000 (to respect limit on requests via application-only auth) per 15 minute time window results in same error.

here mwe (for do, however, need your own api-keys):

library(plyr) # install latest versions github: # devtools::install_github("twitter", username="geoffjentry") # devtools::install_github("hadley/httr") library(twitter) library(httr)      source("twitterkeys.r") # own api-keys setup_twitter_oauth(consumerkey, consumersecret, accesstoken, accesssecret)  # following generate large enough list of user names: searchterms <- c("worldcup", "economy", "climate", "wimbledon",                   "apple", "android", "news", "politics")  # might take while sample <- llply(searchterms, function(term) {   tweets <- twlisttodf(searchtwitter(term, n=3200))   users <- unique(tweets$screenname)   return(users) })  usernames <- unique(unlist(sample))  # function supposed perform lookups in batches  # , mind rate limit: getuserobjects <- function(users) {   groups <- split(users, ceiling(seq_along(users)/6000))   userobjects <- ldply(groups, function(group) {     objects <- lookupusers(group)     out <- twlisttodf(objects)     print("waiting 15 minutes...")     sys.sleep(900)     return(out)   })   return(userobjects) }  # putting action: userobjects <- getuserobjects(usernames) 

sometimes looking smaller subsets manually e.g. via lookupusers(usernames[1:3000]) works; when try automate process, however, error gets thrown.

does have idea reason might be?

according answer i hit rate limit twitter first request , there not limits on total number of users, on number of calls per 15 minute interval. if each call has 100 users, , trying 6000 users, should making 60 calls, more 15 allowed. try putting program sleep , having send out call again after 15 minutes up.


Comments

Popular posts from this blog

javascript - RequestAnimationFrame not working when exiting fullscreen switching space on Safari -

Python ctypes access violation with const pointer arguments -