NOTE: this code is for a previous version of the Twitter API and I will not be updating in the near future. If someone else would like to, I'd welcome that! Feel free to ping me. END NOTE. Here I define a Tweet listener that creates a file called 'tweets.txt', collects streaming tweets as .jsons and writes them to the file 'tweets.txt'; once 100…

tweet_listener.py

class MyStreamListener(tweepy.StreamListener):
    def __init__(self, api=None):
        super(MyStreamListener, self).__init__()
        self.num_tweets = 0
        self.file = open("tweets.txt", "w")

    def on_status(self, status):
        tweet = status._json
        self.file.write( json.dumps(tweet) + '\n' )
        self.num_tweets += 1
        if self.num_tweets < 100:
            return True
        else:
            return False
        self.file.close()

    def on_error(self, status):
        print(status)

datacamp rules- especially hugo.

@hugobowne

As per the comments from @wkkim-se @plumps and @hiliev I believe there are corrections required in this class for downloading streaming Twitter data. I am new to Python so not yet into creating classes. I have made the corrections as per my understanding and it appends the text file. Hope it is correct. Could you review below code and possibly correct the code here in Github since there will be many like me who will be confused and hunting for the solution!

class MyStreamListener (tweepy.StreamListener):
    def __init__(self, api = None):
        super(MyStreamListener, self).__init__()
        self.num_tweets = 0
        self.file_name = "tweets.txt"
        #self.file = open("tweets.txt", "w")

    def on_status(self, status):
        tweet = status._json
        with open(self.file_name, 'a') as file:
            file.write(json.dumps(tweet) + '\n')
        self.num_tweets += 1
        if self.num_tweets < 100:
            return True
        else:
            return False

    def on_error(self, status):
        print(status)

@hugobowne Thanks for sharing this code. I had a query related to Loading and Exploring the twitter data.

I was trying the below code on my laptop. However, it returned an error. Error message has been shown below:

Read in tweets and store in list: tweets_data

for line in tweets_file:
tweet = json.loads(line)
tweets_data.append(tweet)

Error Message: JSONDecodeError: Extra data: line 1 column 5703 (char 5702)

Could you please help me with this issue?

Nipun

Anyone else getting a '401' response from Twitter when you replace the mock access and consumer keys with your own?

This code is the child class, where is the parent class? Post it please

@eric-ahlgren I think it will work just fine

@plumps If it was on_error and not on_status, and the file was opened in on_status not in init, wouldn't the file close? And if it did close, the file was opened in "w" mode but not "a" , wouldn't the content be lost every time the file is reopened?

@hugobowne can you please add the correct code?

@strashynskyi thanks for pinging me. it looks like this the twitter API has changed so that this code doesn't run now. I don't have the bandwidth to go in and figure out what the correct code looks like. If someone else wants to, that would be great. I've made the following note in the description of this gist:

NOTE: this code is for a previous version of the Twitter API and I will not be updating in the near future. If someone else would like to, I'd welcome that! Feel free to ping me. END NOTE.

class MyStreamListener (tweepy.StreamListener):
    def __init__(self, api = None):
        super(MyStreamListener, self).__init__()
        self.num_tweets = 0
        self.file_name = "tweets.txt"
        #self.file = open("tweets.txt", "w")

    def on_status(self, status):
        tweet = status._json
        with open(self.file_name, 'a') as file:
            file.write(json.dumps(tweet) + '\n')
        self.num_tweets += 1
        if self.num_tweets < 100:
            return True
        else:
            return False

    def on_error(self, status):
        print(status)

class MyStreamListener (tweepy.StreamListener):
    def __init__(self, api = None):
        super(MyStreamListener, self).__init__()
        self.num_tweets = 0
        self.file_name = "tweets.txt"
        #self.file = open("tweets.txt", "w")

    def on_status(self, status):
        tweet = status._json
        with open(self.file_name, 'a') as file:
            file.write(json.dumps(tweet) + '\n')
        self.num_tweets += 1
        if self.num_tweets < 100:
            return True
        else:
            return False

    def on_error(self, status):
        print(status)

Thanks