10acad / git-get-started Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
the preprocessing module was imported as py
but was used as p.clean( ) when cleaning the twitter data
107
108 #page attribute in tweepy.cursor and iteration
109 for page in tweepy.Cursor(api.search, q=keyword,count=200, include_rts=False):
name api is not defined.
we can fix it by editing to self.api.search
To resolve this issue will have to add this line api = tweepy.API(auth) before using the api.search
The filtered_tweet variable (in Line 134) in the get_tweet function in the tweet_search class has been called but was defined in the clean_tweets function i.e it has not been defined in the get_tweet function.
Failed to import csv and re modules
Error 'cursor' object is not iterable occurs on the following line:
for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False):
so we add .pages() to go through all the pages or .pages() to go through required number of pages.
for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False).pages():
Class not working due to the following reasons:
The variable name given to the preprocessor library imported is consistent. ppr
was used from the start and p
was used to call method clean
import preprocessor as ppr
#use preprocessor
>> tweet = p.clean(twitter_text)
Regex was used to compile regex patterns but was never imported
We have to do an
import re
The method get_data in ts.get_data is not available in the class tweetsearch
df = ts.get_data(covid_keywords, csvfile=tweets_file)
It appears the intended method to be called is get_tweets.
hence we ought to have:df = ts.get_tweets(covid_keywords, csvfile=tweets_file)
Some modules have either not been imported or their aliases do not match what is in the code.
preprocessor
: aliased as ppr
during import but referenced as p
in the tweetSearch()
classget_tweets should be called in place of get_data which was used in the code as nothing like get_data exists in the class
filename in the tweetfile variable as the name was not consistent
with the filename sent to us on the slack channel
tweetfile variable used ethiopia/........ but the filename didnt contain the word ethiopia
please help me fix this.
Thanks
fixed the call to TweetSearch.get_data() to TweetSearch.get_tweets
No self parameter passed into tweet search in def_init function for the tweets_search class.
def init(cols=None,auth=None):
if not cols is None:
self.cols = cols
In line 7, the self parameter is missing
In class tweetsearch(), the api can not be read by the function get_tweets().
Saved numbers of tweets from availed data was 1000.
Resolved this issue by
tweets = []
for line in open('covid19_23june2020.json', 'r'):
tweets.append(json.loads(line))
line 6 xyz should be zyx because
There was no self parameter for the init constructor
There was also no concluding else statement for one of the if clauses in the init constructor
This was fixed by adding .page() to this line of code
#page attribute in tweepy.cursor and iteration
for page in tweepy.Cursor(self.api.search,q=keyword,count=200, include_rts=False).pages():
The def get_tweets(self,keyword,csvfile=None) utilizes the api.search.
which culminates to the line below in the twitter_mining code:
#page attribute in tweepy.cursor and iteration for page in tweepy.Cursor(api.search, q=keyword,count=200, include_rts=False):
The correct reference is stated below:
#page attribute in tweepy.cursor and iteration for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False):
When using the pandas read_json to read in the json file:
Incorrect syntax:
if not csvfile is None:
#If the file exists, then read the existing data from the CSV file.
if os.path.exists(csvfile):
df = pd.read_json('covid19_23june2020.json')
the code above will give the trailing data error, you can simply add lines = True as shown below
Correct syntax:
if not csvfile is None:
#If the file exists, then read the existing data from the CSV file.
if os.path.exists(csvfile):
df = pd.read_json('covid19_23june2020.json', lines=True)
The cursor object is used to loop through data.
Therefore we append .pages() to the end of the line since we are looping through pages.
Nice description of the problem and error you found
Hi,
When I try to run the Stream part of the Code, I find an error which is" No such file or directory". And I am having issues on adding the json file path.
In the cursor object the argument is not correctly named and the pages are not called for iteration and the twitter function was not called.
After debugging the code, some crucial part that can be solve, I corrected them immediately but by running the code I keep on getting ValueError: Only unicode objects are escapable. Got None of type <class 'NoneType'>.
By: [email protected]
tweetsearch
does not instantiate with self
tweetsearch
has no method get_data
replace with get_tweets
for page in tweepy.Cursor(api.search, q=keyword,count=200, include_rts=False):
with for page in tweepy.Cursor(self. api.search, q=keyword,count=200, include_rts=False).pages():
tweet = p.clean(twitter_text)
to tweet = ppr.clean(twitter_text)
clean_tweets
to @staticmethod
since it does not require the instance of the class clean_text = status['text'] filtered_tweet= self.clean_tweets(clean_text)
import re #regular expression
nltk.download('punkt')
from nltk.tokenize import word_tokenize
StdOutListener
No module named 'tweetpy'
It says 'file is not in Json format'
Some libraries were not imported for cleaning the tweet data and also the clean data was not referenced, it was supposed to be referenced as self but it is not
107
108 #page attribute in tweepy.cursor and iteration
109 for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False):
line 109 raised the error and it can be fixed with
for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False).pages():
This particular line of code in the get_tweets(self,keyword,csvfile=None) references the use of api.search.
It says:
#page attribute in tweepy.cursor and iteration for page in tweepy.Cursor(api.search, q=keyword,count=200, include_rts=False):
But it should be
#page attribute in tweepy.cursor and iteration for page in tweepy.Cursor(self.api.search, q=keyword,count=200, include_rts=False):
I initially got a TypeError: 'Cursor' object is not iterable
and so I added .pages()
attribute to the object then I got the TweepError described above and I don't know what the issue is exactly...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.