thisisjaid / 4am Goto Github PK

Here's the error message I get when running the script on the cultural studies tweets:

Traceback (most recent call last): File "analyze.py", line 86, in <module> tweets.drop(tweets.index[mdtweets], inplace=True) File "/Users/stacy_altmetric/anaconda/lib/python3.6/site-packages/pandas/core/indexes/range.py", line 521, in __getitem__ return super_getitem(key) File "/Users/stacy_altmetric/anaconda/lib/python3.6/site-packages/pandas/core/indexes/base.py", line 1650, in __getitem__ result = getitem(key) IndexError: index 1677 is out of bounds for axis 1 with size 1338

I don't encounter this issue when running the script on the other JSON files in the dataset.

Would you mind taking a look, @thisisjaid?

I need to pull an updated set of descriptive stats, once we've pulled the journal-level metrics and full-text twitter data.

I noticed some words that would be in our dataset might not actually have the existing, coded negative or positive associations (e.g. the use of "queer" in our gender studies tweet dataset). We should review all the SentiStrength lists to be sure terms are coded correctly for our purposes. I'm happy to do that.

In looking more closely at the .txt versions of the datasets that Maciej pulled, it seems as though there are more than just Twitter mentions mixed in (some blog posts, some news). Would you mind pulling the data again for all three disciplines, so we'll have reasonably similar datasets pulled around the same timeframe? Thanks!

Once we have those new json datasets, it should be pretty easy to create the new .txt files to run through Sentistrength (which I'm experimenting with now).

Let's review the raw tweet data for each dataset, once re-exported, to see if it's cleaned up enough (w.r.t. having most paper titles removed, etc) that we won't have to do a manual data cleanup as a final pass, after we run the analyze.py script.