status_id variable is behaving correctly? about comparing-sentiment HOT 9 OPEN

Indeed, I have recognized this before. My best guess is either it has to do with attributes of the tweets or with how old the tweets are. In any way, this varying length has been apparent in other data sets I have seen as well.

from comparing-sentiment.

got it - sounds good. okay if I keep this open for a bit to remind us to check this at some point? As you've addressed by using .rds files, if we use CSVs, sometimes these get treated as doubles by default, causing 0s to be erroneously removed. Shouldn't be an issue but would want to double-check.

from comparing-sentiment.

Oh... Was not aware of this, I agree and will check later as well!

from comparing-sentiment.

I think there is never a problem when we use .rds files - but when we save and the read .csv files (without specifying the column type), this can emerge!

from comparing-sentiment.

Hi! This became an issue again now that I wanted to left join the combined LIWC results with the final raw data frame (including the state based hashtags). The number of unique status IDs for the raw data files and the LIWC data you gave me are not equal. The quickest solution might be to just re-run LIWC over the combined raw data files (NGSS and state-based hashtags), see the current branch "conrad2". Would you mind running LIWC one more time or looking at this again? Thanks...

from comparing-sentiment.

Oye sorry. Lemme take the short/quick option for now. One moment.

from comparing-sentiment.

just uploaded a new file (a compressed file as the size was causing some problems): state-hashtags-liwc-results.csv.zip

from comparing-sentiment.

also pushed a small commit to the conrad2 branch

from comparing-sentiment.

Thanks! All seems to work.

from comparing-sentiment.