himura / twitter-types Goto Github PK

I see that this change added some support for extended entities, but it doesn't include information about the videos posted with the tweet (see this code: https://github.com/himura/twitter-types/pull/41/files#diff-72926f72678f887165e9b64fcfa65423320e8b182b22c95c9997c42aa3dea758R888). Link to the relevant code in the master branch: https://github.com/himura/twitter-types/blob/master/twitter-types/Web/Twitter/Types.hs#L899.

I see that there's example API responses including the video_info already in the repo: https://github.com/himura/twitter-types/blob/master/twitter-types/tests/fixtures/tweet-updates/extended_classic_hidden_13761.json

Link to the Twitter API docs with more information about the data types: https://developer.twitter.com/en/docs/twitter-api/v1/data-dictionary/object-model/extended-entities#tweet-video

I'll see if I can manage to create a PR for this myself. Just creating the issue as documentation :)

Even though tests/Fixtures.hs is included, the tests still don't build:

Preprocessing test suite 'tests' for twitter-types-0.2.20140406...

tests/Fixtures.hs:11:8:
    Could not find module `Text.Shakespeare.Text'
    Perhaps you meant
      Text.Shakespeare.Base (from shakespeare-1.2.1.1)
      Text.Shakespeare (from shakespeare-1.2.1.1)
    Use -v to see a list of the files searched for.

It looks like the tests depend on shakespeare, but still reference modules from shakespeare-text.

Twitter has updated its policy to not include certain content types in the 140 character count. Therefore, many tweets are truncated to 140 characters in the so-called "Classic mode". twitter-types (and twitter-conduit) would need to support the "extended mode", since it may one day become the default.

Twitter's documentation is located here: https://dev.twitter.com/overview/api/upcoming-changes-to-tweets

This will enable people to have the lenses and constructors in scope at the same time without qualification. This will make things like implementing instances for the data types simpler.

The userWithheldInCountries field is defined to be Maybe Text. It seems from e.g. https://blog.twitter.com/2012/new-withheld-content-fields-api-responses that the field should be Maybe [Text].

This is terribly inconvenient and prevents me doing something as simple as serializing the data into JSON that will "just work" with the library.

I just got a failure to parse, and after making a change as described in #21, I was able to produce the error message:

FromJSONError "couldn't parse contributor from: Number 2.55883322e9"

Trying to find information about this field, I came upon the document Overview: Tweets, which provides the following example:

"contributors":
[
    {
        "id":819797,
        "id_str":"819797",
        "screen_name":"episod"
    }
]

I also found GET status/show/:id, which says:

If there are no contributors for a Tweet, then there will be an
empty or "contributors" : {}. This field will only be populated if
the user has contributors enabled on his or her account — this is
a beta feature that is not yet generally available to all.

This object contains an array of user IDs for users who have
contributed to this status (an example of a status that has been
contributed to is this one). In practice, there is usually only
one ID in this array. The JSON renders as such
"contributors":[8285392].

I tried to get the raw JSON output for the request I'm making, but it's for the retweets_of_me timeline so calling the API "manually" using the API tool that Twitter provides isn't possible.

This feature doesn't seem completely finished yet on Twitter's end, to the point that there are two different API specifications for how the field should look or what it does, and different endpoints seem to behave differently. Does it make sense to remove parsing for this attribute for now, until it's generally available?

aeson 0.10 has changed the semantics of the .:? combinator, so twitter-types is currently broken for at least the contributors field. Using aeson 0.9.0.1 seems fine, so twitter-types needs either a dependency version upper-bound or an update to use aeson 0.10

We are in the process of moving Stackage Nightly to aeson-0.11.

I see one test failure in the Stackage Nightly build when rebuilding with aeson-0.11:

  case_parseErrorMsg: [Failed]
expected: "Not authorized"
 but got: "Error in $: Not authorized"

The package doesn't compile on Mac OS X Maverick, the problem apparently come from the new preprocessor. (first point of http://www.haskell.org/ghc/docs/7.8.1-rc1/html/users_guide/release-7-8-1.html).

Apparently I can't tell ghc to use cpphs instead of clang. So I had to use cpphs manually on the file Lens.hs. The problem, is that (using clang -E), the clang preprocessor has problems with the ' char of Lens'. I renamed it Lens.m and ran clang -E Lang.m here is the output:

Lens.m:201:10: warning: missing terminating ' character [-Winvalid-pp-token]
type Lens' s a = Lens s s a a
         ^
Lens.m:204:13: warning: missing terminating ' character [-Winvalid-pp-token]
name :: Lens' (s) (a);\
            ^
Lens.m:338:22: warning: missing terminating ' character [-Winvalid-pp-token]
    status_id :: Lens' s StatusId
                     ^
Lens.m:339:17: warning: missing terminating ' character [-Winvalid-pp-token]
    text :: Lens' s Text
                ^
Lens.m:340:17: warning: missing terminating ' character [-Winvalid-pp-token]
    user :: Lens' s User
                ^
Lens.m:341:23: warning: missing terminating ' character [-Winvalid-pp-token]
    created_at :: Lens' s DateString
                      ^
Lens.m:368:20: warning: missing terminating ' character [-Winvalid-pp-token]
    user_id :: Lens' u UserId
                   ^
Lens.m:369:17: warning: missing terminating ' character [-Winvalid-pp-token]
    name :: Lens' u UserName
                ^
Lens.m:370:24: warning: missing terminating ' character [-Winvalid-pp-token]
    screen_name :: Lens' u Text
                       ^
9 warnings generated.

I then renamed Lens' by LensTMP the clang preprocessor outputted another error:

Lens.m:206:5: error: '#' is not followed by a macro parameter
{-# INLINE name #-}
    ^
1 error generated.

But the cabal install command worked.

Do you want me to make a pull request? I will simply replace the Lens' by LensTMP or another more suitable name.

The twitter-types description mentions the enumerator package.

When parsing fails, it's unclear what type wasn't parsed or what the value should have been. Using 'fail' provides more information to the developer which leads to better bug reports ;)

Instead of this:

instance FromJSON Contributor where
    parseJSON (Object o) =
        Contributor <$>  o .:  "id"
                    <*>  o .:  "screen_name"
    parseJSON _ = mzero

Do something like this:

instance FromJSON Contributor where
    parseJSON (Object o) =
        Contributor <$>  o .:  "id"
                    <*>  o .:  "screen_name"
    parseJSON v = fail $ "couldn't parse contributor from: " ++ show v

The generated tar file doesn't contain all the files needed to run tests.

I can't use twitter-types 0.6.1 because twitter-types-lens 0.6.0 requires twitter-types 0.6.0.

Certain Twitter APIs, for example https://dev.twitter.com/rest/reference/get/users/lookup, come with the user's "last status", if you have permission to read it. It would be nice if this "status" were available.

It might make sense to separate the types of User so that /users/lookup returns a UserWithStatus, since some APIs will never have this field available. I can try to implement this if you like.