Giter VIP home page Giter VIP logo

twitter-monitor's People

Contributors

banka2017 avatar dependabot[bot] avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

twitter-monitor's Issues

homeline怎么办

是推主的主页时间线,这种应该要鉴权了吧,请问大佬怎么做到这种, 用curl复制能跑通

批量获取推文详情

如果我有大量推文ID,我想批量大规模获取这些推文的详情信息,有方式实现吗?我看statuses/lookup.json等接口已经无法使用了

Signature builder returns wrong signature.

你好 BANKA, I am pretty sure that your signature builder (https://banka2017.github.io/twitter-monitor/apps/online_tools/oauth_signature_builder.html) generates wrong signatures when using GraphQL links.

import asyncio

import aiohttp
import orjson

async def test() -> None:
    """Inputs used in signature builder.

    1:
        oauth_consumer_key = "3nVuSoBZnx6U4vzUxf5w"
        oauth_consumer_secret = "Bcs59EFbbsdF6Sl9Ng71smgStWEGwXXKSjYvPVt7qys"
        oauth_token = "1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD",
        oauth_token_secret = "TmgPWdJomJWfXZKKpMkKCFjCDk8LFrTcyaqEtKe4NYNzv"
        method = "GET"
        url = "https://api.twitter.com/1.1/users/show.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&user_id=783214"
        body = ""
        timestamp = "1706110770"
        nonce = "OWZiNWZiZjY4OTkxNDdkZmIyMWY5ZjU2YTJhODZhZWM"

        Generated signature: 'OAuth realm="http://api.twitter.com/", oauth_version="1.0", oauth_token="1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD", oauth_nonce="OWZiNWZiZjY4OTkxNDdkZmIyMWY5ZjU2YTJhODZhZWM", oauth_timestamp="1706110770", oauth_signature="GZ3cEyTVhLp9NtSQYW%2BebDCH%2BaU%3D", oauth_consumer_key="3nVuSoBZnx6U4vzUxf5w", oauth_signature_method="HMAC-SHA1"'

    2:
        oauth_consumer_key = "3nVuSoBZnx6U4vzUxf5w"
        oauth_consumer_secret = "Bcs59EFbbsdF6Sl9Ng71smgStWEGwXXKSjYvPVt7qys"
        oauth_token = "1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD",
        oauth_token_secret = "TmgPWdJomJWfXZKKpMkKCFjCDk8LFrTcyaqEtKe4NYNzv"
        method = "GET"
        url = "https://api.twitter.com/graphql/CO4_gU4G_MRREoqfiTh6Hg/UserByRestId?variables=%7B%22withSuperFollowsUserFields%22%3Atrue%2C%22withSafetyModeUserFields%22%3Atrue%2C%22userId%22%3A%22783214%22%7D&features=%7B%22hidden_profile_likes_enabled%22%3Atrue%2C%22hidden_profile_subscriptions_enabled%22%3Atrue%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22highlights_tweets_tab_ui_enabled%22%3Atrue%2C%22responsive_web_twitter_article_notes_tab_enabled%22%3Afalse%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%7D"
        body = ""
        timestamp = "1706110880"
        nonce = "YzI3YzQxMjBjYzlkNDE0OTliNGMxM2YxNDEzZWEwMDU"

        Generated signature: 'OAuth realm="http://api.twitter.com/", oauth_version="1.0", oauth_token="1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD", oauth_nonce="YzI3YzQxMjBjYzlkNDE0OTliNGMxM2YxNDEzZWEwMDU", oauth_timestamp="1706110880", oauth_signature="9miWy0rpZbYibURJTM4zKBasPVc%3D", oauth_consumer_key="3nVuSoBZnx6U4vzUxf5w", oauth_signature_method="HMAC-SHA1"'

    """

    test_cases = [
        {   
            "name": "restful",
            
            "url": "https://api.twitter.com/1.1/users/show.json?include_profile_interstitial_type=1&include_blocking=1&include_blocked_by=1&include_followed_by=1&include_want_retweets=1&include_mute_edge=1&include_can_dm=1&include_can_media_tag=1&skip_status=1&user_id=783214",
            
            "authorization": 'OAuth realm="http://api.twitter.com/", oauth_version="1.0", oauth_token="1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD", oauth_nonce="OWZiNWZiZjY4OTkxNDdkZmIyMWY5ZjU2YTJhODZhZWM", oauth_timestamp="1706110770", oauth_signature="GZ3cEyTVhLp9NtSQYW%2BebDCH%2BaU%3D", oauth_consumer_key="3nVuSoBZnx6U4vzUxf5w", oauth_signature_method="HMAC-SHA1"'
        },
        {
            
            "name": "graphql",
            
            "url": "https://api.twitter.com/graphql/CO4_gU4G_MRREoqfiTh6Hg/UserByRestId?variables=%7B%22withSuperFollowsUserFields%22%3Atrue%2C%22withSafetyModeUserFields%22%3Atrue%2C%22userId%22%3A%22783214%22%7D&features=%7B%22hidden_profile_likes_enabled%22%3Atrue%2C%22hidden_profile_subscriptions_enabled%22%3Atrue%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22highlights_tweets_tab_ui_enabled%22%3Atrue%2C%22responsive_web_twitter_article_notes_tab_enabled%22%3Afalse%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%7D",
            
            "authorization": 'OAuth realm="http://api.twitter.com/", oauth_version="1.0", oauth_token="1749847962413617152-wZGYi1AGGPFWpq5mxU767lMKn6WrCD", oauth_nonce="YzI3YzQxMjBjYzlkNDE0OTliNGMxM2YxNDEzZWEwMDU", oauth_timestamp="1706110880", oauth_signature="9miWy0rpZbYibURJTM4zKBasPVc%3D", oauth_consumer_key="3nVuSoBZnx6U4vzUxf5w", oauth_signature_method="HMAC-SHA1"'
        }
    ]

    for case in test_cases:

        headers = {
            "Authorization": case["authorization"],
            "Content-Type": "application/json",
            "User-Agent": "TwitterAndroid/10.24.0-release.0 (310240000-r-0) sdk_gphone64_x86_64/12 (Google;sdk_gphone64_x86_64;google;sdk_gphone64_x86_64;0;;1;2013)",
            "X-Twitter-API-Version": "11",
            "X-Twitter-Client": "TwitterAndroid",
            "X-Twitter-Client-Version": "10.24.0-release.0",
            "OS-Version": "31",
            "System-User-Agent": "Dalvik/2.1.0 (Linux; U; Android 12; sdk_gphone64_x86_64 Build/SE1A.220826.008)",
            "X-Twitter-Active-User": "yes",
        }

        async with aiohttp.ClientSession() as session:
            async with session.get(
                    url=case["url"], 
                    headers=headers
                    ) as response:
                resp = await response.content.read(-1)
                print(f'[{case["name"]}] Response:\n {orjson.loads(resp)}\n')

                try:
                    print(f'Rate limits:\n{response.headers["x-rate-limit-limit"]}:{response.headers["x-rate-limit-remaining"]}\n')
                except KeyError:
                    print("Failed to get ratelimits.\n")
    
    await asyncio.Future()

if __name__ == "__main__":
    asyncio.run(test())

As a a result, response for restful will return expected info in json as expexcted:

[restful] Response:
 {'id': 783214, 'id_str': '783214', 'name': 'X', 'screen_name': 'X', 'location': 'everywhere', 'profile_location': None, 'description': "what's happening?!", 'url': 'https://t.co/bGcvaMApJO', 'entities': {'url': {'urls': [{'url': 'https://t.co/bGcvaMApJO', 'expanded_url': 'https://about.x.com/', 'display_url': 'about.x.com', 'indices': [0, 23]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 67436194, 'fast_followers_count': 0, 'normal_followers_count': 67436194, 'friends_count': 0, 'listed_count': 88670, 'created_at': 'Tue Feb 20 14:35:54 +0000 2007', 'favourites_count': 5909, 'utc_offset': None, 'time_zone': None, 'geo_enabled': True, 'verified': False, 'statuses_count': 15124, 'media_count': 2413, 'lang': None, 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': 'ACDED6', 'profile_background_image_url': 'http://abs.twimg.com/images/themes/theme18/bg.gif', 'profile_background_image_url_https': 'https://abs.twimg.com/images/themes/theme18/bg.gif', 'profile_background_tile': True, 'profile_image_url': 
'http://pbs.twimg.com/profile_images/1683899100922511378/5lY42eHs_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/1683899100922511378/5lY42eHs_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/783214/1690175171', 'profile_link_color': '1B95E0', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'F6F6F6', 'profile_text_color': '333333', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'pinned_tweet_ids': [], 'pinned_tweet_ids_str': [], 'has_custom_timelines': True, 'can_dm': False, 'can_media_tag': True, 'following': False, 'follow_request_sent': False, 'notifications': False, 'muting': False, 'blocking': False, 'blocked_by': False, 'want_retweets': False, 'advertiser_account_type': 'promotable_user', 'advertiser_account_service_levels': ['dso', 'dso', 'dso', 'analytics', 'analytics', 'media_studio'], 'analytics_type': 'enabled', 'profile_interstitial_type': '', 'business_profile_state': 'none', 'translator_type': 'regular', 'withheld_in_countries': [], 'followed_by': False, 'require_some_consent': False}

Rate limits:
900:895

where GraphQL will return:

 [graphql] Response:
 {'errors': [{'message': 'Could not authenticate you', 'code': 32}]}

Failed to get ratelimits.

历史数据翻页

universal.json接口无法翻页,尝试max_id和cursor均无法翻页

twitter跟新问题

请问定向用户采集和关键词搜索采集,目前依旧可以使用游客认证X-Guest-Token,进行数据采集嘛?

cannot download twitterspace

Hi, BANKA2017!
I am thankful to your blog post about twitter.

I found error about downloading/reading twitterspace.

now, my script's AudioSpaceId (graphql:audiospace) return error.

  • I using fa0311/TwitterInternalAPIDocument for 'features' field.

Also, twitter-monitor has error now.
test spaceurl : https://twitter.com/i/spaces/1djGXldPqNyGZ

That spaceurl is ended. I know.
Yesterday, I can read information using twitter-monitor.
But. Cannot read that now using twitter-monitor.

<< graphql:audiospace response >>
{"errors":[{"message":"The following features cannot be null: rweb_video_timestamps_enabled","extensions":{"name":"BadRequestError","source":"Client","code":336,"kind":"Validation","tracing":{"trace_id":"censored"}},"code":336,"kind":"Validation","name":"BadRequestError","source":"Client","tracing":{"trace_id":"censored"}}]}

Thank you for read it.

some questions about get_and_upload_guest_account.mjs

hi all,

My first question is can we put HTTPS proxies in proxy.txt ???

when I run 'get_and_upload_guest_account.mjs' it always finds 0 account .
I always have the same answer:

autonia@wayne:~$ node /home/autonia/opt/banka/twitter-monitor/apps/open_account/scripts/get_and_upload_guest_account.mjs [Wed Mar 27 2024 01:29:09 GMT-0700 (Pacific Daylight Time)]: #GuestToken Update open account [Wed Mar 27 2024 01:29:09 GMT-0700 (Pacific Daylight Time)]: #GuestToken Update guest token #Bearer AAAAAAAAAAAAAAAAAAAAAFXzAwAAAAAAMHCxpeSDG1gLNLghVe8d74hl6k4%3DRUMF4xAQLsbeBhTSRrCiQpJtxoGWeyHrDb5te2jpGskWDFW82F TypeError: Cannot read properties of undefined (reading 'oauth_token') at GuestToken.openAccountInit (file:///home/autonia/opt/banka/twitter-monitor/libs/core/Core.function.mjs:74:90) at processTicksAndRejections (node:internal/process/task_queues:96:5) at async file:///home/autonia/opt/banka/twitter-monitor/apps/open_account/scripts/get_and_upload_guest_account.mjs:33:9 count: 1 open_account: No guest account

I wonder if I missed something in my configuration

in 'Backend (Cloudflare Workers)'
i must edit the file 'backend/wrangler.toml' and replace '<YOUR_ID>' by the ID given by cloudflare after creating the kv space named 'twitter-monitor-workers-kv' like this :
kv_namespaces = [ { binding = "open_accounts", preview_id = "NOT_NECESSARY_TO_CHANGE_THIS_VALUE_KV", id = "20e51c53ad77777ba75a7b6f44cf1f2a" }

the <SECRET_TOKEN> is a key completely invented by myself, I just have to put the same key in both files, like this

in backend/.dev.vars
SECRET_WORKERS_KEY="7buwum7PFiFumFRQinZBhz5qWDk4qakjo$QoTi5f"

in scripts/get_and_upload_guest_account.mjs
const key = '7buwum7PFiFumFRQinZBhz5qWDk4qa*kjo$QoTi5f'

in scripts/get_and_upload_guest_account.mjs
if the const endpoint= link replies 'hello world' then everything is good with the configuration on Cloudflare
const endpoint = 'https://worker-winter-bush-156a.oleaster.workers.dev/upload/account'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.