Giter VIP home page Giter VIP logo

reyemb / aisexplorer Goto Github PK

View Code? Open in Web Editor NEW
26.0 3.0 8.0 237 KB

This tool allows the identification and tracking of marine vessels by geolocation, MMSI, or via MarineTraffic URLs. It ensures user privacy via proxy support, and includes fallback strategies for optimal performance.

Python 100.00%
vessel vessel-detection ship-tracking vessel-trackimg ais-data ais proxy marine-data

aisexplorer's Introduction

  • ๐Ÿ‘‹ Hi, I'm @reyemb.
  • ๐Ÿ‘€ I'm interested in machine learning, statistics, data engineering, and IIoT.
  • ๐ŸŒฑ I'm currently enhancing my skills in web development.
  • ๐Ÿ’ž๏ธ I'm looking to collaborate on non-monetized projects.

Reyemb's Github stats

aisexplorer's People

Contributors

reyemb avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

aisexplorer's Issues

Still working?

Hi there
I just found this repo and now I'm unsure if it's still working or not due to the captcha obstacle?
Or just a part? I want to create an layer for the Grafana Leaflet plugin https://github.com/pR0Ps/grafana-trackmap-panel to show AIS Vessel in the map together with the Seamarks from the OpenSeaMap.
Can you clarify a bit please?

No response with script

With an easy debug script, nothing happens. It just hangs and waits and no response is retrieved.

`from aisexplorer.AIS import AIS

print(AIS().get_location(255805840))`

After a while:

aisexplorer.Exceptions.NoResultsError: After 10 attempts still no results are given. If you think this is an error in the module raise an issue at https://github.com/reyemb/AISExplorer

Request: login functionality

Could you add a login function, so we can use more function of marinetraffic?

From what I found, the following payload is needed to POST to https://www.marinetraffic.com/en/users/ajax_login.

payload = {
'action': 'login',
'email': 'emailaddress',
'password': 'password',
'is_ajax': 'true'
}
A cookie is then generated and saved in the session for further use.

Scraper not working properly

Hi.
I think that something is broken.
I'm receiving bad answers for all the queries I tried.

Using first code in example:

from aisexplorer.AIS import AIS
AIS().get_location(211281610)

get this:

Response code: 403 - headers: {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:93.0) Gecko/20100101 Firefox/93.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Vessel-Image': '005bf958a6548a79c6d3a42eba493e339624'} <!DOCTYPE html>
<html lang="en-US">
<head>
    <title>Just a moment...</title>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=Edge">
    <meta name="robots" content="noindex,nofollow">
    <meta name="viewport" content="width=device-width,initial-scale=1">
    <link href="/cdn-cgi/styles/challenges.css" rel="stylesheet">
    

</head>
<body class="no-js">
    <div class="main-wrapper" role="main">
    <div class="main-content">
        <noscript>
            <div id="challenge-error-title">
                <div class="h2">
                    <span class="icon-wrapper">
                        <div class="heading-icon warning-icon"></div>
                    </span>
                    <span id="challenge-error-text">
                        Enable JavaScript and cookies to continue
                    </span>
                </div>
            </div>
        </noscript>
        <div id="trk_jschal_js" style="display:none;background-image:url('/cdn-cgi/images/trace/managed/nojs/transparent.gif?ray=7c3b2a8ab8e60e83')"></div>
        <form id="challenge-form" action="/en/reports?asset_type=vessels&amp;columns=time_of_latest_position,flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,mmsi,ship_type,show_on_live_map,area,area_local,lat_of_latest_position,lon_of_latest_position,fleet,status,eni,speed,course,draught,navigational_status,year_of_build,length,width,dwt,current_port_unlocode,current_port_country,callsign&amp;mmsi%7Ceq%7Cmmsi=211281610&amp;__cf_chl_f_tk=5lumYa8u0Q0vGXEyl7aDdqDED60HcE1cQby3PRkjsu4-1683480367-0-gaNycGzND3s" method="POST" enctype="application/x-www-form-urlencoded">
            <input type="hidden" name="md" value="T0A9Q_oWWTNJ90qfXxdKt7Kwd5GWi2HMmEERwiWf.YE-1683480367-0-AcanNC-UP989ezPBKwc5slRCauGpYvNedJ-BUPQzWF062vpPdVQsTT1-tJJ8Osqp8__2cxZPliy1Ikb-_r84MeU4foBC4a3kX6aEddMtr1g-4bmzsJcRgknGOFa4QkwSNi5D5xov-Emlh7b5lRRPrOAUue3thMEyHvEbJhMG2fSif1xQ3U24QM2uJR94kshPObC7tPE_GYY0wO4F4OjwT8Ej2-dRmxtyGweEo9TvqiQFUB7ocJOqE_GL1_wCXJ3aBrLurBkpgAOXl5368HK_twvHbRK0G3fUbQqfVWDQ8Yy3SuZmUV2T00wgK1nocpJMsSNiO2FbeORCtyNXQY4Lkr0V13PhjlFdKJjD0kfLXawIcNwLpBxF6HqBp2d5RP73AtCZ7pAVwzAGIQL7Dxvc5SJ_wxkCcmsH3hIT_wilOAZ234lntIZNxQGmlTSgs-D6OJf-OUibuONhBtE15byTA1c_WS4xbmQom8Tz0HFzNb14WmJPF9q8ZqUPTNhPVxb_7p6yDSVAxaUdZhnW58PvJQQvE09QYnOCBoaJh8uYKBfaCuJO9ckW8PRPrBfyOnULTxuGb2y234fGfIQH3HORYoeC4u8rIQ5FUuW9pJilmgJBIt-7gtoKS4DC72x456nAaqAgDKN7HKvzmQNVPBwhI0QghVelsRO_JH1q79lrGTLITGPKNOo2S7Yr4Zl19XN_R8WE5fX1hGMmtJhFTytSHyNcAu9apqfQ7VSxxZl2LVH_Uoc3Sb3Mi2vamUozqiJ30KO5dz-zR1UqyirSxDwXNZCXVsB1mLQftQKoa-XSln8z3lGLd6Yi5hGDjDuFFU9Q-VDF1JZq8a3rKDZse8ch2kqct7hxk27UTmR2178V5yHjpcd9Mpoz_DVGu5LD2tq0AiMGe-fsEpQOLR2FSNLKiAiOxMsSLDO5C2btgbjNfVojohliYMHlTR2SLE-vGOFFKqU2xtm8S61IZgYcsiyAyagVAetUg4X5-pCiNLN9BwQF8ocCnfwUg9xlNFPnpb-uYMZXZM58QinR5oQLMEwTQD31MylSCVasWfKlpHKls0mG3g1nBPDOiAzmaTBWXTCrDuBVSAcYv0LTL5HI4-Dk0exkJjGsDFVnzXAF3XuVdfT6-Nn4lANkT6UqjbjezIJ62Gwsh14VphvD5Ii2QVcbqeRNpJuokr4x0GK_HSPUg6Q1C1OLnf3LRbVh4JJUe55H0EH5zFBTzyUfSFOGM23OisSvUWH1wiYVgYH0qVLVKrXLe_6srHU5qCYxc6vILriYSo8EWfxIpvjS6C_9Zcvha2icj5_5rFfEdV4oT8jdlIEkdTIbZ1udYi6ikclh3pOQPnuTaejmiH37Fk7XZq48fpFqzNunT2p4emw30h_Je-TKJ37JGtmG31WlIagNI3Agyos9e-a2okUFn5KHEomxhwGX-iuMs0Pf-d-80pmtlZSVox__5odxbPkPraIlv3jQH7ESTVJUAc2dtSPR59SAEpHgG12H0_whplmpFX6y0YLAAE_iKipswwMG-5ld4tCxK1dL1urDTujt4ohvRQscgVJnFvU8CkYVHaT3-y681B0ZA50u3nLfe5HHzYsXnbyQlRnslUViA7f7e3-AoMoiHdn_Kqs0tu6O1gK94KmqDG1nlikLle7zUgOUyEG7FiU5K_HJ_2jYGjOnbq4_VIomU6xpxiejiwwbbMG48HhK4U_G2Apz7_Fyhi2xzbTiTSpr4kHoS_1gAILBs637Fd4Ym1WBfjMKv7dW1fN8iuOYmreo07v54qvvHwJmzbaVm9DNSJM3Ef0TmJCUpFAqONt2KgHm1iPXxoiK0I6ZSgLA_C-iwRpU_q6txXG_vvBc9csAEgIvlH2AvDLDhmLxkJXXEYuuUzB5jwpcDcRimXuQfedNSWVBpAEE1eExRP7mPk7jknU9A24sX1ptbv_ztoti1Lztxgaj8Q9S9aq0oONoAjttkwrEF3gUznLlTURlUc2TovAe5WgwLQfl3nCkXkyYjpmKKaqJwLQos3W95nj0drk_QE664jRvtOvqgGbtNTkm1Ng9YMUeVrJpMgNqR4MZE7iFzBW1tR4SRa0CatM4FVUoKDp5qyU3s0hH7xenjFveyk3IP9Y1xoay1_yLY3q5NF4mmkUET-_ujvwZyXbvB4ssWoPGSlkDjsK4uWO0JYkSMOLQTj_x6UJvvEp6s7TqSmU7OEMfnNxnqDAQ1Xy5vnxM0hnbNSUaxa8LaD96beFFcSqxCMhY-4Nee_oLp17sUwy7UJMP-OBGaIzH3fgGvhovrSIECKcPRooaScHTuvFYIexxYcv2_dK6UQrzbiGTUmjGf2jqhpcZ7oL2LN3AG4lrlIFr8t18lCikLuHipVLZhcTY4T-5bBSxIq0LLPjFfYJAIly-GZCQAmiuI8AU16FiRYTZIV3dGtXZGHG_xvIIV4QY-LnBKC2zK3UqoJEwVotDjYjtahQ0U16KURQXfAAvCQm4Hztw4emF94yJp4TgumCN_V2PjK9tbo3N-a6IVghUogTI4saqAPIakxt6FCP26EZNnt2islUEGdyjnLl7DF3mV23q_wTBu8zSPdRTpP_88VB2z1RhRsy6RBr8riygz3C4kI1E2W0E3K4leFpGLIBPP_1PimAdZ0ZRpQVSAoAB2lgXe-zK2hLuGUlWwslmeBioVIFsZdO3ougya14PX94YcypCJX0yKsqA2AjxdAUNzJe1oA2NbjVLkH7v2lAThkPp_-u5nKuB3R4VYkHPZT4aVYIq628Nno8thHEmsZhuZ9mOX7WoYJ82V1HubS2dBmJX6bSm8k6pU4eFs2djiefMDEHdydxxXNN9SnrL3Nsg-w5577lQxYY4Q1GLTa1pyY1aeDrqznR8lOMl0jaKjY_3mDkGSBUVNHBIDJBFnZ_RfaWBvjew-QkfBAbFv03rmXkeeAm5zziBGT8EZat0wMKLXiIpIXCFqi7WHguNMjMpNm7DAGA59PKkfg_Me5wHt5dNLfO-fto45YEAeNKxqiA8T6EhMUbO7MHEsWv9boRVpEmsebe4JHo1FnymUs9siIuHm851V55POXZzLcwZLGnzlfLmqcqZvOruPEpYPOjVa5w8atzzZRnIeGDcRNn9_RDy5sPFo6art-1_FrN46xrXdZSLW9KOguNeV4DAJmjcDqS3rqeLGnXbVpzzbIroRTH6v_mhkWeznPjU0EkImXJSSluMXgRwxQsLxtTcvkW0pfmhkMxTWsmqqPv7tl-sAfaNb7Vgb0-i96NUIrZ3NJLZ4uM3s4iH8vVRZ5VSh1U8X-vt18LPPPh4hB9NkoPM1bAGR1xQdSX8gemRn6H0peRv3R0lC8iBf7DmTjJUF23pKd22mtwW32YMbp7Im9lSVIhgUzv-v32-oDOtdhaTEUOnbNb4mhfYn2YyavmbhNc2o8GQ9XrAgAmO2nFdpEzSmyDOBaEexJc2JWGMqt_c1QZfudZTV7KtVZ6hBXwAx8nLXMQw52mBi5ynlcFl8FkAzNPqt_Y-ZGBg6FDR9GYJMZsk5aXkJvwE8jNxP8jcG98KsJtNMr7Y8p_7kxgtvolyXGbH7jrNg3j5mdXP--ToNgXfO1Uxhr3Me58HgR7miXw-HzbIAH_QOsI8KDAnQ4gYFHW5uj5narzO9vHqyVIegBe763BSQlx-5fFN9BkaNm5JzRVl-Rp1iTeMe6cpAe1NQJvdDMFKwTO_TZ3BeGq8PDZhMZxfRxhpjc8kEmDO-6K0xHzA5olzcYgGmH55r67ToKqcd7apj42Oyd4f0_M36ipP3-nP2-3kdP3vtJi-8PXmkGSUcPHvkvjKvyjse4WtZvVxASm2jOeyfAhrnv0K_XLb8ksL7u5tu9aCHTX3QW2M-i85pGt97cq7wOO3OnmFZ5BTgB9h4dtM0IpsC52p0A">
        </form>
    </div>
</div>
<script>
    (function(){
        window._cf_chl_opt={
            cvId: '2',
            cZone: 'www.marinetraffic.com',
            cType: 'managed',
            cNounce: '80480',
            cRay: '7c3b2a8ab8e60e83',
            cHash: 'fbedeeeda5c255e',
            cUPMDTk: "\/en\/reports?asset_type=vessels&columns=time_of_latest_position,flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,mmsi,ship_type,show_on_live_map,area,area_local,lat_of_latest_position,lon_of_latest_position,fleet,status,eni,speed,course,draught,navigational_status,year_of_build,length,width,dwt,current_port_unlocode,current_port_country,callsign&mmsi%7Ceq%7Cmmsi=211281610&__cf_chl_tk=5lumYa8u0Q0vGXEyl7aDdqDED60HcE1cQby3PRkjsu4-1683480367-0-gaNycGzND3s",
            cFPWv: 'g',
            cTTimeMs: '1000',
            cMTimeMs: '0',
            cTplV: 5,
            cTplB: 'cf',
            cK: "",
            cRq: {
                ru: 'aHR0cHM6Ly93d3cubWFyaW5ldHJhZmZpYy5jb20vZW4vcmVwb3J0cz9hc3NldF90eXBlPXZlc3NlbHMmY29sdW1ucz10aW1lX29mX2xhdGVzdF9wb3NpdGlvbixmbGFnLHNoaXBuYW1lLHBob3RvLHJlY29nbml6ZWRfbmV4dF9wb3J0LHJlcG9ydGVkX2V0YSxyZXBvcnRlZF9kZXN0aW5hdGlvbixjdXJyZW50X3BvcnQsaW1vLG1tc2ksc2hpcF90eXBlLHNob3dfb25fbGl2ZV9tYXAsYXJlYSxhcmVhX2xvY2FsLGxhdF9vZl9sYXRlc3RfcG9zaXRpb24sbG9uX29mX2xhdGVzdF9wb3NpdGlvbixmbGVldCxzdGF0dXMsZW5pLHNwZWVkLGNvdXJzZSxkcmF1Z2h0LG5hdmlnYXRpb25hbF9zdGF0dXMseWVhcl9vZl9idWlsZCxsZW5ndGgsd2lkdGgsZHd0LGN1cnJlbnRfcG9ydF91bmxvY29kZSxjdXJyZW50X3BvcnRfY291bnRyeSxjYWxsc2lnbiZtbXNpJTdDZXElN0NtbXNpPTIxMTI4MTYxMA==',
                ra: 'TW96aWxsYS81LjAgKFdpbmRvd3MgTlQgMTAuMDsgV2luNjQ7IHg2NCkgQXBwbGVXZWJLaXQvNTM3LjM2IChLSFRNTCwgbGlrZSBHZWNrbykgQ2hyb21lLzEwOC4wLjAuMCBTYWZhcmkvNTM3LjM2',
                rm: 'R0VU',
                d: '9UmkfU54/yHlLK2blqdZT0IBQgM/eh1L6qHHbLgbN6GLx8JhKbjmQtJDI7wKYHzJSn6VakJMB4vZ5DpcdSF7mUztNmomx0LAL7vgHzjx+vtHVbzfulmOctt8QAYLgBVhF1Iot7+Qkst/9bRSMtQqDFuNNjjJvy7mVx34K92OiWq83ANVOgXyQq0yGhKPX4GjBG4JS1aTiU0O1PZJr8A9DTsuYEHRhfBK6kwypmrQoNIFFBDBhL1O9vDtMUiJ3BxAuG4mLgzxWvkcELdi2asnN0c1TT6GNRSh+tfgp3fGTTO75NL7ERKW3tEoA7wqeaG6LqV+zoTs4qTAAfNE9mY6l0v/B780P8gdoxm4OSNv31sAZMiwS9iCh2u9aS0QmpGX90qlLO5zXD1Bd/DYS0nwilHUuuZjCAcPisM6NveXA3dPEEBqbaa5L6kLwQLfI7kgSqv/F9X98KWR4aBmvEoLza0zC33b3MK4bipJ0fgbEjzGPEcsyxK2q7jG/zmutz5CUoelwxxvdBuX3ebd9IkaWkJ5zzbn5hdCtEoK8niZO9rLD4svIy3Ndi3ekDf7gTiSwDEReVVL3s/jpxC1E4NlaKozoBVNn7ekF/cKizTZ145H6VDsW3xHqq3IYrRLv7nR',
                t: 'MTY4MzQ4MDM2Ny44MDkwMDA=',
                m: 'jW3sPvu77ZpogKHxA0t0C7d8rM2XLHjjwSXEkSbWy0o=',
                i1: 'uQt3aWHaMbHxXSVN54HIPg==',
                i2: 'pDEbQ9i8KJhOeILugKX5Uw==',
                zh: '9ODJ9wUnoPjm/IAh4FoxBVEPcuhbtkHqgDVY7Mg1mFw=',
                uh: 'JkAkY486s8b7quQvdaIQRGNNNpmu+ejtUZNcz6o7wFM=',
                hh: 'qi1pc4iF5r4NNN+eu2J/M9aMOtWqOev4stD1qgOIFxo=',
            }
        };
        var trkjs = document.createElement('img');
        trkjs.setAttribute('src', '/cdn-cgi/images/trace/managed/js/transparent.gif?ray=7c3b2a8ab8e60e83');
        trkjs.setAttribute('alt', '');
        trkjs.setAttribute('style', 'display: none');
        document.body.appendChild(trkjs);
        var cpo = document.createElement('script');
        cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/managed/v1?ray=7c3b2a8ab8e60e83';
        window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;
        window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;
        if (window.history && window.history.replaceState) {
            var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;
            history.replaceState(null, null, "\/en\/reports?asset_type=vessels&columns=time_of_latest_position,flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,mmsi,ship_type,show_on_live_map,area,area_local,lat_of_latest_position,lon_of_latest_position,fleet,status,eni,speed,course,draught,navigational_status,year_of_build,length,width,dwt,current_port_unlocode,current_port_country,callsign&mmsi%7Ceq%7Cmmsi=211281610&__cf_chl_rt_tk=5lumYa8u0Q0vGXEyl7aDdqDED60HcE1cQby3PRkjsu4-1683480367-0-gaNycGzND3s" + window._cf_chl_opt.cOgUHash);
            cpo.onload = function() {
                history.replaceState(null, null, ogU);
            };
        }
        document.getElementsByTagName('head')[0].appendChild(cpo);
    }());
</script>


</body>
</html>
 - referer_url: https://www.marinetraffic.com/en/data/?asset_type=vessels&columns=time_of_latest_position,flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,mmsi,ship_type,show_on_live_map,area,area_local,lat_of_latest_position,lon_of_latest_position,fleet,status,eni,speed,course,draught,navigational_status,year_of_build,length,width,dwt,current_port_unlocode,current_port_country,callsign&mmsi|eq|mmsi=211281610 - request_url: https://www.marinetraffic.com/en/reports?asset_type=vessels&columns=time_of_latest_position,flag,shipname,photo,recognized_next_port,reported_eta,reported_destination,current_port,imo,mmsi,ship_type,show_on_live_map,area,area_local,lat_of_latest_position,lon_of_latest_position,fleet,status,eni,speed,course,draught,navigational_status,year_of_build,length,width,dwt,current_port_unlocode,current_port_country,callsign&mmsi|eq|mmsi=211281610

What's wrong?
Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.