Giter VIP home page Giter VIP logo

4bytedirectory-data's Introduction

4byte.directory data

A full collection of signatures scraped from https://www.4byte.directory/

Exports

All signatures can be found in exports/signatures.json

Due to signature collisions, the signatures are stored as an array.

Usage

usage: export [-h|--help] [-p|--page <integer>] [-t|--threads <integer>]
              [-r|--retries <integer>] [-m|--missing] [-f|--failed-only]
              [-c|--counts]

              Exports function signature data from 4byte.directory

Arguments:

  -h  --help         Print help information
  -p  --page         The page to start scraping from. Default: 1
  -t  --threads      The number of threads. Default: 10
  -r  --retries      The number of times a page should be retried before it is
                     considered failed. Default: 25
  -m  --missing      Checks the completed pages array and looks for any pages
                     that may be missing and fetches them
  -f  --failed-only  If set only pages that have previously failed will be
                     processed
  -c  --counts       Counts the number of scraped pages, failed pages and
                     signatures

Standard scrape

To scrape all the pages starting from the first page

export -p 1

Failed pages only

Due to the instability of the 4byte.directory API, if a page could not be fetched after the retry value specified it is considered failed.

To retry all previously failed pages:

export --failed-only

Missing pages only

Check if any pages are missing from the result set and rescrape them if needed.

export --missing

View result totals

export --counts

Result totals as of 2022-08-17:

Completed pages: 8115, Failed pages: 0, Total Signatures: 811193

TODO

  • Add --latest arg to fetch the most recently updated signatures

4bytedirectory-data's People

Contributors

mt-code avatar

Stargazers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.