sapkotaruz11 / typeevalpy Goto Github PK

• 📜 Contains 154 code snippets to test and benchmark.
• 🏷 Offers 845 type annotations across a diverse set of Python functionalities.
• 📂 Organized into 18 distinct categories targeting various Python features.
• 🚢 Seamlessly manages the execution of containerized tools.
• 🔄 Efficiently transforms inferred types into a standardized format.
• 📊 Automatically produces meaningful metrics for in-depth assessment and comparison.

Below is a comparison showcasing exact matches across different tools, coupled with top_n predictions for ML-based tools.

_{(Auto-generated based on the the analysis run on 20-10-23 14:51)}

git clone https://github.com/secure-software-engineering/TypeEvalPy.git

docker build -t typeevalpy .

🕒 Takes about 30mins on first run to build Docker containers.

📂 Results will be generated in the results folder within the root directory of the repository. Each results folder will have a timestamp, allowing you to easily track and compare different runs.

docker run \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v ./results:/app/results \
      typeevalpy

🔧 Optionally, run analysis on specific tools:

docker run \
      -v /var/run/docker.sock:/var/run/docker.sock \
      -v ./results:/app/results \
      typeevalpy --runners headergen scalpel

🛠️ Available options: headergen, pyright, scalpel, jedi, hityper, type4py, hityperdl

Thank you for your interest in contributing! To add support for a new tool, please utilize the Docker templates provided in our repository. After implementing and testing your tool, please submit a pull request (PR) with a descriptive message. Our maintainers will review your submission, and merge them.

Give a ⭐️ if this project helped you!