Request for mAP of different scale about ultralytics HOT 8 OPEN

👋 Hello @ZYJIQVV, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

from ultralytics.

@ZYJIQVV hello,

Thank you for your feature request and for using YOLOv8 in your project! We appreciate your suggestion to include the capability to calculate mAP for small, medium, and large objects in the OBB detection model, similar to the HBB detection model.

To proceed effectively, could you please provide a minimum reproducible code example that demonstrates your current setup and how you are calculating mAP with the HBB detection model? This will help us understand your requirements better and ensure that we can reproduce the scenario accurately. You can refer to our guide on creating a minimum reproducible example here: Minimum Reproducible Example.

Additionally, please ensure that you are using the latest versions of torch and ultralytics. If not, kindly upgrade your packages and try again to see if the issue persists.

We look forward to your code example and any additional details you can provide. Your contribution to submitting a PR is highly appreciated, and we are here to assist you through the process.

from ultralytics.

@glenn-jocher
Thanks for replying!

I noticed that I mistakenly thought there was the feature in the original YOLOv8 HBB detection model that mAP for small, medium and large objects can be calculated. Well, I just save the predictions in json format and transform it into COCO format json file, then use the COCOapi tools to calculate mAPs, mAPm and mAPl when using HBB detection model.
That is, I have the ground truth and prediction file, gt.json and pd.json respectively both in COCO format. Then I use the classes COCOeval and COCO in pycocotools to create an evaluator: evaluator = COCOeval(COCO(pd.json), COCO(gt.json)). Then call the evaluation function in COCOeval to calculate the metrics, which includes mAP50, mAP50:95 for all classes and all images, and mAP for small, medium and large objects of all classes.

But I'd like to ask, if the same feature as COCOapi could be implemented and assembled into YOLOv8 intrinsicly in both HBB and OBB detection model, and it can be enabled during training, validating and predictions.

Could you please take this request into consideration? Looking forward to your kind reply!

from ultralytics.

Hello @ZYJIQVV,

Thank you for the detailed explanation and for clarifying your use case! It's great to hear that you are leveraging the COCO API to calculate mAP for small, medium, and large objects.

Integrating this feature directly into YOLOv8 for both HBB and OBB detection models is indeed a valuable suggestion. This would streamline the process and make it more convenient for users to obtain these metrics without additional steps.

We will take your request into consideration for future updates. In the meantime, your current approach using the COCO API is a robust method to achieve your desired metrics. If you have any further questions or need assistance with your current setup, feel free to ask.

Thank you for your continued support and valuable feedback! 😊

from ultralytics.