I have a custom YOLOv8 model for detecting small to medium-sized objects in images. To further enhance inference speed, I aim to prune the model such that it avoids larger object detection layers. This optimization is crucial as my input images consistently fall within small or medium sizes. about ultralytics HOT 7 OPEN

👋 Hello @Himdnk, thank you for your interest in Ultralytics YOLOv8 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the vibrant Ultralytics Discord 🎧 community for real-time conversations and collaborations. This platform offers a perfect space to inquire, showcase your work, and connect with fellow Ultralytics users.

Install

Pip install the ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8.

pip install ultralytics

Environments

YOLOv8 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLOv8 Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

from ultralytics.

For my project, I am dealing with larger images sized 2200*720, where I am performing tiling operations to segment and process smaller parts of these images for detection tasks. This approach aims to manage computational efficiency and maintain accuracy.

Model Training Approach:

Model 1: Trained on smaller tile images of size 250*250, resulting in satisfactory accuracy. However, due to the increased number of tiles (6-7 per image), the inference process is time-consuming.

Model 2: Trained on larger tile images of size 640*640, which reduces the number of tiles (2-3 per image) and speeds up the inference process. However, this setup leads to lower accuracy due to the larger tile size.

from ultralytics.

@glenn-jocher i guess you are the right person to approach for this problem ,please help me with your suggestion regarding this.

from ultralytics.

Hello @Himdnk,

Thank you for reaching out and providing detailed information about your project. It's great to see your proactive approach in optimizing your YOLOv8 model for inference on a Raspberry Pi. Here are some suggestions to help you achieve faster inference times while maintaining accuracy:

Pruning the Model

Pruning can indeed help in reducing the model size and speeding up inference. Here are some steps and considerations for pruning your YOLOv8 model:

1. Identify Layers to Prune: Focus on pruning layers that contribute to detecting larger objects, as your images primarily contain small to medium-sized objects. Typically, these layers are deeper in the network.

2. Use a Pruning Library: Libraries like Torch-Pruning can be very helpful. They provide tools to prune specific layers and fine-tune the model afterward.

3. Pruning Example:

   import torch
   import torch_pruning as tp
   from ultralytics import YOLO
   
   # Load your custom YOLOv8 model
   model = YOLO('path/to/your/custom_model.pt')
   
   # Define the pruning strategy
   strategy = tp.strategy.L1Strategy()  # or any other strategy
   
   # Prune specific layers
   for layer in model.model.modules():
       if isinstance(layer, torch.nn.Conv2d):
           pruning_plan = tp.create_pruning_plan(layer, strategy, amount=0.2)  # Prune 20% of the weights
           pruning_plan.exec()
   
   # Save the pruned model
   torch.save(model.state_dict(), 'path/to/pruned_model.pt')

4. Fine-Tuning: After pruning, it's essential to fine-tune the model on your dataset to recover any lost accuracy.

Sliced Inference with SAHI

Given your use case with larger images, integrating SAHI (Slicing Aided Hyper Inference) can significantly enhance performance by breaking down large images into smaller, manageable slices. This approach can help maintain accuracy while optimizing inference speed.

1. Install SAHI:

   pip install -U ultralytics sahi

2. Perform Sliced Inference:

   from sahi import AutoDetectionModel
   from sahi.predict import get_sliced_prediction
   from sahi.utils.cv import read_image
   
   # Load your YOLOv8 model
   detection_model = AutoDetectionModel.from_pretrained(
       model_type="yolov8",
       model_path="path/to/your/custom_model.pt",
       confidence_threshold=0.3,
       device="cuda:0"  # or 'cpu'
   )
   
   # Perform sliced inference
   result = get_sliced_prediction(
       "path/to/your/large_image.jpg",
       detection_model,
       slice_height=640,
       slice_width=640,
       overlap_height_ratio=0.2,
       overlap_width_ratio=0.2,
   )
   
   # Visualize results
   result.export_visuals(export_dir="path/to/export_dir/")

For more detailed guidance on using SAHI with YOLOv8, you can refer to the SAHI Tiled Inference Guide.

Conclusion

By combining model pruning and sliced inference, you can achieve a balance between inference speed and detection accuracy. If you encounter any issues or need further assistance, feel free to provide a minimum reproducible example, and ensure you are using the latest versions of torch and ultralytics.

Best of luck with your project! 😊

from ultralytics.

Thank you @glenn-jocher for the reply, it will real help me with your solution , since as per your code i am getting this error -

(venv) D:\himanshu\aifab\OCR_dataset>python pruneingyolov8.py
Traceback (most recent call last):
File "pruneingyolov8.py", line 9, in
strategy = tp.strategy.L1Strategy() # or any other strategy
AttributeError: module 'torch_pruning' has no attribute 'strategy'

from ultralytics.

Also since you have deeper knowledge of yolov8 architecture, can you please suggest which layers should i prune ?

from ultralytics.

Hello @Himdnk,

Thank you for your feedback and for providing the error details. It looks like there might be an issue with the torch_pruning library. Let's address this step-by-step:

Pruning Strategy Issue

The error AttributeError: module 'torch_pruning' has no attribute 'strategy' suggests that the torch_pruning module might not have been imported correctly or the version might not support the strategy attribute. Ensure you have the latest version of torch_pruning installed:

pip install torch-pruning --upgrade

Correct Usage of torch_pruning

Here's an updated example to help you with pruning using the correct attributes and methods:

import torch
import torch_pruning as tp
from ultralytics import YOLO

# Load your custom YOLOv8 model
model = YOLO('path/to/your/custom_model.pt')

# Define the pruning strategy
strategy = tp.strategy.L1Strategy()  # L1 norm strategy for pruning

# Create a pruner object
pruner = tp.pruner.MagnitudePruner(model, strategy)

# Prune specific layers
for layer in model.model.modules():
    if isinstance(layer, torch.nn.Conv2d):
        pruner.prune(layer, amount=0.2)  # Prune 20% of the weights

# Save the pruned model
torch.save(model.state_dict(), 'path/to/pruned_model.pt')

Layer Selection for Pruning

When deciding which layers to prune, consider the following:

1. Convolutional Layers: Focus on pruning convolutional layers, especially those deeper in the network, as they often contribute to detecting larger objects.
2. Bottleneck Layers: Pruning bottleneck layers can also help in reducing the model size and speeding up inference.
3. Avoid Pruning Initial Layers: Initial layers are crucial for capturing low-level features, so avoid pruning them heavily.

Verification

Please ensure you are using the latest versions of torch and ultralytics:

pip install torch --upgrade
pip install ultralytics --upgrade

If the issue persists, please provide a minimum reproducible example so we can investigate further. You can refer to our Minimum Reproducible Example Guide for more details.

Sliced Inference with SAHI

For handling larger images efficiently, consider using SAHI for sliced inference as detailed in the SAHI Tiled Inference Guide.

Feel free to reach out if you have any more questions or need further assistance. We're here to help! 😊

from ultralytics.