Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Yolo multilabel #17809

Draft
wants to merge 7 commits into
base: main
Choose a base branch
from
Draft

Yolo multilabel #17809

wants to merge 7 commits into from
+179 −48

Conversation

Danil328
Copy link

@Danil328 Danil328 commented Nov 26, 2024 •
edited by UltralyticsAssistant
Loading

YOLO fair multilabel.

NOT FOR MERGE

The idea was as follows:

  1. The dataset format unchanged (label, cx, cy, w, h):
0 0.43335 0.154441 0.114746 0.308883
4 0.43335 0.154441 0.114746 0.308883
10 0.43335 0.154441 0.114746 0.308883
17 0.43335 0.154441 0.114746 0.308883
18 0.43335 0.154441 0.114746 0.308883
0 0.393555 0.580754 0.183594 0.26245
4 0.393555 0.580754 0.183594 0.26245
15 0.393555 0.580754 0.183594 0.26245
17 0.393555 0.580754 0.183594 0.26245
18 0.393555 0.580754 0.183594 0.26245
  1. When loading the dataset, we find unique boxes and aggregate the labels into OHE format.
  2. Change the loss, tal, metrics, plotting for multilabel support.

Training starts, metrics converge, the only thing is that the model prediction after NSM from the multi-label format is turned back into the multi-class format.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

This PR introduces support for multi-label object detection in YOLO, enhancing the flexibility and accuracy of the model in handling datasets where objects may belong to multiple classes simultaneously.

📊 Key Changes

  • New Multi-Label Setting:
    • Added a multilabel option in the configuration file (default.yaml) to enable multi-label classification.
  • New Dataset Class:
    • Introduced YOLOMultilabelDataset to handle multi-label datasets effectively.
  • Improvements to Data Processing:
    • Updated data loading and augmentation to allow multi-label handling.
  • Metrics and Loss Adjustments:
    • Revised metrics and loss functions to appropriately handle multi-label targets, including precision-recall calculations and heatmap generation.
  • Visualization Updates:
    • Adjusted plots and visualizations to reflect multi-label annotations correctly.

🎯 Purpose & Impact

  • Purpose:
    • Enable the YOLO model to support multi-label datasets, addressing a significant limitation in scenarios where objects can belong to multiple categories.
  • Impact:
    • Greater versatility for real-world applications like medical imaging or retail, where multi-label classifications are common.
    • Improved model flexibility without disrupting the functionalities of single-label or traditional YOLO tasks.
    • Users can now seamlessly train on multi-label datasets, unlocking new use cases and expanding YOLO's adaptability. 🛠️💡

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 26, 2024 •
edited
Loading

All Contributors have signed the CLA. ✅
Posted by the CLA Assistant Lite bot.

@UltralyticsAssistant UltralyticsAssistant added detect Object Detection issues, PR's enhancement New feature or request labels Nov 26, 2024
@UltralyticsAssistant
Copy link
Member

👋 Hello @Danil328, thank you for submitting a ultralytics/ultralytics 🚀 PR! Here's a quick checklist to help ensure the seamless integration of your contribution:

  • ✅ Clearly Define the PR Purpose: In your PR description, please ensure the purpose is clearly outlined and, if possible, link to any relevant issues or feature requests. It’s helpful to summarize how this enhancement benefits the project.
  • ✅ Synchronize with Source: Ensure your branch is up-to-date with the latest main branch of the Ultralytics repository. If needed, update your branch by clicking 'Update branch' or running git pull and git merge main locally.
  • ✅ Pass CI/CD Tests: Confirm that all Continuous Integration (CI) checks pass successfully. If any fail, please address the accompanying errors.
  • ✅ Update Documentation: Update any relevant documentation associated with the changes, such as YAML configuration descriptions, new arguments, or method usage.
  • ✅ Add Tests & Validation: If applicable, add tests to validate your multilabel functionality and include outputs to demonstrate its effectiveness.
  • ✅ Sign the CLA: Ensure you have signed our Contributor License Agreement (CLA). If you haven’t already, please comment "I have read the CLA Document and I sign the CLA" in this PR thread.
  • ✅ Minimize Changes: While adding significant new functionality like multilabel handling is impressive 🎉, strive to keep changes concise and focused on your feature to reduce the overhead during review.

For further guidance, please check out the Ultralytics Contributing Guide.

Please note, this is an automated response 🛠️. An Ultralytics engineer will carefully review your PR soon. If you have additional questions or require clarification, feel free to comment below. Thank you for contributing to Ultralytics! 🚀

@Danil328 Danil328 mentioned this pull request Nov 26, 2024
Closed
1 task
@Danil328
Copy link
Author

I have read the CLA Document and I sign the CLA

Copy link

YOLO fair multilabel.

NOT FOR MERGE

The idea was as follows:

  1. The dataset format unchanged (label, cx, cy, w, h):
0 0.43335 0.154441 0.114746 0.308883
4 0.43335 0.154441 0.114746 0.308883
10 0.43335 0.154441 0.114746 0.308883
17 0.43335 0.154441 0.114746 0.308883
18 0.43335 0.154441 0.114746 0.308883
0 0.393555 0.580754 0.183594 0.26245
4 0.393555 0.580754 0.183594 0.26245
15 0.393555 0.580754 0.183594 0.26245
17 0.393555 0.580754 0.183594 0.26245
18 0.393555 0.580754 0.183594 0.26245
  1. When loading the dataset, we find unique boxes and aggregate the labels into OHE format.
  2. Change the loss, tal, metrics, plotting for multilabel support.

Training starts, metrics converge, the only thing is that the model prediction after NSM from the multi-label format is turned back into the multi-class format.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

This PR introduces support for multi-label object detection in YOLO, enhancing the flexibility and accuracy of the model in handling datasets where objects may belong to multiple classes simultaneously.

📊 Key Changes

  • New Multi-Label Setting:

    • Added a multilabel option in the configuration file (default.yaml) to enable multi-label classification.

  • New Dataset Class:

    • Introduced YOLOMultilabelDataset to handle multi-label datasets effectively.

  • Improvements to Data Processing:

    • Updated data loading and augmentation to allow multi-label handling.

  • Metrics and Loss Adjustments:

    • Revised metrics and loss functions to appropriately handle multi-label targets, including precision-recall calculations and heatmap generation.

  • Visualization Updates:

    • Adjusted plots and visualizations to reflect multi-label annotations correctly.

🎯 Purpose & Impact

  • Purpose:

    • Enable the YOLO model to support multi-label datasets, addressing a significant limitation in scenarios where objects can belong to multiple categories.

  • Impact:

    • Greater versatility for real-world applications like medical imaging or retail, where multi-label classifications are common.
    • Improved model flexibility without disrupting the functionalities of single-label or traditional YOLO tasks.
    • Users can now seamlessly train on multi-label datasets, unlocking new use cases and expanding YOLO's adaptability. 🛠️💡

Is there any progress on this work? Can it be used? grateful

@glenn-jocher
Copy link
Member

Thank you for your interest in this work! As the PR is marked NOT FOR MERGE, it is currently not part of the official Ultralytics repository and cannot be directly used. However, you can monitor any updates or discussions on the PR in the repository. If multi-label support is officially integrated in the future, it will be announced in the release notes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
detect Object Detection issues, PR's enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@Danil328 @UltralyticsAssistant @glenn-jocher @ambitious-octopus