anuragxel / salt Goto Github PK
View Code? Open in Web Editor NEWSegment Anything Labelling Tool
License: MIT License
Segment Anything Labelling Tool
License: MIT License
thanks for the amazing work, may i know where is the code for converting the mask into segmented support format coco or yolo format data? i m trying to load my custom detector & want to detect objects & get its coordinates. thanks
When I use the generate_onnx.py to generate the onnx file of SAM, it would return the exception:
Traceback (most recent call last):
File "generate_onnx.py", line 177, in
run_export(
File "generate_onnx.py", line 149, in run_export
torch.onnx.export(
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/init.py", line 305, in export
return utils.export(model, args, f, export_params, verbose, training,
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/utils.py", line 118, in export
_export(model, args, f, export_params, verbose, training, input_names, output_names,
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/utils.py", line 719, in _export
_model_to_graph(model, args, verbose, input_names,
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/utils.py", line 503, in _model_to_graph
graph = _optimize_graph(graph, operator_export_type,
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/utils.py", line 232, in _optimize_graph
graph = torch._C._jit_pass_onnx(graph, operator_export_type)
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/init.py", line 354, in _run_symbolic_function
return utils._run_symbolic_function(*args, **kwargs)
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/utils.py", line 1061, in _run_symbolic_function
return symbolic_fn(g, *inputs, **attrs)
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/symbolic_opset13.py", line 267, in repeat_interleave
return torch.onnx.symbolic_opset9.repeat_interleave(g, self, repeats, final_dim)
File "/HOME/scz0bck/.conda/envs/seg-tool/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py", line 2152, in repeat_interleave
for idx, r_split in enumerate(r_splits):
TypeError: 'torch._C.Value' object is not iterable
(Occurred when translating repeat_interleave).
I don‘t konw how to solve the exception. May I beg a favour?Thanks for your help.
If possible then please integrate this repo with labelstudio because there are lots of other things that they have got it right.
python segment_anything_annotator.py --onnx-models-path models --dataset-path seg_imgs --categories dog
Traceback (most recent call last):
File "segment_anything_annotator.py", line 26, in <module>
editor = Editor(
File "C:\pyprojects\salt\salt\editor.py", line 45, in __init__
self.dataset_explorer = DatasetExplorer(
File "C:\pyprojects\salt\salt\dataset_explorer.py", line 126, in __init__
max(self.coco_json["annotations"], key=lambda x: x["id"])["id"] + 1
ValueError: max() arg is an empty sequence
When I view the image, there is a pink line generated from the top left corner of the image. I tried to investigate the issue by printing variable "sc" from the file dataset_explorer.py, and I found a random value "6.91669498947497e-310, 6.91669498947497e-310" generated at the beginning. To overcome this issue, I added this line "sc = [x for x in sc if x > 0.1]" in the file dataset_explorer.py.
Please refer to the attached screenshots for better understanding.
The pink line should not be generated from the top left corner of the image.
Traceback (most recent call last):
File "……/segment_anything_annotator.py", line 27, in
editor = Editor(
File "……\salt\editor.py", line 57, in init
) = self.dataset_explorer.get_image_data(self.image_id)
File "……\salt\dataset_explorer.py", line 144, in get_image_data
image_name = self.coco_json["images"][image_id]["file_name"]
IndexError: list index out of range
Hi!
when I press the middle mouse button (usually by accident) I get this error:
QLayout::addChildLayout: layout "" already has a parent
Traceback (most recent call last):
File "/home/maciej/PycharmProjects/salt/salt/interface.py", line 97, in mousePressEvent
self.editor.add_click([int(x), int(y)], label, selected_annotations)
UnboundLocalError: local variable 'label' referenced before assignment
Aborted (core dumped)
I will try to add fix of these bug later
Issue1: TypeError: Argument 'bb' has incorrect type (expected numpy.ndarray, got list).
Issue2: ValueError: list.remove(x): x not in list
This is actually crashing the py page everytime
Hi, I was looking at this tool and was wondering if (and how) it would be possible to use my own finetuned SAM model instead of the base one?
Ideally, I would like to draw the bounding box and let my model predict a segmented mask.
Any insight or help would be highly appreciated.
Package libffi conflicts for:
certifi=2022.12.7 -> python[version='>=3.7,<3.8.0a0'] -> libffi[version='>=3.2.1,<3.3a0|>=3.3,<3.4.0a0']
libffi=3.4.2
wheel=0.38.4 -> python[version='>=3.10,<3.11.0a0'] -> libffi[version='>=3.2.1,<3.3a0|>=3.3,<3.4.0a0']
setuptools=65.6.3 -> python[version='>=3.7,<3.8.0a0'] -> libffi[version='>=3.2.1,<3.3a0|>=3.3,<3.4.0a0']
pip=23.0.1 -> python[version='>=3.9,<3.10.0a0'] -> libffi[version='>=3.2.1,<3.3a0|>=3.3,<3.4.0a0']
Add ability to provide box input along with point input (with nice looking mousedown animation). Though I'm not sure if it's needed to be honest, point input works really well in general.
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running Reshape node. Name:'Reshape_149' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:41 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, onnxruntime::TensorShapeVector&, bool) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{1,7,256}, requested shape:{1,10,8,32}
i set 5 classes in it ,but i dont know how to choose these class. it seems no thing happend whether click the class button on the right of the windows
Sorry for the ignorance, can I run this remotely? This is, using a sever - client paradigm or similarly?
i took 5 images in folder,
i clicked on object its showing mask on it, then i clicked on save button, reted same process for all images & genrated annotations.json is not correct
{"info": {"description": "SAM Dataset", "url": "", "version": "1.0", "year": 2023, "contributor": "Sam", "date_created": "2021/07/01"}, "images": [{"id": 0, "file_name": "images\\Cigaretee_i_Weapon_Pragati137.jpg", "width": 1000, "height": 1500}, {"id": 1, "file_name": "images\\Cigaretee_i_Weapon_Pragati138.jpg", "width": 1600, "height": 2400}, {"id": 2, "file_name": "images\\Cigaretee_i_Weapon_Pragati139.jpg", "width": 1600, "height": 1067}, {"id": 3, "file_name": "images\\Cigaretee_i_Weapon_Pragati140.jpg", "width": 1600, "height": 2400}, {"id": 4, "file_name": "images\\Cigaretee_i_Weapon_Pragati141.jpg", "width": 1600, "height": 2400}], "annotations": [{"id": 0, "image_id": 0, "category_id": 0, "bbox": [468.0, 675.0, 53.0, 197.0], "area": 10441.0, "iscrowd": 0, "segmentation": [[505.0, 754.5, 500.0, 754.5, 498.0, 753.5, 493.5, 748.0, 489.5, 737.0, 488.5, 736.0, 484.5, 726.0, 475.5, 707.0, 473.5, 699.0, 471.5, 696.0, 467.5, 684.0, 467.5, 678.0, 471.0, 674.5, 477.0, 674.5, 480.5, 681.0, 485.5, 686.0, 489.5, 692.0, 492.5, 700.0, 495.5, 703.0, 495.5, 705.0, 498.5, 711.0, 500.5, 713.0, 504.5, 721.0, 506.5, 727.0, 510.5, 734.0, 511.5, 738.0, 513.5, 740.0, 513.5, 747.0, 507.0, 753.5, 505.0, 754.5], [518.0, 871.5, 514.5, 870.0, 516.0, 867.5, 519.0, 867.5, 520.5, 869.0, 518.0, 871.5]]}], "categories": [{"id": 0, "name": "'cigaretee'", "supercategory": "'cigaretee'"}]}
Reference: https://github.com/ChaoningZhang/MobileSAM
Our project performs on par with the original SAM and keeps exactly the same pipeline as the original SAM except for a change on the image encode, therefore, it is easy to Integrate into any project.
MobileSAM is around 60 times smaller and around 50 times faster than original SAM, and it is around 7 times smaller and around 5 times faster than the concurrent FastSAM. The comparison of the whole pipeline is summarzed as follows:
Best Wishes,
Qiao
Thanks for your nice work!
I don't have any image that it's shape is [1080,1920],therefor i get this erro when i call segment_anything_annotator.py :
Traceback (most recent call last):
File "D:\Users\26655\Desktop\CAT\library\others\SAM\segment_anything_annotator.py", line 26, in <module>
editor = Editor(
File "D:\Users\26655\Desktop\CAT\library\others\SAM\salt\editor.py", line 58, in __init__
self.onnx_helper = OnnxModels(onnx_models_path)
File "D:\Users\26655\Desktop\CAT\library\others\SAM\salt\onnx_model.py", line 16, in __init__
self.set_image_resolution(image_width, image_height)
File "D:\Users\26655\Desktop\CAT\library\others\SAM\salt\onnx_model.py", line 29, in set_image_resolution
self.__init_model(onnx_model_path)
File "D:\Users\26655\Desktop\CAT\library\others\SAM\salt\onnx_model.py", line 19, in __init_model
self.ort_session = onnxruntime.InferenceSession(
File "C:\Users\26655\.conda\envs\pt2\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 360, in __init__
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "C:\Users\26655\.conda\envs\pt2\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 397, in _create_inference_session
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from model data\sam_onnx.1080_1920.onnx failed:Load model model data\sam_onnx.1080_1920.onnx failed. File doesn't exist
It's clear that it happens when class Editor initializes
Thanks for your tools. I am trying to use this tool in my equipment. And i meet some problems during run the segment_anything_annotator.py.
I have a server with GPU, which is a Linux system without GUI. It's totally terminal system. I use my Macbook to link the server with SSH tool. And I have created embedding and onnx model in server.
When I want to annotate my images, the PyQt doesn't work well and export error:Could not connect to any X display and QLibraryPrivate::unload succeeded on "/usr/lib/x86_64-linux-gnu/qt5/plugins/platforms/libqxcb.so". I have tried to fix it by reinstall pyqt and install PYQT Integration in my VSCode. But it is not useful for me.
I would like to ask two questions:
I am looking forward to your answer
Error: Request failed with status code 500. "datumaro.components.errors.AnnotationImportError: Failed to import item (0, 'default') annotation: Polygon has invalid value count 4. Expected at least 3 (x, y) pairs.\n"
Error Message:
...//salt/salt/display_utils.py", line 33, in __convert_ann_to_mask
rles = coco_mask.frPyObjects(poly, height, width)
File "pycocotools/_mask.pyx", line 308, in pycocotools._mask.frPyObjects
Exception: input type is not supported.
.../salt/salt/display_utils.py", line 33, in __convert_ann_to_mask
rles = coco_mask.frPyObjects(poly, height, width)
File "pycocotools/_mask.pyx", line 294, in pycocotools._mask.frPyObjects
TypeError: Argument 'bb' has incorrect type (expected numpy.ndarray, got list)
My image is pretty obscure so sometimes SAM segmentation include very small mask which has poly
of 4 elements.
This seems to coincide with this issue.
cocodataset/cocoapi#139
Maybe there need to be some changes that make len(poly[i]) > 4
@anuragxel @Daa98
I faced this issue when running the app in windows system. The function was not updated with the click event.
def get_side_panel(self):
panel = QWidget()
panel_layout = QVBoxLayout(panel)
categories = self.editor.get_categories()
for category in categories:
label = QPushButton(category)
label.clicked.connect(lambda: self.editor.select_category(category))
panel_layout.addWidget(label)
return panel
the lambda function inside the loop is capturing the reference to the same category variable every time, and when the lambda function is executed later, it uses the last value that category had in the loop.
To fix this, create a separate function that takes the category as an argument and returns a lambda function that calls self.editor.select_category(category). Now lambda function will capture the value of category at the time it was created.
def get_side_panel(self):
panel = QWidget()
panel_layout = QVBoxLayout(panel)
categories = self.editor.get_categories()
for category in categories:
label = QPushButton(category)
label.clicked.connect(self.create_category_handler(category))
panel_layout.addWidget(label)
return panel
def create_category_handler(self, category):
return lambda: self.editor.select_category(category)
I want to change the categories from 'cat' and 'dog' to other labels, but as I want to change the categories with the command:
python segment_anything_annotator.py --onnx-model-path sam_onnx.onnx --dataset-path ../shot --categories i,j,k,l,o
It turns out that the categories in the interface are still cat and dog, and I cannot change the categories whatever I type in.
So I wonder if someone who once faced the same problem can give me an explanation and help me slove this problem, thank you so much!!! ^-^
Need to think of a UI (how to select the annotation) and then implement corresponding deletion functionality.
Thank you for sharing the awesome work!
I think it is necessary to adjust the transparency of the current predicted mask during the annotation process, so that it is easier to check the accuracy of the mask boundaries.
Not much code to adjust,
Line 96 in 5ed8652
def step_up_transparency(self):
self.display = self.image_bgr.copy()
self.du.increase_transparency()
if self.curr_inputs.curr_mask is not None:
self.display = self.du.draw_points(
self.display, self.curr_inputs.input_point, self.curr_inputs.input_label)
self.display = self.du.overlay_mask_on_image(self.display, self.curr_inputs.curr_mask)
if self.show_other_anns:
self.draw_known_annotations()
def step_down_transparency(self):
self.display = self.image_bgr.copy()
self.du.decrease_transparency()
if self.curr_inputs.curr_mask is not None:
self.display = self.du.draw_points(
self.display, self.curr_inputs.input_point, self.curr_inputs.input_label)
self.display = self.du.overlay_mask_on_image(self.display, self.curr_inputs.curr_mask)
if self.show_other_anns:
self.draw_known_annotations()
while annotating, the areas which are excluded as background is still getting enclosed within the annotation image after clicking add. If we annotate this image, https://www.electriduct.com/assets/images/water-filled-barricade-female-angle.jpg, the hollow parts in the barricade gets annotated even if its not annotated or manually exclude using right click.
Traceback (most recent call last):
File "D:\pyworkspaces\SAM\SAM-Tool\salt\interface.py", line 67, in mousePressEvent
self.editor.add_click([int(x), int(y)], label)
UnboundLocalError: local variable 'label' referenced before assignment
Models folder would have all onnx models in a specified format e.g. ".<res>.onnx" and the ONNX caller picks up the "right" model for the given image.
Let's say you annotated an object and forgot to change the category, go into this UI mode, change the radio selection and fix the category of that object with a click.
Segment Anything model doesn't always work. Let users add a custom polygon if they realize that's faster to do.
Thank for your work! It's very useful for me!
Would you add fuction to delete existing labels, thanks!
Hi
Thanks for your quick and great job!
I want to test on 3 images but get the following error when calling segment_anything_annotator.py
any insights?
Traceback (most recent call last): File "segment_anything_annotator.py", line 26, in <module> editor = Editor( File "/mnt/data/polymore/salt/salt/editor.py", line 57, in __init__ ) = self.dataset_explorer.get_image_data(self.image_id) File "/mnt/data/polymore/salt/salt/dataset_explorer.py", line 146, in get_image_data image_name = self.coco_json["images"][image_id]["file_name"] IndexError: list index out of range
Thanks in advance
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.