how to measure the performance. about affordance-net HOT 15 OPEN

from affordance-net.

from affordance-net.

You just need to merge all the outputted affordance maps into one map (please see the paper for details). Then use the Weighted F-measure code from here to evaluate the results with the groundtruths.

from affordance-net.

HI, what does mean by"merge all the outputted affordance maps into one map"? How to calculate one kind of affordance like 'grasp'? There are more than one affordances on one pic? How to calculate F measure separately?

from affordance-net.

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

from affordance-net.

HI， Nqanh~ about the Fmeasure for each affordance, I still have questions. In the Fmeasure code , the type of GT is logical, which means the background and the foreground. But in foreground, we have different objects and affordances. Do I have to seperate all the affordances and make new GTs?

from affordance-net.

We can save all the affordances in 1 map. Then, for each affordance, we select its id and the background --> it becomes logical when compares with the GT.

from affordance-net.

Well,ok! Thank you for your answer and patience! But I am still a little confused. Would you plz share this part code,like how to save in 1 map and select id?

from affordance-net.

Please see #19
(also please use the search function, most of the problems we have already be answered. Thanks!)

from affordance-net.

Wow~ Thank you!!

from affordance-net.

The network outputs each affordance map for each detected object (as in the demo). Since the image may has many objects --> we need to merge them into "one predicted" image. Then the F measure will calculate the accuracy for each affordance (on this "merged" output).

Hi nqanh,
Please help me by teaching me how to "merge affordance map" into one map. Could you tell me more details about that? For example, we have 3 file .pnp (contains affordance maps), and 3 file .sm, so which extension file (sm or pnp) we will use to merge and how to merge it, please?

from affordance-net.

from affordance-net.

Wow~ Thank you!!

Hi ClaireTun,
I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?

from affordance-net.

Wow~ Thank you!!

Hi ClaireTun,
I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please?
Hi chaundm, did you have solved the problem?

from affordance-net.

Well, I finished the project half a year ago. Some details,  I have already forgot. Next week,I will return to school .I can review the code and try to answer your questions 
…
------------------ Original ------------------ From: chaundm <[email protected]> Date: Fri,Aug 30,2019 2:54 PM To: nqanh/affordance-net <[email protected]> Cc: ClaireTun <[email protected]>, Comment <[email protected]> Subject: Re: [nqanh/affordance-net] how to measure the performance. (#18) Wow~ Thank you!! Hi ClaireTun, I have the same trouble as you. Could you tell me how to "merge all of affordances into a map"? And which extension file is used for F_measure code (sm or png). I found that that the ground truth is sm file with black and white, while the prediction in demo_img.py code outputs the png file with each predicted masks with affordance colour. How to combine these files to run the F_measure code in matlab, please? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Hi ClaireTun, my ground truth and predicted map are png files with each masks with affordance color. How to use the weighted F_measure?

from affordance-net.