Can you set both hq_token_only = True and multiask_ Output= False? And can you show your training loss log? did you train with the box prompt as input?

from sam-hq.

This is the training loss as follows:
At the beginning

Finished epoch: 0
Averaged stats: training_loss: 0.4607 (0.6271) loss_dice: 0.3408 (0.4383) loss_mask: 0.1234 (0.1887
Validating...
valid_dataloader len: 59
[ 0/59] eta: 0:01:52 val_boundary_iou_0: 0.3348 (0.3348) val_iou_0: 0.4443 (0.4443) time: 1.899 max mem: 10998
[58/59] eta: 0:00:00 val_boundary_iou_0: 0.3429 (0.3641) val_iou_0: 0.4028 (0.4534) time: 0.702 max mem: 10998
Total time: 0:00:42 (0.7280 s / it)

Averaged stats: val_boundary_iou_0: 0.3429 (0.3641) val_iou_0: 0.4028 (0.4534)
valid_dataloader len: 35
[ 0/35] eta: 0:00:16 val_boundary_iou_1: 0.5491 (0.5491) val_iou_1: 0.6967 (0.6967) time: 0.472 max mem: 10998
[34/35] eta: 0:00:00 val_boundary_iou_1: 0.5697 (0.5639) val_iou_1: 0.7347 (0.7131) time: 0.160 max mem: 10998
Total time: 0:00:06 (0.1767 s / it)

Averaged stats: val_boundary_iou_1: 0.5697 (0.5639) val_iou_1: 0.7347 (0.7131)
valid_dataloader len: 36
[ 0/36] eta: 0:01:05 val_boundary_iou_2: 0.5984 (0.5984) val_iou_2: 0.7327 (0.7327) time: 1.813 max mem: 10998
[35/36] eta: 0:00:00 val_boundary_iou_2: 0.5396 (0.5335) val_iou_2: 0.6921 (0.6936) time: 0.742 max mem: 10998
Total time: 0:00:33 (0.9207 s / it)

Averaged stats: val_boundary_iou_2: 0.5396 (0.5335) val_iou_2: 0.6921 (0.6936)
valid_dataloader len: 63
[ 0/63] eta: 0:00:54 val_boundary_iou_3: 0.2103 (0.2103) val_iou_3: 0.3468 (0.3468) time: 0.864 max mem: 10998
[62/63] eta: 0:00:00 val_boundary_iou_3: 0.2943 (0.2850) val_iou_3: 0.4127 (0.4070) time: 0.302 max mem: 10998
Total time: 0:00:19 (0.3136 s / it)

Averaged stats: val_boundary_iou_3: 0.2943 (0.2850) val_iou_3: 0.4127 (0.4070)
come here save at /home/quchunguang/datasets/sam-hq/output1/epoch_0.pth

At the end

epoch: 99 learning rate: 1.0000000000000006e-12
[ 0/50] eta: 0:01:27 training_loss: 0.2335 (0.2335) loss_dice: 0.1677 (0.1677) loss_mask: 0.065: 1.7565 data: 0.9814 max mem: 11005
[49/50] eta: 0:00:00 training_loss: 0.2519 (0.2657) loss_dice: 0.1849 (0.1967) loss_mask: 0.067: 0.7031 data: 0.0039 max mem: 11005
Total time: 0:00:36 (0.7278 s / it)
Finished epoch: 99
Averaged stats: training_loss: 0.2519 (0.2657) loss_dice: 0.1849 (0.1967) loss_mask: 0.0675 (0.0690
Validating...
valid_dataloader len: 59
[ 0/59] eta: 0:01:51 val_boundary_iou_0: 0.3998 (0.3998) val_iou_0: 0.4573 (0.4573) time: 1.888 max mem: 11005
[58/59] eta: 0:00:00 val_boundary_iou_0: 0.2846 (0.3288) val_iou_0: 0.3507 (0.3955) time: 0.681 max mem: 11005
Total time: 0:00:42 (0.7137 s / it)

Averaged stats: val_boundary_iou_0: 0.2846 (0.3288) val_iou_0: 0.3507 (0.3955)
valid_dataloader len: 35
[ 0/35] eta: 0:00:15 val_boundary_iou_1: 0.4165 (0.4165) val_iou_1: 0.4103 (0.4103) time: 0.455 max mem: 11005
[34/35] eta: 0:00:00 val_boundary_iou_1: 0.4708 (0.4933) val_iou_1: 0.5607 (0.5773) time: 0.158 max mem: 11005
Total time: 0:00:06 (0.1734 s / it)

Averaged stats: val_boundary_iou_1: 0.4708 (0.4933) val_iou_1: 0.5607 (0.5773)
valid_dataloader len: 36
[ 0/36] eta: 0:00:59 val_boundary_iou_2: 0.5588 (0.5588) val_iou_2: 0.6405 (0.6405) time: 1.651 max mem: 11005
[35/36] eta: 0:00:00 val_boundary_iou_2: 0.5378 (0.5127) val_iou_2: 0.6370 (0.6172) time: 0.755 max mem: 11005
Total time: 0:00:32 (0.8922 s / it)

Averaged stats: val_boundary_iou_2: 0.5378 (0.5127) val_iou_2: 0.6370 (0.6172)
valid_dataloader len: 63
[ 0/63] eta: 0:00:52 val_boundary_iou_3: 0.2597 (0.2597) val_iou_3: 0.4398 (0.4398) time: 0.835 max mem: 11005
[62/63] eta: 0:00:00 val_boundary_iou_3: 0.3006 (0.3051) val_iou_3: 0.3893 (0.4143) time: 0.301 max mem: 11005
Total time: 0:00:19 (0.3076 s / it)

Averaged stats: val_boundary_iou_3: 0.3006 (0.3051) val_iou_3: 0.3893 (0.4143)
come here save at /home/quchunguang/datasets/sam-hq/output1/epoch_99.pth

The training loss ranges from 0.46 to 0.23, and the network converges; Or increase epoch to obtain smaller training loss?
My training input did not use box prompt. Just use box prompt for coarse target localization during local testing. According to your prompt, the segmentation effect after modifying the parameters is as follows, but the effect is not very ideal:

from sam-hq.

Then what's the used prompt during your network training? How many training images are there?

from sam-hq.