A few questions about the structure design of ESPNet about espnet HOT 1 CLOSED

Thanks for your interest in our work.

1. Architecture in Figure 4 of the paper is generic. First skip connection between encoder and decoder makes sense if C is different than number of classes in the dataset. For cityscapes, number of classes is close to 19 so it does not make sense to add it. We experimentally found that this connection is irrelevant for the Cityscapes dataset.

2. Usually, semantic segmentation architectures use a pretrained encoder such as ResNet which is trained on the ImageNet. We did not use a pretrained encoder, that is why we need to adhere to two stage strategy. Also, we found that training end-to-end models from scratch are less accurate than 2 stage accuracy.

3. You could upsample the feature map and compute the loss at original image resolution instead of 1/8th of the image.

4. In general, ignoring the background is not a good idea, especially when considering the generalizability. I would like to emphasize that the aim of ESPNet is to build a network that is efficient with reasonable accuracy.

from espnet.