Combine Multiple Models With BERT Server? about spago HOT 2 CLOSED

Hey @bonedaddy,

thanks for your time experimenting with spaGO!

For now, I implemented the BERT server to work with a single model:

type Server struct {
	model *bert.Model
	port  int
}

The server structure may be modified to support multiple models simultaneously. For example, using a map whose keys are the names of loaded models. Something like:

type Server struct {
	models map[string]*bert.Model
	port  int
}

However, this would complicate the loading phase and especially the routes.

This

http://127.0.0.1:1987/answer

would become something like

http://127.0.0.1:1987/MODEL_NAME/answer

I'm not sure adding this functionality is worth it. What do you think?

from spago.

Ah I see. Is there any reason why it wouldn't be advantageous to run the requests made to the BERT server against multiple models, and in cases like answer requests, the client-side application could then use whatever response has the highest confidence score.

It would allow you to either initiate the BERT server with one model, or more than one model and have pseudocode like this:

var answers []answerStruct
for _, model := range Server::models {
   result := model.askQuestion
   answers = append(answers, result)
}

I'm not sure if this would be useful at all, but it might allow more accurate results to a wider variety of questions and answers, at the cost of lower performance

from spago.