Use generator as translator? about ctranslate2 HOT 3 CLOSED

Opened in #1665

from ctranslate2.

I think my question was too broad because that issue doesn't seem related to my issue. For my case, I have this working:

import ctranslate2
import sentencepiece as spm
translator = ctranslate2.Translator("../models/sugoi-v4-ja-en-ct2", device="cuda")
spsource = spm.SentencePieceProcessor("../models/sugoi-v4-ja-en-ct2/spm.ja.nopretok.model")
sptarget = spm.SentencePieceProcessor("../models/sugoi-v4-ja-en-ct2/spm.en.nopretok.model")
input_text = "日本語上手"
input_tokens = spsource.encode(input_text, out_type=str)
results = translator.translate_batch([input_tokens])
detokenized=""
for i in range(len(results)):
    detokenized+=sptarget.decode(results[i].hypotheses[0])
print(detokenized)

But I want to know if it's possible to do this with ctranslate2.Generator instead.

Does ctranslate2.Translator and ctranslate2.Generator do the same thing in the backend? Is it possible pass a prompt into ctranslate2.Generator in such a way which will return the same result as ctranslate2.Translator?

from ctranslate2.

Sorry for the misunderstanding. I am not sure I can understand your purpose. As I understand, you want to run a Generator to do a translation as Translator? Actually, we don't support the model like that. Then, the backends of Generator and Translator are the same but the architecture behind in Translator is Encoder-Decoder, otherwise the architecture behind Generator is Decoder only, so you can't use the actual model to run with Generator.

from ctranslate2.