Truncation of generated results about opera HOT 3 CLOSED

Thanks for your interest!

When early truncation appears, it is recommended to reduce penalty_weights like penalty_weights=0.1, or reduce length_penalty in generate args. These will help to avoid early truncation in beam search.

from opera.

Thank you for your early reply!
I have tried to modify the penalty_weights and length_penalty parameters, and the part of the code that calls the generate function is shown below:

out = model.generate(
                    {"image": norm(image), "prompt": qu},
                    use_nucleus_sampling=args.sample,
                    num_beams=5,
                    max_new_tokens=4096,
                    output_attentions=True,
                    length_penalty=0.1,
                    opera_decoding=True,
                    scale_factor=50,
                    threshold=15,
                    num_attn_candidates=5,
                    penalty_weights=0.1,
                )

but the re-inference still results in 1/5 truncations.

from opera.

Maybe we should also reduce the scale_factor, like:

out = model.generate(
                    {"image": norm(image), "prompt": qu},
                    use_nucleus_sampling=args.sample,
                    num_beams=5,
                    max_new_tokens=4096,
                    output_attentions=True,
                    length_penalty=0.1,
                    opera_decoding=True,
                    scale_factor=20,
                    threshold=15,
                    num_attn_candidates=5,
                    penalty_weights=0.1,
                )

Hope it works well :)

from opera.