Hi, I was looking over the code, and it seems on a null answer you remove "50 % samples":

However, I was wondering why you did this; I think the original paper that BigBird based their processing off (here) downsampled 50x less, not 50% less.

Hi,

thanks for your good work!

I'm testing your model (BigBirdRobertaQA) in a retriever-reader architecture. I retrieve the top 5 paragraphs (they are a couple of sentences each), merge them (joining them with a space between) and feed them as the context to the model.

1. Is that the best way to approach this? If so, what is the recommended max length for the tokenizer?
2. I expect not, but is there a way to influence what kind of answer will you get? (short, long, yes/no)

Hi Vasudevgupta,

I am interested in predicting category (long, short) answers, but as per your suggestion from the hugging face website (https://huggingface.co/vasudevgupta/bigbird-roberta-natural-questions), I am trying to use BigBirdForNaturalQuestions, but not able to find this class/model anywhere. Could you please help me on that where to get BigBirdForNaturalQuestions and how to get the long and short answers from that model?

Regards,
Kishore

Hi Gupta,

Thank you for your hard work.
I want to use Bigbird for my own dataset in another language. But google/bigbird-roberta-base checkpoint was trained in English language. So can I load another checkpoint (for example roberta-base) to your model?

Hi @vasudevgupta7,

First of all nice job with merging BigBird to huggingface transformers!
I wanted to try training my own BigBird Pegasus model, and I saw that you traiend multiple of these. Browsing both huggingface and your repositories, I did not meet an example of these, and wanted to ask if you have a simple example that you want to share that i can get started on?

Hi there,

it seems like the huggingface online demo does not work at the moment, even with the default example, could u help to fix it?

thanks

Hi, I am getting import error in params for the Category_Mapping. Could you please help me on that?

import torch
import numpy as np
from bigbird.src.train_nq_torch import BigBirdForNaturalQuestions
from params import CATEGORY_MAPPING
from transformers import BigBirdTokenizer

CATEGORY_MAPPING = {v: k for k, v in CATEGORY_MAPPING.items()}
CATEGORY_MAPPING

ImportError Traceback (most recent call last)
in ()
2 import numpy as np
3 from bigbird.src.train_nq_torch import BigBirdForNaturalQuestions
----> 4 from params import CATEGORY_MAPPING
5 from transformers import BigBirdTokenizer
6

ImportError: cannot import name 'CATEGORY_MAPPING' from 'params' (/usr/local/lib/python3.7/dist-packages/params/init.py)