Trouble training temporal cycle consistency model on a new dataset about google-research HOT 9 OPEN

Apologies for the inconvenience.

JSON should be in following format:

{
  "VIDEO_NAME1": {
    "LABELER_ID": {
      "PHASE LABEL 1": [
        START TIME IN MICROSECOND, 
        END TIME IN MICROSECOND
      ], 
     "PHASE LABEL 2": [
        START TIME IN MICROSECOND, 
        END TIME IN MICROSECOND
      ], 
    }
  },
"VIDEO_NAME2": {
    "LABELER_ID": {
      "PHASE LABEL 1": [
        START TIME IN MICROSECOND, 
        END TIME IN MICROSECOND
      ], 
     "PHASE LABEL 2": [
        START TIME IN MICROSECOND, 
        END TIME IN MICROSECOND
      ], 
    }
  },
} 

An example of JSON entry looks like this:

{
  "clearsoda_to_white12_real_view_0": {
    "0": {
      "0": [
        42246.0, 
        1232406.0
      ], 
      "1": [
        1265775.0, 
        2111123.0
      ], 
      "2": [
        2144492.0, 
        4469198.0
      ], 
      "3": [
        4502567.0, 
        4847380.0
      ], 
      "4": [
        4880749.0, 
        6226631.0
      ]
    }
  }, 
}

Please note that by default we extend the first label to beginning of video and the last label to the end of video. We also assign the nearest timestamp's label to to any timestamp that doesn't fall in the segments in the JSON file. This is done to ensure that each frame has a label. You might want to provide your own dummy labels for frames not covered by any segments. Also consider modifying the annotation generating function.

from google-research.

No problem. Happy to help.

from google-research.

This means that your dataset doesn't have per-frame labels but your config.py does. You can set

DATA.FRAME_LABELS: false

in config.py. Also remove 'classification', 'few_shot_classification' , 'event_completion' from EVAL.TASKS in config.py. These tasks also need per-frame labels for evaluation.

from google-research.

Hi @debidatta

I'm trying to create my own dataset too, but have trouble to convert per frame labels and converted to the tfrecord? I saw the create_tfrecords function which will take a JSON label file, do you have an example of how this JSON file will look like?

from google-research.

Really appreciate the quick response!! That example helps a lot, thanks!

from google-research.

Hi @debidatta sorry to pin you again. Maybe you can help with this issue too when you have time: #52 as well. Basically do we actually need per frame label to train TCC? Seems like per frame label is just for evaluation, right? Thank you!

from google-research.

Found it, need to set CONFIG.DATA.FRAME_LABELS = False

from google-research.

I thought that is unsupervised model, but it requires to label something :-\

from google-research.

Per-frame labels are only required to evaluate the model. You can still train a model without those labels. The default config has CONFIG.DATA.FRAME_LABELS = True. Setting it to False should allow you train a model and evaluate on 2 tasks: algo_loss and kendalls_tau.

from google-research.