Hi,

Thanks for the great work!

I tried to download the dataset following the page here: https://ego4d-data.org/docs/start-here/#cli-download. Which directed me to https://github.com/facebookresearch/Ego4d/blob/main/ego4d/cli/README.md.

After I configured AWS, I tried the next step

python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations

However, I find that this requires me to install the package "ego4d", which a new user doesn't expect to see. I also didn't see the install instruction in the main page: https://github.com/facebookresearch/Ego4d. I only realized that this is a pip package by seeing the setup.py file in the main folder.

I think it would be better if you can provide a one-line instruction for installation for the pip package itself. This would make the user experience much smoother. Thanks!

When downloading the data, is it feasible to download the clips with specific narration? For example, like downloading all clips with the narration "grasp the bottle".

Thanks for sharing your work.
You guys provided a slowfast feature for each video. I just wondering how did you sample the frames in the raw videos as the input of slowfast model. Casuse I am doing some experiment but I need the exact index of the corresponding slowfast input frames.It would be great if you could provide a function for sampling.
Thanks for your time, have a nice day~.

Hi Ego4d team,

While I am downloading the dataset and have almost finished all of them, I suddenly encounter error botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://ego4d-consortium-sharing.s3.amazonaws.com/public/v1/ego4d.json".

Is it because of the aws s3 issue? Could you please have a check?

First of all, congratulations for the Dataset provided.

Regarding the Gaze data provided by Ego4D, only around 31 videos contain this type of modality. I am interested in the usage of Gaze data for the Forecasting Human-Object Interaction Challenge, but none of the videos of these challenge contain this data.

Therefore, is all the gaze data for the EGO4D videos already been provided, or it is planned to be released more data for a bigger subset of videos and chllenges?

(Videos from EGO4D Videos with Gaze: 31/9645 --> Only 0.321% of videos contain gaze)

Windows 11, Python 3.8.5

Tried running

python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata --universities="bristol"
python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata

In both cases the error I recieved was

...\lib\site-packages\botocore\httpsession.py", line 468, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://ego4d-consortium-sharing.s3.None.amazonaws.com/public/v1/ego4d.json"

I think the 'None' in the URL is incorrect and may come from a default value like self.s3_bucket: str = None in manifest.py (example from line 55).

Full Error: 
python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata --universities="bristol"
datasets: ['full_scale', 'annotations']
Download Path: C:\Users\annes\ego4d_data\v1

Downloading Ego4D metadata json..
Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\util\connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "C:\Users\annes\anaconda3\lib\socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\httpsession.py", line 439, in send
    urllib_response = conn.urlopen(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\util\retry.py", line 386, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\packages\six.py", line 735, in reraise
    raise value
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 309, in connect
    conn = self._new_conn()
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x000002C7BBF3DC10>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\annes\anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\cli.py", line 220, in <module>
    main(config)
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\cli.py", line 63, in main
    metadata_path = download_metadata(
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\manifest.py", line 189, in download_metadata
    _metadata_object(version, s3).download_file(str(download_path))
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\inject.py", line 359, in object_download_file
    return self.meta.client.download_file(
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\inject.py", line 190, in download_file
    return transfer.download_file(
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\transfer.py", line 320, in download_file
    future.result()
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\futures.py", line 103, in result
    return self._coordinator.result()
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\futures.py", line 266, in result
    raise self._exception
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\tasks.py", line 269, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\download.py", line 354, in _submit
    response = client.head_object(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 415, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 731, in _make_api_call
    http, parsed_response = self._make_request(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 751, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 107, in make_request
    return self._send_request(request_dict, operation_model)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 183, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 305, in _needs_retry
    responses = self._event_emitter.emit(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 358, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 229, in emit
    return self._emit(event_name, kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 212, in _emit
    response = handler(**kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 194, in __call__
    if self._checker(**checker_kwargs):
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 267, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 294, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 333, in __call__
    checker_response = checker(attempt_number, response,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 233, in __call__
    return self._check_caught_exception(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 376, in _check_caught_exception
    raise caught_exception
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 249, in _do_get_response
    http_response = self._send(request)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 321, in _send
    return self.http_session.send(request)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\httpsession.py", line 468, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://ego4d-consortium-sharing.s3.None.amazonaws.com/public/v1/ego4d.json"
PS C:\Users\annes\ego4d\Ego4d> python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata
datasets: ['full_scale', 'annotations']
Download Path: C:\Users\annes\ego4d_data\v1

Downloading Ego4D metadata json..

Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\util\connection.py", line 61, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "C:\Users\annes\anaconda3\lib\socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\httpsession.py", line 439, in send
    urllib_response = conn.urlopen(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 726, in urlopen
    retries = retries.increment(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\util\retry.py", line 386, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\packages\six.py", line 735, in reraise
    raise value
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 309, in connect
    conn = self._new_conn()
  File "C:\Users\annes\anaconda3\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <botocore.awsrequest.AWSHTTPSConnection object at 0x000002DB9D2EDC40>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\annes\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\annes\anaconda3\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\cli.py", line 220, in <module>
    main(config)
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\cli.py", line 63, in main
    metadata_path = download_metadata(
  File "C:\Users\annes\ego4d\Ego4d\ego4d\cli\manifest.py", line 189, in download_metadata
    _metadata_object(version, s3).download_file(str(download_path))
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\inject.py", line 359, in object_download_file
    return self.meta.client.download_file(
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\inject.py", line 190, in download_file
    return transfer.download_file(
  File "C:\Users\annes\anaconda3\lib\site-packages\boto3\s3\transfer.py", line 320, in download_file
    future.result()
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\futures.py", line 103, in result
    return self._coordinator.result()
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\futures.py", line 266, in result
    raise self._exception
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\tasks.py", line 269, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\s3transfer\download.py", line 354, in _submit
    response = client.head_object(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 415, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 731, in _make_api_call
    http, parsed_response = self._make_request(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\client.py", line 751, in _make_request
    return self._endpoint.make_request(operation_model, request_dict)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 107, in make_request
    return self._send_request(request_dict, operation_model)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 183, in _send_request
    while self._needs_retry(attempts, operation_model, request_dict,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 305, in _needs_retry
    responses = self._event_emitter.emit(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 358, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 229, in emit
    return self._emit(event_name, kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\hooks.py", line 212, in _emit
    response = handler(**kwargs)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 194, in __call__
    if self._checker(**checker_kwargs):
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 267, in __call__
    should_retry = self._should_retry(attempt_number, response,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 294, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 333, in __call__
    checker_response = checker(attempt_number, response,
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 233, in __call__
    return self._check_caught_exception(
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\retryhandler.py", line 376, in _check_caught_exception
    raise caught_exception
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 249, in _do_get_response
    http_response = self._send(request)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\endpoint.py", line 321, in _send
    return self.http_session.send(request)
  File "C:\Users\annes\anaconda3\lib\site-packages\botocore\httpsession.py", line 468, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://ego4d-consortium-sharing.s3.None.amazonaws.com/public/v1/ego4d.json"

I was assuming that the 'has_gaze' field in the metadata indicates whether the corresponding video has the gaze information. However, this field is not consistent with the csv files in the gaze folder. For example, while 0d271871-c8ba-4249-9434-d39ce0060e58.csv exists, the 'has_gaze' field is false in the metadata.

I wonder if this means that the complete gaze data is not released yet or if I misunderstand the metadata?

When I run the command: python-m ego.cli.cli ....., the error Error when finding module specification for ego.cli.cli, ModuleNotFoundError: No Module named 'ego4d occurs. Could you please tell me how I can fix this?

Dear EGO4D team,

Thank you for your great work.

The following video clips are used in 'fho_sta_train.json' or 'fho_sta_val.json' , however they are not downloaded by the commandpython -m ego4d.cli.cli --datasets clips --output_directory="~/ego4d_data" --benchmarks FHO.

['0d421f58-18cc-4bb2-ab69-160fde32aeff', '0ec5d437-9b08-4b8b-bd60-4145b8f009f5', '2411f12c-4160-4aba-8c03-aedd5ec01a6d', '268bf954-37ca-4a34-8efc-874f58c18fa5', '3ff5ab03-3b8f-4e47-b2c9-e9c10e3d17be', '583ff0cb-4697-4f63-b525-bf5bc2318117', '596218ea-6672-472e-aeff-e5fde94562c7', '6798785f-76f9-4c9c-9519-1143f0bc0a83', '70d0b892-057d-4beb-a384-7fc07dbec1f7', '75ed98db-7f09-48d2-9dba-8bf5d19b1f2a', '79ddd897-af72-4831-9e7a-690bc3216c99', '7bbd0124-91dc-4ec6-bda0-f3b6671eecd2', '7d439601-e62e-4107-8c0b-ba0f8554e044', '87fc0cae-0e41-4139-8f49-cc966a1e5a15', '9d58bb9b-0905-4499-a40b-12d659739030', '9e89877c-cd58-43b3-9981-80f62c67e720', 'a0d6bd8e-c477-442b-bab3-731775b44b43', 'ae12594d-09de-474b-a1f7-ada6f105a9da', 'af4609e7-ea6e-4057-8ddd-9f8e76c300d2', 'c9f5958f-ea3f-452e-913d-4f6ede7decea', 'cbb9d6d2-7101-4a94-9d08-29f45919af4d', 'cf6cb4f0-11e6-4968-829a-9725bb677c68', 'd19faa97-b823-4c1d-b00b-e9839f1a2391', 'd246c676-cb66-421f-bf35-dd20240998de', 'd4fc1fd7-c725-454e-9d2b-0fbc774e81d6', 'dd002ced-eab0-4942-aded-303aefd0d627', 'e08c9103-d6a0-4a43-8276-c8f5e7cfe57f', 'e252c493-e587-403d-bd01-22713a1d5a57', 'e964bb42-f596-4dca-96de-0940b52f0c75', 'ea87324e-d129-425f-b247-e6bcc4ff332c', 'fefb0892-7e1a-4a34-9418-ea1c96705edc', '08a7fe30-3f6c-42f7-8b92-88df11e59175', '151dbdab-9705-4641-a8f7-9f432cec8c2e', '2cc33c07-45d7-4d3f-9ef5-46289c4d3394', '302cc305-2c85-4c74-888a-21fbe53bb7a3', '322b2a61-30ce-4973-8d3f-83cd30e4f4ed', '337f8d6a-b079-4b1d-827a-43061d130288', '53808b1d-36e6-44f7-bf84-dbffe702187b', '5c9791a6-ebc2-42e8-b776-ea69f42217ed', '6f5776bf-3403-49c5-949f-03fe38212a6d', '8709efb8-d4f2-40c8-b34a-8a6524d5e7f6', 'bf59f51d-a8c6-4cae-bf4f-415b53fdbbdd', 'd1d4a1b3-a651-4eb8-bb7f-8d66982854fa', 'db5eba61-05b0-4cf5-96ba-979c6f94e267', 'ded39483-8be8-4f8a-a3de-c86b86fd1e7c', 'f801d038-60b9-4957-bd9d-87263202cbb7']

I have to download them manually.

So I think you can add these clips to the benchmark download script

Dear Ego4D team,

Thank you for the excellent dataset.
According to https://github.com/EGO4D/hands-and-objects, there supposed to be three annotation files for the hand object interaction task. but 'fho_oscc-pnr_.json' for Object state change classification seems to be missing in the CLI.

The code only works with Python>=3.8 due to "missing_ok=True" in file_path.unlink (e.g., in download.py line 273).

Maybe it is worth adding this requirement to the README.

Unable to download clips whith the following command:

python -m ego4d.cli.cli --output_directory="~/ego4d_data_clips" --datasets clips annotations --benchmarks fho

I obtain a lot of warnings similar to the ones below and no clip is actually downloaded. I suspect the regex defined in manifest.py:104 is not matching anything.

WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: fho                                                                                                 WARNING:root:Invalid benchmarks manifest entry ignored: em                                                                                                  WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: fho
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: em
WARNING:root:Invalid benchmarks manifest entry ignored: fho

First of all, thanks for the dataset!

On your website, you mention a --features flag for the CLI. However, there is no such flag implemented in config.py. Seems like one should use --dataset slowfast8x8_r101_k400. So I guess you should either add the --features flag as a shortcut for this or adjust the website.

I want to do some research regarding egocentric attention prediction. It should be under social benchmark, but I can't find it in av_val.json. I only found tracking paths of participants and whether they are looking at the wearer. How can I read annotations of the wearer's gaze?

Hello,
I was wondering what is the difference between the --clips and the --full_scale flags. I could not find any source mentioning it..
Thanks!

I was downloading the clips dataset and I noticed that the following file seems to be missing.

Boto 403 Exception For exists: 476870ad-d779-423d-86ec-b8c9c4c54df2 | 476870ad-d779-423d-86ec-b8c9c4c54df2.mp4
Missing s3 object (ignored for download): 476870ad-d779-423d-86ec-b8c9c4c54df2 | 476870ad-d779-423d-86ec-b8c9c4c54df2.mp4

Hi,

Thanks for the dataset! I'm currently facing this issue and I'm not sure what may be causing it:

Running
python -m ego4d.cli.cli --aws_profile_name="ego4d" --output_directory="./ego4d_data" --datasets full_scale annotations --metadata
gives a bunch of warnings that some mp4s are missing on S3:

Boto 403 Exception For exists: a0b8bc0a-9ed2-489b-a31c-48e1afac1bc1 | a0b8bc0a-9ed2-489b-a31c-48e1afac1bc1.mp4
Missing s3 object (ignored for download): a0b8bc0a-9ed2-489b-a31c-48e1afac1bc1 | a0b8bc0a-9ed2-489b-a31c-48e1afac1bc1.mp4

Boto 403 Exception For exists: eaa560d5-6432-4030-a03a-b6ba512e621d | eaa560d5-6432-4030-a03a-b6ba512e621d.mp4
Missing s3 object (ignored for download): eaa560d5-6432-4030-a03a-b6ba512e621d | eaa560d5-6432-4030-a03a-b6ba512e621d.mp4

Boto 403 Exception For exists: 19a7cce8-5771-4ec9-bdd0-189cc34c082d | 19a7cce8-5771-4ec9-bdd0-189cc34c082d.mp4
Missing s3 object (ignored for download): 19a7cce8-5771-4ec9-bdd0-189cc34c082d | 19a7cce8-5771-4ec9-bdd0-189cc34c082d.mp4
...

No existing videos to filter.
ERROR:root:1515/9670 missing S3 downloads will be ignored
Downloading 8155/9670..
Expected size of downloaded files is 5448.6 GB. Do you want to start the download? (y/n)

Is this expectable behavior?

name: ego4d
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=4.5=1_gnu
  - ca-certificates=2021.10.26=h06a4308_2
  - certifi=2021.10.8=py38h06a4308_2
  - ld_impl_linux-64=2.35.1=h7274673_9
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.3.0=h5101ec6_17
  - libgomp=9.3.0=h5101ec6_17
  - libstdcxx-ng=9.3.0=hd4cf53a_17
  - ncurses=6.3=h7f8727e_2
  - openssl=1.1.1m=h7f8727e_0
  - pip=21.2.4=py38h06a4308_0
  - python=3.8.12=h12debd9_0
  - readline=8.1.2=h7f8727e_1
  - setuptools=58.0.4=py38h06a4308_0
  - sqlite=3.37.2=hc218d9a_0
  - tk=8.6.11=h1ccaba5_0
  - wheel=0.37.1=pyhd3eb1b0_0
  - xz=5.2.5=h7b6447c_0
  - zlib=1.2.11=h7f8727e_4
  - pip:
    - boto3==1.21.3
    - botocore==1.24.3
    - ego4d==1.0
    - jmespath==0.10.0
    - python-dateutil==2.8.2
    - s3transfer==0.5.1
    - six==1.16.0
    - tqdm==4.62.3
    - urllib3==1.26.8

Hi,

I was downloading the data from a subset with "python -m ego4d.cli.cli --output_directory="/ego4d_data" --datasets full_scale annotations --metadata --universities bristol ", but got the following errors:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/cxu-serve/p1/chuang65/dataset/Ego4d/ego4d/cli/download.py", line 302, in download_video
obj.download_file(str(file_path), Callback=callback)
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/boto3-1.21.5-py3.8.egg/boto3/s3/inject.py", line 319, in object_download_file
return self.meta.client.download_file(
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/boto3-1.21.5-py3.8.egg/boto3/s3/inject.py", line 173, in download_file
return transfer.download_file(
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/boto3-1.21.5-py3.8.egg/boto3/s3/transfer.py", line 315, in download_file
future.result()
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/futures.py", line 103, in result
return self._coordinator.result()
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/futures.py", line 266, in result
raise self._exception
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/tasks.py", line 139, in call
return self._execute_main(kwargs)
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/tasks.py", line 162, in _execute_main
return_value = self._main(**kwargs)
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/download.py", line 583, in _main
for chunk in chunks:
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/download.py", line 728, in next
chunk = self._body.read(self._chunksize)
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/s3transfer-0.5.1-py3.8.egg/s3transfer/utils.py", line 599, in read
value = self._stream.read(*args, **kwargs)
File "/home/cxu-serve/u1/chuang65/anaconda3/envs/ego4d/lib/python3.8/site-packages/botocore-1.24.5-py3.8.egg/botocore/response.py", line 100, in read
raise ResponseStreamingError(error=e)
botocore.exceptions.ResponseStreamingError: An error occurred while reading from response stream: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer'))

Checking file integrity...
ERROR:root:ERROR: 8 files failed download: d38f271c-53f3-4771-8c17-a5157e723067
31ea56e6-7a99-4b91-b74b-3b1e6de05e99
504315c0-17fd-4259-9d2c-fd2b25fdcbea
ae9cb412-7b93-49ae-bb46-306dc74d5fe3
a855cacc-0c0f-4ad3-9991-90cbd761637a
f3a9685f-0238-45a8-b7f5-30660e4d3acf
fbaa9e09-62c9-460a-b44b-d1db9e5f4039
05df9f6f-90b8-4ed0-81c7-c0e98a730f6f

WARNING:root:upsert_version: ignoring 0 byte download: d38f271c-53f3-4771-8c17-a5157e723067
WARNING:root:upsert_version: ignoring 0 byte download: 31ea56e6-7a99-4b91-b74b-3b1e6de05e99
WARNING:root:upsert_version: ignoring 0 byte download: 504315c0-17fd-4259-9d2c-fd2b25fdcbea
WARNING:root:upsert_version: ignoring 0 byte download: ae9cb412-7b93-49ae-bb46-306dc74d5fe3
WARNING:root:upsert_version: ignoring 0 byte download: a855cacc-0c0f-4ad3-9991-90cbd761637a
WARNING:root:upsert_version: ignoring 0 byte download: f3a9685f-0238-45a8-b7f5-30660e4d3acf
WARNING:root:upsert_version: ignoring 0 byte download: fbaa9e09-62c9-460a-b44b-d1db9e5f4039
WARNING:root:upsert_version: ignoring 0 byte download: 05df9f6f-90b8-4ed0-81c7-c0e98a730f6f

It seems that I have already downloaded some of the videos successfully. Not sure what's the reason, could you please help me check this? Thanks!

Hi Ego4d Team,

Thanks for the great work!

May I ask how many videos I should expect when downloading with full_scale, i.e. running python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata --aws_profile_name ego4d?

I am downloading 9674 videos, is that correct?

How about clips dataset option?

Hi - unfortunately, our Ego4D access keys seem to have expired; is there any way to renew them so we can download the dataset?

Hi,

I wonder if it's possible to provide an explanation for each field in the annotation files? Specifically, I wonder what's the relationship between clip_parent_start_sec, interval_start_sec, action_clip_start_sec, etc.

I'm trying to get a subset of FHO benchmark data whose observations also have associated narrations.

In particular, for each triplet of critical frames in FHO-SCOD, I would like to be able to get the corresponding narration.

Is there an explicit mapping between FHO annotations and narrations that would allow me to achieve my goal? Or, if it does not exist, is it possible to derive this mapping through some procedure?

Hi, thanks for providing this wonderful dataset.
The fho.json file is required for annotation_visualization.ipynb but I can't find it in the annotations directory (and there is no other file with similar fields that can be used instead).
Thanks in advance,
Elad.

Hi Ego4d team,

Thanks for the dataset, it's great!
I have a doubt regarding the hand annotations in the "fho_hands.json" file.

The documentation seems to indicate the presence of the bounding boxes of the hands (https://ego4d-data.org/docs/benchmarks/hands-and-objects/), but in my annotation files I have something like this:

In which each box of a hand is represented by an array of two elements, while I would have expected an array of four elements (x0, y0, w, h) to indicate the bounding box.

Am I doing something wrong?

Thank you in advance.

Hi Ego4d Team,

Thanks for your great work.

In the cli/README.md, it states the dataset options are [annotations, full_scale, clips]. But in the documentation here https://ego4d-data.org/docs/data/videos/ , it states there are 3 types of videos, Canonical videos, Canonical Clips, and Video Components.

May I ask how they correspond to each other and how to download them?

Thanks for sharing your work. I notice that the training data for state change object detection are PRE + PNR + POST image pairs. However, when I download the data by using command line as shown below, all I got are videos not images. How can I get those images? Do I need to process the raw video to got the image pairs?

python -m ego4d.cli.cli --output_directory=/home/naliu/Data/Datasets_tmp/ego4d/fho --dataset full_scale --benchmark fho

Hi there,

In the Ego4D website, you mentioned the locomotion prediction in forecasting benchmark. But it seems there's no data for this task? I wonder if you have the corresponding video id for the locomotion videos and the GT for this task. Thanks a lot!

Thanks for the data release. The paper detailed 3D mesh for 491 hours of the dataset, but I was unable to find anything related to meshes or which videos came with meshes in the release docs. Will this be released? Thanks!

Update: there are 3d and 3d_scans options for the dataset that provide a set of json files with annotations and dataset splits (3d) and a set of 16 tar-files that contain 3D mesh and point cloud information for 16 scenes. Is this the full dataset or are files still being added to this? Thanks again for all your work!

Hi,

When I try to download the dataset by following the instructions, I have encountered this problem:

botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Any idea about how to fix this? Thanks in advance!

Dear EGO4D team,

Thank you for your great work.

Does the '--benchmark' flags only support "EM, FHO, AV" so far?
I would like to download the parts for the benchmark " Short-term object interaction anticipation ". I tried "--benchmark sta", it just showed 0 objects are available. Am I using wrong option, or the benchmark has not been released yet

Sorry, I didn't find the way or doc to download the test set for ego4d challenge.

python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets full_scale annotations --metadata --benchmarks EM
datasets: ['full_scale', 'annotations']

Filtering by benchmarks: ['em']
Retrieving object metadata from S3...
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 3254/3254 [00:03<00:00, 1044.25object/s]
Checking if latest file versions are already downloaded...
0%| | 0/3254 [00:00<?, ?file/s]already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/narration.json
0%| | 1/3254 [00:00<05:41, 9.54file/s]already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_sta_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_sta_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/av_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/vq_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_lta_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_scod_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/moments_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/av_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/nlq_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/vq_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/narration_noun_taxonomy.csv
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/nlq_val.json
0%| | 2/3254 [00:00<22:04, 2.46file/s]already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/moments_train.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_scod_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_hands_val.json
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-consortium-sharing.s3.us-west-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-consortium-sharing.s3.us-west-1.amazonaws.com. Connection pool size: 10
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_lta_val.json
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_lta_taxonomy.json
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-consortium-sharing.s3.us-west-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-consortium-sharing.s3.us-west-1.amazonaws.com. Connection pool size: 10
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/narration_verb_taxonomy.csv
0%| | 3/3254 [00:00<17:09, 3.16file/s]WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-consortium-sharing.s3.us-west-1.amazonaws.com. Connection pool size: 10
already_downloaded: no version entry for existing file: /GPUFS/nsccgz_ywang_ywj/ego4d_data/v1/annotations/fho_hands_train.json
1%|█▎ | 43/3254 [00:32<40:27, 1.32file/s]
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-unict.s3.eu-central-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-unict.s3.eu-central-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-unict.s3.eu-central-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-unict.s3.eu-central-1.amazonaws.com. Connection pool size: 10
WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: ego4d-unict.s3.eu-central-1.amazonaws.com. Connection pool size: 10
Traceback (most recent call last):
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 994, in _prepare_proxy
conn.connect()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connection.py", line 371, in connect
self._tunnel()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/http/client.py", line 905, in _tunnel
raise OSError("Tunnel connection failed: %d %s" % (code,
OSError: Tunnel connection failed: 503 Service Unavailable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/httpsession.py", line 405, in send
urllib_response = conn.urlopen(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 785, in urlopen
retries = retries.increment(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/util/retry.py", line 525, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connectionpool.py", line 994, in _prepare_proxy
conn.connect()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/urllib3-1.26.8-py3.8.egg/urllib3/connection.py", line 371, in connect
self._tunnel()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/http/client.py", line 905, in _tunnel
raise OSError("Tunnel connection failed: %d %s" % (code,
urllib3.exceptions.ProxyError: ('Cannot connect to proxy.', OSError('Tunnel connection failed: 503 Service Unavailable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/cli.py", line 220, in
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/cli.py", line 136, in main
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/download.py", line 241, in filter_already_downloaded
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/tqdm-4.63.0-py3.8.egg/tqdm/std.py", line 1195, in iter
for obj in iterable:
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/concurrent/futures/_base.py", line 619, in result_iterator
yield fs.pop().result()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/concurrent/futures/_base.py", line 444, in result
return self.__get_result()
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/download.py", line 244, in
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/download.py", line 193, in already_downloaded
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/ego4d-1.0-py3.8.egg/ego4d/cli/download.py", line 65, in exists
File "/GPUFS/nsccgz_ywang_ywj/.local/lib/python3.8/site-packages/boto3/resources/factory.py", line 505, in do_action
response = action(self, *args, **kwargs)
File "/GPUFS/nsccgz_ywang_ywj/.local/lib/python3.8/site-packages/boto3/resources/action.py", line 83, in call
response = getattr(parent.meta.client, operation_name)(*args, **params)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 395, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 711, in _make_api_call
http, parsed_response = self._make_request(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 731, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 107, in make_request
return self._send_request(request_dict, operation_model)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 183, in _send_request
while self._needs_retry(attempts, operation_model, request_dict,
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 305, in _needs_retry
responses = self._event_emitter.emit(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 357, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/utils.py", line 1484, in redirect_from_error
new_region = self.get_bucket_region(bucket, response)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/utils.py", line 1541, in get_bucket_region
response = self._client.head_bucket(Bucket=bucket)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 395, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 711, in _make_api_call
http, parsed_response = self._make_request(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/client.py", line 731, in _make_request
return self._endpoint.make_request(operation_model, request_dict)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 107, in make_request
return self._send_request(request_dict, operation_model)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 183, in _send_request
while self._needs_retry(attempts, operation_model, request_dict,
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 305, in _needs_retry
responses = self._event_emitter.emit(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 357, in emit
return self._emitter.emit(aliased_event_name, **kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 228, in emit
return self._emit(event_name, kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/hooks.py", line 211, in _emit
response = handler(**kwargs)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 192, in call
if self._checker(**checker_kwargs):
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 265, in call
should_retry = self._should_retry(attempt_number, response,
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 292, in _should_retry
return self._checker(attempt_number, response, caught_exception)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 331, in call
checker_response = checker(attempt_number, response,
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 231, in call
return self._check_caught_exception(
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/retryhandler.py", line 374, in _check_caught_exception
raise caught_exception
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 249, in _do_get_response
http_response = self._send(request)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/endpoint.py", line 321, in _send
return self.http_session.send(request)
File "/GPUFS/nsccgz_ywang_ywj/anaconda3/envs/cjd_ego4d/lib/python3.8/site-packages/botocore/httpsession.py", line 436, in send
raise ProxyConnectionError(proxy_url=mask_proxy_url(proxy_url), error=e)
botocore.exceptions.ProxyConnectionError: Failed to connect to proxy URL: "http://10.20.18.21:3128"

HI,I want to submit my results about ego4d nlq challenge on evalai , but it keep reporting a failed status.Could you please fix it ?

Downloading Ego4D metadata json..
Traceback (most recent call last):
File "E:\SoftWare\Anaconda\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\SoftWare\Anaconda\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\Project\project\Ego4d\ego4d\cli\cli.py", line 216, in
main(config)
File "E:\Project\project\Ego4d\ego4d\cli\cli.py", line 63, in main
metadata_path = download_metadata(
File "E:\Project\project\Ego4d\ego4d\cli\manifest.py", line 189, in download_metadata
_metadata_object(version, s3).download_file(str(download_path))
File "E:\SoftWare\Anaconda\lib\site-packages\boto3\s3\inject.py", line 319, in object_download_file
return self.meta.client.download_file(
File "E:\SoftWare\Anaconda\lib\site-packages\boto3\s3\inject.py", line 173, in download_file
return transfer.download_file(
File "E:\SoftWare\Anaconda\lib\site-packages\boto3\s3\transfer.py", line 315, in download_file
future.result()
File "E:\SoftWare\Anaconda\lib\site-packages\s3transfer\futures.py", line 103, in result
return self._coordinator.result()
File "E:\SoftWare\Anaconda\lib\site-packages\s3transfer\futures.py", line 266, in result
raise self._exception
File "E:\SoftWare\Anaconda\lib\site-packages\s3transfer\tasks.py", line 269, in _main
self._submit(transfer_future=transfer_future, **kwargs)
File "E:\SoftWare\Anaconda\lib\site-packages\s3transfer\download.py", line 354, in _submit
response = client.head_object(
File "E:\SoftWare\Anaconda\lib\site-packages\botocore\client.py", line 391, in _api_call
return self._make_api_call(operation_name, kwargs)
File "E:\SoftWare\Anaconda\lib\site-packages\botocore\client.py", line 719, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Hi,

I wonder if it's possible to provide an explanation for each field in the annotation files? Some particular problems I encountered while reading the annotation file includes:

1. What's the relationship between clip_uid, source_clip_uid, and video_uid? It seems like that clip_uid refers to canonical clip while video_uid refers to canonical video, but what about source_clip_uid? Similarly, what's the relationship between (video_start_sec, clip_start_sec), (frame, video_frame, clip_frame, start_frame), etc.?

2. Following the previous question, does this means that canonical videos are a concatenation of multiple canonical clip, and if so, how to determine which clips are concatenated together and their boundaries in canonical videos?

3. What does the following filed means?

• videos/clips/valid
• videos/clips/persons/tracking_paths/unmapped_frames
• videos/clips/persons/tracking_paths/suspect
• videos/clips/social_segments_talking/target

4. Regarding the person_id, it seems like the ids are zero-based where 0 indicating the camera wearer. However, the benchmark challenge description indicates that the camera wearer should have id -1, and -1 also appears in social_segments. Is it possible to clarify the meaning of person_id?

command line: python -m ego4d.cli.cli --yes --datasets clips --benchmarks FHO --metadata --output_directory ~/ego4d_data/
when I run this command,I met this problem(showing in figures)

I downloaded gaze data using command python3 -m ego4d.cli.cli --output_directory=/Data/Ego4D --datasets gaze, but I still have two issues about gaze data.

(1) I don't find annotations in downloaded data. How can I get them?

(2) I tried to play these videos, but there's always an error from player. I can play videos of other benchmarks. Could you check whether these videos regarding gaze are damaged?

Hi Ego4d team,

Thanks for the dataset.

I'm trying to download videos with gaze annotations. Following the instructions in official website, I should download them using the command python3 -m ego4d.cli.cli --output_directory=/Data/Ego4D --datasets gaze. I downloaded all gaze data, but I failed to play them. There's an error - 'The file isn’t compatible with QuickTime Player'. I wonder whether those videos have been damaged.

As for the annotation, the official document shows some samples of gaze annotations in csv, but I can't find it after downloading gaze dataset. Could you tell me how to obtain them?

I have posted these issues twice (#79 #54 ), but they were closed and the issues are still unsovled. Some replies even conflict with the instructions on official website. I can't reopen them, so I post this new issue. I really need those data and labels to continue my research. I would appreciate it if you keep this issue open until it's addressed.

Thank you in advance.

Dear Ego4D-Team,

thanks again for the dataset! I am currently working with the NLQ data, and it seems the number of samples does not match the numbers reported in the paper.

>>> import json
>>> x = json.load(open('nlq_train.json'))
>>> len([c for v in x['videos'] for c in v['clips']])
998
>>> len([q for v in x['videos'] for c in v['clips'] for a in c['annotations'] for q in a['language_queries']])
11296

When doing the same with nlq_val.json, I get the following numbers:

In contrast, in the paper you report (Table 6 in the appendix, p. 27)

Am I missing something?

Furthermore, I noticed that a small amount of NLQ annotations have no query:

>>> [a['annotation_uid'] + '_' + str(i) for v in x['videos'] for c in v['clips'] for a in c['annotations'] for i, q in enumerate(a['language_queries']) if 'query' not in q or not q['query']]
['7b63ccdc-9246-448b-9068-d6e05d6cbe5f_4', '51b44048-723b-4101-b1f1-0c73c35800b1_2', '390ded20-fb92-482a-8c65-89b517a69706_5', '43381ede-b3d9-4d66-8d7e-7ae2f9cf1f0d_5', '2a2a09d6-fe45-4176-9350-c8870aada15d_3']

in train and 6d490d22-8ba3-4f13-82ad-ab1ec9074cfe_8 in val

Thanks for your help in advance!

Hi,

Thanks for the code release. I noticed that the filter_already downloaded function seems to have 2 issues:

• it doesn't current operate as expected since the files seem to be missing a version which gets triggered as a mismatch. As a result, all files are redownloaded as noted by #48
• Even if we comment out the version checks in cli/download.py (L207-212, L219-225). The check for number of downloads is incorrect since to_download at that point refers to an array of booleans which will always have the same length and is only used to filter the statements in the return statement using list(compress(downloads, to_download)). This only affects the log as the function returns the correctly filtered set.

I can submit a PR to remove the version checks and fix the to_download, however, I wasn't sure if there were plans to use version checking.

Thanks,
Mohamed

Hi Ego4D team,

I am tring to download everything as this table shows: https://github.com/facebookresearch/Ego4d/blob/main/ego4d/cli/README.md#datasets

However, right now I cannot download fut_loc, lta_models and vq2s_models. The rest of the dataset tags all work fine.

The error message I received (for all three cases) is

botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Will these datasets be released soon, or it's the problem from my side?

Thanks!

Hi Ego4D team,

Thank you so much for releasing this benchmark! It's great!

I downloaded the annotation file yesterday. After taking a short scan of the data, I've noticed that for av_train.json and av_val.json, all "social_segments_looking" are labeled as false. It's a bit odd, since it should contain both true and false labels. Would you please help take a look at that and see if I'm missing anything?

Best,

Thanks for this wonderful work!
The paper mentioned that "Audio-Visual Diarization" and "Talking to me" benchmarks both have audio and visual inputs. So I want to know that is all else benchmarks DON'T have audio inputs? If not, which else benchmarks are audiovisual benchmarks?

Hi,
I keep getting the error "An error occurred (403) when calling the HeadObject operation: Forbidden" when I run the CLI command to download the dataset. I am already an IAM user with root privileges and added the key & the id for AWS cli.

Has anyone else received the problem?
Thanks

Thank you for the great work. I was reading about the dataset and while trying to find the metadata details, I discovered that the link directing to the overview of annotations and metadata does not exist yet. I did not find the repository for Ego-4D docs to report it there (sorry if I missed it). Here's the link that I am talking about (did not find a better way to represent it)

Could you please update it to the correct link so that is easier to find it in the documentation. ( I suppose it is this and this?). Thank you.

If download is interrupted for any reason (example computer reboot due to OS updates), next run of CLI script is attempting to download all files instead of download only non exist files in the file system. It's not efficient way of download for this size of data.

I got the following warning message: Benchmarks specified but ignored without a benchmarks field in manifest.
I used the following command line:

python -m ego4d.cli.cli --output_directory="/datasets/ego4d" --datasets clips annotations viz slowfast8x8_r101_k400 av_models --metadata --benchmarks AV -y

Dear Sir
adding --universities="bristol" to the download query results in an error.

could you please advise on the correct way to request:
python -m ego4d.cli.cli --output_directory="~/ego4d_data" --datasets clips --universities="bristol"