Cannot use cached dataset without Internet connection (or when servers are down) about datasets HOT 3 OPEN

There are 2 workarounds, tho:

1. Download datasets from web and just load them locally
2. Use metadata directly (temporal solution, since metadata can change)

import datasets
from datasets.data_files import DataFilesDict, DataFilesList

data_files_list = DataFilesList(
    [
        "hf://datasets/allenai/c4@1588ec454efa1a09f29cd18ddd04fe05fc8653a2/en/c4-train.00000-of-01024.json.gz"
    ],
    [("allenai/c4", "1588ec454efa1a09f29cd18ddd04fe05fc8653a2")],
)
data_files = DataFilesDict({"train": data_files_list})
c4_dataset = datasets.load_dataset(
    path="allenai/c4",
    data_files=data_files,
    split="train",
    cache_dir="/datesets/cache",
    download_mode="reuse_cache_if_exists",
    token=False,
)

Second solution also shows where to find the bug. I suggest that the hashing functions should always use only original parameter data_files, and not the one they get after connecting to the server and creating DataFilesDict

from datasets.

Hi! You need to set the HF_DATASETS_OFFLINE env variable to 1 to load cached datasets offline, as explained in the docs here.

from datasets.

Just tested. It doesn't work, because of the exact problem I described above: hash of dataset config is different.
The only error difference is the reason why it cannot connect to HuggingFace (now it's 'offline mode is enabled')

from datasets.