A collection of Korean Text Datasets ready to use using Tensorflow-Datasets.
pip install tfds-korean
import tensorflow_datasets as tfds
import tfds_korean.nsmc # register nsmc dataset
ds = tfds.load('nsmc')
train_ds = ds['train'].batch(32)
test_ds = ds['test'].batch(128)
# define model
# ....
# ....
model.fit(train_ds)
model.evaluate(test_ds)
The license for this repository and licenses for datasets are applied separately. It is recommended to use each dataset after checking the dataset's website.