fail to run reddit_tft example preprocess part on google cloud about cloudml-samples HOT 10 CLOSED

Did you specified --output_dir, that seems to be the problem. I'll make sure to make it required.

from cloudml-samples.

I use

python preprocess.py --training_data fh-bigquery.reddit_comments.2015_12 \
                     --eval_data fh-bigquery.reddit_comments.2016_01 \
                     --predict_data fh-bigquery.reddit_comments.2016_02 \
                     --output_dir $GCS_PATH/preproc \
                     --project_id $PROJECT \
                     --cloud

in https://github.com/GoogleCloudPlatform/cloudml-samples/blob/master/reddit_tft/README.md.

from cloudml-samples.

Just by looking at the symptoms is $GCS_PATH defined on your current environment?

In other words were all these statements successful?

PROJECT=$(gcloud config list project --format "value(core.project)")
BUCKET="gs://${PROJECT}-ml"

GCS_PATH="${BUCKET}/${USER}/reddit_comments"

from cloudml-samples.

echo $GCS_PATH
gs://megvii-test-ml/dongziming/reddit_comments

from cloudml-samples.

Thanks for the information by any chance are you running tensorflow version 1.2?

from cloudml-samples.

>>> import tensorflow as tf
>>> tf.__version__
'1.2.0'
>>> 

I tried tf 1.1.0, error is same.

from cloudml-samples.

Hi @suiyuan2009, I debugged this issue on a clean environment, could you please try the following?

This ensures that the preprocessing code is always executed on a clean directory in case that's the problem.

JOB_ID="reddit_${USER}$(date +%Y%m%d%H%M%S)"
PREPROCESS_OUTPUT="${GCS_PATH}/${JOB_ID}"
python preprocess.py --training_data fh-bigquery.reddit_comments.2015_12
--eval_data fh-bigquery.reddit_comments.2016_01
--predict_data fh-bigquery.reddit_comments.2016_02
--output_dir "${PREPROCESS_OUTPUT}"
--project_id "${PROJECT}"
--cloud

from cloudml-samples.

One more thing could you please verify if your output bucket is regional?

from cloudml-samples.

Also if you could indicate us if you are running on Mac (OS X)?

from cloudml-samples.

Please take a look at #56, this should be fixed.

from cloudml-samples.