Alternative/current state of tf_cnn_benchmark about benchmarks HOT 3 OPEN

The benchmark is now unmaintained and untested. I do not recommend using it anymore. I think it still is functionally correct and I doubt it will perform worse than it previously did (but it's very possible I'm wrong). However, I recommend using the official models, as you pointed out.

And the documentation points towards to the new TF2 models for benchmarking. Are you aware of an implementation of an actual benchmark based on the models that could be an alternative?

Running the official models prints out the performance numbers, so it can be used as a benchmark. For example, you can run the official resnet50 model from source by following the instructions here with Method 2, navigating to the path <official models repo>/official/vision, then running

python train.py --logtostderr --model_dir=/tmp/model_dir --experiment=resnet_imagenet --mode=train --params_override=runtime.num_gpus=1,task.train_data.global_batch_size=64,task.train_data.input_path=<path-to-imagenet-tfrecords>/train*,task.validation_data.input_path=<path-to-imagenet-tfrecords>/valid* --config_file configs/experiments/image_classification/imagenet_resnet50_gpu.yaml

In the command above, you need to replace both instances of <path-to-imagenet-tfrecords> with the path to Imagenet in tfrecords format. Unlike tf_cnn_benchmarks, synthetic data is not supported.

from benchmarks.

I miss you @reedwm

from benchmarks.

I miss you too @tfboyd! (this comment is unrelated to this issue BTW @kessel)

from benchmarks.