zhengying-liu / autodl_starting_kit_stable Goto Github PK

View Code? Open in Web Editor NEW

40.0 40.0 23.0 2.28 MB

Starting kit for AutoCV/AutoDL challenge (https://autodl.chalearn.org)

Python 95.33% Jupyter Notebook 4.67%

autodl_starting_kit_stable's People

Contributors

Stargazers

Watchers

autodl_starting_kit_stable's Issues

no metadata needed

Remove metadata from https://github.com/zhengying-liu/autodl_starting_kit_stable/tree/master/AutoDL_sample_code_submission

Stephane's feed-back

une remarque générale : il y a beaucoup d’informations à récupérer depuis les différents codes du github. Une documentation sur les fonctions attendues de model.py et l’interaction entre le programme "ingestion" et le model.py pourrait éclaircir certains points.
Une question qu’on s’est posé : quelle est l’ordre de grandeur du time budget ? Sur la page des résultats sur Codalab, on voit des soumissions allant de quelques heures à plusieurs jours.. Il faudrait peut-être préciser ça, qui aiderait à cibler des approches d’intérêt pour le challenge. Par exemple, est-ce qu’il peut être envisagé des approches grid-search pour déterminer tailles et nombres de couches ?
Le notebook dit que le Tmax en log scale est 1 ; alors que sur les figures (sur la page principale et le notebook), ce n'est pas le cas. En revanche sur codalab, les timestamp des learning curves sont bien normalisées.
Que voit on dans les metadata d’un dataset lorsque les données sont de tailles différentes? None ? Faut parcourir toutes les données d’entrainement pour connaitre les tailles min et max ? Est-ce que les données de test seront nécessairement dans cette même intervalle ?

ValueError when running local tests: Tensor [...] must be from the same graph as Tensor [...]

Describe the bug
When I run local test I am having this error:

ValueError: Tensor("buffer_size:0", shape=(), dtype=int64, device=/device:CPU:0) must be from the same graph as Tensor("MapDataset_2:0", shape=(), dtype=variant).

The same error occurs when I try to use the notebook

To Reproduce

python run_local_test.py

Expected behavior
Logs obtained:

2020-01-02 11:22:52,457 INFO score.py: ===== Start scoring program. Version: v20191219 =====
2020-01-02 11:22:53,929 INFO ingestion.py: ************************************************
2020-01-02 11:22:53,929 INFO ingestion.py: ******** Processing dataset Miniciao ********
2020-01-02 11:22:53,929 INFO ingestion.py: ************************************************
2020-01-02 11:22:53,929 INFO ingestion.py: Reading training set and test set...
2020-01-02 11:22:56,215 INFO ingestion.py: Creating model...this process should not exceed 20min.
2020-01-02 11:22:56,231 INFO ingestion.py: Initialization success, time spent so far 0.016284942626953125 sec
2020-01-02 11:22:56,231 INFO ingestion.py: ===== Start core part of ingestion program. Version: v20191204 =====
2020-01-02 11:22:56,231 INFO ingestion.py: Begin training the model...
2020-01-02 11:22:56,232 INFO model.py: Counting number of examples on train set.
2020-01-02 11:22:56,476 INFO score.py: Detected the start of ingestion after 4 seconds. Start scoring.
2020-01-02 11:22:56,517 INFO model.py: Finished counting. There are 82 examples for training set.
2020-01-02 11:22:56,517 INFO model.py: Model already trained for 0.0000 epochs.
2020-01-02 11:22:56,517 INFO model.py: Begin training for another 10 steps...
2020-01-02 11:22:57,513 INFO api.py: Tensor shape before preprocessing: (1, 32, 32, 3)
2020-01-02 11:22:58,716 INFO api.py: Tensor shape after preprocessing: (1, 32, 32, 3)
2020-01-02 11:22:58,723 INFO ingestion.py: Failed to run ingestion.
2020-01-02 11:22:58,723 ERROR ingestion.py: Encountered exception:
Tensor("buffer_size:0", shape=(), dtype=int64, device=/device:CPU:0) must be from the same graph as Tensor("MapDataset:0", shape=(), dtype=variant).
Traceback (most recent call last):
  File "/Users/romainegele/Documents/AutoDL/auto-nlp-#1/autodl_starting_kit_stable_TF1/AutoDL_ingestion_program/ingestion.py", line 381, in <module>
    remaining_time_budget=remaining_time_budget)
  File "/Users/romainegele/Documents/AutoDL/auto-nlp-#1/autodl_starting_kit_stable_TF1/AutoDL_sample_code_submission/model.py", line 181, in train
    self.classifier.train(input_fn=train_input_fn, steps=steps_to_train)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 370, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1161, in _train_model
    return self._train_model_default(input_fn, hooks, saving_listeners)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1188, in _train_model_default
    input_fn, ModeKeys.TRAIN))
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1025, in _get_features_and_labels_from_input_fn
    self._call_input_fn(input_fn, mode))
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_estimator/python/estimator/estimator.py", line 1116, in _call_input_fn
    return input_fn(**kwargs)
  File "/Users/romainegele/Documents/AutoDL/auto-nlp-#1/autodl_starting_kit_stable_TF1/AutoDL_sample_code_submission/model.py", line 177, in <lambda>
    train_input_fn = lambda: self.input_function(dataset, is_training=True)
  File "/Users/romainegele/Documents/AutoDL/auto-nlp-#1/autodl_starting_kit_stable_TF1/AutoDL_sample_code_submission/model.py", line 309, in input_function
    dataset = dataset.shuffle(buffer_size=self.default_shuffle_buffer)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 1873, in shuffle
    buffer_size, seed, reshuffle_each_iteration))
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 930, in shuffle
    return ShuffleDataset(self, buffer_size, seed, reshuffle_each_iteration)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/data/ops/dataset_ops.py", line 3108, in __init__
    **self._flat_structure)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/ops/gen_dataset_ops.py", line 5607, in shuffle_dataset
    name=name)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/framework/op_def_library.py", line 367, in _apply_op_helper
    g = ops._get_graph_from_inputs(_Flatten(keywords.values()))
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 5979, in _get_graph_from_inputs
    _assert_same_graph(original_graph_element, graph_element)
  File "/Users/romainegele/opt/anaconda3/envs/auto-dl/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py", line 5914, in _assert_same_graph
    (item, original_item))
ValueError: Tensor("buffer_size:0", shape=(), dtype=int64, device=/device:CPU:0) must be from the same graph as Tensor("MapDataset:0", shape=(), dtype=variant).
2020-01-02 11:22:58,724 INFO ingestion.py: Wrote the file end.txt marking the end of ingestion.
2020-01-02 11:22:58,724 INFO ingestion.py: [-] Done, but encountered some errors during ingestion.
2020-01-02 11:22:58,724 INFO ingestion.py: [-] Overall time spent  2.49 sec
2020-01-02 11:22:58,728 INFO ingestion.py: [Ingestion terminated]
2020-01-02 11:22:59,487 INFO score.py: Detected ingestion program had stopped running because an 'end.txt' file is written by ingestion. Stop scoring now.
2020-01-02 11:22:59,488 INFO score.py: Final area under learning curve for miniciao: 0.0000
2020-01-02 11:22:59,489 INFO score.py: Computing error bars with 10 scorings...
2020-01-02 11:22:59,489 INFO score.py:
Latest prediction NAUC:
* Mean: -1
* Standard deviation: -1
* Variance: -1
2020-01-02 11:22:59,489 INFO score.py: Computing ALC error bars with 5 curves...
2020-01-02 11:22:59,489 INFO score.py:
Area under Learning Curve:
* Mean: 0.0
* Standard deviation: 0.0
* Variance: 0.0
2020-01-02 11:22:59,491 ERROR score.py: [-] Some error occurred in ingestion program. Please see output/error log of Ingestion Step.
2020-01-02 11:22:59,491 INFO score.py: [Scoring terminated]

Desktop (please complete the following information):

OS: macOS Catalina version 10.15.2
Browser: Chrome
Version: Version 79.0.3945.88 (Official Build) (64-bit)

Additional context
Python 3.7.5 (Clang 4.0.1)

Python packages:

absl-py==0.9.0
appnope==0.1.0
astor==0.8.1
attrs==19.3.0
backcall==0.1.0
bleach==3.1.0
certifi==2019.11.28
cycler==0.10.0
decorator==4.4.1
defusedxml==0.6.0
entrypoints==0.3
gast==0.2.2
google-pasta==0.1.8
grpcio==1.26.0
h5py==2.10.0
importlib-metadata==1.3.0
ipykernel==5.1.3
ipython==7.11.1
ipython-genutils==0.2.0
ipywidgets==7.5.1
jedi==0.15.2
Jinja2==2.10.3
joblib==0.14.1
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==5.3.4
jupyter-console==6.0.0
jupyter-core==4.6.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.0
kiwisolver==1.1.0
Markdown==3.1.1
MarkupSafe==1.1.1
matplotlib==3.1.2
mistune==0.8.4
more-itertools==8.0.2
nbconvert==5.6.1
nbformat==4.4.0
notebook==6.0.2
numpy==1.18.0
opt-einsum==3.1.0
pandas==0.25.3
pandocfilters==1.4.2
parso==0.5.2
pexpect==4.7.0
pickleshare==0.7.5
powerline-status==2.7
prometheus-client==0.7.1
prompt-toolkit==2.0.10
protobuf==3.11.2
psutil==5.6.7
ptyprocess==0.6.0
Pygments==2.5.2
pyparsing==2.4.6
pyrsistent==0.15.6
python-dateutil==2.8.1
pytz==2019.3
PyYAML==5.2
pyzmq==18.1.1
qtconsole==4.6.0
scikit-learn==0.22
scipy==1.4.1
Send2Trash==1.5.0
six==1.13.0
tensorboard==1.15.0
tensorflow==1.15.0
tensorflow-estimator==1.15.1
termcolor==1.1.0
terminado==0.8.3
testpath==0.4.4
tornado==6.0.3
traitlets==4.3.3
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.16.0
widgetsnbextension==3.5.1
wrapt==1.11.2
zipp==0.6.0

Cannot download public dataset

Describe the bug
The download page in the python script is unavailable

[Example issue] Create an issue in this page whenever you encounter a bug

Describe the bug
This is an example issue created using issue template, which you can find at 'Issues' tab of the repo --> 'New issue' --> 'Get started' for 'Bug report'

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Support PyTorch models

Hi, must we use tensorflow for autoDL competition? Is it possible to use PyTorch in models? If so, would you mind to provide an example model code?

In addition, the website (https://autodl.lri.fr/competitions/146) seems down and can not be connected?

Hi Isabelle,
thanks... ;)
I downloaded the data and tried to run our baseline using the new
starting kit from your git and got an error related to score.py, as
shown below (by the way, you missed an apostrophe ' in the following
instruction:

you have instructed: -code_dir=AutoDL_simple_baseline_models/linear'

where it should be: -code_dir='AutoDL_simple_baseline_models/linear'

the error:

root@9f8e3b89de81:/app/codalab/autodl_starting_kit_stable# python
run_local_test.py -dataset_dir='../data/'
-code_dir='AutoDL_sample_code_submission_resnet_v2_50-image/'
2019-05-11 19:16:41 INFO run_local_test.py:
##################################################
2019-05-11 19:16:41 INFO run_local_test.py: Begin running local test using
2019-05-11 19:16:41 INFO run_local_test.py: code_dir =
AutoDL_sample_code_submission_resnet_v2_50-image
2019-05-11 19:16:41 INFO run_local_test.py: dataset_dir = data
2019-05-11 19:16:41 INFO run_local_test.py:
##################################################
2019-05-11 19:16:41 INFO run_local_test.py: Cleaning existing output
directory of last run:
/app/codalab/autodl_starting_kit_stable/AutoDL_sample_result_submission
2019-05-11 19:16:41 INFO run_local_test.py: Cleaning existing output
directory of last run:
/app/codalab/autodl_starting_kit_stable/AutoDL_scoring_output
2019-05-11 19:16:41,821 INFO ingestion.py: ===== Start ingestion
program. Version: v20190508 =====
2019-05-11 19:16:44,225 INFO ingestion.py:
************************************************
2019-05-11 19:16:44,225 INFO ingestion.py: ******** Processing dataset
Hippocrate ********
2019-05-11 19:16:44,225 INFO ingestion.py:
************************************************
2019-05-11 19:16:44,225 INFO ingestion.py: Reading training set and test set...
WARNING:tensorflow:From
/app/codalab/autodl_starting_kit_stable/AutoDL_ingestion_program/dataset.py:232:
sparse_to_dense (from tensorflow.python.ops.sparse_ops) is deprecated
and will be removed in a future version.
Instructions for updating:
Create a `tf.sparse.SparseTensor` and use `tf.sparse.to_dense` instead.
2019-05-11 19:16:44,805 INFO score.py: Detected the start of ingestion
after 1 seconds. Start scoring.
2019-05-11 19:16:44,806 WARNING score.py: 0 solution files found: []!
Return `None` as solution.
Traceback (most recent call last):
 File "/app/codalab/autodl_starting_kit_stable/AutoDL_scoring_program/score.py",
line 682, in <module>
   is_multiclass_task = is_multiclass(solution)
 File "/app/codalab/autodl_starting_kit_stable/AutoDL_scoring_program/score.py",
line 202, in is_multiclass
   return all(is_one_hot_vector(solution, axis=1))
 File "/app/codalab/autodl_starting_kit_stable/AutoDL_scoring_program/score.py",
line 187, in is_one_hot_vector
   norm_1 = np.linalg.norm(x, ord=1, axis=axis, keepdims=keepdims)
 File "/usr/local/lib/python3.5/dist-packages/numpy/linalg/linalg.py",
line 2283, in norm
   return add.reduce(abs(x), axis=axis, keepdims=keepdims)
TypeError: bad operand type for abs(): 'NoneType'

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

zhengying-liu / autodl_starting_kit_stable Goto Github PK

autodl_starting_kit_stable's People

Contributors

Stargazers

Watchers

Forkers

autodl_starting_kit_stable's Issues

no metadata needed

Stephane's feed-back

ValueError when running local tests: Tensor [...] must be from the same graph as Tensor [...]

Cannot download public dataset

[Example issue] Create an issue in this page whenever you encounter a bug

Support PyTorch models

Bug found by Julio

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent