hipster-philology / pandora Goto Github PK
View Code? Open in Web Editor NEWThis project forked from mikekestemont/pandora
A Tagger-Lemmatizer for Natural Languages
License: MIT License
This project forked from mikekestemont/pandora
A Tagger-Lemmatizer for Natural Languages
License: MIT License
Would it not be good too to be able to test only every nth epoch ? Training takes some time, and maybe we don't need to do it each time.
We are awaing on @mikekestemont feeling if this is gonna break anything. My own feeling is that it would not.
philology-pandora
(?) on http://pypi.python.org/ ( See http://peterdowns.com/posts/first-time-with-pypi.html )Once this is done, it will be up to me to setup the repository for a setup.py
( See #6 )
Dear all,
I think it would be better to enforce, for everyone, the need to go through PR. I am willing to enact this by setting up the repo for this (github allows to block push to specific branch). I think it would allow a more streamlined development and also allows people to be aware of changes. The four or five of us are active enough to get back in an acceptable timeframe to merge the PR.
Are you okay with this ?
The current approach of using '%', '$' and '|' is very brittle if the input text includes one of those characters.
I think this could be abstracted using less ambiguous symbols and putting as constants somewhere in a utils.py file instead of hardcode them.
@mikekestemont : This is a result from the merge operation that should be settled: I'm
guessing main.py should become obsolete.
Add back the functions here : #30
Based on the data set of @Jean-Baptiste-Camps : https://docs.google.com/spreadsheets/d/1uSnLrkouxCkHIzZqr0nTR7-u75WuW2dtXs-Opuywl7A/edit?usp=sharing
Not all epoch have been run. Before I used to run only 100 epochs on Chrestien. Maybe I have badly set the max_len_lemma (?). But it clearly shows superiority of Keras model over PyTorch. It outperforms it starting epoch 25.
It might be an issue with my old config but it seems max lemma len was not in my config.
I would propose to have it computed automatically on "auto" value in the config.
I feel like this is required to promote the thing and for people that would find it
Training, I encountered an error that caused the program to stop, because of a missing font (I usually get a warning about this, but I never saw it crashing everything). Curiously, it happened only at epoch 15.
main(**vars(parser.parse_args()))
File "main.py", line 95, in main
tagger.epoch(verbose=curr_verbose, autosave=True)
File "/home/jbcamps/02_lemmatisation/pandora_testing/pandora/tagger.py", line 579, in epoch
self.save()
File "/home/jbcamps/02_lemmatisation/pandora_testing/pandora/tagger.py", line 492, in save
bbox_inches=0)
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/pyplot.py", line 697, in savefig
res = fig.savefig(*args, **kwargs)
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/figure.py", line 1573, in savefig
self.canvas.print_figure(*args, **kwargs)
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backend_bases.py", line 2252, in print_figure
**kwargs)
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backends/backend_pdf.py", line 2533, in print_pdf
file.close()
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backends/backend_pdf.py", line 547, in close
self.writeFonts()
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backends/backend_pdf.py", line 650, in writeFonts
fonts[Fx] = self.embedTTF(realpath, chars[1])
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backends/backend_pdf.py", line 1124, in embedTTF
return embedTTFType3(font, characters, descriptor)
File "/home/jbcamps/02_lemmatisation/pandora_testing/env/lib/python3.5/site-packages/matplotlib/backends/backend_pdf.py", line 910, in embedTTFType3
filename.encode(sys.getfilesystemencoding()), glyph_ids)
RuntimeError: Failed to open TrueType font
This is similar to the bug described in #12 . When I launch training on the Geste sample data, with
include_morph = label
in the config, I get the following error,
Traceback (most recent call last):
File "train.py", line 6, in <module>
cli_train()
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/cli.py", line 144, in cli_train
train_func(**vars(parser.parse_args()))
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/cli.py", line 94, in train_func
tagger.setup_to_train(**data_sets)
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/tagger.py", line 230, in setup_to_train
morph=self.train_morph)
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/preprocessing.py", line 516, in transform
X_morph, nb_classes=len(self.morph_encoder.classes_))
TypeError: to_categorical() got an unexpected keyword argument 'nb_classes
This does not happen, though, with
include_morph = multilabel
which seems to work quite fine (I'm actually impressed by the results obtained with this option).
From #55
I am not sure this one is worth an issue, but since there might be something I am missing here:
why do we use
for i in range(int(params['nb_epochs'])):
tagger.epoch()
tagger.save()
in Main.py, instead of the existing
tagger.train()
?
it's useful to have example config files but they clutter the topdir a little. @Jean-Baptiste-Camps should we move into a toplevel dir of their own?
I opened a Wiki page about best methods for sharing corpora : https://github.com/hipster-philology/pandora/wiki/Sample-or-example-data-for-Pandora-:-contributing-a-repository
This is for your information.
To open a repository, juste create one or ask for the rights with the name of the repository you want to create
This is just to keep track of model improvements.
Currently, pytorch model with default params in 3 epochs achieves:
::: Train scores (lemmas) :::
Message I have when running the cli
RuntimeError: module compiled against API version 0xb but this version of numpy is 0xa
Couldn't import PyTorch. PyTorch implementation not available
Pip Freeze
boto==2.48.0
bz2file==0.98
certifi==2017.7.27.1
chardet==3.0.4
cycler==0.10.0
editdistance==0.3.1
gensim==2.0.0
h5py==2.7.0
idna==2.6
Keras==2.0.4
matplotlib==2.0.2
mock==2.0.0
nltk==3.2.2
numpy==1.12.1
olefile==0.44
pandas==0.20.3
-e [email protected]:hipster-philology/pandora.git@8f71dbca8d9d5eb335eb43ed9fb17705d65b5113#egg=pandora
pbr==3.1.1
Pillow==4.2.1
protobuf==3.4.0
pyparsing==2.2.0
python-dateutil==2.6.1
pytz==2017.2
PyYAML==3.12
requests==2.18.4
scikit-learn==0.18.1
scipy==0.19.1
seaborn==0.7.1
six==1.11.0
smart-open==1.5.3
tensorflow-gpu==1.1.0
Theano==0.9.0
torch==0.2.0.post3
torchvision==0.1.9
urllib3==1.22
Werkzeug==0.12.2
Functions used by tagger.py will be deprecated in future Keras versions, and we might need to update them to the Keras 2 API. Here are the warnings:
model.py:142: UserWarning: Update your `Conv1D` call to the Keras 2 API: `Conv1D(activation="relu", padding="valid", input_shape=(20, 28), filters=100, strides=1, kernel_size=3, name="focus_conv", kernel_initializer="glorot_uniform")`
name='focus_conv')(token_input)
model.py:174: UserWarning: The `merge` function is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
joined = merge(subnets, mode='concat', name='joined')
env/lib/python3.5/site-packages/keras/legacy/layers.py:460: UserWarning: The `Merge` layer is deprecated and will be removed after 08/2017. Use instead layers from `keras.layers.merge`, e.g. `add`, `concatenate`, etc.
name=name)
model.py:274: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=[<tf.Tenso..., outputs=[<tf.Tenso...)`
model = Model(input=inputs, output=outputs)
-> epoch 1 ...
tagger.py:559: UserWarning: The `nb_epoch` argument in `fit` has been renamed `epochs`.
self.model.fit(train_in, train_out, nb_epoch=1, shuffle=True, batch_size=self.batch_size)
I wonder if it make sense to do it now, or if we should wait for the time where we will merge back the pytorch branch.
From #69
Ok, this solves the first bug, but it leads to another one:
@Jean-Baptiste-Camps
Traceback (most recent call last):
File "train.py", line 6, in <module>
cli_train()
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/cli.py", line 154, in cli_train
train_func(**vars(parser.parse_args()))
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/cli.py", line 104, in train_func
tagger.setup_to_train(**data_sets)
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/tagger.py", line 230, in setup_to_train
morph=self.train_morph)
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/preprocessing.py", line 516, in transform
X_morph, num_classes=len(self.morph_encoder.classes_))
File "/home/jbc/Data/F/pandora/pandora_testing/hipster-philology/pandora/pandora/utils.py", line 353, in to_categorical
y = np.array(y, dtype='int').ravel()
ValueError: invalid literal for int() with base 10: 'NOMB._p|GENRE_f|CAS_r'
-> Epoch 1 ...
203300/203300 [==============================] - 33s - loss: 2.4688 - lemma_out_loss: 1.4247 - pos_out_loss: 1.0441
+ all acc: 0.8012268491211514
+ kno acc: 0.8214169909208819
+ unk acc: 0.14492753623188406
+ all acc: 0.8187645000196611
+ kno acc: 0.8274967574578469
+ unk acc: 0.5349143610013175
::: Train Scores (lemma) :::
+ all acc: 0.8055841134137763
+ kno acc: 0.8055841134137763
+ unk acc: 0.0
::: Dev Scores (lemma) :::
+ all acc: 0.7857818784007571
+ kno acc: 0.8105465553919162
+ unk acc: 0.12636165577342048
::: Test scores (lemma) :::
+ all acc: 0.8012268491211514
+ kno acc: 0.8214169909208819
+ unk acc: 0.14492753623188406
::: Train scores (pos) :::
+ all acc: 0.8166913439355407
+ kno acc: 0.8166913439355407
+ unk acc: 0.0
::: Dev scores (pos) :::
+ all acc: 0.8099913256052362
+ kno acc: 0.8238422516773032
+ unk acc: 0.4411764705882353
::: Test scores (pos) :::
+ all acc: 0.8187645000196611
+ kno acc: 0.8274967574578469
+ unk acc: 0.5349143610013175
When running unseen.py on a directory, the output files have no new lines.
We should pass params to the model defining whether to use gpu and which gpu to use.
For now, when a model is saved, config.txt is just a copy of the original config.txt, instead of being saved from the parameters being used. This prevents, for instance, modifying a parameter during run (for instance a parameter counting the number of epochs ran so far, as needed for #7 ). Here is the code:
# save config file:
if self.config_path:
# make sure that we can reproduce parametrization when reloading:
if not self.config_path == os.sep.join((self.model_dir, 'config.txt')):
shutil.copy(self.config_path, os.sep.join((self.model_dir, 'config.txt')))
else:
with open(os.sep.join((self.model_dir, 'config.txt')), 'w') as F:
F.write('# Parameter file\n\n[global]\n')
F.write('nb_encoding_layers = '+str(self.nb_encoding_layers)+'\n')
F.write('nb_dense_dims = '+str(self.nb_dense_dims)+'\n')
F.write('batch_size = '+str(self.batch_size)+'\n')
F.write('nb_left_tokens = '+str(self.nb_left_tokens)+'\n')
F.write('nb_right_tokens = '+str(self.nb_right_tokens)+'\n')
F.write('nb_embedding_dims = '+str(self.nb_embedding_dims)+'\n')
F.write('model_dir = '+str(self.model_dir)+'\n')
F.write('postcorrect = '+str(self.postcorrect)+'\n')
F.write('nb_filters = '+str(self.nb_filters)+'\n')
F.write('filter_length = '+str(self.filter_length)+'\n')
F.write('focus_repr = '+str(self.focus_repr)+'\n')
F.write('dropout_level = '+str(self.dropout_level)+'\n')
F.write('include_token = '+str(self.include_context)+'\n')
F.write('include_context = '+str(self.include_context)+'\n')
F.write('include_lemma = '+str(self.include_lemma)+'\n')
F.write('include_pos = '+str(self.include_pos)+'\n')
F.write('include_morph = '+str(self.include_morph)+'\n')
F.write('include_dev = '+str(self.include_dev)+'\n')
F.write('include_test = '+str(self.include_test)+'\n')
F.write('nb_epochs = '+str(self.nb_epochs)+'\n')
F.write('halve_lr_at = '+str(self.halve_lr_at)+'\n')
F.write('max_token_len = '+str(self.max_token_len)+'\n')
F.write('min_token_freq_emb = '+str(self.min_token_freq_emb)+'\n')
F.write('min_lem_cnt = '+str(self.min_lem_cnt)+'\n')
F.write('curr_nb_epochs = '+str(self.curr_nb_epochs)+'\n')
I suggest removing the 6 first lines.
Hello everybody,
We are planning to retake work on the pytorch branch of the repository. In the meantime there has also been some work on the master branch (kudos!). I checked for compatibility and the branches wont merge automatically, but I have the feeling it shouldn't be too much work to make it work.
If you agree, I could open a PR and somebody else could review it (otherwise I could do it myself, but it wont be as clean and transparent).
We know that this is a big move but we are confident that the model will improve in performance given that we will be able to implement things like attention, etc...
What do you think?
Things needed for abstracting over the Keras vs Pytorch API:
It seems that keras needs one-hot format for the targets in order to compute the loss,
Pytorch however requires integer format. It makes sense to default to the pytorch format
and have the keras implementation call to_categorical on it rather than having pytorch
undoing the categorical (which means twice as much computation)
Using --dev files is supposedly not mandatory, since the config parameter include_dev can be set to False. But the cli needs --dev path to be defined, even not to use it afterwards. So, right now, even with config
include_dev = False
You need to use
--dev path/to/files
I have recently tried to use Pandora with Python 3 on Ubuntu 16.04, with a newer install of all packages. Doing this, I encountered an error, that might be related to a change in Keras (?):
TypeError: to_categorical() got an unexpected keyword argument 'nb_classes'
I will try to examine it more closely soon.
Traceback (most recent call last):
File "main.py", line 59, in <module>
main(**vars(parser.parse_args()))
File "main.py", line 42, in main
dev_data=dev_data
File "/home/jbcamps/02_lemmatisation/vanilla_pandora/pandora/pandora/tagger.py", line 242, in setup_to_train
morph=self.train_morph)
File "/home/jbcamps/02_lemmatisation/vanilla_pandora/pandora/pandora/preprocessing.py", line 490, in transform
nb_classes=len(self.lemma_encoder.classes_))
TypeError: to_categorical() got an unexpected keyword argument 'nb_classes'
NOTE: this all refers to the pytorch branch
Currently I am using a dict as output of the pytorch model
https://github.com/hipster-philology/pandora/blob/pytorch/pandora/impl/pytorch/model.py#L256-L267
This should be matched by the keras branch as well as the client code in tagger.py (example: https://github.com/hipster-philology/pandora/blob/pytorch/pandora/tagger.py#L379)
I am having a hard time to find out what the model output should be during training and testing. Unfortunately, the docstrings aren't helping.
For instance, for the lemmas at training time. Docstring says:
- list of lemma-indices (`include_lemma` = 'label')
- list of lemma-matrices at character-level (when
`include_lemma` = 'generate').
But if I print it looks like it is taking rather class probabilities...
I am trying out both models for config with lemma == 'generate'
and there seems to be a bug in preprocessing:
Traceback (most recent call last):
File "train.py", line 47, in <module>
tagger.setup_to_train(train_data=train_data, dev_data=dev_data)
File "/home/enrique/code/python/pandora/pandora/tagger.py", line 321, in setup_to_train
self.save()
File "/home/enrique/code/python/pandora/pandora/tagger.py", line 410, in save
self.preprocessor.save(self.model_dir)
File "/home/enrique/code/python/pandora/pandora/preprocessing.py", line 655, in save
for idx, c in self.lemma_char_idx.items():
AttributeError: 'Preprocessor' object has no attribute 'lemma_char_idx'
I started checking this out and it seems there is an issue with the naming of a couple of variables involving "lemma_char_idx", "lemma_char_vocab", and "lemma_char_lookup".
Perhaps @mikekestemont can have a look at this?
I had to add some other changes so make sure you pull first from the pytorch branch.
That part of the code also looks a bit brittle given all the if self....
-statements that make it hard to keep track of things while debugging, perhaps some refactoring could come fine.
Currently, you can't add unknown lemma for post correction. Just add one
I have noticed something weird while retraining a model, and I think it has to do with learning rate. During the first training on the corpus (halve lr at 50), after 100 epochs, my results were seemingly hitting a ceiling:
-> epoch 100 ...
- Lowering learning rate > was: 0.004999999888241291 , now: 0.0025
Epoch 1/1
37853/37853 [==============================] - 200s - loss: 2.6546 - lemma_out_loss: 1.7012 - pos_out_loss: 0.9534
::: Train scores (lemmas) :::
+ all acc: 0.8658758883047579
+ kno acc: 0.8658758883047579
+ unk acc: 0.0
::: Dev scores (lemmas) :::
+ all acc: 0.8330515638207946
+ kno acc: 0.8961684011352885
+ unk acc: 0.30357142857142855
::: Train scores (pos) :::
+ all acc: 0.8463529971204395
+ kno acc: 0.8463529971204395
+ unk acc: 0.0
::: Dev scores (pos) :::
+ all acc: 0.8229078613693999
+ kno acc: 0.8512298959318827
+ unk acc: 0.5853174603174603
::: Test scores (lemmas) :::
+ all acc: 0.8271996615905245
+ kno acc: 0.8962331201137171
+ unk acc: 0.252465483234714
::: Test scores (pos) :::
+ all acc: 0.8189509306260575
+ kno acc: 0.8538261075574508
+ unk acc: 0.5285996055226825
::: ended :::
Since I was not satisfied, I tried reloading and training again for 50 more epochs. I was then very surprised to see the various rates, which had not increased noticeably for some epochs, starting to increase 1 % per epoch, as well as training time going up. I wondered why, but now I think it is because of a «bug» in my loading function (?) which set learning rate very high (notice the change in learning rate and training time):
-> epoch 150 ...
- Lowering learning rate > was: 1.0 , now: 0.5
Epoch 1/1
37853/37853 [==============================] - 266s - loss: 1.3604 - lemma_out_loss: 0.9304 - pos_out_loss: 0.4300
::: Train scores (lemmas) :::
+ all acc: 0.992365202229678
+ kno acc: 0.992365202229678
+ unk acc: 0.0
::: Dev scores (lemmas) :::
+ all acc: 0.9234995773457312
+ kno acc: 0.9739829706717124
+ unk acc: 0.5
::: Train scores (pos) :::
+ all acc: 0.983462341161863
+ kno acc: 0.983462341161863
+ unk acc: 0.0
::: Dev scores (pos) :::
+ all acc: 0.9275147928994083
+ kno acc: 0.9477294228949859
+ unk acc: 0.7579365079365079
::: Test scores (lemmas) :::
+ all acc: 0.9141285956006768
+ kno acc: 0.9701492537313433
+ unk acc: 0.4477317554240631
::: Test scores (pos) :::
+ all acc: 0.9282994923857868
+ kno acc: 0.9559346126510305
+ unk acc: 0.6982248520710059
Have you encountered something like that in your trainings? How high is lr supposed to be at startup ?
In my case, it was at 0.009999999776482582
for the original run, and at 1
for the second (after loading). I guess it has something to do with the optimizer used ?
Trying to train with a PyTorch model, I encounter this error at the time of evaluation
Building model...
-> epoch 1 ...
176400/176400 [==============================] - 1570s - loss: 1.5666 - lemma_out_loss: 1.5666
Traceback (most recent call last):
File "main.py", line 6, in <module>
cli_train()
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/cli.py", line 144, in cli_train
train_func(**vars(parser.parse_args()))
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/cli.py", line 112, in train_func
tagger.epoch(autosave=True, eval_test=tagger.include_test)
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/tagger.py", line 504, in epoch
score_dict = self.logger.epoch(self.curr_nb_epochs, run_eval)
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/logger.py", line 69, in epoch
score_dict = callback()
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/tagger.py", line 501, in run_eval
score_dict.update(self.test())
File "/home/jbcamps/02_lemmatisation/pandora_edge/pandora/tagger.py", line 369, in test
test_preds = self.model.predict(test_in, batch_size=self.batch_size)
TypeError: predict() got an unexpected keyword argument 'batch_size'
Indeed, if we go to pandora/impl/base_model.py, l. 125,
def predict(self, input_data):
does not have the batch_size argument.
Compared to previous Keras implementation, the evaluation takes a really long time (on GPU). It takes few minutes to evaluate while it takes ~ 30/35 seconds to train...
This could be related to #54. Could it be things are run twice ?
Currently, there are different embedding matrices for token and lemma characters. This is not a huge increase in model params, since the char vocab is typically very small, but it is probably wasteful in terms of updates. I can imagine that using the same embedding space for both could help...
We should stop supporting Theano as a keras backend -- partly because the current cannot support it anymore, partly because it won't be developed any further...
[ ] add something about this to the README
Make the printing of tests during training optional.
Might be related to #8
It seems that param dict is not loaded anymore according to test results in https://travis-ci.org/hipster-philology/pandora/builds/283083635 and some code reading
Just using this outside would allow for lighter API and centralized "logging". I can see the use of an export to csv for example.
This one has been around for a long time and does not cause much problems:
RuntimeWarning: More than 20 figures have been opened. Figures created through the pyplot interface (`matplotlib.pyplot.figure`) are retained until explicitly closed and may consume too much memory. (To control this warning, see the rcParam `figure.max_open_warning`).
max_open_warning, RuntimeWarning)
Solving it is simple, I think. We just need to add
sns.plt.close()
in save().
There are some issues with max_token_len
being None.
Additionally, there is a save_params
call inside the tests that doesn't seem to have a correlate (anymore) in the model api. We should fix this... Could somebody check if there is still stuff missing after the merge?
Perhaps we can use a logo? An icon with a gift-wrapped box perhaps? :)
Additional step from #20 : clean up the cli so it does not use that much defaults variable and does not assume too much.
I am encountering some strange bugs, with the current pytorch branch, with some data (and not with others). I suspect it might be related to some character encoding problem, perhaps. I have been trying to solve it, but could not pinpoint the cause. Basically, after the first epoch, I have an error due to a difference in length between predictions and input tokens:
-> epoch 1 ...
Epoch 1/1
25845/25845 [==============================] - 12s - loss: 3.5287 Traceback (most recent call last):
File "main.py", line 61, in <module>
main(**vars(parser.parse_args()))
File "main.py", line 46, in main
tagger.epoch()
File "/home/jbcamps/02_lemmatisation/pytorch_MAJ/pandora/tagger.py", line 480, in epoch
train_preds = self.model.predict(train_in)
File "/home/jbcamps/02_lemmatisation/pytorch_MAJ/pandora/impl/keras/model.py", line 297, in predict
assert len(out) == len(labels)
AssertionError
Here is a sample of the problematic training data:
Item item
IX 9
£ liura
per pẹr
los lo2
borzes borgẹs
de de
Paris Paris
Item item
XXV 25
£ liura
que que
ac avẹr
P P
Albanels Albanel
de de
part part3
P P
Bonome Bonome
lo lo2
jorn jọrn
de de
la lo2
Saint_Peire Sanh_Peire
de de
feureir febrier
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.