ofnote / tsalib Goto Github PK
View Code? Open in Web Editor NEWTensor Shape Annotation Library (numpy, tensorflow, pytorch, ...)
License: Apache License 2.0
Tensor Shape Annotation Library (numpy, tensorflow, pytorch, ...)
License: Apache License 2.0
Explain different forms of shorthand, and their usage.
Old:
x = np.random.randn(10,50,300)
x = x[None, None, None, :].permute(3, 0, 1, 2)
New:
from tsalib import dim_vars, get_dim_vars, alignto, warp
# B, T, D, C = dim_vars('B(b):10 T(t):50 D(d):300 C(c):4') #one time, in __main__
B, T, D, C = get_dim_vars('b t d c')
x: 'btd' = np.random.randn(B,T,D)
x = alignto((x, 'btd'), 'btdc')
x = warp(x, 'btdc -> cbtd', 'p') #'p' for permute
combine last two lines as:
x = warp(x, 'btd -> btdc -> cbtd', 'ap') #'a' for align
Error in warp(x, 'b*t,d -> b,t,n,d//n', 'v')
dvs('Lq(lq):100')
warp(x, [',t,d -> ,t,2,d//2', ',t,2, -> ,2,t, '])
tsalib is tested with Python 3.6 but the core API implementation should work with Python 2.x with minor changes.
tensorflow: https://github.com/google-research/bert
pytorch: https://github.com/huggingface/pytorch-pretrained-BERT
Should compress the model description significantly.
May expose other shape transformation instances where tsalib can help.
There is already a setup.py file which defines the requirements. tsalib probably doesn't need a requirements.txt file since it's not used as a standalone application.
Right now join
spec consists of a single shape annotation with *
or ^
Allow join spec to be in l -> r
form, e.g. (b,t,d)* -> (^,b,t,d)
. Then, a warp
sequence can contain a join
.
Can tsalib be compared with datashape? This is assuming there is a basis for comparison.
warp(x, ['btd->bthn->btnh', 'btnh -> b*t,n*h'], tfm_names=['vp', 'v'])
Currently, you have to do an expand_dims
and then expand_transform
.
Is there a crisper way of combining them using shorthand notation?
Does it make sense to use tsalib for Pandas series and/or dataframes?
So I'm declaring global DimVars
(without a hardcoded value) and later at runtime want to update their values based on some hyperparams to do shape assertions.
I'm aware I can just manipulate their ._val
propery, but that seems a bit hacky to me, what's the intended workflow for this kind of problem?
This is a great library! I am wondering whether 'b,1,d'
should be 'b,l,d'
in the following assignment that I found in README.md:
b: 'b,1,d' = alignto((a,'bd'), 'bld')
Avoid carrying dim vars in config dictionary. Just look them up wherever needed by name.
update
size of dim vars.
Hi,
I'm trying to instantiate OpenaiTransformer class from openai_transformer.py code in transformer toy problem setting to learn your tsalib a bit. I'm curious to learn how useful tensor shape annotations would be in practice.
However, I've noticed some syntax level errors in code, e.g.:
How would you go about correcting these problems?
Best,
Wojtek
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.