holindauer / tensorgo Goto Github PK

View Code? Open in Web Editor NEW

2.0 1.0 1.0 385 KB

Minimal GoLang Tensor Library

Go 98.82% Python 1.18%

tensorgo's Introduction

TensorGo

TensorGo is a minimal Tensor library for the Go Programming Language.

- Neural Networks

Supported Features

Operations on Arrays of n dimmensions
Reverse Mode Autodiff for Backpropgation
Linear Algebra Functionality
Neural Networks (limited)

Setup

To set up this use this package in your own project, first install the package using the following command:

go get github.com/Holindauer/TensorGo

Then import the package into your project:

import . "github.com/Holindauer/TensorGo/TensorGo"

Don't forget the . before the import statement. This allows you to call the functions and methods in this package without having to specify the package name.

Documentation

The documentation for this library can be found here: TG Doc.

Contributing

If you would like to contriibute to this poroject, take on an issue and submit a pull request. If you would like to add a new feature, please open an issue first to discuss the feature you would like to add.

The two most import rules are:

1: All changes must pass testing 
2: Reduce code duplication as much as possible

For more information on the contribution procedure and coding standards, please visit our Contribution Guidelines

Also, take a look at the Project Structure file to get a sense for how the project is organized:

Discord

For more information, or to ask questions about functionality or contribution, join our Discord server: TG Discord

Extra Thoughts, Reflections

This was my first project that involved major software engineering challenges. The overal goal going into the project was to understand Tensors better, collaborate with other developers, improve at software engineering by pushing at something really difficult.

I think that given the time I started it, it has turned out better than I expected, but were I to go back and reimplement it there would be a lot of changes. This project was where I discovered the need for testing, when the project began I did not know much about how to test code and as such, only began writing tests midway through the project. Because of this, the Tests directory is not as thorough as I'd want it to be. Another issue I ran into that I had not experienced before was watching the codebase grow to the point of being unruly, my projects, up until starting this one were all realively self contained and minimal. Currently, TensorGo is abouy 1750 lines of code. I found myself with a bit too many features that were difficult to maintain and, as such, found myself refactoring often and the code being a bit messier than I like.

Overall, I'm pretty happy with how this turned out and the features I was able to implement. Though much improvement is to be made wrt execution on the next.

-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
Go                              16            734            975           1744
-------------------------------------------------------------------------------
SUM:                            16            734            975           1744
-------------------------------------------------------------------------------

tensorgo's People

Contributors

Stargazers

Watchers

Forkers

yuchiu54

tensorgo's Issues

Load Data From Disk

There is currently a need for a process to load data in form a CSV file into tensors.

Some types of data should be tackled as independent features. Such as:

Loading tabular data
Loading sequential data
Loading batched img data

ie: if it is a common data format for ML or data science, there should be a way load it in from file storage.

Simplify Extend_Shape()

I realized that Extend_Shape() can be simplified a lot by replacing the current mechanism it uses (Recursion) with a process that:

Creates a copy of the input Tensor
Adds a singleton dim to the Input Tensor
Then concats the Singleton added Tensor to itself however many times is specified in num_elements.
Should be relatively simple to implement.

Also it would be good if you could specify which dimension the added dim should occupy, rather than just being appended to the last position in the shape slice. Probably use Permute() at the end of the function after the concatenation to implement this bit.

Download Dataset Extension

The task is to implement a function in extensions.go that runs a bash script to download data from online and store it in a batched tensor. It doesn't really matter from where, the python kaggle api might be the simplest implementation.

Standard datasets should be easy for the user to download, like MNIST and CFAR10/100 and have their own function.

Expand All-Ops and Axis-Ops

Currently, there are not that many operations within the axis_ops.go and all_elelemnt_ops.go files.

The task is to expand these files to include more operations:

Some initial implementations are:

-Geometric mean
-Harmonic mean
-Argmax /Argmin

Normalization of a Tensor into a specific range (all op)

Better Test Coverage Needed for Statistical Operations

Currently, operations that involve statistical operations, such as finding the mean/std don't have great test coverage.

Specifically those that involve taking the standard deviation, variance, and Normalization.

An idea I had would be to go into numpy or torch and create some test cases using analogous operations with a reproducible random seed. Then, implement those into the tests in TG.

Implement Batching

Currently, only a portion of the functions in this library are capable of batched operations.

Functions such as MatMul() and Display_Matrix() are good examples of the standard mechanism that is being implemented for batched operations.

Long story short, for each function, the operation of that function is made into an interface and sent over to one of the functions in the batching.go file to split apart each element of the batch and process them all concurrently.

Implementing this across all other functions in this library to which it makes sense is the task.

Implement Boolean Masking

Boolean Masking needs to be implemented.
ie, a binary op BooleanMask() that places a 1 at equal values, and a 0 at unequal values.

https://towardsdatascience.com/boolean-masking-with-pandas-b21b7714d0b6

Add Singleton to Include Arbitrary Placement

Currently, the AddSingleton() function only adds singletons to the end of the shape.

However, it should really be able to do this at any dim of the tensor.

Modify AddSingleton() to include an argument that will then change insert a 1 at the specified location.

The contiguous data does not need to change for this implementation