cazala / mnist Goto Github PK

mnist.set(60000, 10000) does not output datasets the right size. I got different lengths in my datasets.

First, thank you for helping people load the MNIST data set!

Now onto replicating the problem, if you download your distribution to a hosted environment and link to mnist.js the following way: (I'm using NodeJS and a statically served folder with express.)

import mnist from './mnist.js';

you will get the following Syntax Error when the browser (Chrome in my case) tries to parse your code.

Uncaught SyntaxError: The requested module './mnist.js' does not provide an export named 'default'

I was able to get past this Syntax Error by adding "export default" to the first line of mnist.js:

export default (function e(t,n,r){function...

I hope this helps you make edits that will help others avoid this troubleshooting.

It looks like this lib is practically in archive mode, so I figure I'll just post this ambient declaration here in a issue instead of a PR. Whoever wants to scratch the same itch like me, copy the following and paste into src/global.d.ts (filename actually doesn't matter, as long as it ends in .d.ts) in your project's directory.

declare module "mnist" {
  type Output = [number, number, number, number, number, number, number, number, number, number];

  type Datum = {
    input: number[];
    output: Output;
  };

  interface Digit {
    id: number;
    raw: number[];
    length: number;
    get: (index?: number) => number[];
    range: (start: number, end: number) => number[][];
    set: (start: number, end: number) => Datum[];
  }

  namespace MNIST {
    export function set(
      trainingSetSize: number,
      testSetSize: number
    ): {
      training: Datum[];
      test: Datum[];
    };

    export function get(count: number): Datum[];

    export function draw(
      digit: number[],
      context: CanvasRenderingContext2D,
      offsetX: number,
      offsetY: number
    ): void;

    export function toNumber(array: number[]): number;
  }

  export = MNIST as typeof MNIST & Digit[];
}

Hi,
I am trying to use your library in the browser and while building the project I get:

...
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
...
Aborted (core dumped)
npm ERR! code 134

I am using webpack for bundling and as I const mnist = require('mnist') I get the error.
I even tried loading dynamically with webpack:

  import(
    /* webpackChunkName: "mnist" */
    /* webpackMode: "lazy" */
    /* webpackPrefetch: false */
    /* webpackPreload: false */
    'mnist'
  ).then(mnist => {
    console.log('mnist');
  });

but this leads to the same error.

It would be nice if this package provides a way to load the dataset asynchronously in the browser.

Hi,

I'm trying to use your library but it seems that when I try to get a digit for 0, 2, 3, 4, 6, 7, 8, 9 I always get a wrong digit.
It seems that the mapping of real digits is wrong.

I always get this mapping:

Here the code that generates the following image:

  for (var i = 0; i < 10; i++) {
    var digit = mnist[i].get();

    var el: HTMLCanvasElement = <HTMLCanvasElement>document.getElementById('canvas' + i);

    var context = el.getContext('2d');

    context.fillStyle = 'white';
    context.fillRect(0, 0, el.width, el.height);

    console.info('Drawing...');
    mnist.draw(digit, context, 0, 0);

    // (<any>context).imageSmoothingEnabled = true;
    context.drawImage( el, 0, 0, 4*el.width, 4*el.height );
  } 

Did I miss something ?

Btw, I am using JSPM with typescript.

I am using your library in my own application to feed a neural network. Thanks for your efforts!

It runs without any problems locally. But when I try to build my application with npm with mnist included, it fails to build the project.

Creating an optimized production build...

<--- Last few GCs --->

[77896:0x102801600]    76919 ms: Mark-sweep 1408.1 (1463.6) -> 1408.0 (1445.6) MB, 3084.7 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 3085 ms) last resort
[77896:0x102801600]    79960 ms: Mark-sweep 1408.0 (1445.6) -> 1408.0 (1445.6) MB, 3040.6 / 0.0 ms  last resort


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x1e648c89cca1 <JSObject>
    2: token(aka token) [0x1e648c882241 <undefined>:1729] [bytecode=0x2efc0f2f56a1 offset=264](this=0x1e648c882241 <undefined>,type=0x2efc0f2eb451 <String[4]: punc>,value=0x1e648c8c1ed1 <String[1]: ,>,is_comment=0x1e648c882241 <undefined>)
    3: arguments adaptor frame: 2->3
    4: next(aka next) [0x1e648c882241 <undefined>:~2147] [pc=0x9b1f1e22da6](this=0x1e648c882241 <undefined>)
    5: expr_...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/mydspr/.nvm/versions/node/v8.3.0/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/mydspr/.nvm/versions/node/v8.3.0/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/mydspr/.nvm/versions/node/v8.3.0/bin/node]
...

These are my current node/npm versions:

→ node -v
v8.3.0
→ npm -v
5.6.0

Did you into this for your own application too?