A few errors in production.... looks like someone was trying to create a project with a name made up of Unicode characters

2018-07-16T09:47:30.69+0100 [APP/PROC/WEB/2] ERR {"name":"ml-for-kids","hostname":"032dfbf0-71c7-4343-46d6-8b3a","pid":51,"level":50,"err":{"message":"Incorrect string value: '\\xEF\\xBC\\x9F' for column 'name' at row 1","name":"Error","stack":"Error: Incorrect string value: '\\xEF\\xBC\\x9F' for column 'name' at row 1\n at PromisePoolConnection.PromiseConnection.execute (/home/vcap/app/node_modules/mysql2/promise.js:102:20)\n at Object.storeProject (/home/vcap/app/dist/lib/db/store.js:100:47)\n at <anonymous>\n at process._tickDomainCallback (internal/process/next_tick.js:228:7)","code":"ER_TRUNCATED_WRONG_VALUE_FOR_FIELD"},"msg":"DB error","time":"2018-07-16T08:47:30.698Z","v":0}

Problem: If a group of students all use the Try It Now authentication mechanism for creating their projects, they can't then log into another computer to demonstrate their work. This is because the authentication is tied to the browser session.

Possible solution: allow a mechanism to "share" their work by generating a token that is easy to read (such as green-gazelle) so they can then switch to another computer to demo/continue their work.

The back-end and REST APIs have reasonable test coverage, but there are no tests for the UI.

It would be good to identify the most critical areas to start introducing tests for.

Auth0 is used to provide the back-end for auth and user management, but the setup is a little involved. I should write down exactly how to do it.

A common request for secondary school students is to be able to migrate from projects in Scratch to writing their own projects in Python.

This can be enabled by providing a simple Python SDK that can call the endpoints used by Scratch (so as not to require any new server-side endpoints/APIs).

The UI can display a customized snippet for how to use the SDK for a particular project, as well as helper code to explain how to send images.

I should write down the steps needed to run the whole site locally.

This includes:

• Starting and configuring a local MySQL DB
• Creating Object Storage instance
• Creating a developer instance of Auth0
• Running a local taxinomitis-numbers instance
• Compiling the TypeScript for the back-end
• Building the Angular front-end
• Setting up the environment variables
• Downloading a copy of ScratchX

Hello, when you are training, the title of the small topic will not be changed, only the option will be changed. This is very bad.

dot-env is a great package for pulling in .env files and populating variables into process.env at run time. This would be a nice solution to have

Long overdue, really. bower is a bit of a zombie walking. I should move away from it before it breaks and causes me problems.

https://github.com/dalelane/ml-for-kids has been moved to https://github.com/IBM/taxinomitis-docs

The references throughout the site need to be updated to match

e.g.
links to project worksheets -

links to Help docs -

The structure of the taxinomitis-docs repo is different (as it is the result of merging three previous repos) so the URLs will need to be updated to match.

As the number of developers increases, it’s going to be important to make it easier for developers to make code contributions to the project.

The code compile and linting that is currently being run manually using gulp should be extracted and made into a status check that can be run from Travis as part of pull request reviews.

The code is hard-coded based on the workspaces limit of the Standard tier

A little bit of technical debt that it would be good to improve.

There are a bunch of long-running background tasks now. It would be good to have something that will handle SIGTERM by closing all of them cleanly.

IBM Cloud have shuffled around their menus again, so I need to re-do the screenshots in https://github.com/IBM/taxinomitis-docs/blob/master/docs/msword/machinelearningforkids-apikeys.key to reflect the latest screens.

Everything in the site that is done manually (like setting up new Managed Classes) isn't going to happen while I'm on vacation.

It's probably worth adding a note on the site so people don't get annoyed waiting for a fortnight.

Repeated attempts to train a machine learning model all failed with a generic "training failed" error message.

The server logs show that the training data was rejected by the Visual Recognition API because the request payload was too large.

"err":{"message":"413 - \"<html>\\r\\n<head><title>413 Request Entity Too Large</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>413 Request Entity Too Large</h1></center>\\r\\n<hr><center>nginx</center>\\r\\n</body>\\r\\n</html>\\r\\n\"","name":"StatusCodeError","stack":"StatusCodeError: 413 - \"<html>\\r\\n<head><title>413 Request Entity Too Large</title></head>\\r\\n<body bgcolor=\\\"white\\\">\\r\\n<center><h1>413 Request Entity Too Large</h1></center>\\r\\n<hr><center>nginx</center>\\r\\n</body>\\r\\n</html>\\r\\n\"\n at new StatusCodeError (/home/vcap/app/node_modules/request-promise-core/lib/errors.js:32:15)\n at Request.plumbing.callback (/home/vcap/app/node_modules/request-promise-core/lib/plumbing.js:104:33)\n at Request.RP$callback [as _callback] (/home/vcap/app/node_modules/request-promise-core/lib/plumbing.js:46:31)\n at Request.self.callback (/home/vcap/app/node_modules/request/request.js:185:22)\n at emitTwo (events.js:126:13)\n at Request.emit (events.js:214:7)\n at Request.<anonymous> (/home/vcap/app/node_modules/request/request.js:1157:10)\n at emitOne (events.js:116:13)\n at Request.emit (events.js:211:7)\n at Gunzip.<anonymous> (/home/vcap/app/node_modules/request/request.js:1079:12)\n at Object.onceWrapper (events.js:313:30)\n at emitNone (events.js:111:20)\n at Gunzip.emit (events.js:208:7)\n at endReadableNT (_stream_readable.js:1055:12)\n at _combinedTickCallback (internal/process/next_tick.js:138:11)\n at process._tickDomainCallback (internal/process/next_tick.js:218:9)"}

Annoyingly the response came back as a stack trace in an HTML page, so it's not going to be the nicest thing to parse. But the 413 Request Entity Too Large is an obvious enough eye-catcher to be able to search for.

As the student(s) didn't know if it was a temporary glitch/error, they just kept hitting the training button. A lot. Repeatedly. (Which actually ended up crashing the whole app. But that's another issue)

idea 1:
If they got an informative error message, they might've stood a chance of going back to their training data and removing a very large image.

We could parse the error coming back from the VR API and returning a nicer version of it.

idea 2:
Presumably their training data was all/mostly URLs, so when we download an image to check it's accessible, we could try to check it's size - and reject super-large high-res images. That would stop this problem before it happens.

I just tried to compule the application and got this error

/Users/adaniel/dev/taxinomitis/dist/lib/restapi/api.js
  5:22  error  "cors" is not found  node/no-missing-require

Looks like the application requires cors but it is not specified in the package.json

Managed classes are currently created manually.

This involves:

1. Creating a new entry in the tenants DB table, with the restrictions to be applied to the new class
2. Creating Watson Assistant and Visual Recognition service instances and service credentials in IBM Cloud
3. Creating new entries in the bluemixcredentials DB table, to store the newly created service credentials
4. Creating a new user in Auth0 for the teacher
5. Email the teacher with their new username/password

Steps 1 and 3 involve running manually written SQL queries against the live production DB.

This isn’t without risk. It’s also a time-consuming process that will only get more challenging to complete if the site popularity increases.

This should be automated, with a simple web interface for performing all of these steps in one easy go. The interface should be available only to site admins.

I'm currently doing this by running SQL commands against the site's live production MySQL DB. This is risky, so it would be better to put a little tooling around this, accessible only to siteadmin users.

Machine learning models are automatically deleted, after a configurable number of hours. This is explained in the Help page.

The number of hours before machine learning models are deleted is stored as an unsigned TINYINT in MySQL

This means it has a maximum value of 255 (cf. https://dev.mysql.com/doc/refman/8.0/en/integer-types.html )

Today was the first time I was asked to set the deletion period to something bigger than this (30 days).

Only seen this on one machine, but the /scratchx page remained blank - no errors in the browser console and no network errors on the network tab

The training page for images doesn't allow drag and drop when using IE.

Uhhhhhhh.......

What do I do here?

Thanks

Find a lot of information do not know how to build their own ScratchX, please help me.

I want to deploy the website locally, download the source code in GitHub, and I can't find "bluemix-autoscaling-agent" after NPM start.

There are a number of emails that are sent to teachers (e.g. letting them know if one of their API keys looks invalid, explaining training errors due to insufficient IBM Cloud credentials, notifying them of unmanaged classifiers, etc.). These are currently sent manually, based on templates checked into the github project.

This process should be automated.

It’d be nice to have both HTML and plain text versions of the emails, and sending emails with both.

The error section on most pages is within isAuthenticated sections, which means any messages can't be seen.

The existing gulp build is showing it's age and needs replacing.

The new version should:

• Support istanbul for coverage
• Distinguish between dev and prod output
• Only clean dev output

When entering the Watson API keys for Watson Assistant, when entering username/password credentials, the password field is not hidden. I have provided a picture below to demonstrate this.

In addition, when adding an API key for the Watson Visual Recognition, you are not given the choice to log in with username/password credentials - only an API key. Not sure if this is intentional, or an issue.

https://twitter.com/Math_CS_Teach/status/1016001945641811970

MIT AppInventor is a bit Scratch-like but lets you build apps that can be deployed to Android devices. And according to this, soon to iOS devices, too.

In the same way that it was easy to add Python support by re-purposing the existing APIs for the Scratch extension, it might be worth looking into how AppInventor could drive the same API.

If it's not too complicated, it could be very nice to allow students to write machine-learning-powered apps that they can run on a mobile phone.

There isn't a bulk modification API in Auth0 so I'm resetting the passwords for users individually. And to avoid hitting the rate-limiting, I'm doing this one at a time. This is insanely slow with large classes.

Reported by a user:

In Scratch, you can use the globe icon in the upper-left corner to set your desired interface language. But English isn't in the drop-down. So, if you set it to something else (being, naturally, a teen poking around and making trouble), there's no way to set it back to English short of clearing cookies.

The Models UI page displays a machine learning concepts multi-choice quiz for students to do while they're waiting for a model to train. This is particularly useful for images projects which can take 4-5 minutes to finish training a model sometimes.

The quiz questions were written in a hurry for the initial site release, with the plan that I'd come back and write some proper ones.

Except I mostly never did.

I should double-check if they're as bad as I suspect.

Adding new ones would be good, too - as I've seen students get through all of them and still have a minute to wait.

extension API
https://github.com/LLK/scratch-vm/wiki/Scratch-3.0-Extensions-Specification

current build
https://preview.scratch.mit.edu/

The add-training-data block is documented by examples in the worksheets, but never actually explained in the site Scratch documentation.

It would be good to add it to the scratchblocks doc.

The list of tenants that use their Watson credentials outside of Machine Learning for Kids is growing, so the current hard-coded list in the code has become unwieldy. It should be moved into a new DB table, to make it easier to maintain.

As the number of developers increases, it’s going to be important to make it easier for developers to make code contributions to the project.

The integration test that is currently being run manually using gulp should be extracted and made portable.

This means there will be a need for a test DB (either by running a test DB in Compose, or wrapping it in something that prepare a new local MySQL DB) and a test taxinomitis-numbers.

There is currently no way for teachers to delete their entire account. The Help pages offer that the workaround is for teachers to email me and I will do it for them. (Noone has ever taken me up on this). This is something I should remedy anyway. There should be a red "Delete everything" button that deletes everything in the DB associated with a class id.

This will need some new DB functions (to clear up DB rows associated with a class), new image store functions (to clear up images uploaded by a class), and new Auth0 functions (to delete user accounts that are members of a class, including finally the teacher requesting the delete themselves).

The UI will need to make it clear that this is unrecoverable / irreversible.

The app is dependent on valid configuration to support storing / retrieving images. But what should the behaviour be if the config is incorrect? Refuse to start? Slack notification to report the error?

Browsers which block third-party cookies are not able to log in.

This needs fixing.

Possible approaches:

• Enabling custom domain support with auth0 (This isn’t free. It’s not clear whether this would be included in the Open Source Program)
• Implementing a fall-back login page for these browsers
• Displaying an error explaining that the user should enable third-party cookies

I'm soon going to exceed the number of active users that is allowed on the Auth0 free tier. The OSS program looks like an appropriate way to address this, but I'll need to add an Auth0 badge to the login screen to be eligible.

I should write down the steps that I go through to add a new managed class. This would be a helpful pre-req for automating it.

Tests are calling out to the numbers service when they shouldn't. "Delete entire project" in particular keeps calling it in tests. This would be better mocked.

I've had a few offers to translate the UI - but can't accept them until I enable NLS support in the UI.

This will need to include the web UI text, the messages returned by the REST API that are currently displayed in the UI as-is, and the labels on the custom Scratch blocks.

When a teacher clicks on the verify-email link in their welcome email, they go to auth0.com and are immediately bounced to machinelearningforkids.co.uk. They end up on the site home page (not logged in).

There have been a couple of teachers who didn’t realise they had successfully verified their email address. Some did this multiple times because of their confusion.

It would be good if they got a message on landing on the site to confirm their email address was verified.

At the moment, all pending job processing is stopped when one fails.

We could improve on this, so that other jobs can still be attempted if there is a single blip.

Now that the github issues list is available in public Github instead of private Github Enterprise, I could start pointing people at it as a way of reporting problems.

A link from the Help page to the issue tracker, and explaining that it is an option for reporting problems or requesting new features, would be enough to start.

Crash in production :-(

This happened after a (very) large number of failed attempts to train an image machine learning model. (see #40 for background explaining why many of these happened... but not all - there were other reasons, such as VR API credentials being rejected, failure to download some images, etc. etc.).

The cause was that the disk space allocated to the Node.js app in Cloud Foundry was full. And the next call to downloadAndZip failed, because there was no room to write the zip file to.

{"name":"ml-for-kids","hostname":"bd45599a-9080-4613-44f3-cbe1","pid":49,"level":50,"err":{"message":"ENOSPC: no space left on device, write","name":"Error","stack":"Error: ENOSPC: no space left on device, write","code":"ENOSPC"},"stack":"Error: ENOSPC: no space left on device, write","msg":"Crash","time":"2018-07-04T07:26:53.334Z","v":0}