wikiotics / wikiotics.org Goto Github PK

It will be nice to have some modern favicons on this new static site. It's not essential, so deferring for now. But there is a pretty good guide at https://stackoverflow.com/questions/48956465/favicon-standard-2020-svg-ico-png-and-dimensions

• 16x16 favicon.ico
• 32x32 and 48x48 favicon png files
• 180x180 apple-touch-icon
• 192x192 Chrome Web App Manifest
• Edge and IE 12 browserconfig

The TOML data structures currently used for lessons should be documented.

Currently, 5 gigabytes is required to store all lesson media on the site. This is a large amount of data. If we were to add it directly to this repository, it would significantly slow down cloning, and there would be no way of removing this later aside from rewriting git history. For this reason, we have decided to store lesson media outside this repository for now.

Current solution: a separate git repository

Currently, all lesson media is stored in a separate repository at https://github.com/wikiotics/wikiotics.github.io. 5 GB is still well below github's limit of 100 GB per repository. However, the limit of a GitHub Pages site is 10 GB, so if were to cross this threshold we would have to find another web host.

More important, having lesson media in a separate repository means that all changes to lesson media must be coordinated across two repositories. This is just fine if our goal is to archive the site, but it provides a major barrier to improving and developing additional content.

Additionally, it is important to note that this repository contains both the original media and any other files derived from it. For pictures, this includes thumbnails at smaller sizes. For audio files, this includes conversions to different formats, as well as concatenated "podcast" files.

One possible solution: git "large file storage" (LFS)

One possible solution is to use git large file storage. This would mean that the large files are hosted by a LFS provider outside the repository. GitHub allows 1 GB of storage and 1GB per month of free bandwidth, so their free plan already does not meet our needs.

Alternatively, the LFS host could be another provider, such as Netlify. We might be able to get a free Netlify plan under their open source plan policy, but doing so would mean linking to Netlify from our main page. With this, we could also use their proprietary infrastructure for image thumbnailing, so we would not have to explicitly store thumbnails, but this comes with all the downsides of relying on proprietary web services. I don't think they have proprietary infrastructure for dealing with converting and concatenating audio files, though. Also, it is unclear to me whether random people contributing to our repository will be able to upload their "large files" directly to Netlify as part of making a pull request. Somehow I doubt this would be straightforward.

Another possible solution: IPFS

Another possible solution is to use IPFS, which is a distributed global namespace of content-addressable files. Think of it as the original vision of how we stored media in Ductus (urn:sha386:[...]) with a distributed network like BitTorrent. Already, I have "pinned" all lesson media to IPFS and created a preview that hosts all lesson media through Cloudflare's IPFS gateway (#6).

However, given acceptable performance and reliability, this could be taken a few steps further. Each lesson's TOML structure could contain a hash that resolves to an IPFS directory which contains all the media elements for a lesson. Then, a pull request could simply update this hash to a new one, thus solving the problem of coordination with a second repository.

Pinning infrastructure

Because of IPFS's architecture, content on the network can vanish at any time unless it is "pinned" by at least one computer connected to the network. So in addition to a contributor to a lesson providing a hash, we would need to make sure that the hash remains accessible.

One could imagine a virtual server running the IPFS daemon and configured to respond to events via the github API. When something is pushed to any branch, or when a pull request is opened, one could imagine this machine running a simple script (e.g. grep) that outputs the full list of hashes relevant for that version of the site. The daemon could then pin the relevant hashes by adding them to IPFS's "mutable file system". Anytime a pull request is updated, this pin could be updated, and when a pull request is closed, this pin could be freed.

What I describe is very general infrastructure that could be useful to many projects using IPFS. But I don't know of anything like it, at the moment. The "pinning services" in existence right now are pretty rudimentary.

Working with transformed elements

Above, I said that a lesson could contain a hash to a directory which contains all its media elements. However, we would have to decide: does this directory contain only the original media files, or does it also contain transformed elements (such as thumbnails and conversions)? If it contains only the original files, we would need infrastructure somewhere to deal with the conversions. If the directory is meant to contain everything, this puts additional burdens on the human who is editing the TOML or any potential future editing interface.

After the migration to github is finalized and the GUI lesson editor is retired, a few pages will need a rewrite. Those include (at least):

• https://wikiotics.org/en/main_page/
• https://wikiotics.org/en/Wikiotics_Foundation/
• https://wikiotics.org/en/Lesson_Building_Tips/
• https://wikiotics.org/en/help/home/
• https://wikiotics.org/en/help/Modifying_content
• https://wikiotics.org/en/help/Creating_a_lesson
• https://wikiotics.org/en/help/Using_a_lesson_as_a_template
• https://wikiotics.org/en/Contribute/

and removing links to our old special pages:

• https://wikiotics.org/special/create_account
• https://wikiotics.org/en/make_a_lesson

Once the data structures for lessons are documented (#9), the data structures should be improved, and the documentation should be updated in coordination.

Currently, there are three separate lesson interactions:

1. choice
2. storybook
3. podcast

These lesson interactions are unrelated, but they were stored in Ductus using the same data structure, which we called a "flashcard."

Here is a rough plan:

• Identify any lessons that use multiple of the above interaction types. Are multiple interaction types ever used or actually needed?
• Split the lessons into three types rather than having each lesson be type = "lesson" with the actual lesson type under [[interactions]].
• Refactor the "choice" interaction to have multiple groups of cards, rather than dividers. This will be more pleasant and conceptually simpler when editing the TOML.
• Remove headings where it is unnecessary. The role of columns is already hard-coded for storybook lessons, and the best structure for them might be even more straightforward.
• Further improve the lesson TOML for each lesson type, with the goal of making it a pleasure to edit by hand.
• Deal with media elements. We probably don't need both href and blob_href. The lesson elements might not be the best place for credit elements. (Perhaps they can be in a separate section of the TOML, or stored alongside the media elements themselves.)

In the end, we should make sure that the "grid" view continues to work for each lesson type when setting grid = true in a lesson's front matter.

One could imagine bringing back GUI lesson editing on the static site by having the editor work entirely in the browser via Javascript, with the end result being a pull request via the github API. If we choose to host lesson media on IPFS (#15), this system could even potentially use js-ipfs to get the lesson media available to others.

I only vaguely recall the rules we followed when choosing what to name pages within the wiki portion of the site, i.e., pages outside the group and user namespaces. The best I can remember is that we chose to place something, e.g., in the "English wiki" (content/en/) if the document's title/location was written in English. This is independent of whether a lesson actually teaches English, which is instead coded by the target-languages taxonomy.

A few questions:

• Am I indeed remembering this correctly?
• Does the existing convention make the most sense going forward?
• How, in particular, do we want to handle the user and group sections from the old site?
• If I am having trouble remembering the guidelines on this, I doubt it was followed by all contributors to the site, so there is surely cleanup to do.
• Can our ideal scheme be made to work with Hugo's multilingual feature?

When content is moved or consolidated, an alias should be left at any old location(s). Since we first started the wiki, we have consistently followed the principle that cool URIs don't change.

• Decide and document conventions
• Implement site-wide conformance

Right now, there is an overwhelming amount of content on the site. Much of it is incomplete or unlikely to be what a user is actually looking for. Somehow, we should improve this experience to make sure they are most often directed to the high quality content on the site.

Let's try to achieve 50 stars on github. If we were to achieve this right now, it would put us in ...

• the number 1 spot for language-acquisition.
• the number 25 spot for language-learning (this list also includes many machine learning projects).
• the number 9 spot (tied) for creative-commons.
• the number 4 spot for foreign-language.
• the number 1 spot for foreign-language-learning (actually, we are already there).
• the number 20 spot for flashcards.
• the number 2 spot for second-language-acquisition.

Right now, Wikiotics has three lesson types: choice, storybook, and podcast. However, over the years of talking to actual language teachers, we have encountered requests for various other types of lessons. It would be great to compile a list of these along with their desired features, which could be useful both to us (if this project is to continue) and to others working on projects with similar goals.

@wikey, since you spent much more time talking to actual language teachers, would you be up for drafting such a list? (or even responding here with comment(s) to get things started off)