Perform a automatic cache refresh when a page isn't found about tldr-node-client HOT 27 CLOSED

Agree with @Ostera here. I am not a big fan of the interactive approach. Puts more cognitive load on the user and leads to unexpected behavior. (You wouldn't expect a man command to suddenly start giving you a list of options to act on.)

Anyways, yes the retry logic makes sense to me. And we can definitely work on improving the logging messages the tool gives. I will open a separate issue to track the logging part.

from tldr-node-client.

Then the tool should silently fail, and directly go to showing the log message. Users do not expect to see any network error when they look up a page.

from tldr-node-client.

I would prefer full zip downloads. Avoids yet another dependency on git.

from tldr-node-client.

I understand fractional updates will be fast. But still I think we are better without git. Here are my points:

• To be actually fast, ssh needs to be used. So that would require an user account in github.
• As we download more commits, the git history will keep bloating. (git gc is an option but that brings in extra complexity)
• Moreover, the zipped file is just 188K compared to the initial git clone which is about 2M and will download extra non-pages content too.

However, I am not saying that we shy away from the problem. I just remembered that we have this tldr-pages/tldr#343 from .zip to .tar.gz) as an open issue. Its a really simple fix to add building an additional .tar.gz archive(Just need to add couple of extra lines in the build.sh script). And we gain about 56% compression ratio ! More if we use .bz2 !!

I just tested and here are my results-

-rw-rw-r--  1 agniva agniva  60K Oct  5 14:57 tldr.tar.bz2
-rw-rw-r--  1 agniva agniva  84K Oct  5 14:54 tldr.tar.gz
-rw-rw-r--  1 agniva agniva  66K Oct  5 14:58 tldr.tar.xz
-rw-rw-r--  1 agniva agniva 188K Oct  5 14:51 tldr.zip

I am always a fan of doing small and simple things to solve problems that give incremental gains rather than move to a different architecture to go to a perfect solution. And if we do this, we close yet another long standing issue ! Profit 😃

from tldr-node-client.

I'm not sure we can require all clients to use git. For instance, I wonder if the Android clients would be able to do it that way. Unless we can make sure that the git-based workflow is viable for all current clients, we should perhaps consider making that a strong recommendation, rather than a requirement.

Note that we would still require (wherever applicable) the actual user UX to follow this pattern --tell the user which pages were updated, attempt to update silently on page not found, etc.--, so the effect should be the same for the end user regardless of how it is implemented.

from tldr-node-client.

Given the new cache mechanism, the interaction could be like:

No TLDR page for "ssh" found in your cache.

- Press [u] to try updating from the TLDR remote
- Press [o] to open https://github.com/tldr-pages/tldr to contribute!
- Press [m] to try the man page instead
- Press any other key to exit

Maybe there should be a new interactive: true config setting, and if false it just displays the current message:

No TLDR page for "ssh" found in your cache.
Try updating from the TLDR remote with --update.

from tldr-node-client.

I know I am a bit necro, but this issue is very interesting. What do you think about this @agnivade & @waldyrious?

from tldr-node-client.

I love this idea 👍 would definitely provide a better UX! /cc @Ostera

from tldr-node-client.

From the top of my head, if the page is not found, we should ping git to see if there is indeed something to fetch from the remote
and if it is we can pull it immediately.

Either way we retry the lookup.

If there isn't anything new to pull that means we actually don't have the page (or the person has been fiddling with the local repo) and we can show a nice message with a link to open a PR or whatnot.

Else the page will be found on the retry, and the user didn't even notice.

from tldr-node-client.

That would be fine by me: updating silently and providing a direct link to create the page. Just one question: what if the network is down and the git pull fails because of that?

from tldr-node-client.

Sure, agreed that exposing network error messages aren't exactly good UI. I meant that the tool could suggest adding a page that could potentially already exist...

from tldr-node-client.

My approach in general is, if failure is transient or fixable by the app
without bothering the user, then fix it and don't bother the user; if
failure is fatal, let the user know everything he needs to know.

In this case, if there is a problem contacting git, git will give us an
appropriate error code and we can then tell the user "hey, git says you
don't have Internet: ".
On Tue, Oct 4, 2016 at 8:07 PM Waldir Pimenta [email protected]
wrote:

Sure, agreed that exposing network error messages aren't exactly good UI.
I meant that the tool could suggest adding a page that could potentially
already exist...

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0IzvB6McjmRx7t6cMBAnmMaamnNhwpks5qwpXTgaJpZM4CpzPL
.

from tldr-node-client.

Sure, showing up a polished-up message based on the result of the git command sounds good to me. I just wouldn't want the tool to say something that is potentially misleading ("there's no page for this command" when the truth could be "couldn't check whether there's a page for this command").

from tldr-node-client.

Actually I'd rather excuse ourselves and show exactly what happened. With
"" I meant Full git error output.
On Wed, Oct 5, 2016 at 12:01 AM Waldir Pimenta [email protected]
wrote:

Sure, showing up a polished-up message based on the result of the git
command sounds good to me. I just wouldn't want the tool to say something
that is potentially misleading ("there's no page for this command" when the
truth could be "couldn't check whether there's a page for this command").

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0Izhm3npg9wltasEAQmmMWJ3rT1JXfks5qwszAgaJpZM4CpzPL
.

from tldr-node-client.

Well, we don't exactly do a git pull. The tool just downloads the zip file at tldr-pages.github.io. So, it will just be a plain network connectivity error from our code.

This is what I am thinking -
Error scenario

Page not found locally. Trying to fetch a fresh copy ..
Network error- <bla bla ..>

Pass scenario

Page not found locally. Trying to fetch a fresh copy ..
Page found !
<page output>

from tldr-node-client.

Ah, I thought we were a bit more like brew in that case! Do we want that?
Full zip downloads versus git fetches?
On Wed, Oct 5, 2016 at 6:07 AM Agniva De Sarker [email protected]
wrote:

Well, we don't exactly do a git pull. The tool just downloads the zip file
at tldr-pages.github.io. So, it will just be a plain network connectivity
error from our code.

This is what I am thinking -
Error scenario

Page not found locally. Trying to fetch a fresh copy ..
Network error- <bla bla ..>

Pass scenario

Page not found locally. Trying to fetch a fresh copy ..
Page found !

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0Izr1134O5Zy3euPFGkFsndSjGz9k4ks5qwyKFgaJpZM4CpzPL
.

from tldr-node-client.

Well, downloading anything relies on wget, curl, or platform dependably
libraries for networking.

Besides, git is by far more common nowadays than ever before, and if it
gives us a fraction of a time for an update, then it means I don't have to
mysteriously wait for however long it'll take to download the zip file of
all pages need my network.

Potentially a shitty network, and thus a shitty experience.

Just saying that I think git is perfect for content tracking, and that's
exactly the problem at hand. Do we have new content? And can we get it as
fast as possible?
On Wed, Oct 5, 2016 at 6:14 AM Agniva De Sarker [email protected]
wrote:

I would prefer full zip downloads. Avoids yet another dependency on git.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0Izs0iwpbnOIMweDaWAWvXwhNfjtHtks5qwyQigaJpZM4CpzPL
.

from tldr-node-client.

@agnivade, I beg to disagree. Let me try and make a case for switching to git.

tl;dr We get compressed, SHA-1 integrity verified, incremental changes for free, made even faster if the user sets up ssh.

Speed: Incremental changes

We get incremental changes pulled in: a fetch can download as many revisions as required, 1 being what's considered a shallow clone. Alternatively --depth is provided at the git clone (and thus git fetch as well) level to specify the amount of revisions to be fetched for the initial setup.

Note: running git fetch --depth 1 https://github.com/tldr-pages/tldr master:master only downloads 205.17 KiB on a clean git init . repository. Fetching in the last merged change (from tldr-pages/tldr@35b685f to tldr-pages/tldr@093bb6f) means fetching in 2 compressed commits. I've measured using this script and the output is below:

repos/tldr λ ./size.sh 093bb6f..35b685f
3       package.json
total 3

3 mere uncompressed bytes. Now throw in Packfile compression.

Speed: Smart Protocol

If the user setups ssh access, we get the benefits of using the "start" protocol: https://git-scm.com/book/en/v2/Git-Internals-Transfer-Protocols#The-Smart-Protocol

Speed: Compression

The content comes stored in compressed form by default: https://git-scm.com/book/en/v2/Git-Internals-Packfiles

Integrity

Not only content is SHA-1 verified upon transfers (https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) but can be further verified very cheaply at any point in time to ensure consistency.

We are already introducing tooling that does one thing and does it great. We can pretend all we want that we can craft a better way than git for keeping the content up to date in all of our clients, but ultimately this is the obvious better alternative.

from tldr-node-client.

I'll also add that using git would give us the list of changes for free :D (I added that as a requirement for the --update operation on tldr-pages/tldr#1065 (comment) a few days ago). Otherwise we'd have to manually compare the old and the new content and figure out what changed.

from tldr-node-client.

Fair enough. I have made my case, but the --depth=1 does make a convincing counter-argument.

Just wanted to clarify a couple of things before we seal the deal -

1. Should we mandate git fetch to be the standard pull mechanism to be an officially sanctioned client ?
2. What about the fate of tldr-pages/tldr#343 ? Should we close it saying that clients should use git or add building of a .tar.gz archive and resolve it ?

from tldr-node-client.

Agree on both. What about the 2nd point ?

from tldr-node-client.

I'll let @Ostera comment on that one. I have no strong opinions either way.

from tldr-node-client.

If we choose git, it would supersede that issue, IMO.
On Wed, Oct 5, 2016 at 6:55 PM Waldir Pimenta [email protected]
wrote:

I'll let @Ostera https://github.com/ostera comment on that one. I have
no strong opinions either way.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA0IzmnLyl8RA_UBA0cwwCtpvbhvTHPSks5qw9aMgaJpZM4CpzPL
.

from tldr-node-client.

This is taken care of in #122.

from tldr-node-client.

Adding git as a dependency for operating tldr clients is ugly. I'm sure everybody has unzip already, but git??

from tldr-node-client.

Its a decision we took after much deliberation as you can see in the discussion. And its not implemented yet, FWIW. But git does give some advantages for features we want to implement like the list of changes that has happened in an update. There were other comments to make the native C++ client the default one for tldr which would take care of these problems of git dependency.

from tldr-node-client.

In any case, this is specific to the node client -- I agree we shouldn't force all clients to depend on git. I've added some comments regarding the update process in the client spec discussion: tldr-pages/tldr#1065 (comment)

from tldr-node-client.