Curious about purpose of this module about npmjs.us HOT 6 CLOSED

Another common purpose for private registries is to publish private packages. But if your fetch for a tarball always goes to fullfatdb... then you're only serving public stuff...

So confuse?

from npmjs.us.

Hey @isaacs. Great questions and I appreciate your interest and your desire to confer purpose to what must definitely seem like a strange endeavor. I'll try to illuminate. Here's what I'm not doing:

• trying to be faster or better than the official npm repo
• trying to geo-coincide our infrastructure and assets

Here's what I could be said to be doing:

• experimenting with and learning how npm repository hosting works
• setting up some infrastructure that we might want to in the future use for private module hosting. I'd like to offer a similar service to all NorfolkJS members for free (re: https://www.nodejitsu.com/pricing/#npm-main)
• we're proxying all tarball requests to fullfatdb, however we also have nginx set up with a disgustingly large cache (32gb I think) with like a 6mo LRU expiration. (the qualifier "strictly" isn't actually accurate in practice since the npm client fails over to npmjs.org, but you're correct in terms of the architecture itself).
• finding an excuse to put a bitcoin donation badge on a github readme

Let me know what we're doing that still doesn't make sense. Will you be at jsconf?

from npmjs.us.

But so anyway, you're talking a lot about what's in etc/nginx. This here npm module is just a client-side configurator and a statement of our own existence.

from npmjs.us.

Awesome! Well, sounds interesting, at least :)

Falling back to fullfatdb is probably not the best option. You're missing out on the HA setup that we've been building, and introducing a single point of failure unnecessarily. I'd recommend fetching tarballs from registry.npmjs.org if you want to do that. It's closer, faster, and more likely to be available.

A 32gb cache should work well if you don't have as broad a distribution of requests as the public registry does, and don't drop it often (for server restarts etc.) The overall tgz size is something like 300GB at this point.

finding an excuse to put a bitcoin donation badge on a github readme

Nice :)

By the way, one of the explicit goals of npm, Inc. is to figure out how to help people writing tons of valuable modules to make money from them somehow. It's always tricky to introduce money into the equation, though. Motivations are weird.

from npmjs.us.

And yes, I'll be at JSConf, as will a few other npm, Inc folks. We're sponsoring the coffee! :)

from npmjs.us.

300GB is a lot of GB. I'm thinking that there's a pretty long tail in this data, so my wild guess is that 32gb will be sufficient, but I haven't tried it. That is, the cache isn't full yet. Then I'll be pretty interested in tracking misses.

Falling back to fullfatdb is probably not the best option

I had since changed the config to proxy registry.npm.org already; maybe I've neglected to update the config file in the repo.

In addition to the public caching repo, my plan here is just to set up a true private repo -- as in not world readable at all -- as like personal and corporate analog to github private repos. For testing and hardening, I'd like to give accounts away for free to basically whoever wants one, starting with the members of my local JS user group. A single dev team could easily produce O(100) modules, and I think making it easier and freer to host them securely will incentivize their creation over other strategies, e.g. git submodules or internal static asset hosting, perhaps for eventual sale or publish.

@lynnaloo and I (we co-organize the NorfolkJS meetup group) will be at JSConf, so I'll try to come say hi.

from npmjs.us.