Extend `contributing.md` with a bird's eye view of server and services about openstreetmap-website HOT 7 OPEN

• What is the production server architecture of the website (# of application server, deployment process)

I don't actually know this. Although I'm a maintainer it's not really relevant to the codebase or the work we do on it. Last time I was involved in OWG matters there were 6 machines running rails (and some? with cgimap) a primary database and I think 2 read-only database replicas. But that was years ago and I think it has changed substantially since. But I haven't needed to know at any point, since it's not relevant to pretty much any of the development on the codebase, and as Tom pointed out it's different for different deployments too.

• Is there a queuing system / which and some relevant specs

We use the standard ActiveJob abstraction layer, with delayed_job_active_record as the implementation. I don't know any stats on how many jobs per day are processed, I expect it's a substantial number.

• Is there a system in place to use Action Cable? AFAIK we don't but could we?

At the moment we aren't using ActionCable for anything, but I suspect it will be used at some point in the future, likely as an implementation detail for Turbo.

• Relevant notes on the database server

It's PostgreSQL, but I don't know anything else about the database server. Other than it's huge.

• Relevant notes on the staging / testing server

We use Github Actions for testing. We don't have any test servers i.e. servers running automated tests or for QA teams.

We don't have staging servers, in the typical use of the term (i.e. as the final step before production). OSMF has a development server, and a small number of branches and/or configurations have been deployed on that, but it's very rarely used (compared to the amount of PRs we handle - I'd say once every 2-3 years a branch is published there) and I've never used it.

If you are curious you can see the list of deployments on the OSMF dev server at https://apis.dev.openstreetmap.org/

I'm happy to answer any other questions!

from openstreetmap-website.

As @gravitystorm says we do have the ability to deploy a branch for testing on the dev server though it's mostly only used for large changes we want to be able to give them more testing or invite comment on UI changes etc.

You're welcome to ask for a branch to be added there though, or open a PR against chef to add one to https://github.com/openstreetmap/chef/blob/8cb84a3535943f8e24c9ff187dd90ceb83316ae9/roles/dev.rb#L110.

from openstreetmap-website.

A lot of this seems out of scope for the website. When developing a feature it shouldn't matter how its deployed unless you hit resource constraints, which is fairly rare.

We could use better documentation on the deployment in chef in the relevant cookbooks, but it would be written to help users get oriented in the chef code, not this code.

from openstreetmap-website.

I tend to agree with @pnorman that this is not the place for details of the OpenStreetMap deployment of this code.

from openstreetmap-website.

I disagree that the questions I raised are irrelevant for development of features. I even provided two examples where they came up recently.

I am sure most info can be found somewhere. For example I saw a list of hardware once. And the cookbooks are somewhere as well. I could research all this myself, but maybe someone who knows where to look can share the links here?

from openstreetmap-website.

Well your position seems to be that development should be constrained by the current operational environment rather than that the operational environment should be adjusted to support what the code requires.

Of course the reality probably lies somewhere between the two but we wouldn't want people to walk away without talking to us just because the current setup doesn't support what they want to do.

There is also of course the question of other users of this code base - while openstreetmap.org is the primary user it's certainly not the only one.

from openstreetmap-website.

When you're developing features on Rails only without much external dependencies, you can do a lot of testing locally. Things get much more involved once you start integrating with other components.

Speaking of other components, we have a few external applications with direct read and/or write access to some APIDB tables. The chef repo would be a good starting point to learn more. Any changes to the layout of these db tables should be carefully reviewed with externals apps in mind.

Last time I was involved in OWG matters there were 6 machines running rails

We're down to 3 machines, web backend servers are no more. (I'm only counting spike-0[678], since spike-0[123] look rather idle these days)

from openstreetmap-website.