georchestra / datadir Goto Github PK

Currently, the datadir (master branch) is not in a coherent state with the security-proxy code (master branch too) , since the sec-org header is set twice:

• once to the inetOrgPerson's o field with https://github.com/georchestra/datadir/blob/master/security-proxy/headers-mapping.properties#L4
• a second time to the cn attribute of the linked org with https://github.com/georchestra/georchestra/blob/master/security-proxy/src/main/java/org/georchestra/security/LdapUserDetailsRequestHeaderProvider.java#L156 (eg https://github.com/georchestra/georchestra/blob/master/ldap/georchestra.ldif#L208)

Several options here:

• remove the sec-org=o header mapping,
• keep the line, but rename the sec-org header (the one mapping to the user's o field) to sec-orgname

One question to @Vampouille : what links the user's o field and his org's cn field ?

There are no default configuration provided for this microservice.

Having two different branches adds some overhead and is time-consuming when maintaining the datadir for the containerized environments (Docker and Kubernetes). We have to merge the changes for two different solutions.

What I propose is to add variables for as many configurations files as possible so that the same datadir for Docker and Kubernetes can be used.

@jeanmi151 is already working on it

cf urls on https://groups.google.com/g/georchestra/c/5f7f1KYTOBk et https://geoservices.ign.fr/services-beta-geoplateforme-diffusion pour les urls temporaires...

• mettre ces urls 'béta' en plus des existantes pour permettre les tests d'interop au fur et a mesure que l'ign alimente les nouveaux flux ?
• -faire la bascule vers les urls définitives avant le débranchement des anciennes urls fin 2023 ?

les existants ont été ajoutés en georchestra/mapstore2-georchestra#410 et #224

cc @fvanderbiest

See georchestra/georchestra#1850

As discuted previously with other administrators, we should setup the way example catalogs are configured in order to have faster tiles loading.

Here is an example of what can be done :

[...]
"newService": {
    "url": "",
    "type": "wms",
    "title": "",
    "isNew": true,
    "autoload": false,
    "format":"image/vnd.jpeg-png",
    "layerOptions":{
        "tileSize":512,
        "singleTile":false,
        "legendOptions":{
            "legendWidth":30,
            "legendHeight":30
        }
    }
},
[...]
"services": {
    "localgs": {
        "url": "/geoserver/ows",
        "type": "wms",
        "title": "le geoserver local",
        "autoload": true,
        "format":"image/vnd.jpeg-png",
        "layerOptions":{
            "tileSize":512,
            "singleTile":false,
            "legendOptions":{
                "legendWidth":30,
                "legendHeight":30
            }
        },
    }
[...]

The use of image/vnd.jpeg-png should allow to faster tiles loading as it rely on GS to choose between JPEG or PNG (transparency or not).

In the last Mapstore release, image/vnd.jpeg-png8 is also available, so when it will be integrated in Georchestra, we could use it.

Any opinions about this sample conf ?

@MaelREBOUX @landryb @catmorales ?

there are still https://georchestra.mydomain.org occurences in https://github.com/georchestra/datadir/blob/master/security-proxy/security-proxy.properties#L9 - with all the work that's been done to sanitize properties & use values from defaults.properties, can those entries use publicUrl from the defaults ?

Not sure proxy.defaultTarget is still required in https://github.com/georchestra/config/blob/master/security-proxy/security-proxy.properties#L2

Clarify the use of moderatorEmail in https://github.com/georchestra/datadir/blob/master/ldapadmin/ldapadmin.properties:

[email protected]

How is it related with (below):

# Mails-related properties
emailHtml=false
[email protected]
[email protected]
language=en
subject.account.created=[geOrchestra] Your account has been created
subject.account.in.process=[geOrchestra] Your new account is waiting for validation
subject.requires.moderation=[geOrchestra] New account waiting for validation
subject.change.password=[geOrchestra] Update your password
subject.account.uid.renamed=[geOrchestra] New login for your account
warnUserIfUidModified=true

Definitely needs more doc ...

eg:

atlas/atlas.properties:atlas.temporaryDirectory=/tmp/georchestra/atlas
cadastrapp/cadastrapp.properties:tempFolder=/tmp
geonetwork/geonetwork.properties:geonetwork.dir=/tmp/gn_data

https://github.com/georchestra/datadir/blob/18.06/console/templates/newaccount-notification-template.txt misses the Org name

Useful to grant rights

Now that @bortzmeyer is looking at us, I think we should care the variables names :)

should be named headerPath and not headerUrl.

See https://en.wikipedia.org/wiki/URL#Syntax:

URI = scheme:[//authority]path[?query][#fragment]

I'm wondering if we should not have header0.value=.* in https://github.com/georchestra/datadir/blob/master/security-proxy/removed-xforwarded-headers.properties ie: x-forwarded headers should not be sent to any server.

since the upgrade to GN 4.2, there's no log4j2.xml in the geonetwork/log4j subdir. import it from https://github.com/georchestra/geonetwork/blob/georchestra-gn4.2.x/web/src/main/webapp/WEB-INF/classes/log4j2.xml ?

cc @pmauduit ;)

(really minor issue)

The datadir new.json extent is center on a specific region.
It could be nice to use a more largest area instead :

    "maxExtent": [
      -1076658.754450426,
      5134999.16310118,
      2166310.6398374303,
      6771685.279280833
    ],

We historically use a dedicated user for geonetwork, so that the webapp does not mess up tables in the public schema. We actually can define the following property in the geonetwork.properties file:

jdbc.connectionProperties=currentSchema=geonetwork

and use the geochestra user instead.

In https://github.com/georchestra/datadir/blob/master/security-proxy/security-proxy.properties we're missing the defaultTarget property and its associated documentation (that was recently added)

eg:

defaultTarget=/portail/

All the properties are (should be) loaded with Spring placeholders in the applications, with a default value in case the property does not exist.

In order to facilitate the tasks of the administrator, we could use that following pattern in the properties file (it's the common way of doing in .conf files in /etc/ in GNU/Linux distributions):

# The maximum in memory size allowed
# Default is 10240
# See also: see https://docs.spring.io/autorepo/docs/spring-framework/3.1.0.RELEASE/javadoc-api/org/springframework/web/multipart/commons/CommonsFileUploadSupport.html#setMaxInMemorySize(int)
#maxInMemorySize=10240

• the property is commented (the administrator will uncomment if they want to change the value)
• it's preceded by a comment on what the property is, and on its default value, that must be equal to the value in the commented line

And obviously, this value should also correspond to the default value in the code when it's accessed via a placeholder, eg:

    <property name="maxInMemorySize" value="${maxInMemorySize:10240}"/>

Vu avec @catmorales ce jour :

Le fichier newgeostory.json est manquant dans ce dépôt :

https://github.com/georchestra/mapstore2-georchestra/tree/master/configs

follows #312 & #313, cf Configuration changes section in https://github.com/georchestra/mapstore2-georchestra/releases/tag/2023.02.00-RC2-geOrchestra - required for upcoming permalink support

After some discussion and investigation (and browsing through issues from the past) I am wondering if we could not simplify the process of keeping up with default configurations for mapstore.

Of what I could identify, the main ups and downs of the currently "externalized" configs and patching the default configs shipped with the webapp are:

This makes me wonder if it would not make sense to switch from "externalizing" to "patching" in the georchestra datadir (at least for the two configs that are most impacted by new developments). This would allow staying automatically aligned with these developments.

To be more precise, I'm thinking of this:

• remove localConfig.json and pluginsConfig.json from datadir
• motivate and explain the use of patching these two configs (via localConfig.json.patch and pluginsConfig.json.patch) in the readme
• mention the possibility to externalize these configs, but clarify what this means for updates in the readme (and point to the doc that already treats this)

I do not know the complete history of the mapstore datadir in georchestra and might be missing reasons, why certain files have been included. So please correct me.

Note: One potential drawback that I've noticed playing around with patching the localConfig.json is the technical limitation of targeting elements in an array other than by their position, which makes replacing plugins via a patch (without potentially breaking future configs) difficult for example.

Question: I saw that overriding is also a possibility, but had some trouble applying this in georchestra. Is it meant to work within mapstore2-georchestra, as well?

I understand that the following files are never used and therefore do not allow to override the default behavior.

• https://github.com/georchestra/datadir/blob/18.06/mapfishapp/log4j/log4j.properties

see georchestra/mapstore2-georchestra@58f2b99 & georchestra/mapstore2-georchestra#629

Could be something like:

# Note: ldapAdminDn and ldapAdminPassword properties allow to create an
# authenticated connection to the LDAP server.
# But, as the access required by CAS is read-only, depending on the LDAP server
# configuration, you may only want to give CAS an anonymous access. In that
# case, uncomment ldapAdminDn and ldapAdminPassword properties, and let an
# empty value.

See #149 (comment) comment and below.

here : https://github.com/georchestra/datadir/blob/docker-master/geonetwork/microservices/ogc-api-records/config.yml#L23

Following #234, the suggested configuration for the datafeeder will make it unbootable, because the $scheme variable no longer resolves.

Quick fix could be to have a default variable passed (like ${scheme:https} in the DF configuration files, but it's still not clear if this syntax is ok for mail templates.

Current config:

  <intercept-url pattern="/ldapadmin/private/.*" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/public/.*" access="IS_AUTHENTICATED_ANONYMOUSLY" />
  <intercept-url pattern="/ldapadmin/console/public/.*" access="IS_AUTHENTICATED_ANONYMOUSLY" />
  <intercept-url pattern="/ldapadmin/console/.*" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/account/userdetails" access="IS_AUTHENTICATED_FULLY" />
  <intercept-url pattern="/ldapadmin/account/changePassword" access="IS_AUTHENTICATED_FULLY" />
  <intercept-url pattern="/ldapadmin/.*/emails" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/.*/sendEmail" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/attachments" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/emailTemplates" access="ROLE_MOD_LDAPADMIN" />
  <intercept-url pattern="/ldapadmin/emailProxy" access="ROLE_MOD_EMAILPROXY" />

To be tested:

  <intercept-url pattern="/ldapadmin/public/.*" access="IS_AUTHENTICATED_ANONYMOUSLY" />
  <intercept-url pattern="/ldapadmin/console/public/.*" access="IS_AUTHENTICATED_ANONYMOUSLY" />
  <intercept-url pattern="/ldapadmin/account/userdetails" access="IS_AUTHENTICATED_FULLY" />
  <intercept-url pattern="/ldapadmin/account/changePassword" access="IS_AUTHENTICATED_FULLY" />
  <intercept-url pattern="/ldapadmin/emailProxy" access="ROLE_MOD_EMAILPROXY" />
  <intercept-url pattern="/ldapadmin/.*" access="ROLE_MOD_LDAPADMIN" />

The mapstore/README.md contains a extensions.json detail :

extensions.json: dynamic registry of currently installed extensions, in JSON format

But we have nothing about that when we clone the datadir.
From last release, we have a new folder extensions to contains extensions.json file.

Btw, I think that we could provide this folder and a ready extensions.json file to avoid default install error.

I will create a PR next.

See

here we could use env var : https://github.com/georchestra/datadir/blob/docker-master/cas/config/cas.properties#L49

this env var for example : https://github.com/georchestra/docker/blob/master/.envs-ldap#L5

do you think that adding the local geoserver in mapfishapp/wms.servers.json should be a good idea ?

    {
        "name": "My local WMS geoserver",
        "url": "https://georchestra.mydomain.org/geoserver/wms"
    }

I think it would be very useful for everyone.

I understand that the line:

 <context:property-placeholder location="file:${georchestra.datadir}/geonetwork/geonetwork.properties" file-encoding="UTF-8" ignore-unresolvable="true" order="0"/>

present in three files:

• config-datadir-georchestra.xml
• config-db-georchestra.xml
• config-security-georchestra.xml (note that the line is slightly different here, but I think that it does not change the idea)

should be moved to config-overrides-georchestra.xml in order to avoid multiple loading operations of the same file. And it could allow to add a second file (default.properties, see georchestra/georchestra#2299) with more trust in the priority order (see georchestra/georchestra#2123).

See:

cf georchestra/mapstore2-georchestra#516

https://github.com/georchestra/template/blob/master/cas-server-webapp/WEB-INF/view/jsp/default/ui/includes/top.jsp#L46 looks for a homepage.url value in cas.properties, which isnt present in https://github.com/georchestra/config/blob/master/cas/cas.properties. Either homepage.url should be added there, or the top.jsp could use server.name.

I think this breaks redirect-to-homepage when you login directly on /cas/login.

Hello,

I'm having troubles while configuring Georchestra! I cannot create account.
When I try to generate a password for one user by sending email =>{"success":false,"error":"com.sun.mail.smtp.SMTPSendFailedException: 530 5.7.1 Authentication required\n"}

I correctly configured my SMTP parameters, I can receive test message.
According to the "Authentification required" message, I would say that my configuration in administration parameter isn't used.
So I tried to force them in default.properties and console.properties but there is nothing about a password I should specify...

Like an authentification e-mail and password on the SMTP server...

Is there something I am missing?

Thanks for help!

With the recent introduction of rabbit MQ, some new config parameters have been introduced in the default.properties file.

Since Rabbit MQ for now only addresses the usage of Gateway, I would say that we should consider it optional, since Gateway is still not the mainstream option. Considering that, it would be good if those config parameters didn't break the apps.

Right now, using the helm chart and docker-master datadir, the console fails with java.lang.NumberFormatException: For input string: "${rabbitmqPort}".

Apparently, the issue is not rabbit MQ not being available, but simply a string to Int conversion error (when RABBITMQ_PORT is not configured). I propose we provide a default value to this env var, to prevent this error. I didn't check it on properties file, but ${RABBITMQ_PORT:-5672} works in the shell and should also work here, I'd say. And at least prevent this error.

For the upcoming 22.1 or 23 release, mapfishapp's config should be exported to a subfolder in https://github.com/georchestra/mapfishapp

It currently appears in many different modules:

analytics/analytics.properties:#excludedUsers=geoserver_privileged_user,monitoringUser
analytics/analytics.properties:# default: geoserver_privileged_user
analytics/analytics.properties:#excludedUsers=geoserver_privileged_user
console/console.properties:# default: geoserver_privileged_user
console/console.properties:#protectedUsersList=geoserver_privileged_user
datafeeder/datafeeder.properties:#datafeeder.publishing.geoserver.auth.basic.username=geoserver_privileged_user
gateway/security.yaml: protectedUsers: geoserver_privileged_user
security-proxy/security-proxy.properties:# default: geoserver_privileged_user
security-proxy/security-proxy.properties:#trustedUsers=geoserver_privileged_user

We're not using the environment variables in the geonetwork/microservices/ogc-api-records/config.yml config file, which will lead to a broken service. Since we are applying envsubst to this file, it should be using the env vars

In config-security-georchestra.xml, we have:

<ctx:property-override location="WEB-INF/config-security/config-security-overrides.properties" order="2"/>
<ctx:property-override location="file:${georchestra.datadir}/geonetwork/geonetwork-security-overrides.properties" order="1" ignore-resource-not-found="true" />
<!-- using the one from the geOrchestra datadir first (if available) -->
<ctx:property-placeholder location="file:${georchestra.datadir}/geonetwork/geonetwork.properties" ignore-resource-not-found="true" ignore-unresolvable="true" order="1"/>

and the comments seem to mean that file:${georchestra.datadir}/geonetwork/geonetwork.properties would be the file of highest priority to override the beans properties.

But the property-placeholder tag used is not intended for that, it only provides the XML Spring configuration file with values for the placeholders. Meanwhile, the property-override tag used in the previous two lines is intended to directly replace the properties of bean, without using a placeholder. To do that, the properties inside the file:${georchestra.datadir}/geonetwork/geonetwork-security-overrides.properties file, for example, explicitly mention the bean to be modified (see , it will replace the mapping[name] property inside the ldapUserContextMapper bean).

I don't know if it's the expected behavior, but clearly, the file:${georchestra.datadir}/geonetwork/geonetwork.properties file shouldn't be used with the property-override tag, since the format of the strings is different between the "placeholder" and the "override" properties files.

Note that it's somewhat related to georchestra/georchestra#2123.

i know mfapp subdir should be ditched from this repo, but there are still mentions of IGN services such as wxs.ign.fr/opbnnt8s6esy6680ptjfqmwo (cf wms/wmts.servers.json which are deprecated/broken since february, and those should be moved to new urls as done for ms2 in #224

cf georchestra/georchestra#3218 georchestra/geonetwork@dcc7507e and georchestra/geonetwork@7c47f10b

some properties needs to be added to geonetwork.properties to point at elasticsearch and kibana:

• es.host
• es.featureproxy.targeturi
• kb.url

some new logging classes for log4j.xml ?

Hello Guys..
can any one help me understand the path here....is this from a Git or a localmachine of the developer ?
much appreciated .
Thank you

Query: geonetwork.dir=/mnt/geonetwork_datadir

I get an error 500 :

mapstore/rest/config/uploadPlugin 500

java.io.FileNotFoundException: /etc/georchestra/mapstore/configs/pluginsConfig.json.patch (Permission denied)

Indeed, i have no file /etc/georchestra/mapstore/configs/pluginsConfig.json.patch.

Indeed, this doc show configs/pluginsConfig.json path but seems missing in the datadir :

https://docs.mapstore.geosolutionsgroup.com/en/latest/developer-guide/context-editor-config/

As far as I know, no debian package is configured to claim the ownership of the /etc/georchestra/default.properties file, which means that a default setup based on these packages might be broken (or we have a good luck with default parameter values).

Maybe the Security-proxy should take care of the file ?

this had to happen ofc :)

should be kept in sync with https://github.com/georchestra/mapstore2-georchestra/blob/master/localConfig.json ...

added in #232 but to my understanding it isnt referenced by cas/config/cas.properties (unless hidden defaults ? i've seen https://fawnoos.com/2018/10/22/cas6-springboot-admin-server/#spring-boot-admin-server-configuration) and the cas webapp itself doesnt handle SSL, so i think this file should be removed from the repo.

@pmauduit ?

https://github.com/georchestra/datadir/blob/master/geonetwork/geonetwork.properties should use defaults from https://github.com/georchestra/datadir/blob/master/default.properties