Giter VIP home page Giter VIP logo

sddi-ckan-k8s's People

Contributors

beneschwab avatar bwibo avatar eidottermihi avatar klml avatar marijaknezevic avatar thomashkolbe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

sddi-ckan-k8s's Issues

Remove default resource settings

The current pre-defined resource settings can avoid rolling out the chart, e.g in a test environment with low resources.
Resource requirements and limits should be set by users dependent on the hardware they use and not be pre-definded in this chart.

Replace DataPusher

DataPusher should be replaced

CKAN DataPusher is not a good choice for pushing data into CKAN datastore.
One core reason to replace DataPusher is that it is complicated to setup and extremly slow. Some more arguments are listed here.
I identified two candidates to replace DataPusher.

ckanext-xloader

Pros

  • Comes as a CKAN extension and is easy to setup
  • Up to 10x faster than Datapusher

Cons

  • Needs to be included in the CKAN-SDDI image
  • Can only be autoscaled by scaling CKAN instances
  • All columns defined as text, and the Data Publisher will need to manually change the data types in the Data Dictionary and reload the data again.

DataPusher+

Pros

  • Built on qsv, an ultra fast processing tool wirtten in Rust.
  • Lives in a separate container and can be scaled individually

Cons

  • Complicated setup

Chart releaser: Don't set alpha/beta releases as latest

What happened?

See title.

Steps to reproduce

Release a beta/alpha version

Configuration

none

Helm chart version

all

SDDI CKAN Docker image version

all

Other software versions and environment

No response

If applicable, what browsers are you seeing the problem on?

No response

Relevant log output

No response

Additional information and comments

No response

Option to replace background/header image

Feature description

Is there an easy way to replace the background/header image? And, if not, it would be great to have one.

Suggested solution

No response

Code snippets

No response

SMTP connection failures

What happened?

my smtp configuration doesnt work. I have used the smtp configuration of ionos (See details in the configuraiton setting). I tried the "forgotten password?" but the email wasnt send (see below the log of the ckan's pod)

Steps to reproduce

  1. Deploy HelmCharts with SMTP Configuration
  2. Create a User with email
  3. Test the "Password Forgotten?", The email doesnt come.

Configuration

ckan:
 smtp:
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    server: "smtp.ionos.de:587" or "smtp.ionos.de:465"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    user: "******"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    password: "**********"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    mailFrom: "admin@********"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    tls: "enable"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    startTls: "true"
    # -- [CKAN SMTP settings](https://docs.ckan.org/en/latest/maintaining/configuration.html#email-settings)
    replyTo: "None"

Versions and environment

Version information

  • helm version: version.BuildInfo{Version:"v3.11.3", GitCommit:"323249351482b3bbfc9f5004f65d400aa70f9ae7", GitTreeState:"clean", GoVersion:"go1.20.3"}
  • Kubernetes: v1.25.2-r0

Environment information

  • VARIABLE_NAME: values

If applicable, what browsers are you seeing the problem on?

No response

Relevant log output

**LOGS CKAN**

[ckan.views.user] Password reset requested for user "********@******.de"
2023-06-09 15:42:30,607 INFO  [ckan.views.user] Emailing reset link to user: nsoule
2023-06-09 15:42:30,705 ERROR [ckan.config.middleware.flask_app] Server not connected
Traceback (most recent call last):
  File "/srv/app/src/ckan/ckan/lib/mailer.py", line 96, in _mail_recipient
    smtp_connection.starttls()
  File "/usr/lib/python3.8/smtplib.py", line 788, in starttls
    self.sock = context.wrap_socket(self.sock,
  File "/usr/lib/python3.8/site-packages/gevent/_ssl3.py", line 114, in wrap_socket
    return self.sslsocket_class(
  File "/usr/lib/python3.8/site-packages/gevent/_ssl3.py", line 312, in __init__
    raise x
  File "/usr/lib/python3.8/site-packages/gevent/_ssl3.py", line 308, in __init__
    self.do_handshake()
  File "/usr/lib/python3.8/site-packages/gevent/_ssl3.py", line 666, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_ILLEGAL_PARAMETER] sslv3 alert illegal parameter (_ssl.c:1131)

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/lib/python3.8/site-packages/flask/views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/flask/views.py", line 163, in dispatch_request
    return meth(*args, **kwargs)
  File "/srv/app/src/ckan/ckan/config/middleware/../../views/user.py", line 662, in post
    mailer.send_reset_link(user_obj)
  File "/srv/app/src/ckan/ckan/lib/mailer.py", line 188, in send_reset_link
    mail_user(user, subject, body)
  File "/srv/app/src/ckan/ckan/lib/mailer.py", line 133, in mail_user
    mail_recipient(recipient.display_name, recipient.email, subject,
  File "/srv/app/src/ckan/ckan/lib/mailer.py", line 124, in mail_recipient
    return _mail_recipient(recipient_name, recipient_email,
  File "/srv/app/src/ckan/ckan/lib/mailer.py", line 116, in _mail_recipient
    smtp_connection.quit()
  File "/usr/lib/python3.8/smtplib.py", line 1002, in quit
    res = self.docmd("quit")
  File "/usr/lib/python3.8/smtplib.py", line 429, in docmd
    self.putcmd(cmd, args)
  File "/usr/lib/python3.8/smtplib.py", line 376, in putcmd
    self.send(f'{s}{CRLF}')
  File "/usr/lib/python3.8/smtplib.py", line 361, in send
    raise SMTPServerDisconnected('Server not connected')

Additional information and comments

No response

Path prefix warning

What happened?

On deployment (heml upgrade or heml install), this warning is shown sometimes:

W0807 09:43:50.396997    2602 warnings.go:70] path /(.*) cannot be used with pathType Prefix

It needs to be investigated what this is about and why it's not shown everytime, just every now and then.

Steps to reproduce

  1. Deploy or upgrade helm chart and see warning

Configuration

ckan-instance-this-happened:
  - catalog.savenow.de

Helm chart version

sddi-ckan-1.1.5 and older versions too

SDDI CKAN Docker image version

ghcr.io/tum-gis/ckan-sddi:1.1.3

Other software versions and environment

Helm

version.BuildInfo{Version:"v3.12.2", GitCommit:"1e210a2c8cc5117d1055bfaa5d40f51bbc2e345e", GitTreeState:"clean", GoVersion:"go1.20.5"}

Kubernetes

kubectl version  --output=yaml
clientVersion:
  buildDate: "2023-05-17T14:20:07Z"
  compiler: gc
  gitCommit: 7f6f68fdabc4df88cfea2dcf9a19b2b830f1e647
  gitTreeState: clean
  gitVersion: v1.27.2
  goVersion: go1.20.4
  major: "1"
  minor: "27"
  platform: linux/amd64
kustomizeVersion: v5.0.1
serverVersion:
  buildDate: "2023-06-19T16:12:25Z"
  compiler: gc
  gitCommit: 8cfcba0b15c343a8dc48567a74c29ec4844e0b9e
  gitTreeState: clean
  gitVersion: v1.25.11
  goVersion: go1.19.10
  major: "1"
  minor: "25"
  platform: linux/amd64

If applicable, what browsers are you seeing the problem on?

No response

Relevant log output

No response

Additional information and comments

No response

No icon at default favicon path

What happened?

The default favicon path points to a location without an image. This should be adapted to the default path in tum-gis/ckan-docker: /base/image/favicon.ico

Steps to reproduce

Check the logs for an instance with default settings.

Configuration

ckan:
  favicon: "/webassets/favicon.ico"

Helm chart version

sddi-ckan-1.0.2

SDDI CKAN Docker image version

docker pull ghcr.io/tum-gis/ckan-sddi:1.0.1

Other software versions and environment

No response

If applicable, what browsers are you seeing the problem on?

No response

Relevant log output

No response

Additional information and comments

No response

SMTP tls option unsed

What happened?

The ckan.smtp.tls option is not used and should be removed.

Versions and environment

Version information

  • Helm chart: v1.0.1

Decouple database initialization

CKAN and its tool chain require databases that need specific initialization steps (mainly roles and permissions) for security reasons. Several init steps require Postgres superuser rights. At the same time, unprivileged database users need to be set for CKAN and other services (e.g. Datapusher), that are supposed to use the databases. The different databases may potentially live in separte instances (e.g. ckan-db on one server, ckan datastore on a different server).

Currently, those step can only be performed when the postgis sub-chart is used.
External databases (e.g. a managed database from a cloud provider) cannot be initialized successfully, because the native CKAN init fail, because it does not have sufficient rights.

In the future, the database initialization should be moved to a (maybe multiple?) separate Helm chart/subchart, the ckan-database-initializer.

This inititalizer should be capable to initialize one or many databases for CKAN and it's services:

  • Specify a DB superuser for each required DB instance, that can do all initialization steps:
    • Create roles and users
    • Create databases
    • Grant required access to the different DBs for each role

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.