Wagtail recommends to use redis for caching purposes. It supports various caching mechanisms like caching parts of views, sessions etc. It looks like by enabling redis, we can heavily speed up our sites without much effort.

Related links:

• http://niwinz.github.io/django-redis/latest/
• http://django-redis-cache.readthedocs.io/en/latest/intro_quick_start.html
• https://www.revsys.com/blog/2015/may/06/django-performance-simple-things/
• http://docs.wagtail.io/en/latest/advanced_topics/performance.html
• http://michal.karzynski.pl/blog/2013/07/14/using-redis-as-django-session-store-and-cache-backend/

TimeStampedModel.modified is None if the item has never been modified. This makes sense, especially considering UI where edited items get an additional "this has been edited on …" label.

However, this makes it impossible to sort by "recent changes". It might be better to set both modified and created on creation. To know weather an item has been edited you would then check for equality of these fields.

If you are not logged in the comment widget will offer a link to login. This link is without next parameter leaving you at the front page after an successful login.

In our current implementations, if a model instance is submitted using the RestAPI, that models clean method is never called.

This doesn't affect any code in a4 itself (unless I missed something). But documents in opin are affected.

It seems that DRF (since 3.0) deliberately removed the calling of full_clean. The reason is, that they don't want to mix validation and model declaration.

As a result I see three options:

• implement validators in <app_name>/validators.py and use them from Model.clean() (used by django-admin) and from the ModelSerializer
• somehow call full clean from serializer
• implement the validators in pre_save instead. That would make sure that there is no way to bypass them from any part of the code.

To find the email logo the finders.find('images/email_logo.png') method is used.

This is currently broken on the servers, as the finders function does not (cannot) look into the static/ folder created by collectstatic and the original static folders are not included in the deployment.

This can be fixed by:

1. Always rely on the static folder created by collectstatic (also in development) and open the file there without using the finders functionality
2. Include original static folders in deployment in addition to the static folder generated by collectstatic (duplicates the static files)

• add alt attribute with the filename
• if there is no image uploaded the image tag should not be there
• the label's for attribute doesn't reference the file input's id.

The first part causes the HTML to be invalid, which is part of the BITV Test "4.1.1a Valides HTML".
The third part is part of the BITV Test "3.3.2a Formularfelder richtig beschriftet".

• make attribution configurable
• validate GeoJSON (in GeoJSON field)
• rethink if we actually want GeoJSON, or store Points and MultiPolygons in Leaftlet representation
  • we want to stick to GeoJSON (decided in Scrum of Scrums)
  • was only an option to simplify validation
• document isMarkerInsidePolygon algorithm properly
• replace view mixins for list and detail with template tag to use in detail and list view
• rename display_points attribute from point to points
• don't use map_url and base_url as synonyms
• prefix all settings value with A4 or A4_MAPS
• set a better default help text for GeoJSONField
• do not hardcode height
  • add as optional parameter to template tag and widget, with old value as default, and none if None
• consistent usage of form controls in different templatetags/widgets (display_points currently uses a custom solution)
• Show rates in pins only if the rating feature is enabled

This issue is meant to track follow up tasks from the multi module changes. They result mostly from comments in #144 #153 and offline discussions.

• Update phases.md
  • Rename to modules_and_phases.md
  • Unify structure (Adapt modules structure to phases)
  • Ensure "hide features if not usable in a module" is really solved (see: #153 (comment))
• Decide if timeline should become a core feature
• Decide if phases should be sorted by 'start_date' by default (will be breaking for dashboards) (see: #153 (comment))
  • Decide and document where to expect Phases without a date. For example after every phase with a set date. This is relevant for Project previews
• Remove unused PhasesQuerySet.finished_phases
• Set request.project and document that request.module and request.project are valid request properties (see: #153 (comment))
• Decide if overlapping phases should be conceptually allowed
  • if yes: remove Project.active_phase and adapt Project.days_left
  • if no: remove deprecated comment and document the restriction
• Decide on 'days_left' display on project tile and 'phase over soon' actions.
  • Decide on days_left of project (=last phase) or current phase
  • Decide when to send phase over soon emails wit multiple overlapping modules and how to word them
• Decide if Actions should point to modules instead of projects

UX/UI says, that the big alert is much to dominant in the forms and should be shown in the helptext instead.

#53 adds code that is specific for embedding in meinberlin. This feature is currently in a test phase. After the test phase, there are two options:

• Remove the code from this repository and find a way to do embedding in meinberlin without changes to core (e.g. by defaulting to the "external" embed target.)
• Move the embedding feature (most importantly documentation) to core.

Future pull requests that contain similar changes should reference this issue so we can keep track.

A Content Security Policy (CSP) tells browsers to restrict some features. For example, it can be used to disallow inline JavaScript. django-csp provides django integration.

It would be nice if we could define a CSP in order to enhance security. The biggest step required for this is to remove all inline JavaScript. In opin, most of the work has already been done in liqd/a4-opin#775. Similar changes should also be implemented here.

Currently report emails are broken because, the templates of the app are not included.

make install fails with the following error:

./bin/python3 setup.py development
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'development'
Makefile:17: recipe for target 'install' failed
make: *** [install] Error 1

Time is optional in the backend but the html input field still gets the required attribute if the the DateTimeField is initializes with required=True
The time Widget should always be initialized without required.

Currently the comment react components are missing any error handling when a comment is submitted.
If there is a validation or connection error the user will not be informed that her comment has not been saved.
(see also liqd/a4-meinberlin#362)

On all models, methods that QuerySet should use Django's cached_property instead of property.
One example: https://github.com/liqd/adhocracy4/blob/master/adhocracy4/projects/models.py#L155

One example where this is noticed:
When showing 3 project tiles in a4-meinberlin, he claims a query is 6 times duplicated, but i think it should be "only" 3 times. You could workaround it with the template with syntax, but we should fix it here.

Currently, all client side code for API calls is in a single, monolothic file (api.js). This means that individual projects that build on adhocracy4 have no good way of extending this. As a result, api.js already contains some code that is specific to individual projects (e.g. follow, document).

We should refactor api.js to be extendable from individual projects.

The ordering feature of the used django_filter.FilterSet does not provide a default/initial ordering.

This is needed to be able to show the initial ordering in the ordering widget and to be able to have an initial ordering for models that do not have it on the model level.

A model field for richtext could bundle up the following settings:

• clean non-allowed html from input, or output or both (using bleach)
• set ck-editor as widget
• idea: mark loaded strings as safe
• idea: meinBerlin team is wrapping contents with a div (assigning a common) class. This is currently happening in the template. We could also do it in the ModelField itself.

1. https://mein.berlin.de/projects/arbeit-40-made-in-berlin/ or https://opin.me/de/projects/zbiranje-idej-za-delovanje-mladinskega-kluba-stik/
2. Follow: 0
3. Click Follow
4. Follow: 3 (or whatever it currently is)

also the comments are not displayed at all: I get an error in the developer console "Uncaught TypeError: window.adhocracy4.getCurrentPath is not a function
at Object.getLoginUrl (config.js:6)"

Might be a connected to this: af7043e

What is this?

In the Scrum of scrums on (18th July 2017) it was decided that we collect a list of bigger refactorings that we want to todo to give them back into the scrum process.

project/module mixin

Most operations in a typical adhocracy4 platform happen inside in the context of a project or a module (which also implies a project). Currently we have some ways to find those contexts

• read module_slug parameter from url (eg. creation of ideas and other items)
• read module property of a project mixin, which in turn reads the project_slug of the url (eg. listing of ideas)
• read project from project_slug slug of the url (eg. project detail view / all views extending project mixin)
• read module or project from self.object if the object is an item

The same issue exists to some extend also for forms, that want to create an item.

What would be a good solution:

• implement an view(-mixin/baseclass) that provides self.project and self.module
  • it should automatically try to get the values from their respective url slugs, or self.object if it has an project or module property
  • it should allow to override the project or module property if needed
  • it should provide those values to the template via get_context
  • it should provide the most specific value to the form via get_form_kwargs (possibly introspecting the form, if it needs that value)
• implement an model form(-mixin/baseclass) that receives the project or module parameter from the view
  • it should set that value as a property of a for
  • it should on save set the project or module property of the saved object (maybe that needs to be configurable)
  • it should target create forms primarily (maybe something similar could be done for update forms, but that should read the value from the instance instead)

makeing item an abstract model

Currently we are using item as an concrete model (with it own table). This type of inheritance is discouraged in many Django guides. In addition we never (or seldom) use that model, so is there really a need for it.

pro

• don't use multi table inheritance

contra

• lose ability to give an unqiue number to each item

If for example I fill the form of the face to face module wrongly and submit (with inspect element remove the browser validation before), the http status code is 200

In which other places that happens must be investigated.

This is relevant for example in testing, where I can check by the http status code if the form validation was successfully.

See the screenshot:

But most important of all:
I already spend so much time looking for a paint program for linux, how can i draw red circles on an image? Gimp is completely overkill for this.

The way comments are structured has been bothering me for quite a long time. In my opinion, it's weird how the "answer" link is positioned in mid air, losing all visual connection.

Here is a screenshot for comparison.
On the left hand side: status quo on Opin
Right hand side: design for Advocate Europe

I think keeping all actions on one side will make the whole thing easier to digest and visually more consistent. At the end of the day, more motivating to participate!

This way we could - in addition to the changes in the FE code - also implement a server side secure saving of text fields. I would suggest to add the code to the save method of the UserGeneratedContentModel https://github.com/liqd/adhocracy4/blob/master/adhocracy4/models/base.py#L20

@slomo @xi @2e2a what do you think?

Once they are show/hidden with javascript, the aria-hidden attribute is set correctly. But we need to set the initial value manually.

One of the restrictions for embedding is "Do not trigger navigation from JavaScript". This is broken in the rating code:

This results in the following issue:

• Go to a project where a normal user can rate
• Log out
• Try to rate

The expected outcome would be to get a login popup. Instead the iframe gets redirected to /accounts/login/ on the embedding domain.

We have to set tile_image in the IMAGE_ALIASES in the settings. There should be a default/fallback. @slomo

but it should be streamed

here is the code https://github.com/liqd/adhocracy4/blob/master/adhocracy4/exports/views.py#L19
It should be ok for a while, since we are hosting our own servers we can have plenty of memory per request and its a feature that is not used a lot at once. So this is also a problem where we wait for it come to us before solving it.
Ha but on the other hand since this should be an open source base component we could fix it because in other setups this would not work (more db entries, different hosting)

The easiert solution is probably to buffer it in a file, send the file, then delete it

version number is always the same: adhocracy4-0.0.0.dev1
when using this library in requirements.txt, and another version is pinned via a git commit hash, it does not get updated by pip because the version number stays the same.

Quick solution: always bump the version number when significant changes are made

When posting a comment or rate, object_pk and content_type_idcan be set to any project item. The phase permission of the project are not being checked.

Currently the API is checking if a user is authenticated. This is not sufficient to disallow subscribing to private projects.

Usind ViewSetRulesPermission was not possible with the current API, as it consolidates two endpoints in one view with different permission objects:

1. List follows of a user
2. Put follows for a projects

Need to refactor the API to fix the permissions.

Consider discussion under: liqd/a4-opin#510

error occurs when installing requirements

When using models.query.CommentableQuerySet.annotate_comment_count only direct comments on the item are counted, not the replies to comments.
I would expect the comment_count to include the sum of all comments, including the replies.
What is the intended behavior?

Currently there is the PhasesQuerySet which defines active_phases and finished_phases, the Project model with future_phases and past_phases and the Module model with future_phases and past_phases properties that are doing very similiar things
I think we should unify the properties to use the PhasesQuerySet and extend the QuerySet with default filters and a future phases method

Take error message from REST response and display in existing error message alert. On top it probably makes sense to limit the field as well.

Currently Uploading|RichTextFields are cleaned from unallowed html by using the models save() method [1].
As this is prone for errors because devs forget to call the clean method manually we should extend Uploading|RichTextFields to auto clean their html.
This could be achieved by using a custom field that overwrites the pre_save method [2] as described by [3].

[1] https://github.com/liqd/a4-meinberlin/blob/master/meinberlin/apps/ideas/models.py#L44
[2] https://docs.djangoproject.com/en/1.11/ref/models/fields/#django.db.models.Field.pre_save
[3] https://docs.djangoproject.com/en/1.11/howto/custom-model-fields/#preprocessing-values-before-saving

The categories form could only add the category field if there are any categories to choose from. So the field can also be make required removing the need to set the empty_label to None. Also the show_categories helper could go and be replaced by {% if form.category %} .

For OPIN there is also the need make categories optional. Either the form should have a property. An alternative maybe an method that will be passed the module, so we can decide on a per module basis if categories are optional.

Sometimes tests fail because emails don't allow newlines in there subject. This results from fakers behavior for generating "text". If the requested text should have more or equal then 100 chars it adds newlines between sentences. If no char limit is set it defaults to 200.
We are using the text generator with a char limit of 120 for the Organisation and Project name where we expect single line texts. As the project name is part of email notification subjects the tests fail.
In other places we use the "name" faker instead of the the "text" faker but the "name" faker does not allow to limit the number of chars. And it returns relatively short human names which is not what we expect for project names (which are actually titles).

I see 4 fixes for the test problems:

1. Replace "text" with "name" or "sentence" when expecting single lines.
   As it is not possible to limit the number of chars, the faker may generate data that is not valid in regard to the database.
2. Set the char limit to <100 to prevent newlines, as we have internal knowledge of the faker code. But we may never now
3. Write a custom provider "title" which allows to limit the char number but does not create newlines.
   Unfortunately the custom provider would have to be added in every projects conftest.py and creating the example provider as documented by faker did not work :(
4. File a bug report on faker to add a char limit to the "name" or "sentence" or a new "title" and hope for the best. In the meantime we could fall back to 1.

is true for all images using the image field from a4 and all errors occuring in the respective form