https://react-hook-form.com/migrate-v6-to-v7/

#400

fyi @aaronelliotross It might make the stripe form code nicer (I had to deal with some incompatibility and weird redraw)

deal with the various EC magic text, with or without
, with or without new line...

@marcinkoziej could you test on your side?

I've made the icons a bit smaller and adjusted the Whatsapp color (according to the log of the lib;)

unit tests (jest seems to be the trendy framework this week) and e2e ones

A Code of Conduct for Open Source Projects

https://www.contributor-covenant.org/

@marcinkoziej, any objection?

right now, it's 320k unzipped. generate a zip+nginx rule to serve it if the browser supports compress

We need a clear dashboard for campaigners to understand what's happening on their campaign. On a later stage when we have lots of users, we will probably shift the focus toward benchmarks (how your campaign compare to other similar ones), but right now let's have them analysed separately only

It is especially tricky because we should be providing useful informations, even for campaigners that have no experience in online campaigning

I think one of the most useful key indicator is the bounce rate ( how many visitors do NOT take an action) and being able to see that per medium/source (random visitor is less likely to convert than one coming from a mailing)

We have the number of action taken and supporters (action taker), but obviously we also need to know how many visitors we have

Several options for that:

1. Parse and analyse the nginx log of the widget (filter by referer domain?), using go access or something?
2. Add a counting/log feature on our api server (assumption: visitors == those that hit getCount)
3. Use google analytics
4. Install a web analytics (goatcounter or plausible.io tickle me the right way)

for privacy reasons, 3. has to be discarded, but not sure for the rest. @marcinkoziej what do you think?

We need to find a balance between having a big unmaintainable mess with a gazillion options and something that is too rigid.

On the backend, we have mostly sorted it out (famous last words) with a tool that can process two "abstract" payload from the campaign website: "add action+personal data+consent" (eg a petition signature) and "add action" (eg a share on the previous signature). The core process in then encrypting the data and pushing it it to various other micro-services that will take care of other actions (send an email, add it to the CRM, generate a PDF...)

What we need now to tackle is how to customise the front-end (widget).

We have 3 main options:

1. add css/js to customise from the campaign page (the site that embed the widget)
2. set options to the widget generator to include/exclude some features
3. fork and have your own widget with exactly what you want

from the campaign page

The widget is embedding html+js directly on the page (no iframe), so you are free to modify the layout or add extra features (eg add a lookup on the zipcode to pre-fill the city field)

• warning *: our terms and conditions prevent you to alter the "consent to be contacted part" of the form. Beside violating our term, you'd doing something illegal (GDPR, you risk fines big enough to bankrupt you).

the widget has a few hooks and you can provide your own function at the end of each step (eg "call my function displayFirework() after the user signs). or you can provide your own custom steps (eg "call my donate tool after the share step").

parameters to our widget generator

Right now, the customisation options are put into a config/your-org.yaml file, and used in the build step $widget=your-org yarn build that creates the widget you embed.

What you can customise beside the "static texts" (eg the language, name of your organisation...) is what steps you want in the journey (eg "petition,share" or "floating action button, dialog with petition" without share

At the technical level, it defines what components to include in the widget (each step is a react component)

It allows to have as many "special" steps as needed (eg "ECI signature", "CH initiative"...) and they won't be included in the code of the widget unless it's one of the step

Where I'm not sure is how to enable/disable features within a single step. For instance:

• remove the ip to country geolookup
• make the last name mandatory
• remove facebook from the share option
• ...

one option might be to have a line in the yaml

• {step}_features= list eg.
  petition_features=first_name_optional, last_name_mandatory, country_geolookup
  share_features=facebook,twitter,reddit

But it means that if we introduce a new feature, the widgets won't benefit "out of the box" without having to modify the config.

Might be better to have a multitude of
{step}_{feature} = true | false and if we do not have that line, it's the default?

in the code, we'll have a multitude of if (widget.step.feature) {do the feature}

That might be a bit of a hell to manage too

Don't know @marcinkoziej ?

fork

you can customise the hell out of your widget the way you want, but we strongly suggest you to check with us first because

• you might struggle adding new features that landed on the main branch
• we might change the API (but it's graphql, so we will do our best to keep compatibility)
• your own custom thingie might benefit other organisations. Contributing back to the greater good gives you such a nice and warm feeling, that's be a shame to miss it

X+

Hi,

Right now, it saves successful donations only.

There are plenty of other events that might be useful to record:

• click on a payment provider (initialise the payment processor workflow)
• aborted donations (we might not always get that event, ie if the supporter clicks to donate on paypal and doesn't cancel, but doesn't finish either, we will never know)
• something about clicking on the pre-selection amount?
• something about switching to recurring/one off?

When signing the ECI, the snackbar eci:congratulations.successfully-title isn't clear enough and we display it on step 2 (join/email step)

For some widgets, you want to be able to remove some fields, like the country drop-down or the comment. To do that:

In the config of the actionpage:

  "component": {
    "register": {
      "field": {
        "organisation": true, // by default false, it's for open letters
        "postcode": false,
        "country": false,
        "comment": false,
      }
    },

Trying to bundle together various needs expressed in multiple issues :

• Send signatories/action taken to a crm
• sending emails to the targets of an action (mail 2 target)
• sending progress updates (we reached 50k signatures today) to the organisers, supporters or targets
• double optin emails?

Some, campaigns will need to be able to personalise (eg send both the total of signatures and the one from their constituencies to an mp) or have different thresholds or...)

Rather than trying to have everything done in the core and dealing with a gazillion options in the ui, let's instead have Web hooks and have external services dealing with that

I am not keen on calling the hook for every signature, because it forces every external services to implement some kind of queuing or we risk dos them

What I see is to have a web hook and call it max every X seconds to notify "we got one or more new actions, and let the external system call our API to get the details based on their needs:

• one count API (with filter on timestamp?)" how many actions" (that is the API that can be reused by the frontend for the progress counter)

• one aggregate per action page (option to aggregate per day too?) can be used for the dashboard too

• one fetch actions (with limit and start id/timestamp) used for sync with crm or email 2 target or whatever funky action they want to set up

• one aggregate per target?

Dependabot couldn't parse the package-lock.json found at /package-lock.json.

The error Dependabot encountered was:

Dependabot::DependencyFileNotParseable

View the update logs.

We sort the countries based on the iso code (I think), turns out humans find is not very helpful to sort on a text that isn't even displayed

As reported by Sebastian

In the widget, the choice of countries from the drop-down is non-intuitive because they are not (re-)sorted. In the German version, Germany becomes Deutschland but appears after Tschechien (Czech Republic) and before Dänemark (Denmark). The order of countries seems to be the same in every language and I don’t think the current order is fully correct for any language. For example I don’t know in which language Denmark would appear directly after Germany. German sorting rules do not place Dä-(nemark) after De-(utschland) as the Ä umlaut is sorted as A or AE (sadly two traditions are in use). When trying out a few major languages I could not find any one where Germany would come just before Denmark. Of course Germany becomes Allemagne in French or Alemania in Spanish but that again would produce a very different placement. Even when expressing both these country names in their respective languages – which I don’t think would help anyone – gets you the names Danmark and Deutschland which would again require Denmark to come first.

So this German is now calling for Denmark to come before Germany. For balance, Austria needs to move way below Germany in the German list, as it becomes Österreich. :slight_smile: More to the point, I think the drop-down should always be re-sorted according to the country names and sorting rules of the respective language. I don’t see any value in preserving whatever order across languages.

https://api.country.is/ is privacy friendly and we can run our own server, we should use it ;)

An open source version
https://github.com/gzuidhof/friendly-challenge
a better re-captcha
https://www.hcaptcha.com/

The original english version of strings src/locales/en/eci.json have for some reason underscores instead of periods (full stops). This is what caught my eye while I was translating into Croatian on Weblate. Because of that, sentences don't end with a period, URL addresses don't work.

Two examples:

"oct_error_hr_postalcode": "Invalid format_ Format required: NNNNN (N = digit)",
should be
"oct_error_hr_postalcode": "Invalid format. Format required: NNNNN (N = digit)",

"s2_link": "https://europa_eu/citizens-initiative/faq_en#Giving-support",
should be
"s2_link": "https://europa.eu/citizens-initiative/faq_en#Giving-support",

I don't know about thouse leading "markups", like s2_link - should that be s2.link?

It seems to me, that with the Commits on Oct 25, 2020 ALL periods have been replaced by underscores in the eci.json file. When correcting, whatch out for underscores which are necessary, like in URL /faq_en#Giving-support or in links target="_blank"!

Integrate the widget means having to try the action several time. There is a risk you end up with test data (and wrong counters).

What about we introduce a test mode?
On the frontend, it could:

• be triggered by a "?proca_test" param
• OR if the widget is in "test" mode (eg not validated by the lead organisation yet)
• display an alert "Running on test mode, disable once finished"
• have some kind of visual sign, like a black and yellow stripped background?
  
• send the test flag to the backend (as part of the extension?)
• change the action type (eg instead of register, have "test-register"?)

On the backend, not quite sure how to handle that

• at least have a different count of signatures on test mode or not?
• export the test signatures on the proca-cli (always? only if flag "prod=true" - false by default ?)

As requested as an option, we can

• install a proxy counter on the campaign website (a cron job that fetches an store a local copy of the API from proca)
• modify the widget to use this url to fetch the counter

the rationale being that some orgs don't want proca to know about every visitor they have

However, having the number of visitors as an aggregate is super useful for campaigners, for something as trivial as knowing the conversion rate and try to improve it (we changed the button from blue to red, does it make people sign more?) or more segmentation (we have zero conversion from mobile users, we might have a display problem)

@kantorkel, would you see any way to have a privacy friendly way of doing that?

we do not log the IP address of the visitor, don't put session cookies and are only interested on aggregate data.

what do you think of the approach like
https://plausible.io/ or https://usefathom.com/ ?

The aim is to store encrypted signatures on the server. The server should only have the public key, ie. shouldn't be able to decrypt the signatures, but only to push the encrypted payload to the next application (or allow it to be downloaded and decrypted on the client)

client side encryption

The most secure: the signature is encrypted client side and sent encrypted to the server
Two options: either using an elliptic curse crypto directly, eg:
https://github.com/bitchan/eccrypto
or gpg
https://github.com/openpgpjs/openpgpjs#platform-support

From a quick browsing, it seems that it's only compatible with reasonably recent browsers. Moreover as we do not have the private key on the server, if there is any compatibility issues with a specific browser, we might end up storing undecypherable garbage and not realise it until after several months

server side encryption

the data is sent over https, and the server does the encryption of data before saving it

@marcinkoziej for mvp, I think it's safer to only implement the server side.

They are various options on how to style the form and how to display it.

We need to make them easier and generate the code

We are almost there anyway: instead of encrypting the data as soon as we receive it on the server, do it from the browser

A few changes:

• bundle libsodium on the widget
• have the public key of the organisation(s) in the widget (or fetch it part of the getCount call)?
• the browser sends both the encrypted payload and the "in the clear" action elements that need to stay in the clear (eg firstname+country+comment for the "latest signatories") and email (if the campaign is configured to send a thank you or double opt-in).
• hash/fingerprint? or does it mean we can only limit end2end to dedupe on the email?

Is there a way we can guaranty that we effectively delete the emails once sent?

Is there a way to prevent "spam" with folks that would send random strings as fingerprints?

@marcinkoziej

Dependabot couldn't parse the package-lock.json found at /package-lock.json.

The error Dependabot encountered was:

Dependabot::DependencyFileNotParseable

View the update logs.

This is a new feature to try to decrease the number of invalid emails

0. check if the domain name exists and if it has an mx record
1. build a list of the most common domain names for emails (we have partners that have done that already)
2. check if the address looks like a typo of the name (eg supporter is xavier dutoit and email [email protected])
   ...
3. check if the thank you email is properly received
   ...

and use that to warn the user and offer them the option to correct their email

Goal: we only compile necessary steps, languages into a widget, and on build also some other customizations are passed (branding color, etc)

Proposal to achieve it using dependency injection: instead of using Webpack resolution hack.

Campax widget:

collect.campax.org.js (entry file)

import RefernedumStep from "campax/component/referendum_step"
import ShareStep from "base/component/share_step"
import WidgetBuilder from "base/widget_builder"
import en from "i18n/en"
import it from "i18n/it"

widget = WidgetBuilder({"referendum": ReferendumStep, "share": ShareStep}, ["referendum", "share"], [en,it])
widget.attachToPage().run()
...

^^
|
`-- so this is processed and you cat a compiled bundle with just needed steps and languages, but all "moving parts" are defined in this entry point file.

The entry point file would be generated from tamplete, using the yaml config for each widget.
The advantage here is that there is no resolve magic, and such code can be compiled with any bundler, it can be properly introspected by an IDE, etc

This issue is an overview of next step work on Proca Widget code structure (does not involve discussing features etc)

Builder / bundler

• Choosing a faster bundler then Webpack. We had Esbuild in mind because it is advertised as super fast (like 100x faster). However, after some investigation it looks like the performance gains are due to drastic feature cuts of esbuild. For example, it achieves compiling Typescript fast by ... ignoring types and just stripping them into JS. It also does not have features we want to keep like a dev server (there is a supposedly compatible dev server by modern-web, but it does not work, seems outdated). When comparing Webpack to Parcel, the new feature full bundler, it looks like Parcel does not have a win (depends on the test). There is also Rollup which I have not reviewed nor used much, but seems popular. Considering huge community support and more standard set of plugins, I think Webpack is still unbeatable choice.
• Webpack configuration - We had intention to stop using Create React App and I agree here. CRA is best for it's use case, that is a SPA. We are building custom widgets and JS module at the same time. What CRA is doing now is that we have one set of config for babel compilation when CRA is building the code (the babel config is built in) and another in .babelrc when we build the module. I think we should remove CRA altogether, and use one standard set of build configs for every kind of build. I would use webpack via API for greater control, and being able to pass customization options at compile time (AP config, selecting i18n etc).
• Staged build - The proca widget build is actually staged, but at the moment there is not a clear separation between code running at different stage. For instance, we process AP config both at build time, and at runtime. More over, we use old JS in build stage because it runs in webpack process. My proposal is to: 1. write our build scripts in ESnext and build them with yarn bootstrap into bin/ (only need to be done once). 2. try to keep all the customization in entry file (index.js) 3. have a separate module.js which is an entry file for npm module.

Widget interface (browser and module)

• Standardise Proca Widget JS/browser API. Right now there are a few APIs for Proca widget. They rely on a global window.proca object and use nonstandard callback mechanism. Would be great to use EventTarget API of DOM node that widget is attached to, and using events to communicate with the widget.
• Questions on how stable is the API - can we design an event API that is forward-looking and not likely to change/break?

Just noticed the org name is black in dark mode, see screenshot:

Possibly Material UI has some sort of support/helpers for light/dark mode - are we using it?

Hi!
The form collects address field but it is not sent to the API.

Single name field

• supports any kind of name (many segments, or names with titles, etc), does not have to figure out
• Name is opaque piece of data with no semantic parts

Firstname + Lastname

• gives information on how to address people (Hey Marcin! in the email). Is this kind of personalization a hard requirement in digital campaigning?
• is arbitrary (2 fields, in that order)

Approaches

• Use Firstname and Lastname, if people put the titles in, then we'll have garbage in data
• Use Names + some sort of AI to figure out the name (https://github.com/namsor - but paid)
• Use Names + UI that will ask the user how they would like to be addressed

Form does not validated if captcha is saved.
The server returns an error with code: "bad_captcha" but this is ignored.

To be tried...

When displayed on a mobile, the form (usually set in the second column) is hard to see, it would be great that instead of the form, we have a FAB that opens the signature dialog

Not on the viewport width

at least one per language, but as well various custom ones (eg with different list of fields)

I'm not sure what's the best path: continuing with create-native/webpack, use gatsby, use next.js?

(I was discussing #4 with @francesmcmullin on a "candidate, do pledge X" campaign they ran for the elections)

they have another criteria when selecting their target beside the constituency (via a map): has the target already signed the pledge?

the logic being that if the target has already agreed to the call of the campaign, there is no point asking our supporters to contact them.

For this campaign, or for campaigns asking MPs to vote something specific, we have in essence two campaigns/actions:

1. the "normal" campaign asking our members to contact their representative (or candidate): email/tweet type of action
   In this campaign, we need to be able to filter out the candidates that already agree to do the right thing and support the campaign (eg. their is no point having our supporters sending them tons of emails if they already took position and said they would vote what we want)

so what we need is...

2. the "for target" campaign tool

This is kind of a petition with a twist: who can sign this pledge is restricted to a list of pre-identified target

and we need to have an API to fetch the targets that have signed the pledge (and possibly those that we are sure will never be convinced to do the right thing either)

Possible implementation:
upload a list of targets (name + email?), create them as "pledge action" with a status "not taken"
the action page is a simple select/autocomplete (instead of being able to fill the name like a normal petition)

once selected, I have a "consent like" radio button:

• yes, I support this campaign and will do X
• no, I oppose this campaign and will not do X
• comment field

here I can't see how to avoid a double opt-in (send an email to the target "someone, hopefully you has signed the pledge, click here to confirm)

Once done, the status is updated from "not taken" to either "support" or "oppose"

the API returns the list of targets with their status, so the UI of the main campaign can take that into account and remove from the target list those that support the campaign (and display the target comment)

I'm not sure at all it's for MVP, but would be a darn cool thing to have ;)

Have a better system to keep a global state

Hi,

right now, the key is the actionPage, it's probably not ideal as it's easy to copy/paste and get the wrong one, therefore adding it to the wrong campaign/organisation ;(

New rule: each action page has their own widget, eg on a url widget.proca.foundation/campaign/org/id

the widget contains actionPage details, (eg language, what consent to collect, campaign, organisation,url) and check if the url of the page embedding the widget is matching the url of the actionPage

If it's the case, everything works as expected

if not matching, redirect to a "widget builder page" that offers two options:

1. create a new actionPage for the organisation owning the previous widget (eg because it's embedded in a different page/website)
2. register as a new organisation

then the widget is created and it's back to the normal case

What kind of workflow to validate?

By default, it should:

• create the widget js code
• send an email notification to the partner organisation (if new actionPage)
• send an email notification to the campaign owner (if new organisation)

Makes sense @marcinkoziej?

To make it easier to configure the widget (from outside the widget) introduce hooks:

<div "proca-form" data-loader="myLoader"></div>

What it does is pause the display and call the myLoader function.
In that function, you can add "proca.hook" that will be called by various widgets

@marcinkoziej what do you think?

  function myLoader () {
    proca.hook("twitter","load", function(list){
      list[0].description ="bozo";
      console.log(list);
    });
    proca.set("iplocate",true);
    console.log("loaded");
    proca.go(); // continue to the next step
  }

Several things:

1. add a js library to manage it (i18next?)
2. modify the form/things to use this library
3. generate/translate. We can probably find a way to extract from existing free software that have the same fields (first name, last name...). civicrm? is there a public/generic one (eg using wikipedia?)
4. generate static widgets per language

Hi,

There is a need to change elements in the page embedding the widget - beside displaying the widget obviously-

Things like updating a counter in the content of the page, put the title of the petition or the text. In React, there is a notion of "portal", that is displaying a component outside of the main container and we are lifting it

We introduce a "portal" item in the config, that lists couple (components to be displayed, the dom selector to display them, {optional param})

eg [["Counter",".total-counter"],["eci/Title",".eci-title"]]

Is there a way of displaying server errors in the front end? As an example, the backend could have a validation mechanism for email addresses. If it fails, how does that get passed back to the front end and displayed?

Why we collect area info?

• To better target further CTAs – relevancy wrt to political constituency (my representative on level X) or space (an event happening in a place reachable from me)
• We do not need a very exact location beyond that.

Caveats using postcodes

• Postcodes may not exist - In some countries postcodes do not exists, and we had to ask people for province name instead - they are very big and might not be useful for relevancy
• Postcodes may not imply constituency (Brussels example where (postcode, language) tuple implies constituency
• Postcodes can be ambiguous in inferring political areas (Poland example - there are postcodes that overlap even the biggest administrative division level (Voyevodship). For the given administrative division level, postcode -> area mapping is not defined well

Other approaches

• using device location to get a proper coordinates?
• using a map UI for people to find their proper address?
• instead of getting an area hint, and calculating tags from it, exposing the "tags" somehow (like, asking them for language, their constiuency etc). This could be hard/annoying (people do not always know their constituency for instance)
• getting access to some good administrative data to ask better question? (maybe street number + city)
• Should we support alternative characteristics that are relevant for this? Similar to language? Workplace (Organize use case with speakout), religion, ethnicity in some countries would be relevant?

for now EU28?
V2: make it configurable per petition

At the moment, when a client opens a campaign site, the number of signatures is requested from the API. This leaks the IP addresses of all visitors to the API provider. It would be nice if, instead, the server-side of the front-end would make this request.

(Sorry if ended up posting this in the wrong repo)

Two things to tune on webpack:

• have a different prefix to the default class (go away from MuiXXX and use ProcaXXX)
• have a different prefix to the custom style (go aways from jssNN and use procaNNN)

This mostly impacts when the containing website is done on react (eg next.js or gatsby), so nothing urgent ;)

As discussed in #374, it would be good to allow multiple widgets to be embedded into the same page.

This is a non exclusive list of things that might be hardcoded (ie clash between two widgets)

• webpack wraps everything under the global object "proca", it needs to be made customisable, somehow
• webpack embed react+reactdom+materialui into the widget. it might be good to have a mode to say they are external? (+ a way to reference/reuse them?).
• the communication between the external world and proca are channeled through a "proca-listener"
• to prevent loading twice the widget, it checks if there is already a script "#proca"
• the widget is displayed into the dom ".proca-widget"
• I read the proca-text dom elements to overwrite the locales in the widget. It might be good to have the same overwrite for all the embedded widgets, or it might not

It seems that quite a few users struggle to do what I thought would be a basic thing: have a two layout on their website and put the text of the campaign on the left and the signature on the right

To solve that, I'll set up a mode so they write on their site:

<div class="proca-widget">
This is the text that needs to appear on the left hand column, bla sign bla
</div>

and it will generate and add the needed css

<div container>
<div left column>
This is the text that needs to appear on the left hand column, bla sign bla
</div>
<div right column>
the widget
</div>

if the proca-widget div is empty, it will still only add the widget, as previously

if mobile (and FAB), the left column is full width

Required fields for each country:
https://github.com/fixthestatusquo/proca-backend/blob/feature/eci_data/utils/ECI/required.csv

notes:

• for id property group, one of them is required - and needs a select if more then one for user to specify

Rules for each country/fields:
https://github.com/fixthestatusquo/proca-backend/blob/feature/eci_data/utils/ECI/rules.yaml

notes:

• "common" applies to all countries.
• not sure what "skippable" means. It does not correlate with the required status (in CSV) - eg. postal.code is not required by all countries, and is marked as skippable, and street address is also not required by all countries, but is marked as non-skippable.
• I think the rules are applied if the property was asked in the form, so 1. check if all rules required in csv are collected 2. from those collected. check if they satisfy the rules.

It would be nice if one could configure whether the client's IP address is send to the country API or not. As an alternative, a setting like default country could be introduced.

This is only the front end part. It takes care of adding the petition form into your campaigning website, but you need a backend to collect the actions and push them to your CRM (or let you download them)

To make it more versatile and flexible, the front end and the back end are hosted separately, with a clear API between the two.

@marcinkoziej is looking at the backend and will share the url