This covers @andylolz's #120 and adds more functionality. It could also be used to record winners and I expect this would be useful upstream too.

The idea is to have a candidate_status value in c.dc with a drop-down so users can choose:
-incumbent*
-rumoured
-expected
-announced
-confirmed (only once SoPN has been published aka post has been locked)
--successful
--unsuccessful
--withdrawn
*auto add all existing MPs for GE2017, then if required users can remove them and add a source for the definitely-not-standing info eg http://www.bbc.co.uk/news/uk-politics-39658426

The real value would come if WCIVF displayed 'expected', 'announced', 'confirmed' etc so users of that site could quickly and easily assess the quality/validity of that candidate's inclusion.

Perhaps there's no need for both rumoured and expected, but I feel the overall principle is sound and would prove useful.

https://candidates.democracyclub.org.uk/moderation/suggest-lock/
for review of areas with multiple divisions on a single SOPN PDF, it would be useful to have the list of areas in alphabetical order on a single page to avoid repeated searches of the same PDF

I'm receiving the following errors in Chrome developer tools when trying to add a new candidate to Bristol West:

https://candidates.democracyclub.org.uk/election/parl.2017-06-08/post/WMC:E14000602/bristol-west

all.b85c78f532e6.js:26 Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience. For more help, check https://xhr.spec.whatwg.org/.
send @ all.b85c78f532e6.js:26

all.b85c78f532e6.js:451 Uncaught TypeError: $(...).not(...).not(...).select2 is not a function
    at k (all.b85c78f532e6.js:451)
    at Object.success (all.b85c78f532e6.js:451)
    at cf (all.b85c78f532e6.js:26)
    at Object.fireWith [as resolveWith] (all.b85c78f532e6.js:26)
    at co (all.b85c78f532e6.js:26)
    at XMLHttpRequest.cf (all.b85c78f532e6.js:26)

VM533:451 Uncaught TypeError: $(...).not(...).not(...).select2 is not a function
    at k (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:451:181344)
    at Object.eval [as success] (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:451:183083)
    at cf (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:26:26306)
    at Object.fireWith [as resolveWith] (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:26:27315)
    at co (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:26:93222)
    at XMLHttpRequest.cf (eval at <anonymous> (all.b85c78f532e6.js:15), <anonymous>:26:97769)

The information I'm trying to add is as follows:

Full name: Sarah Margaret 'Molly' Scott Cato
Email: [email protected]
Party in 2017 General Election: Green Party
Add new election: 2017 General Election (2017-06-08)

(source: http://www.bristolgreenparty.org.uk/bristol-green-party-announce-parliamentary-candidates)

Shortly after selecting from the "add new election" dropdown I see what appears to be a version of the full page being rendered, as if the response was expecting a HTML snippet but was instead loading a full page layout. This seems to match up with what the Network tab is showing:

I've managed to replicate this in a private browsing window. My first thought was the quotation marks in the name but I've tried all variations of those and the issue persists. Please let me know if there's anything else I can do to help.

on pages for people who were past candidates, e.g.
https://candidates.democracyclub.org.uk/person/47/julian-huppert
at the top of the page, it says "Candidate for Cambridge in 2015 General Election".

If the election has passed in which that person was a candidate, it should say "previously candidate for..."

(this would make it simpler when deduplicating people in the bulk add system)

where there are multiple wards on a single nomination paper (e.g. https://candidates.democracyclub.org.uk/upload_document/5008/ ) that page should say at the top Nomination Paper for XXX ward (and YY others), to reduce transcription errors where people don't notice there are multiple wards in a single paper and start in the middle

Not totally convinced by this one, but interested in others' thoughts:

I've done the mission, from the Statement as to Persons Nominated. I didn't know what to put as source, so used the name of the file, SATPN_-_CCS2.pdf - if you could add some clarity on the form as to what should be used here, it would be good.

When bulk entering candidates and showing matches for previous candidates, show the body to which they stood last time (a Sam Smith on the other side of the country is unlikely to be the same one, but in the ward next door it is likely to be)

On https://candidates.democracyclub.org.uk/help/api, the 2015 candidate data download is listed under the 2010 list:

@andylolz reckons this is an error in Election.group_and_order_elections (https://github.com/DemocracyClub/yournextrepresentative/blob/12aff56a/elections/models.py#L99-L176) somewhere.

These are in EveryElection, but not in Candidates/WCIVF. While it's unlikely to be very publicly useful, I think these should be included for the sake of completeness.

They are
parl.stoke-on-trent-central.2017-02-23
parl.copeland.2017-02-23

There's a mistake in the data model that @symroe pointed out: it's posts that are locked (or suggested for locking), and posts exist in multiple elections. So if a post for a constituency MP was locked after finding all the candidates in the 2015 General election, that locks the post even for adding candidates in the 2017 General election.

Really the candidates_locked attribute should be on PostExtraElection, which holds attributes for that post in a particular election. Similarly suggestions for post locking should be linked to a post / election combination, not just a post.

We should add migrations to switch to this fixed data model. (They won't be easily reversible unfortunately.)

Or possibly replace with a message about 2017 plans for results?

As evidenced here: https://candidates.democracyclub.org.uk/person/6735/juliana-brimicombe — might also be happening for other CP candidates..

This is a familiar piece of feedback:

2. I have no idea what my username is and can't figure out how to retrieve it

And several people try to log in using their email and don't even know that they're supposed to be using their username.

Ergo, can we allow people to log in with their email address at the box below?

Low priority, but I've spotted a few issues in the Candidacies section. For example, here the Copeland label should be something like "2017 Copeland parliamentary by-election"

Others miss the date too — this one for 2017 and 2016.

It’s useful to direct people to the candidate search, but it’s really not obvious on the homepage.

Fortunately, the search page is quite nice! But if there’s no query string, the input field value is “None”:
https://candidates.democracyclub.org.uk/search

on https://candidates.democracyclub.org.uk/moderation/suggest-lock/ place any entry where the currently logged in user suggested locking at the bottom of the list. That reduces the likelihood that an individual reviews their own entries.

We've been getting quite a lot of these error emails recently:

Internal Server Error: /bulk_adding/parl.2017-06-08/WMC:S14000004/
Traceback (most recent call last):
  File "/var/www/ynr/env/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 132, in get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/var/www/ynr/env/local/lib/python2.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/var/www/ynr/env/local/lib/python2.7/site-packages/braces/views/_access.py", line 64, in dispatch
    request, *args, **kwargs)
  File "/var/www/ynr/env/local/lib/python2.7/site-packages/django/views/generic/base.py", line 89, in dispatch
    return handler(request, *args, **kwargs)
  File "/var/www/ynr/env/local/lib/python2.7/site-packages/django/views/generic/base.py", line 158, in get
    context = self.get_context_data(**kwargs)
  File "/var/www/ynr/code/bulk_adding/views.py", line 68, in get_context_data
    'source': context['official_document'].source_url,
AttributeError: 'NoneType' object has no attribute 'source_url'

This is from a GET request with an empty QueryDict.

I've just merged two versions of Prem Goyal together: https://candidates.democracyclub.org.uk/person/5417/prem-goyal (formerly also /20319)

It looks sadly like we've lost the result (0 votes - elected!) attributed to Prem for the Bishopsgate — City of London election.

A rare example, I suppose!

https://candidates.democracyclub.org.uk/person/2111

The following candidacy (and result) is missing:

Shepway South ward Green Party – Maidstone local election
Not elected (25 votes).

(I broke this, sorry!)

The version history for https://candidates.democracyclub.org.uk/person/2111/stuart-jeffery includes the following revert:

local.maidstone.2016-05-05 was a locked post at the time of performing the reversion, so perhaps I should have been prevented from performing the operation.

Refs #118.

I was investigating why the fix for #51 was being immediately undone by uk_create_elections_from_every_election.

I believe this is related to some of those posts being linked to more than one election initially. For example, let's look at the the post with label "Anderston/City":

In [3]: post = Post.objects.get(label='Anderston/City')

In [4]: post.extra.elections.all()
Out[4]: [<Election: Glasgow by-election: Anderston City ward>, <Election: Glasgow City Council local election>]

When importing from EveryElection, this post comes up for local.glasgow-city.2017-05-04 (from https://elections.democracyclub.org.uk/api/elections?group_type=organisation ) This post is found to exist, but all its PostExtraElection objects are deleted by

I think the fix here is something like changing those lines to:

PostExtraElection.objects.update_or_create(
    postextra=post_extra,
    election=election,
    defaults={'winner_count': winner_count}
)

What do you think, @symroe? I'm not completely confident about what the scope of data coming from Every Election is, so I'm not sure if this the right fix, or a sign of something wrong in the data. (I should have thought about this when reviewing, of course - sorry...)

Feedback:

Couldn't find a way to register

Quick attempt at solution and light copy edits (also goes with issue #84 )

There's no similar problem in WM (this works fine) so I suspect the problem is caused by the list of current posts including:
Greater Manchester
Liverpool City Region
and also:
Mayor of Greater Manchester
Mayor of Liverpool City Region

Unfortunately this search leads directly to https://candidates.democracyclub.org.uk/areas/combined-authority--combined-authority:greater-manchester-ca
Which states: Known candidates for Greater Manchester in the Greater Manchester - Oh no! We don’t know of any candidates in Greater Manchester for the Greater Manchester yet.

It should lead to this post.

Also, given this election, results for this search should also include this post.

Feedback from a user:

Might be worth pointing out on the form that you want the second name last as it appears first on nomination papers

I think changing Name where it appears to Name (style: Ali Smith, not SMITH Ali) might be good. Any downsides?

At the moment people in the "Trusted To Lock" group are able to change candidacies while the post is locked.

This might lead to mistakes being made, and creates a slightly confusing interface for those users.

We should force a user to unlock a post, make changes and lock it again.

Recent changes have linked the bulk add forms with a SOPN document, meaning that the pages don't work unless there's a document associated with that post.

There are some places where we still link to the bulk add form when it doesn't have a document, thus causing a 500 error.

The candidate review page claims that you have found an existing candidate, and invites me to select them despite not having found an existing candidate. This is confusing.

Steps to recreate:

1. Go here: https://candidates.democracyclub.org.uk/person/5664/update
2. Click ‘Add a new election for this person’
3. Select ‘2017 General Election’
4. Set ‘Standing in 2017 General Election’ to ‘No’
5. Link to the source
6. Save changes
7. Server error

This probably didn't deserve a gif..

Has this borrowed the ‘seats available’ default from locals, perhaps?

The version history for https://candidates.democracyclub.org.uk/person/2111/stuart-jeffery includes the following merge:

You can see that historical candidacies for GE2010 and GE2015 were removed in this merge (which shouldn’t happen!)

Note that this can also cause candidacies for locked posts to be removed (as in the above example, in fact) which is particularly bad (because the list of candidacies for the post has been checked and verified by a person).

Currently the only option for photo uploads is to select an image from your local storage using a "Choose file" system.

Ideally there would also be options to:
Paste an image from your clipboard directly into the browser
Enter the URL of an online image so DC wgets it, saving users the trouble of downloading/saving then finding then uploading. (can you tell I've done that too many times already?)

See above demonstration. Bit of an odd one this as it doesn't happen consistently across all my browser/adblock setups. It might be that I have some particularly aggressive settings applied on this instance. I suspect this is also related to the same problem @davidmiller is hitting over in DemocracyClub/WhoCanIVoteFor#89 (unconfirmed). Maybe there's nothing actionable to do here: If a user's content blocker is blocking our content then...meh. On the other hand:

1. Maybe there is something we can do to make our 'social' content look less spammy to content blockers
2. It is worth being aware of this in case someone else reports it/is confused by it

@symroe pointed out that a problem with an import script for the UK was also present in jamaica_create_basic_site.py and cr_create_basic_site.py. See #59

I think the only way I can tell a candidate is not standing on the candidate page is by reading the versions. It would be better to show this both on the candidate page and in the election picker on the candidate edit page.

The password reset page can display "The e-mail address is not assigned to any user account" which can inversely be used to confirm valid emails or find out if someone is a user. It would be better as "If we know the email address [email protected] then an email with a password reset link will have been sent to it."

The principle is highlighted in here (via #85):

I've had pushback from dev based on security measures on patterns like this in sensitive data use cases. The argument being we are exposing personal information via the interface (you can ping addresses to see if they exist in the system).

On an election page "Upload an updated Statement of Persons Nominated" is bigger aka more attractive than the "Nomination paper" link. Since people usually want the nomination paper link could the other link be shortened and diminished, eg to "Upload an updated version" ?

For the SoPN page's address Is "/upload_document/" apt or would "/document/" or "/uploaded_document/" be more suitable?
Or c/should a url-rewrite make this eg /election/mayor.greater-manchester.2017-05-04/post/greater-manchester/greater-manchester/upload/6956

Naughty parties.

Election names should include the type of election, not just the location. Otherwise it’s weird when used in context, e.g.:

The Sheffield Brightside by-election is called:
“Sheffield by-election: Sheffield Brightside and Hillsborough constituency”

Also, there’s probably a case for using the current="Don’t know" status here 1.

Probably a good idea...

Feedback from user:

Following an email alerting me to start uploading information about candidates I've just been doing that, however I didn't realise that my username would be plastered all over the pages.

This wouldn't be a problem except I choose to use my email address as my username, now my personal email address is readily stamped all over the site and will be picked up I'm sure by search engines etc.

Sorry, I couldn't find the code to do this myself... but candidate search results to appear 'above', not 'below'. :)

E.g. https://candidates.democracyclub.org.uk/search?q=John+Williams

At the moment, the logic in this command: https://github.com/DemocracyClub/yournextrepresentative/blob/4eb2a85ae656dad3a0d9c2d726bc0ed624cf9d55/candidates/management/commands/candidates_add_twitter_images_to_queue.py isn't quite right, which is stopping us from running it every night (which was the intention). The problem is that if a photo imported on a previous night was moderated and accepted, it will still be added to the queue by this command.

The simplest solution is probably to only queue the image if the candidate has no Images associated with them and they have no QueuedImages in the queue (regardless of whether they were accepted or rejected - the QueuedImage objects aren't ever deleted regardless of the decision made about them - the decision field is just set). That won't queue changed images from candidates, which would be nice, but this fix would be really quick and let us start running the import every night again.

Where there is a major problem with data, it would be useful for verification to be able to explicitly rejected, and the entry is sent back to the main queue with a clear message of what was wrong ("the URL given is the wrong nomination paper").

This isn't really for easily correctable errors, but for larger classes of problems.

It's unclear whether "wrong SOPN data entered" would be in this class of problem. It's something those outside of Review can fix just as easily, and submit back to Verification.

This search directs to this which shows all 24 candidates for the three current elections on one long page.

I'm not usually a fan of extra stages but here I feel it would be better to first display a list of the current elections for that postcode, eg:
Current Elections for M14 5JB
Rusholme Ward - Manchester City Council local by-election
Manchester, Gorton - UK Parliament by-election
Mayor of Greater Manchester - Greater Manchester Combined Authority election

I feel that the search should only ever go directly to the post when there's only one post. Even when there are two posts each with few candidates and they easily fit on one page we should consider and prioritise the needs of general site visitors who I expect are less familiar with the various different types of elections than DC power-users are.

E.g. Here:
https://candidates.democracyclub.org.uk/areas/CED--CAM:market,WMC--gss:E14000617,combined-authority--combined-authority:cambridgeshire-and-peterborough

The posts shown fall on the dates: 4 May, 8 June, 4 May. I guess GE2017 should be shown third.

So this screen does tell lies, when it says 'An existing person with this name has been found...'

Can we change the text to (something like):

Some similar names have been found in our database. If you're sure this is a new candidate, please select 'Add new person'. Otherwise, please check the similar names, and if it is the same person, select their name instead. The system will update them with the new candidacy.

Currently, at all times, it's:
Known candidates for [area] in the [area]
Once locked, this c/should be:
Candidates for [area] in the [area]

No biggie or rush but I think this would help site users and editors, and I don't see any dis-benefits (the database query has already been done as the line above always states if the list is locked or unlocked).

The numbers page is meant to show you comparisons of the numbers with previous elections of the same "type" (broadly - this will need to be better defined in fixing this problem). This is currently sometimes unhelpful, e.g.:

e.g. 2:

The best elections to compare the 2017 general election to are those in 2015 and 2010. It might be best to have to explicitly link elections that should be compared like this. (This might be linked to: #103 )

I guess these numbers show either candidates per party, or candidates per party for current elections?

It would be better to show candidates per party for the election I am currently adding.

We're using the US style date format of MONTH, Day, YEAR, which is upsetting to me.

We could go Chinese, cos they're objectively correct, or we could go British, cos that's what one might expect in Britain. Make Dates Great Again.