Stemming from OpenDataScotland/the_od_bods#26
The ability to filter datasets with working links (vs dead links/ archived items)

Question: do we need this? Should we even list datasets with a dead url anyway? Maybe they should be dropped in processing, so this feature becomes redundant.

We do have an existing function that checks url status (see analytics.ipynb) but because of the time it takes to process the whole listing, we restrict this to local authority published datasets only. So, function exists, probably not the best, but could be modified to be better, attribute does not exist yet as such.

Related to task #32

The resources page https://opendata.scot/resources/ is very busy, split this out into minimum

• projects
• read about open data
• reference guides for UK/Scottish open data

Stemming from OpenDataScotland/the_od_bods#26
The ability to filter a dataset by the time it relates to. e.g. Filter all datasets with data for 2010.

This attribute doesn't currently exist, so would need to extract from dataset name or description. Might need some sort of proxy.

Is your feature request related to a problem? Please describe.
Jekyll builds currently take about a minute on a good PC at the moment. We should see if we can cut this down. Output stats from a build:

Done in 65.089 seconds.

Describe the solution you'd like
Investigate if build can be optimised with any of the techniques in this article: https://forestry.io/blog/how-i-reduced-my-jekyll-build-time-by-61/

Describe alternatives you've considered
N/A

Additional context
N/A

Potentially replace static chart images with D3.js?

Pages/charts to do:

• # of datasets by org
• # of data files by org
• Scorecard
• Data portal sources (e.g. CKAN, ArcGIS, manual etc.)
• File types/file types by org/average file types per dataset
• File size distribution/average data files per dataset per org
• Top 10 most popular tags/Top 10 most shared tags (after data cleaning)
• Licence distribution (after data cleaning)

Is your feature request related to a problem? Please describe.
When navigating our repoes, it's not always clear that the JKAN repo is the site repo for opendata.scot

Describe the solution you'd like
For clarity we should change this to opendata.scot

Describe alternatives you've considered
N/A

Additional context
Some things to consider when we make the change:

• Make sure we keep the attribution to the original JKAN project: we wouldn't be here without them after all!
• Consider impacts on the pipeline scripts in the_od_bods and opendata.scot_pipeline repositories as they may use hardcoded references to this repo name. Consider changing the repo name reference to be stored in a configuration file or as an environment variable.
• Update all documentation to remove references to our JKAN repo and repoint to the opendata.scot repo

We've a great deal of data in Wikidata (10,000's of items at a guess) which are now in PD - CC0.

Allow a dataset to be added for either No organisation (a fake org of 'None' ) or add w WikiData organisation to which to attribute the datasets (as SPARQL end points for queries to retrieve the data in question.

This can be a service that power users can use to monitor updates/changes to datasets they are using

We should probably close OpenDataScotland/the_od_bods#100 and create an issue template as part of OpenDataScotland/the_od_bods#103 before we move this forward

In the interests of making it easier for people/orgs to contribute suggestions for datasets we could scrape, let's create a Google Form instead of forcing them to sign up for GitHub and open a new issue, which they may not be familiar with doing.

    graph TD
    A("👨‍💻 User identifies website to be scraped") --> B("👨‍💻 User submits Google Form");
    B --> C("🤖 GitHub Action picks up new submission")
    C --> D("🤖 GitHub Action opens new issue using template")

Suggestion from Ash McClenaghan

JKAN’s search a little excluding because it returns only those datasets whose metadata contain all of the search words, consecutively.

See thread on slack:
https://opendatascotland.slack.com/archives/C02HEHDL8AY/p1651586908149319

Is your feature request related to a problem? Please describe.
Our dataset pages don't include the name of the organisation in the title which could lead to confusion in search results on search engines like Google (see screenshots below) if the user was just reading the page title.

Describe the solution you'd like
Consider adding organisation to the title metadata tags so that they can be included in search results

Describe alternatives you've considered
None

Additional context

Stemming from OpenDataScotland/the_od_bods#23
Suggested source https://w.wiki/4Kd6, original suggestion from @watty62

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Our dataset pipelines take raw HTML from the descriptions of some datasets which means that they can often be littered with various tags that mess with the styling when outputted on opendata.scot (e.g. <h1> or any tag with a style) property. This can sometimes produce unexpected results like large text being outputted from header tags.

Expected behavior
Some of these styles or tags could be simplified (e.g. we could convert all header tags to just be bold and underlined)

Screenshots

Example from https://opendata.scot/datasets/dundee+city+council-housing+available+now/

Hardware and software used
N/A

Additional context
Whilst unlikely to happen, I have concerns that this could leave us vulnerable to XSS (cross-site scripting) attacks if we ended up loading JavaScript <script> tags in the description of datasets we pull from other websites. See this relevant article where someone registered an XSS attack payload as a company name on Companies House which had the knock on effect of XSSing websites that consumed data from the Companies House API: https://www.theregister.com/2020/10/30/companies_house_xss_silliness/

Allow users (or us) to display a 5-star rating of the dataset.

Allow users to say they have used the data and/or made a project with it + links

Suggested reading:
https://5stardata.info/en/
https://www.w3.org/DesignIssues/LinkedData.html

When the google form is submitted, the table of submitted requests on the right doesn't update as the page doesn't refresh so it feels like the submission hasn't been successful.

Ideally, the page or table should refresh on form submission.

e.g. compare this year to last year - what has changed?

• what's been added?
• what's been removed?
• what has had more data added on?

Is your feature request related to a problem? Please describe.
Some of the datasets being surfaced via local authority open data portals belong to their leisure and culture providers that are often separate organizational entities (e.g. Sport Aberdeen, ANGUSalive, Leisure and Culture Dundee). We need to record these org records in JKAN so the data links up correctly on opendata.scot

Describe the solution you'd like
Create org records in JKAN for each leisure and culture organization

Describe alternatives you've considered
N/A

Additional context
Some research will need to be done to get a list of leisure and culture providers for each local authority area to add them all.
We may need to consider adding a new organization type to JKAN as well as it may not fall under any of the existing ones.

Stemming from OpenDataScotland/the_od_bods#26
The ability to filter datasets by organisation type (Local authority, health board etc...) see existing options: https://github.com/OpenDataScotland/jkan/blob/gh-pages/_data/org_types.yml

Attribute already exists as an Organisations attribute, but not in Datasets, so the data item needs to be pulled through the Organisation-Dataset relationship

Describe the bug
Moving from the old projects board on the the_od_bods repo to the new 2023 board means references and links to the old board need updating

To Reproduce
N/A

Expected behavior
Locations
[x] opendata.scot footer
[x] https://opendata.scot/about/
[x] contributing docs
[x] old wiki (retired)
[x] org repo public README
[x] org repo private readme

Screenshots
N/A

Hardware and software used
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
We have a health report for displaying which orgs don't have a What Do They Know page attached to their record. The problem is, some orgs may never have a page as they are exempt (e.g. due to being a public body). See OpenDataScotland/the_od_bods#71 for discussion

Describe the solution you'd like
A way of marking orgs that are FOI exempt so this shows up in the health report. Should we consider also displaying this on the org's page? E.g. "this org is FOI exempt"

Describe alternatives you've considered
Continuing on as normal - would show as missing data on health report

Additional context
N/A

Describe the bug

When I go to the webpage for the Place Name Gazetteer dataset, I find a link to the dataset under the 'Resources' heading, marked as media type 'WFS', which I presume is Web Feature Service, which seems to be a protocol for carrying out operations on geographic data over the Web. When I click on this link, I get an XML file back describing an error: 'Could not determine geoserver request from http request org.geoserver.monitor.MonitorServletRequest@2c5b19d'.

Here's the full error:

<ows:ExceptionReport version="2.0.0" xsi:schemaLocation="http://www.opengis.net/ows/1.1 https://geo.spatialhub.scot/geoserver/schemas/ows/1.1.0/owsAll.xsd">
    <ows:Exception exceptionCode="MissingParameterValue" locator="request">
        <ows:ExceptionText>
            Could not determine geoserver request from http request org.geoserver.monitor.MonitorServletRequest@2c5b19d
        </ows:ExceptionText>
    </ows:Exception>
</ows:ExceptionReport>

To Reproduce
Steps to reproduce the behavior:

1. Go to the webpage for the Place Name Gazetteer dataset
2. Click on the link to the Place Name Gazetteer data

Expected behavior

Data is returned

Actual behaviour

Error is returned as detailed above

Hardware and software used

• Device: Laptop
• OS: Windows 10
• Browser: Firefox 123.0.1

Is your feature request related to a problem? Please describe.
Adding some browser simulation tests that can be ran to make sure that the website is functioning as normal. To be ran either after a new site publish (usually on a weekly basis) or for testing when adding new features.

Describe the solution you'd like
Browser tests using Selenium to check various aspects of the website:

• Datasets page loads and has datasets
• Orgs page loads and has datasets
• Open a dataset page
• Open an org page

Selenium tests can be written using Python
https://vknight.org/unpeudemath/code/2014/12/23/using_python_and_selenium_for_a_jekyll_site.html

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
No
Describe the solution you'd like
An advent calendar for 2023 would be nice. One door per day, with a data set behind it.

Describe alternatives you've considered
None.

Additional context
I'd like to pretend that this is an original idea but hey, https://opendata.schleswig-holstein.de/adventskalender

Is your feature request related to a problem? Please describe.
For milestone 23Q1. Create a dynamic scorecard, very similar to the original one done in 2021 (bottom of the notebook) which will allow users to see which organisations are doing best in providing open data.

Describe the solution you'd like

• A page in frontend, under Analytics
• sources latest data in JKAN
• preferable: user selection to include/exclude organisations to compare

Describe alternatives you've considered
It could be a static table that is generated every week on refresh, but seems rather archaic.

Additional context
Build of this is 2 parts:

1. The scoring mechanisms
2. The display in frontend