open-contracting / european-union-support Goto Github PK

Definition:

The criteria (or criterion) will (only) be used to select the candidates to be invited for the second stage of the procedure (if a maximum number of candidates was set).

Shall I understand it as:

The criteria or criterion will only be used to select the candidates to be invited for the second stage of the procedure if a maximum number of candidates was set. (all parenthesis removed)

or:

The criteria or criterion will be used to select the candidates to be invited for the second stage of the procedure, and possibly for the rest of the procedure. If a maximum number of candidates was set, they will only be used to select the candidates to be invited for the second stage.

or something else?

Any idea @JachymHercher?

In F02 "Contract notice", the guidance for IV.2.1 to link to the previous release(s) states:

Discard. The previous publication concerning this procedure is the OCDS release with the same ocid as this release and with the nearest earlier date to this release.

Problem: a Prior Information Notice can contain several contracts, each one generating a different ocid. There is apparently no way from F02 to determine which ocid to link back to.

I also don't know how we determine the ocid in F02.

Until now, in the mapping files, I have consistently expressed boolean values as 'true' or 'false', using single quotes.

However, `true` and `false`, with back ticks, are more appropriate.

BT-04 (Procedure identifier)

The European Public Procurement Procedure Identifier, a unique identifier of a procurement procedure. Including this identifier in all published versions of this notice (e.g. published on TED, national publication portals, regional publication portals) allows unique identification of procurement procedures around the EU.

This BT is not present at planning stage (notices 1-9) and only appears from the tendering stage. That means we can't add it in the ocid from the start, unless the procedure starts with a tender notice.

On solution could be to have two ocid: one for the planning stage and one for the rest of the procedure. They would be linked via .relatedProcesses. However, as there can also be a process split between the tender and the award stages (certain lots having their own procedure), we may end up with data that is hard to analyse.

BT-651 "Subcontracting Tender Indication" is an array of codes (values).

Description:

The information about subcontracting that must be indicated in the tender.

In OCDS, the subcontracting extension has tender.subcontracting.description, but it matches better with BT-554 Subcontracting Description.

We could cram BT-651 Subcontracting Tender Indication in tender.subcontracting.description, asking implementers to append the label or labels from the codelist.

Or we add a new field to the subcontracting extension.

Related to #75

Currently, the guidance CSV only has information down to the BT-notice number pair (BT, notice number, notice type, BT description, datatype + standard forms information).

Since many developers probably rely on Xpath in their code, we should add eForms xpath to the guidance CSV.

We must keep in mind that previous study of the xpath-BT cardinality has revealed that BTs can, on average, be bound to many different Xpaths. This high cardinality of xpaths per BT is mainly due to the duplication of xpaths that describe organisations. A few corner cases are due to data modelling decisions, and seldom exceed 4 xpath per BT.

Definition:

Whether tenderer-related information can be supplemented even after the submission deadline.

Basically, whether the information of a tenderer can me modified or added after the submission deadline/tendering period.

I'm not sure whether an existing field would fit (submissionMethodDetails?), or if we need a new one.

It's bound to a short codelist.

A lot of BT that can describe the whole procedure can also describe individual lots. This is very much visible in the XML, where the element that describes the general information of a procedure (cac:ProcurementProject) is also a child of cac:ProcurementProjectLot that describes a lot, bringing all its own children elements.

Rough XML structure of an eForms document:

<[form type]>
     <cac:ProcurementProject>
         general information on the procedure...
     </cac:ProcurementProject>
     <cac:ProcurementProjectLot>
          <cac:ProcurementProject>
               general information on lot 1...
          </cac:ProcurementProject>
          other information on lot 1
     </cac:ProcurementProjectLot>
     <cac:ProcurementProjectLot>
          <cac:ProcurementProject>
               general information on lot 2...
          </cac:ProcurementProject>
          other information on lot 2
     </cac:ProcurementProjectLot>
     ...
</[form type]>

For example, the two possible xpaths for BT-24 (Description) and their respective mapping in OCDS:

• /*/cac:ProcurementProject/cbc:Description => tender.description
• /*/cac:ProcurementProjectLot/cac:ProcurementProject/cbc:Description => Lot.description

From a quick scan in the mapping data, BT that have an xpath that starts with/*/cac:ProcurementProject almost always have also an xpath that starts with /*/cac:ProcurementProjectLot/cac:ProcurementProject. On the other hand, many BT only have an xpath that starts with /*/cac:ProcurementProjectLot, meaning the XML schemas of eForms are rather centered on lots.

@jpmckinney What's the status of the thinking around the extension of the Lot object in OCDS? I think we're going to need it for data such as:

• description (general description of the procurement)
• procurement category (works, services, supplies) + additional
• classification (CPV, etc) + additional
• contract period
• (more examples incoming)

Belarus draft data includes an additional productCountry field in contract/items for the country where goods were created. Helpdesk feedback suggested renaming to countryOfOrigin.

eForms enables the publication of statistics through the BT pair BT-759 Received Submissions Count / BT-760 Received Submissions Type (nb of submissions, type of submissions (codelist)).

Example:

	<efac:ReceivedSubmissionsStatistics>
		<efbc:StatisticsCode listName="received-submission-type">tenders</efbc:StatisticsCode>
		<efbc:StatisticsNumeric>5</efbc:StatisticsNumeric>
	</efac:ReceivedSubmissionsStatistics>
	<efac:ReceivedSubmissionsStatistics>
		<efbc:StatisticsCode listName="received-submission-type">t-sme</efbc:StatisticsCode>
		<efbc:StatisticsNumeric>4</efbc:StatisticsNumeric>
	</efac:ReceivedSubmissionsStatistics>
	<efac:ReceivedSubmissionsStatistics>
		<efbc:StatisticsCode listName="received-submission-type">t-oth-eea</efbc:StatisticsCode>
		<efbc:StatisticsNumeric>0</efbc:StatisticsNumeric>
	</efac:ReceivedSubmissionsStatistics>
	<efac:ReceivedSubmissionsStatistics>
		<efbc:StatisticsCode listName="received-submission-type">t-no-eea</efbc:StatisticsCode>
		<efbc:StatisticsNumeric>0</efbc:StatisticsNumeric>
	</efac:ReceivedSubmissionsStatistics>
	<efac:ReceivedSubmissionsStatistics>
		<efbc:StatisticsCode listName="received-submission-type">t-esubm</efbc:StatisticsCode>
		<efbc:StatisticsNumeric>5</efbc:StatisticsNumeric>
	</efac:ReceivedSubmissionsStatistics>

The OCDS bid extension has codes for most of the eForms codes, but the following eForms codes have no straight match yet (current mapping table):

• t-med (Tenders from medium tenderers)
• t-small (Tenders from small tenderers)
• t-micro (Tenders from micro tenderers)
• t-no-verif (Tenders for which it has not been verified if they are admissible or inadmissible)
• t-verif-inad (Tenders verified and inadmissible)

For the micro/small/med codes, we could suggest implementers that they sum them up and add them as smeBids in OCDS.

• Need to omit first part of XPath, like with MOVE.xsd
• Can use similar code to label:reverse to extract label keys
• Can manually copy mappings from R2.0.9
• Can use label:copy task to copy across to F17-F18
• Add a content (Markdown) file noting that these forms use R2.0.8

Standard forms contained information that don't explicitly exist anymore in eForms, such as (not exhaustive):

• legal basis of the contracting process: this information is not stored in the XML but can be derived from the notice number (/*/ext:UBLExtensions/ext:UBLExtension/ext:ExtensionContent/efext:EformsExtension/efac:NoticeSubType/cbc:SubTypeCode). Providing a mapping table (each notice is bound to a single directive) would enable the extraction of this information.

Since we have an OCDS data structure to receive this information, do we provide guidance to help implementers extract this information from the eForms XML, in spite of the fact it's not explicitly stored in the XML?

Potential resources on this topic:

EU green public procurement criteria: https://ec.europa.eu/environment/gpp/eu_gpp_criteria_en.htm

EU library of use cases: https://ec.europa.eu/environment/gpp/case_group_en.htm

In output/mapping/eForms/BT-sfLevel.csv, certain rows have several notice numbers, such as 01,02,03, but we want granular rows so that we can then add form types based on single notice numbers. This way, we also enable the publication of guidance per notice number.

We consequently want to explode those rows rows in as many rows.

This BT is bound the Main activity code list.

This list contains many values that:

• are not from COFOG
• seem mostly related to private sector

@JachymHercher Is it safe to assume that only the COFOG values will be used with BT-10, since it applies to public organizations? That would simplify the guidance as I'm not sure what sort of code scheme the other values come from.

Merci !

In F01, II.2.3 "Place of performance" can be repeated, but only one .deliveryAddress is allowed in the Location extension.

Shall we add a .deliveryAddresses array in Location or a EU specific extension?

The Standard Form Level is the shared key we will use to script the mapping the existing SF guidance with the eForms Xpath. In order to anticipate mapping duplicates, it would be good to see if we have non-1:1 relationships in the mapping chain:

OCDS guidance for SF => SF Xpath => SF level => eForms BT => eForms Xpath => OCDS guidance for eForms

eForms adds more details to the prizes that are awarded at a contest. The BTs are grouped in BG-44 Prize.

From official examples:

<cac:AwardingTerms>
	<cac:Prize>
  			<cbc:RankCode>1</cbc:RankCode>
  			<cbc:ValueAmount currencyID="EUR">10000</cbc:ValueAmount>
  			<cbc:Description languageID="FRA">Une indemnité forfaitaire d'un montant identique à celui versé aux concurrents non retenus sera payée à l'équipe lauréate sous forme d'avance sur le marché de maîtrise d'œuvre dès l'issue du concours; cette somme étant à valoir sur le montant des honoraires à percevoir ultérieurement au titre du marché de maîtrise d'œuvre.</cbc:Description>
 		</cac:Prize>
 		<cac:Prize>
  			<cbc:RankCode>>= 1</cbc:RankCode>
  			<cbc:ValueAmount currencyID="EUR">10000</cbc:ValueAmount>
  			<cbc:Description languageID="FRA">Les trois concurrents (candidats admis à concourir) recevront une indemnisation forfaitaire maximale de 10 000 EUR HT. Pour les prestations fournies, sous réserve de la recevabilité de leurs prestations au regard du règlement du concours et du respect du programme. Les indemnisations sont forfaitaires, conformément aux dispositions de l'article R. 2172-4 du code de la commande publique, l'acheteur, sur proposition du jury, se réserve le droit, dans le cas d'un projet qu'il jugerait incomplet ou dont les prestations seraient non conformes par rapport au règlement de concours et ou au programme, de supprimer totalement ou partiellement l'indemnité.</cbc:Description>
		</cac:Prize>
</cac:AwardingTerms>

Currently, the design contest extension only has .prizes.description. We may need an array of Prize objects.

Figuring out the structure of an eForms XML document and how the data is distributed into XML blocks may help structuring the guidance in a handy way for implementers.

I'll use the can_24_FRA_comments.xml example document from the eForms-SDK.

In XPATHs provisional release v. 1.0.docx, the following BT-Xpath mappings have unclear semantics:

• /*/cac:ProcurementProjectLot/cac:TenderingTerms/cac:TenderRecipientParty/cbc:EndpointID => BT-18 (BT-509)
• /*/cac:ProcurementProjectLot/cac:ProcurementProject/cac:ContractExtension/cbc:OptionsDescription => BT-54 (BT-53)

We can assume that the presence of BT-54 (Options Description) and the corresponding Xpath imply that BT-53 (Options), of datatype indicator, is true. If cbc:OptionsDescription is absent, BT-53 is false. In this case, BT-53 wouldn't have its own Xpath. However, other BT such as BT-94 (Recurrence) are indicators and have their own Xpath (/*/cac:ProcurementProjectLot/cac:TenderingTerms/cbc:RecurringProcurementIndicator).

Throughout this issue, comments for which the relevant updates to guidance.yaml are done are marked with a ✔️

The same way we did for standard forms, in order to avoid repetitions, we need to document the guidance that applies to any eForms document.

The schema of this extension has changed a lot and the corresponding mappings must be updated.

eForms has a code list for organization roles and subroles. On top of these roles, buyers can be (or not) 'contracting entity' via the BT-740 (Buyer Contracting Entity) boolean.

Definition of 'buyer' in the eForms regulation:

Buyer refers to a contracting authority, a contracting entity, a defence contractor, an international organisation, or an organisation awarding a contract subsidized by a contracting authority; unless the above mentioned are an association of organisations that is not an organisation in itself, in which case each individual organisation is considered a 'buyer'.

Definition of 'contracting authorities' in the directive 2014/24/EU:

contracting authorities means the State, regional or local authorities, bodies governed by public law or associations formed by one or more such authorities or one or more such bodies governed by public law.

I couldn't find an official definition of 'contracting entity'.

'contracting entity' is not mentioned in related issues (open-contracting/standard#825, open-contracting/standard#571)

I'm not sure how to tie BT-740 back to OCDS roles (buyer, procuringEntity):

• what role does the organization get if BT-740 is 'not-cont-ent' (False)?
• if it's 'cont-ent' (True)?

Any idea @JachymHercher?

Definition:

Advanced or qualified electronic signature or seal (as defined in Regulation (EU) No 910/2014) is required.

@JachymHercher I'm not sure what this BT is about: is it the regular electronic signature that applies in virtually any electronic procedure, or is it something more advanced?

In a nutshell:

• can we call the to-be-created OCDS field electronicSignature: true/false, or would it be too broad?
• doesn't BT-17 Submission Electronic: required necessarily imply that BT-744 Submission Electronic Signature: true?

eForms uses xsd:date datatype to express dates that don't require a time. According to the XML Schema recommendation, an xsd:date is the date portion of an xsd:dateTime object, with the possibility to add an indication of the timezone.

Valid examples of xsd:date:

• 2021-10-09
• 2021-10-09+02:00

Question: is this datatype fully compatible with OCDS date, or does a time portion must be added? I see that in the EU standard forms guidance dates (e.g. "20191127") are mapped as-is, without conversion.

Ideally we should require a conversion of the dates both in eForms and SF guidance so that they match the OCDS date format: YYYY-MM-DDThh:mm:ssTZ

What do you think @jpmckinney?

For the description of the procedure in standard forms, we wrote the following guidance:

Map to the lot's .description
F12, F13: Instead, map to tender.description

@jpmckinney Do you remember why design contests (notice and result notice) had special guidance in this case? They are not alotted?

In OCDS we support the following procurement categories:

• works
• services
• supplies

With the possibility to add additional procurement categories via an open codelist.

In eForms, they support the same values + 'combined' ("Provision of any combination of services, supplies and/or works").

How do we reflect that in the mapping?

Can we leave tender.mainProcurementCategory empty and

• add 'combined' to tender.additionalProcurementCategories?
• OR add 'works', 'services' and 'supplies' to tender.additionalProcurementCategories?

I think I saw somewhere that there were discussions around a 'combined' option in OCDS.

@jpmckinney The European Commission has updated the TED XML schema with a few new elements https://webgate.ec.europa.eu/fpfis/wikis/pages/viewpage.action?pageId=876970615

Quoting the blog post:

These 4 elements have been added to R2.0.9.S05:

• PT_REQUEST_EXPRESSION_INTEREST : Request for expression of interest - only for rail (art. 5(3b) of 1370/2007)
• PT_DA_EXCEPTIONAL_CIRCUMSTANCE_RAIL : Direct award, if justified by exceptional circumstances - only for rail (art. 5(3a) of 1370/2007
• PT_DA_MARKET_NETWORK_RAIL : Direct award, if justified by relevant structural and geographical characteristics of the market and the network - only for rail (art. 5(4a) of 1370/2007)
• PT_DA_OPERATOR_MANAGER_RAIL : Direct award, to an operator which manages simultaneously the entire or the major part of the railway infrastructure on which the services are provided - only for rail (art. 5(4b) of 1370/2007)

There will be no change to the R.2.0.8 (defence).

Do we intend to stick to the latest release of the TED XML schemas?

@BenCCS Does your team plan to make use of these new elements?

@JachymHercher The description of eDelivery Gateway URL in the annex to the eForms regulation:

The organisation's uniform resource locator for exchange of data and documents.

It's not the buyer's profile, it's an interface to exchange documents securely using eID (part of the CEF infrastructure):

eDelivery is a building block that provides technical specifications and standards, installable software and ancillary services to allow projects to create a network of nodes for secure digital data exchange. By building with eDelivery, public and private organisations from different sectors can easily create a safe and interoperable channel to transfer documents and data among each other over a public or private network.

(https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/eDelivery)

Do you think the eDelivery system is specific to Europe for document interchange? If specific, I would add and document it in the EU OCDS extension. Otherwise, probably in the Communication extension.

Thanks!

We should clarify in guidance how groups of lots translate from eForms to OCDS.

Groups of lots exist in eForms to:

• let buyers group lots that share the same award criteria (BT-330, BT-1375)
• express the maximum or estimated value of a group of lots within a framework agreement (BT-556)

In order to track the progress of the mapping, we need the following stats:

• eForms xpath with OCDS guidance:
  • total number of xpath
  • number of xpath with guidance
  • number of xpath without guidance
  • percentage of xpath with guidance
    • per form type
    • total
• eForms xpath whose guidance has been validated (either the scripted mapping added guidance ready for peer review, or new guidance has been added and is ready for peer review)
  • total number of xpath
  • number of xpath with validated guidance
  • number of xpath without validated guidance
  • percentage of xpath with validated guidance
    • per form type
    • total

Hi, I have the XML files for the Norwegian Doffin. It's the same as TED. And I well make JSON files of it.
I'm trying to run code with ours data (not mine) but i dont find the docs folder

for i in 01 02 03 04 05 06 07 08 12 13 14 15 20 21 22 23 24 25; rake table LANGUAGE=EN FILES=F$i > path/to/european-union/docs/forms/F$i.md; end
for i in 01 02; rake table LANGUAGE=EN FILES=MOVE FORM=T$i > path/to/european-union/docs/forms/T$i.md; end

When i run
for i in 01 02 03 04 05 06 07 08 12 13 14 15 20 21 22 23 24 25; rake table LANGUAGE=EN FILES=F$i > /home/bjorn/source/european-union-support/output/content/F$i.md; end

I get
index of key in data:
6

omit: icar_noticeref
ignore: internets
enumerations: maintype_ministry
additional: false
rake aborted!
unexpected key: nutscode
/home/bjorn/source/european-union-support/tasks/table.rake:210:in block (2 levels) in <top (required)>' /home/bjorn/source/european-union-support/tasks/table.rake:59:in each'
/home/bjorn/source/european-union-support/tasks/table.rake:59:in `block in <top (required)>'
Tasks: TOP => table
(See full trace by running task with --trace)

Not needed for this project, but may be valuable to the community.

The initial mapping stats indicate only 61 eForms Xpath don't have OCDS guidance. That's surprisingly low given the number of new terms introduced in eForms.

The following link is used in the documentation (e.g. F14) : https://op.europa.eu/en/web/eu-vocabularies/tedschemas, but it's broken.

Here is the right one: https://op.europa.eu/en/web/eu-vocabularies/e-procurement/tedschemas

I believe that we haven't really mapped VI.3 "Additional description". tender.description is mapped to something else.

In eForms there is a BG "Additional information", with an "Additional information" field.

F23 and F25 have fields that look similar:

• F23, in V.2.4) (/AWARD_CONTRACT/AWARDED_CONTRACT/INFO_ADD_VALUE), "Any other details relevant to the value of the concession accord to Art. 8(3) of the directive" (this section is about the method to calculate the value of a concession)
• F25, in II.1.5) (/OBJECT_CONTRACT/CALCULATION_METHOD) "Method used for calculating the estimated value of the concession"

I've mapped CALCULATION_METHOD to award.concession.valueCalculationMethod (the name may be improved), and I wonder whether INFO_ADD_VALUE could also map to it. They seem semantically very close.

Could both F23 and F25 be filled in a concession procedure, and could INFO_ADD_VALUE and CALCULATION_METHOD clash?

Definition:

Information about a number linked to the selection criteria (or criterion) used for selecting the candidates to be invited for the second stage of the procedure.

It contains the following BTs:

• BT-752 Selection Criteria Second Stage Invite Number: A number linked to the selection criteria (or criterion).
• BT-7531 Selection Criteria Second Stage Invite Number Weight: Whether the number linked to a selection criterion (or selection criteria) is a type of weight (e.g. a percentage) => codelist
• BT-7532 Selection Criteria Second Stage Invite Number Threshold: Whether the number linked to a selection criterion (or selection criteria) is a type of threshold (e.g. a minimum score, a maximum number of tenders with the highest score passing) => codelist

This convoluted, multi-purpose data structure enables the expression of any numeric feature of selection criteria.

These semantics could be added to the selection criteria extension. Do we keep the same structure? It would ease the eForms mapping, but might not work for non-EU procedures.

In standard forms, we mapped the information about weight to the criterion's .description.

For the standard forms, for each form, we have used the Xpath as the unit of guidance:

• sporadic level to refer to the paper form
• one xpath with its description
• OCDS guidance that may refer to other Xpath.

For the eforms guidance, for each form type (Planning, Result, etc.), I'm leaning toward using the BT as the unit of guidance and group the matching Xpath with it:

• BT
• Xpath(s), with description if available
• OCDS guidance that may refer to other BT and Xpath

Most BT have a single Xpath, and multi-Xpath BTs seem to have a logic that deserve to be treated as a whole. The information we currently doesn't tell us whether a given BT can have a different logic from a form type or a notice to another.

eForms introduces new roles (I have crossed the roles that are not new):

1. Beneficial owner
2. Buyer
3. Central purchasing body acquiring supplies and/or services intended for other buyers (this used to be a single code in TED, it means wholesaler)
4. Central purchasing body awarding public contracts or concluding framework agreements for works, supplies or services intended for other buyers (this used to be a single code in TED, it means contract intermediary)
5. Organisation providing information concerning the general regulatory framework for employment protection and working conditions applicable in the place where the contract is to be performed
6. Organisation providing information concerning the general regulatory framework for environmental protection applicable in the place where the contract is to be performed
7. Organisation providing information concerning the general regulatory framework for taxes applicable in the place where the contract is to be performed
8. Mediation organisation
9. Organisation (or person) requesting review that has or has had an interest in a contract and who has been or risks being harmed by an alleged infringement
10. Review organisation
11. Procurement service provider
12. Subcontractor
13. TED eSender
14. Tenderer
15. Winner

This topic isn't new: open-contracting/standard#1180 cc @JachymHercher

BT-65 "Subcontracting Tender Obligation" is an array of codes (values).

Description:

An obligation to be met by the tenderer concerning subcontracting.

In OCDS, the subcontracting extension has tender.subcontracting.description, but it rather matches better with BT-554 Subcontracting Description.

We could cram BT-65 Subcontracting Tender Obligation in tender.subcontracting.description, asking implementers to append the label or labels from the codelist.

Or we add a new field to the subcontracting extension.

Related to #74

The BidsStatistics may be used to publish statistics about amount of money. In that case, the value field must be completed with an optional currency field (see II.1.7 "Currency" in F03).

In eForms, organization have a main role that is similar to OCDS roles (winner, buyer, etc.) and it can be refined with a subrole via BT-770. Possible values:

• Group leader
• Organisation executing the payment
• Organisation processing requests to participate
• Organisation processing tenders
• Organisation providing additional information about the procurement procedure
• Organisation providing more information on the review procedures
• Organisation providing more information on the time limits for review procedures
• Organisation providing offline access to the procurement documents
• Organisation receiving requests to participate
• Organisation receiving tenders
• Organisation signing the contract
• Organisation whose budget is used to pay for the contract

We could create an OCDS extension that adds a .subrole array field to the Party object, with an open codelist including the codes above (rephrased with OCDS terms).

On top of the duration in days/months/years and the start and end dates, eForms offers the possibility to express duration according to a wide code list. OCDS has no equivalent.

Users have the possibility to use this field as a replacement for the other types of durations.

Since it's tightly bound to a EU code list, I suggest we add it to the EU extension.

BT-1252 Direct Award Justification Previous Procedure Identifier enables to link a procedure that justifies the use of a direct procedure.

An identifier of a previous procedure that justifies the use of a procedure which allows directly awarding contracts, i.e. justifying a procedure that does not require publishing a call for competition in the Official Journal of the European Union.

This BT would typically work well with the tender.relatedProcesses array, but I'm not sure the existing codes fit the bill.

Is a new code necessary?

Currently, when the guidance has several steps, it looks like this:

"Add 'electronicSubmission' to tender.submissionMethod, and map to tender.submissionMethodDetails

If to the following address is selected, this results in a loss of structure. (WARNING #27)"

To improve readability and make the separation of steps more systematic (no hesitation between commas or line breaks), we could consistently use line breaks and number the steps. The numbering should be automatic and not require renumbering when we remove an intermediate step.

The steps above would look like this:

1. "Add 'electronicSubmission' to tender.submissionMethod

2. Map to tender.submissionMethodDetails

3. f to the following address is selected, this results in a loss of structure. (WARNING #27)"

Although we have created many extensions to cover the fields of standard forms, eForms introduces fields that require more fields to be created, and possibly new extensions:

• Notice Publication Date Preferred (BT-738) => Communication extension
• (Atypical) Tool Name (BT-632) => Communication extension
• Documents Official Language (BT-708) => Document Details (Add it to the document's .languages array.)
• Place Performance Services Other (BT-727) There are other restrictions on the place of performance (e.g. "anywhere in the European Economic Area", "anywhere in the given country"). => item's deliveryLocation.description
• Dispatch Invitation Interest (BT-631) The estimated date of dispatch of the invitations to confirm interest. => Communication. The extension should be extended to lots. In guidance: .communication.invitationToConfirmInterest.startDate
• Organisation Part Name (BT-16) The name of a part of an organisation (e.g. the relevant department of a large buyer). => I can't think of an organisation extension where it would fit, thus it could go to the EU extension
• Submission Nonelectronic Justification (BT-19) The justification for electronic submission of tenders, requests to participate, or expressions of interest not being possible. => .nonElectronicRationale in Submission terms
• Multiple tenders (BT-769) Tenderers may submit more than one tender (for a given lot). => .multipleTendersPolicy in Submission terms

When guidance is validated for a specific context (eForms xpath + BT + ?), it should spread to all other rows that match the same context, and have these rows tagged as validated.

To do:

1. define a context that is broad enough to speed up validation, but narrow enough to avoid over-spreading. This will be defined with practice
2. write the script

• The proposal is to have one page per eforms notice (there are 40) with one BT per row, as a lookup table.
• For BTs like BT-513, we can replace the middle part of the XML path with * to reduce repetition.
• We can add a search box to filter the data table (e.g. to match an XML path or BT).

Miscellaneous

• Before merging the branch, I need to check the changes to the standard form CSVs: https://github.com/open-contracting/european-union-support/commits/eForms/output/mapping/sf
• Run black on Python scripts

I'm also putting notes from a call with Colin here, in case relevant in future.

eForms documentation

• The eForms SDK has sample forms/notices. See readme.
  • fields/fields.json was added more recently and might be useful.
  • Under examples/notices, can_24 means "Contract Award Notice EU/2014/24".
• https://docs.ted.europa.eu/home/index.html
  • https://docs.ted.europa.eu/eforms/0.4.0/index.html is not much used for mapping.
  • https://docs.ted.europa.eu/eforms/0.4.0/index.html#noticesXmlStructureSection gives the general structure.

output/mapping directory

• Indices were added to the standard form CSVs.
• In eforms-guidance.json, each JSON element is a form-BT pair.
  • The status values can be:
    • imported
    • issue and spread: See the issue in the comments.
    • done and spread
    • done (spread from X): this means it was copied from location X
• The spread Python script copies guidance and comments.