Approval of an IIA about ewp-specs-api-iias HOT 56 CLOSED

Perhaps this could be used instead of embedded signatures.

from ewp-specs-api-iias.

Ad. Method I
In our opinion more in line with the spirit of EWP (according to the assumption that we stick to IIA3 distributed, no-master, meaning that everybody is master of its own data) would be the solution where every partner sends his own approval and does not send approvals of the other partners. In such approach I share information which comes from myself, so it is the most current.

Ad. Method II
It is much more complicated than Method I. Are these version numbers local or global? Anyway, sending version number does not solve the problem of credibility. If we really want to be sure that we know what is being approved we should send the copy of IIA being approved.

You write "However, they should use the same IIA-code." This condition is not met in the current specification.

from ewp-specs-api-iias.

We would prefer a method 1 like approach.
I'd suggest to move the status inside the partner element where the signing-date is already placed.
Either with <approved/> / <signed/> Empty flags or a <status>APPROVED/SIGNED</status>
This way partners which don't track the signing data could also indicate that the agreement has been signed.
Also a REJECTED status could be used to indicate that some changes made by the partner were rejected.

from ewp-specs-api-iias.

We also prefer a method 1.

from ewp-specs-api-iias.

I'd suggest to move the status inside the partner element where the signing-date is already placed.

If we keep status for all the partners, than it seems a good idea. @georgschermann What do you think about our proposal where every partner sends only his own approval/status?

from ewp-specs-api-iias.

I am not quite sure about this. It makes sense to only share someones own status, but for some edge cases it could come in handy to also propagate someone elses status, for example if you have 3 or more partners, or multiple faculties of one university, one of multiple partners not using EWP, ...

from ewp-specs-api-iias.

But how can you be sure that you really know the current status of the opinion of the other partner? He might have changed his mind a couple of seconds ago.
I would like to hear a really convincing scenario.

from ewp-specs-api-iias.

From the Minutes from the Technical meeting in Malaga (2019-09-12).

It is decided that we will only support bilateral agreements and do away with multilateral agreements. So each agreement may have only 2 partners.
The discussion will be continued in the GitHub issue #30. Francisco will write the proposal for bilateral agreements and make sure we have some way of signing.

from ewp-specs-api-iias.

As agreed in the technical meeting in Málaga (12/09/2019), when an Institution creates a new IIA (or makes any change), hereinafter referred to as First Party, and sends the notification to the other institution, hereinafter referred to as Second Party, the First Party is automatically agreeing that the data is the correct data and will not be changed. Given this, the First Party must sign the data with a valid certificate, every time the Second Party gets this data.
With this, we guarantee that after the Second Party gets the data for the first time, and checks that it is properly signed from the First Party, every request after should return exactly the same data.
After this, the Second Party may agree or disagree and make changes to the agreement.
If the Second Party agrees with what is specified, signs the same document received and sends a notification to the First Party to get the new document. The First Party gets the document from the Second party and validates his signature and the signature from the Second Party, and if correct, the agreement is closed.
If the Second Party does not agree with the specifications, makes the proposed changes, removes the First Party signature from the document and send the notification, starting the process as if it was “new”, signing it every time a request was made.

The signing process should be exactly the same as specified in he ToR Get API https://github.com/erasmus-without-paper/ewp-specs-api-imobility-tors/blob/master/endpoints/get.md .

In summary, the following steps are needed.

• First Party creates de the new IIA and sends the CNR to Second Party;
• Second Party contacts First Party and get the data of the IIA, verifies the signature of the First Party and saves it locally as proof.
• If Second Party agrees with the IIA:
  • Leaves the signature from First Party in the document and sends a CNR to First Party
  • The First Party gets the data from the Second Party, verifies both his signature and the other and saves it as proof.
• If Second Party does not agree with the IIA:
  • The Second Party removes the First Party signature from the document and makes the intended changes.
  • The Second Party then becomes the First Party and sends a CNR to the Other Party and the process restarts.

from ewp-specs-api-iias.

Thank you. We will update the specification according to this proposal.

from ewp-specs-api-iias.

The signing process should be exactly the same as specified in he ToR Get API https://github.com/erasmus-without-paper/ewp-specs-api-imobility-tors/blob/master/endpoints/get.md .

That specification states that the certificate is not important, but

it seems that the best certificate to use for this, would be the same
certificate which was used to actually serve the ToRs API.

That is not always possible: in our case, the key and certificate to serve the API are managed by another team. What then should be the requirements for the certificate?

from ewp-specs-api-iias.

We see some problems with the solution described by Francisco. We wrote a short google document with the description of what we need and possible solutions (among others our comments to the Francisco's proposal). I will send you an email with a link. Please, comment.

from ewp-specs-api-iias.

I wrote some comments in the document. One of the arguments against signing (only) PDF is that the PDF might not contain the same data as the XML (e.g. due to some error). Since the XML contains the master data (that is the purpose of this API), I think the XML should be signed.

from ewp-specs-api-iias.

I added comments to your comments.

My impression is that you miss the most important part, that - according to our opinion - the Francisco solution cannot be meaningfully implemented. If you think that it can, please explain how you see such an implementation. I hope that this will help us all to either notice deficiencies in the Francisco algorithm or maybe prove that we are wrong and the implementation is possible.

We can discuss whether PDFs are better than XMLs, but that's not the biggest problem. I am afraid that the correct solution with XML signing would be just as or maybe even more complicated than LA, because the contracts are served by both parties.

from ewp-specs-api-iias.

I think it would be easier if both parties just sign their own version of the agreement, and do not include the signature of the other party. That would eliminate most difficulties arising from using different element order, API version,...

We already agreed that the agreement should not change once it is approved by both partners. If an field is to be added or removed, a new agreement must be created.

If the API version is changed, the signature will change also. That should not be a problem, as it would not have an effect once the approval process is over. It is probably a good idea for both parties to keep the original signed XML.
An implementation should be able to parse this old XML's. Since in the case of an API change, different partners will be at a different version for some time, a parser for both versions will be needed simultaneously anyway.

from ewp-specs-api-iias.

Bavo: I think it would be easier if both parties just sign their own version of the agreement, and do not include the signature of the other party. That would eliminate most difficulties arising from using different element order, API version,...

If each partner signs his own version, no partner has a proof that HIS version has been approved by the other party.

Bavo in the Google document: Non-repudation based on HTTP Signature works as long as the old keys from the registry can still be recovered.

This is a real problem and it concerns not only HTTP Signatures. To solve it we should either use qualified certificates or store somewhere all keys in use any time by all partners (for example in the registry). This is a real challenge.

To push the discussion forward we suggest that we first agree on the answers to the following questions:

1. PDF
   option a) sending PDF in IIA is obligatory. This solution is attractive since (1) only this gives us one document signed by both partners, (2) PDf has a nice visual form, (3) if the physical docuemnt is needed in real world anyway, signed PDF is the best replacement
   option b) sending PDF in IIA is optional
   option c) we definitely don't want to send PDFs

2. XML
   ad a) in this option we do not need to sign XML
   ad b and c) in this option we have to sign XML, the only possible solution (in our opinion) is described in the Google document as "Signing partner’s XML". Also it should be remembered that solutions shown here based on signing XMLs are more complex.

from ewp-specs-api-iias.

store somewhere all keys in use any time by all partners

That was one of the reasons why we log every registry change and allow people to replicate this log in a distributed Git-like fashion (although I cannot find the prod log for some reason, perhaps it wasn't configured?)

In other words everyone should have access to fingerprints of all keys ever served by the registry. (Registry keeps this repository locally too, and it could run queries on it to help people find things.)

from ewp-specs-api-iias.

If each partner signs his own version, no partner has a proof that HIS version has been approved by the other party.

Partner A fetches the signed version of partner B, and vice versa. Both versions should match, or there would be no agreement.
How would it work if both signatures need to be present? Since there is no master in the process, both partners would need both signatures. If A starts the process and serves a signed agreement, which B fetches and signs, A would have to include B's signature in its own version. So both A and B's versions would have signatures from A and B. Then B has proof that A has seen its approval and vice versa. But is this necessary? @fmapeixoto, what do you think?

To solve it we should either use qualified certificates

This applies to both XML and PDF. I would prefer CA-signed certificates over self-signed. But we still need to agree on which CA's are allowed. This hasn't even been decided for certificate-based server authentication.

In other words everyone should have access to fingerprints of all keys ever served by the registry.

Agreed, although that would be another discussion.

To push the discussion forward we suggest that we first agree on the answers to the following questions

I have no strong opinion to whether we should or should not have (signed) PDF's. I agree signing XML has its challenges, but signing PDF and not XML incurs the risk of both versions deviating from each other. How would you guarantee that they are in sync?

from ewp-specs-api-iias.

Then B has proof that A has seen its approval and vice versa. But is this necessary? @fmapeixoto, what do you think?

As I understand we started the whole discussion with the assumption that every partner needs to have the approval of his version of IIA of the other partner.
If not, what problem are we solving and why the current solution does not suffice?

Where is Paul? Where is George? Where is Francisco? There were so many opinions when we talked about that in Malaga and now only Bavo is interested in finding the solution?

from ewp-specs-api-iias.

Hello,

For what I have read I think we are mixing problems and different things.

First, I think we should treat the IIA as a contract between parties, and as a contract, after signed, you can't just change content or add lines or any kind of alteration. So, as Bavo said, both versions of the XML should match so both parties can agree that they signed the same contract.

Regarding the two parties signing the same XML or just one party, if both of them send to the other partner the agreement signed by them and it's saved as it is, I don't see a big problem with that. Both parties have proof that the other signed so.

from ewp-specs-api-iias.

For what I have read I think we are mixing problems and different things.

or I would rather say that we do not understand each other

First, I think we should treat the IIA as a contract between parties, and as a contract, after signed, you can't just change content or add lines or any kind of alteration.

That's obvious

So, as Bavo said, both versions of the XML should match so both parties can agree that they signed the same contract.

That's also obvious but the problem is that we are not able to check that on a technical level, since semantically the two versions may be the same but represented by two different XML files.

By the way - for that reason we still think that PDF is as good as XML and even better).

Regarding the two parties signing the same XML or just one party, if both of them send to the other partner the agreement signed by them and it's saved as it is, I don't see a big problem with that.

We do see the problem. It's not enough that every party signs his version, what we need is to approve the partner's version. So every partner should:

1. get the version from the other partner
2. sign it
3. send it back.
   Optionally this version might be then signed by the first partner in step 1.

I would suggest to meet via Skype for a short talk to make sure that we do understand each other.

from ewp-specs-api-iias.

I agree with a Skype meeting to agree to solve these issues.

from ewp-specs-api-iias.

Bavo and all the others, will you be available for Skype on Monday, December 2nd, at 11:00 CET?

from ewp-specs-api-iias.

Works for me.

from ewp-specs-api-iias.

we would prefer PDF signatures for simplicity, the possibility of more than 2 signatures, the possibility of hand written or personal, digital signatures, the possibility for staff to view and verify the signatures, etc.

Skype meeting works for me.

from ewp-specs-api-iias.

Follow up on the Skype Meeting, please correct me if I got something wrong:

The initial requirement we wanted to cover is, that Partner A can approve/verify Partner B's IIA Proposal. We think this should/could be achieved using a solution similar to method 1 and the EWP CNR/GET (vs. POST, discussed multiple times earlier) approach.

We propose similar to method one to introduce an "approved" (or verified if preferred) timestamp.

We can use one shared <approved> in the schema or one for every partner, it souldn't matter in the implementation, since every partner knows about the own approval and can GET every partners approval at anytime later.

The timestamps only add the possibility to check if the version the partner GOT earlier and approved will differ from the current version. (supposing the local last modified date of the IIA is stored)

If a partner not approves an approved IIA, it can just omit the <approved> and the renegotiation continues, the same would be the case if a partner sees that the timestamp refers to an outdated IIA (in which case an CNR should have been sent earlier, but was probably lost).

from ewp-specs-api-iias.

Seems logical and intuitive to me and in line with the Skype meeting, but of course I cannot see all the technical implications

from ewp-specs-api-iias.

In this flow, isn't the partner is approving their own copy and not other partners copy?

from ewp-specs-api-iias.

Georg, thank you very much for your input. I will try to explain why - in our opinion - it will not work.

1 We agree that we need an approval. Strictly speaking we need to know:
1.1 that partner approved
1.2 what has been approved (we don't want to base our solution only on trust)

Because of "what" neither version number nor timestamp is sufficient .

2 We can use:

2.1 hash (or fingerprint) - exactly as we already use in LA
2.2 signed hash - nice to have but is more than what we need

Final proposal to solve (1) --> Let's use hash.

2. Should we use PUSH (meaning POST) or CNR/GET?

Assumption:
We have to assume that in both systems each single IIA is represented as a single object identified with one id. Every system should store local IIA id and partners's IIA-id. (Unfortunately some preliminary tests show that this may not be the case in real life.)

We are in favor of PUSH for several reasons:

2.1 PUSH allows us to obtain two goals in one simple synchronized call:
a) credible delivery of the approval decision to the partner
b) getting his confirmation.
In case of CNR/GET we send CNR (one call), partner invokes GET (second call) but can not react in the same call to the obtained content.

2.2 In case of CNR/GET we always have to keep in database the data we approve (GET can be invoked any time). In case of PUSH we push the data, which can be generated dynamically at our convenience.

2.3 Extensions needed in APIs specification
PUSH --> We had to add one extra endpoint to IIA API, lets call it "approved".
CNR/GET --> We had to add two extra APIs, lets call them PARTNER_IIA with endpoint "get" and PARNTER_IIA_CNR.

2.4 The scenario of data exchange
Partners exchange draft versions for some time. We start the scenario when one of the partners (or both at the same time - they behave in a symmetric way) want to approve.
Partner A calculates hash of his IIA and sends (IIA CNR/GET) IIA and hash-A to partner B.
Partner B wants to confirm that he accepts this IIA, so have to send identifier of this IIA, IIA-id, and the same hash-A to partner A.

2.4.1 PUSH to endpoint approved
Parameters of PUSH: IIA-id, hash-A

PUSH is synchronous, we get the confirmation in a transactional way with error code 200 (everything is OK, A has got approval) or 409 (A has got an approval but the version approved is outdated) or 400/500 (some other error or timeout).

2.4.2 New APIs PARTNER_IIA_CNR/PARTNER_IIA_GET
A send PARTNER_IIA_CNR to B with information "I have new information about version of IIA identified by IIA-id".
B gets this IIA identified by IIA-id, finds there hash-A.
If the version is outdated B has to use another way to say it to A.

Final conclusion on (2) --> GET/CNR could also be used but is more complicated. We opt for PUSH, as it is simpler to specify, simpler to implement and compliant with the current version of LA. If we would replace PUSH with CNR/GET for IIA, we would still have PUSH in LA. If you would argue that PUSH should be replaced by CNR/GET in LA it would make LA even more complicated than it is now.

from ewp-specs-api-iias.

Janina, I am finding it difficult to understand. Is it possible if we can have sample xml get api responses from both partner as a flow. This will clear any confusion.

from ewp-specs-api-iias.

For which version? All of them?

from ewp-specs-api-iias.

using hash with get/cnr for proposed new approval endpoint

from ewp-specs-api-iias.

But we recommend PUSH :)
Marta, Michał, please prepare an example.

from ewp-specs-api-iias.

What do you think about our arguments and our solution? @umesh-qs, @fmapeixoto, @BavoNootaert, @georgschermann, the others? It will be easier to talk on Monday when we exchange arguments before.

I will prepare an example scenario.

from ewp-specs-api-iias.

https://github.com/erasmus-without-paper/ewp-specs-api-iias/tree/acceptanceExamples/example-scenarios/

Currently there is one simple example scenario. If I write more scenarios, I will put them in the same catalogue.

from ewp-specs-api-iias.

In response to Janina's arguments above (#30 (comment)):
2.1
a) true. But CNR/GET is similar: once the CN has been delivered, it is the responsibility of the partner to fetch the approval.
b) true. But what would you do if the partner responds with a 500?
2.2 Do you mean you don't keep your approval in your database, or that you can change the data after it has been approved?
2.3 CNR/GET: In Georg's proposal the approval-element is included in the current API. This works, because the data from both partners has to be the same, or there would be no agreement.
2.4.1 Same remark as 2.1b: what would you do when there was a 500? In the case of CNR/GET, assuming that an error is more likely in processing the approval than in processing the CN, the partner can GET the approval after the bug has been fixed.
2.4.2 If the version is outdated, then B will already have sent a CN. It can do so again, or wait until A has processed the previous one.

from ewp-specs-api-iias.

Your response refers to the "Push vs. CNR" problem. We will try to answer soon.
And what do you think about our proposal to use hash instead of timestamp. Do you think it is correct? Do you agree, that timestamp is not sufficient?

from ewp-specs-api-iias.

After thinking about implementation of our solution on the client side we noticed that the user needs to know:

• which partner agreements have been approved,
• if partner agreement has changed after being approved.

To solve this issues we need to store the fact of approval and what has been approved (hash). We need to store this for every partner agreement and map it by partner agreement ID.

In other words we would have all the information needed to implement GET in Partner Agreements Approval API. No point in not sharing this data with the partner.

As for the question of adding the approval to IIAs GET method we don't think this is possible as it would require the approved partner to already have the mapping from partner's agreement ID to his own.

It makes more sense to create a separate API for approvals and use ID's of the approved agreements. This also doesn't require agreements to always map 1 to 1.

Having a separate API for approvals, it doesn't seem justified to use PUSH for sending an approval, as there aren't enough arguments for it. For example, knowing immediately that the partner agreement has changed doesn't solve an issue of the change happening just moments after the approval.

from ewp-specs-api-iias.

We agreed to use hash and to use CNR and GET for sending approval. The last thing is to decide if we add the acceptance to the existing API or we add new API(s). Here you can find example scenario in this two approaches: https://github.com/erasmus-without-paper/ewp-specs-api-iias/tree/acceptanceExamples/example-scenarios

from ewp-specs-api-iias.

Could you give an example where the agreements don't map 1 to 1?

from ewp-specs-api-iias.

I can write something, but I'm not sure if it will be a real life scenario. I tried to find why we started to consider agreements which don't map 1 to 1 and found emails from tests with SOP. They are/were saving recurring agreements as one.

@georgschermann do you still do this way? How will the acceptance process look like? Saving recurring agreements as one probably means that at some time you change a signed agreement.

Maybe we shouldn't allow this?

from ewp-specs-api-iias.

Yes, our internal agreements roughly correspond to the cooperation conditions, so 10 of our agreements are summarized to one iia, or the other way round.
During agreement rollover for a new year, etc., existing signed agreements would change, or we would map the changes to a new iia. Since we already do a lot of internal mapping it wouldn't be much of a problem.

from ewp-specs-api-iias.

Having a seperate API for approval is OK for us.

from ewp-specs-api-iias.

What about the others? @fmapeixoto, @umesh-qs? As I remember Georg said he can accept any version (separate API or field in existing API). Do we need this skype meeting that we have planned for today?

from ewp-specs-api-iias.

from ewp-specs-api-iias.

The draft version of the new IIA Approval API is available at:
https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval

There is still an issue how to handle PDFs. We see two options:
Option 1

1. We add one PDF to IIAs API and one to IIAs Approval API.
2. In IIAs API we publish PDF that has been created by us.
3. In IIAs Approval API we publish PDF that has been created by partner and signed by us.

Consequences:

1. If we want to take PDF created by partner and an approval of our version, we need to make two requests.

Option 2

1. We add one PDF to IIAs API and one to IIAs Approval API.
2. In IIAs API we publish any PDF we have (it may be PDF that has been created by us or PDf created by partner and sign by us)
3. In IIAs Approval API we publish any PDF we have (it may be PDF that has been created by us or PDF created by partner and signed by us).

Consequences:

1. If we want to have our version of PDF and partner wants to have his version of PDF we can either keep versions with single signatures or need extra requests to obtain our version signed by partner

Which one do you prefer? Do you see other options we should consider or other consequences which should be taken into account?

from ewp-specs-api-iias.

@janinamincer-daszkiewicz if we talk about PDF created by us, does that mean it is signed by us? If both partners have the same data in their system, the PDF's (based on EC template) should look exactly the same whoever generates it right?

from ewp-specs-api-iias.

In theory yes. But visually they may differ e.g. having different logos. For some reason Georg asked for the possibility to send separate PDFs. @georgschermann could you give us examples?

from ewp-specs-api-iias.

I would prefer Option 1, because then the PDFs in each API reflect the meaning of the corresponding XML-document. Both options could possibly support multiple PDFs.

from ewp-specs-api-iias.

In theory yes. But visually they may differ e.g. having different logos. For some reason Georg asked for the possibility to send separate PDFs. @georgschermann could you give us examples?

I think we shouldn't include logos and other additional elements on the agreements and stick to the EC template and the data included on this template. In my view the PDFs are only relevant for IRO users who need to get used to a new way of dealing with agreements. They are used to support users in the transition to a new way of working and can become obsolete once a digital workflow is generally used, trusted and accepted.

from ewp-specs-api-iias.

Both options could possibly support multiple PDFs.
Could you give an example of a case when you would like to send multiple PDFs?

from ewp-specs-api-iias.

Sorry, probably a misunderstanding from my part. I thought

For some reason Georg asked for the possibility to send separate PDFs.

meant that multiple PDFs were required, but perhaps you meant just one PDF from each partner.

In any case, as Paul said, we are not asking for any special features regarding PDF documents. But if they are to be supported by the API, then I think Option 1 is the cleaner solution.

from ewp-specs-api-iias.

I think it would be 1 PDF for each partner at most. Some of our Universities use the E+ template, but some extend or adapt the template, use their logo, add more signature fields, additional information or be more specific on the conditions, etc.

Some Universities won't allow any contracts/agreements which are not signed by various responsible persons. And the IROs cannot force the rectors or staff from other faculties, etc. to use any software tool they have. Getting a digital signature onto a PDF is the most digitization some can hope for at the moment. In some cases it may also end in print & re-scan. So for the time being the lack of PDFs would be a show stopper for some.

I'll ask some of our customers if the have example agreements.

For the implementation I think we would also prefer Option 1

from ewp-specs-api-iias.

Currently SIGMA generates the PDF according to the template, we do not add logos or more information.
We do not know if universities subsequently edit this document and add some elements. At the moment we don't have the possibility of signing digitally this document.
We also prefer Option 1.

from ewp-specs-api-iias.

Marta, I think we can continue with the specification.

from ewp-specs-api-iias.

Done:
https://github.com/erasmus-without-paper/ewp-specs-api-iias, ver. 2.2.1
https://github.com/erasmus-without-paper/ewp-specs-api-iias/tree/stable-v2/example-scenario , ver. 2.2.1
https://github.com/erasmus-without-paper/ewp-specs-api-iias-approval, ver. 0.1
https://github.com/erasmus-without-paper/ewp-specs-api-iia-approval-cnr, ver. 0.1
The last two will be made stable after the first reference implementation.

from ewp-specs-api-iias.