openradx / adit Goto Github PK

Related to issue #19

A java script class which handles the complete process of client side pseudonymization need to be implemented. Build a class with general properties for managing the pseudonymization. Probably just a pseudonymization method needed.

General workflow:

• Users logs in, goes to Upload tab
• Users uploads images
• Users has a text input field where he can provide a pseudonym
• The uploaded images are handled by client side upload code from issue #19
• The incoming data in the browser, is passed to an pseudonymization function which handles everything
• The function should return a Dicom Object (see JS Dicom Implemntation dcmjs), which is pseudonymized

Start implementation as follow:

• Set up a working environment, where you can mimic incoming images to the browser, basically copy the relevant stuff from #19

• You should be able to start an run everything without the overhead of adit, only with node.js

• Look into how the files are handled by client, probably an array containing some blob objects or similar

• Create the pseudonymization object and pass on each blob object separetly to the pseudonymization method

• Most important thing for the pseudonymization object creation will be, to initilaize this object correctly, so every image in that session is pseudonymized the same way

• Create Dicom Objects (dcmjs) out of these blobs

• Pseudonymize Dicom object

• Return the Dicom Object

• Set up mini working envirionment which accepts/loads files like a browser would do with node.js

• Search for pseudonymization solutions in JS, does dcm.js have one?

• If not, implement one, for that look into the Python Implementation Dicognito (same pseudonymization needed), but for now ignore the pseudonymization of UIDs

Desired workflow

Users logs into ADIT and can generate a token via the UI. They can manage their tokens via the UI. An admin can also manage all tokens of all users. Token generation should only be availabe for a subset of users, which an admin can manage. This token is later used to authenicate for the REST service.

• Available with Django? If not, what should we use?
• How to integrate with our permissons? Concept?
• How to implement?
• Optional for now but should be kept in mind: How to restrict token authentication to specific DICOM Servers? New permissons concept necessary?

Current pseudonymization runs on transfer task level. If a transfer task level is executed on only one series, different series from the same study are pseudonymized independently from each other, with a different anonymizer object. Series from the same study transferred on series level lose their connection to each other. At the destination they will be treated as if they come from different studies.

Move anonymizer object to job level?

Currently only the results that succeeded can be exported. For evaluation and correction purposes it would be useful, if all results, also the not successfull ones could be exported. This could me made optional with a toggle button. Users then could see which of their queries were not successful, correct a typo or similar, and then retry it with the same file.

The current case is, that every user can use every dicom node as source and destination. To handle user access to specific dicom nodes more granuarly, we need a new permission infrastructure. An admin should be able to define PACS groups via the Admin UI. A PACS group has a name, for example DIR. A PACS group can contain multiple PACS accesses, which in turn can contain only one dicom node. A dicom node can be related to many PACS accesses, but an access can only be related to one PACS group. The cardinalities are also denoted in the diagramm above. There are only three access types: Source, Destination and Bidirectional, which define if a group can access a specific dicom node only as source or destination, or as both. Users must have at least one PACS group. A PACS group can be empty and contain no accesses. An access must have one dicom node as relation.

The tables PACS groups and PACS access need to be implemented as new Django models in a new Django App called groups. The User model in the Accounts app needs a new field for the relation to PACS groups.

Desired workflow

Users logs into ADIT and next to the existing apps they have access to an upload app. The upload app is also bound to a permission manged by an admin. An admin can grant users access to the upload app. How the upload form should look like can be seen in the diagramm avobe. To process the images on the server side, the existing ADIT infrastructure for receiving images should be used, see BatchTransfer App und ADIT Core. The client side can transfer the images unpseudonymized to th server. But the user can also select an internal pseudonymization and one with a user provided batch file.

General info regarding pseudonymization:

• Happens on client side, using Java Script (dcmjs)
• Happens on-the-fly
  1. Image is received by the client browser on upload
  2. Image is read
  3. Image is pseudonymized
  4. Image is send to ADIT server
  5. Proceed with next image

General info for internal pseudonymization:

• In general the same infroamtion as stated above is also valid for this pseudonymization type
• Users can not assign pseudonyms, the complete process is handled by client side code
• Technical workflow:
  1. Take the department field from the user model of the currently uploading user
  2. Put it in front of the PatientID extracted from the uploaded image (e.g. DIR123456)
  3. Run SHA3-256 hashing function over the resulting string
  4. Use new 64 byte long string as new PatientID and PatientName
  5. Update the DICOM tags with that information
  6. Pseudonymize leftover DICOM tags normally with dcmjs
  7. Send image to server

Upload Job View

• Same general info about the upload job as in a BatchTransferJob view
• Upload tasks list also the same as in BatchTransferJob view
• But additional View Results button as in a BatchQueryJob which the user can download
  • The downloaded CSV file should contain a column with PatientID and the newly assigned pseudonym
  • If the pseudonymization is None, the two columns are basically the same
  • If the pseudonymization is Batch file, the client gets the same list as initially provided by the user
  • If the pseudonymization is Internal, the client gets a list with the generated pseudonyms and PatientIDs
• In the last to cases, no information about PatientID is transferred to the server, only pseudonym
• In the first case, the PatientID is transferred to server, since no pseudonymization is applied

• Implement frontend of upload app
• Implement connection via websockets
• Implement receiving image handling on server side
• Implement new permissions for pseudonymization type on upload (see diagramm above)
• Implement client side pseudonymization with Batch File
• Implement client side pseudonymization with Internal hashing method

Implement batch querying on Series Number Tag as addiontal, optional method to query on series level, next to the already exsting querying on Series Description Tag.

Hey Kai,

Manu asked me to bring this to your attention: in some cases (e.g. Task 88585, Batch Transfer Job 489), transfer fails consistently not due to the usual PACS timeout, but due to dataset encoding. Seems to be a rare occurence, I've encountered it only once now, for said patient, both in Batch as well as Selective transfer, over a few retries.

Not super urgent, but probably worth having documented.