Issue46

Title PHI remaining after anonymization
Priority feature Status resolved
Superseder Nosy List admin
Assigned To
Keywords Orthanc Core

Created on 2017-04-25.00:37:56 by admin, last changed by admin.

Messages
msg205 (view) Author: admin Date: 2017-04-25.00:37:56
[BitBucket user: jmesterh]
[BitBucket date: 2017-04-24.22:37:56]

Tag 0010,1060 PatientMotherBirthName is considered PHI and remains after anonymization.
msg206 (view) Author: admin Date: 2017-04-26.16:22:56
[BitBucket user: Salim Kanoun]
[BitBucket date: 2017-04-26.14:22:56]

Hi there,

I picked up another tag also not correctly removed  : 0032,1033 : Requesting Service

Best regards,

Salim
msg207 (view) Author: admin Date: 2017-05-10.22:55:46
[BitBucket user: Alain Mazy]
[BitBucket date: 2017-05-10.20:55:46]

done in a branch here with multiple other tags following PS 3.15 Table E 1.1: https://hg.orthanc-server.com/orthanc/changeset/c6defdc4c611fca2ab528ba2c6937a742e0329a8

@sjodogne : integration tests are still ok after this change.

A few TODOs to discuss: 
- according to the PS3.15, Study/Series Date/Time shall be 'nulled' .  This might break some stuffs.  And, BTW, I'm not sure what we should use as a 'null' date value
- PatientSex shall be 'nulled' too ...

Once these 5 fields are removed, the integration tests are still ok but this could break stuffs in some users workflows -> to be documented as a potentially breaking change ?

@salimkanoun I think you're kind of anonymization expert.  Do you think we should also remove these 5 tags ?
msg208 (view) Author: admin Date: 2017-05-10.23:13:47
[BitBucket user: Salim Kanoun]
[BitBucket date: 2017-05-10.21:13:47]

Hi @alainmazy 

The anonymization service is perfect as now, the "standard" anonymization anonymize everything but you can play with Replace or Keep to change or avoid to erase some tags.

The question of the Study / Series date / Time is complicated, for CT/MR/US studies it is probably safe to erase them. 
For PET and NM it is forbidden because this time data are used to calculate decay correction in image processing software.

In our Anonymization tool we are building we designed scenario following the NEMA recommendation, we made a "Date" and "Body characteristic" option to keep related tag (because for PET patient wheight, height and even sex are important to calculate SUV/SUL values).

Tomorrow we have a call with Frederic to present the anonymization work we are doing using Orthanc. I will explain you our design for anonymization. 

In short, my advise is too keep your anonymization kernel as it is now : Erase every tag listed in the NEMA documentation and then the Keep and Replace option in the API will make the user able to design different level of anonymization. 

Probably the default anonymization which erase everything is too hard and will lead to loose valuable data, however your Orthanc design for anonymization "kernel" is really good. The solution might be to update OrthancExplorer to manage different option of anonymization with playing with Keep argument (we will give you our source of our anonymization software with the scenario we designed).

Best regards,

Salim
msg209 (view) Author: admin Date: 2017-05-11.09:26:23
[BitBucket user: Alain Mazy]
[BitBucket date: 2017-05-11.07:26:23]

Hi Salim,

To summarize:
- we should probably implement multiple 'profiles' in the user interface.  With the default implementation removing all tags listed by NEMA including Study/Series Date/Time and PatientSex.
- the default Rest API Route would also remove all tags including those 5 ones.  Any user can use the Keep and Replace options to customize the anonymization process.

Would that sound reasonable ?
msg210 (view) Author: admin Date: 2017-05-11.09:50:08
[BitBucket user: Salim Kanoun]
[BitBucket date: 2017-05-11.07:50:08]

Sounds very good !
msg211 (view) Author: admin Date: 2017-06-15.04:17:48
[BitBucket user: Salim Kanoun]
[BitBucket date: 2017-06-15.02:17:48]

Dear Alain,

Since my AWServer reject the anonymized DICOM files by Orthanc (Header incomplete error without further explanaition) I reviewed the list of tags to anonymize so look for type 1 and type 2 tags.

I found several type 2 tags that should be nulled but not removed : 
Accession Number 0008, 0050
Acquisition context sequence 0040,0555
Acquisition date (2C for PET) 0008,0022
Acquisition Time (2C for PET) 0008,0032
Content Creator Name 0070,0084
Contrast Bolus Agent 0018,0010
Filler Order Number of Image service request 0040,2017
Patient ID 0010,0020
Patient Birth date 0010,0030
Patient Name 0010,0010
Patient Sex 0010,0040
Placer Order number of imaging service request 0040,2016
Referenced Study sequence 0008, 1110
Reffering physician name 0008,0090
Requested procedure description 0032,1060
Study Date 0008,0020
Study ID 0020,0010
Study Time 0008,0030
Verifiying Observer Idenditifcation Code sequence 0040,A088

I found then some type 1 and type 1C for which I'm less sure what about is needed to do : 
Concatenation UID (1C) 0020,9161
Content Sequence (1C) 0040,A730
Creator Version UID (type 1) 0008,9123
Digital Signature UID (type 1) 0400,0100
Dimension Organization UID (type 1) 0020,9164
Frame of Reference UID (Type 1) 0020, 0052 => I guess this missing one is the origin of my AWserver rejection
Graphic Annotation Sequence (type 1) 0070,0001
Modified Attributes sequence (type 1) 0400,0550
Person Name (Type 1C) 0040,A123
Referenced SOP instance UID (type 1C) 0008,1155
Requested procedure UID (1C) 0040,1001
Synchronization Frame of Reference UID (type 1C) 0020,0200
Text string (1C) 2030,0020
UID (1C) 0040,A124
Verifiying Observer Name 0040,A075
Verifying Observer Sequence 0040,A073
Verifying Organization 0040,A027
Content Time (1, 1C or 2C depending module) 0008,0033

Best regards,

Salim
msg212 (view) Author: admin Date: 2017-06-21.11:15:22
[BitBucket user: Alain Mazy]
[BitBucket date: 2017-06-21.09:15:22]

Dear Salim,

Thanks for the feedback.  I would translate that into 2 'software' requirements:

- we need a way to null tags instead of removing them
- for many UIDs, like the Frame Of Reference UID, we need to make sure we anonymize them the same way along all instances -> we should keep a map that will remember the anonymized UID of each source UID:

```
UID anonymize(UID& sourceUID) {
  // if this is the first time we encouter this sourceUID, generated a new UID and save it in a 'cache'

 // else, returned the cached UID
}
```
We'll need to define the lifetime of this 'cache'.  I would suggest that the cache is reset for each anonymization job.
msg213 (view) Author: admin Date: 2017-06-21.15:07:20
[BitBucket user: Salim Kanoun]
[BitBucket date: 2017-06-21.13:07:20]

Dear Alain,

I'm not an expert in UIDs things but it sounds logical to me do define the value at the anonymize job level. 
What you are proposing sounds good.

Clearly when you will have experimental versions I will be here to test (especially to check that the generated dicom will be accepted by AWserver).

We have finished an anonymization tool based on orthanc APIs that we will release with the GPL licence. 
It will be fully ready only after the debug of orthanc anonymization kernel, but once ready I have not doubt that we will have the best anonymization tool ever made.
msg214 (view) Author: admin Date: 2017-07-12.09:08:39
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2017-07-12.07:08:39]

Regarding the recursive mapping of UIDs mentioned above by Alain on 2017-06-21, this is an advanced construction that should be implemented as a plugin or as an external application. I have added an item in the [official roadmap](https://hg.orthanc-server.com/orthanc/changeset/06ff7e86638a466572639a3a76464ddc3a27036c).
msg215 (view) Author: admin Date: 2017-07-12.22:13:31
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2017-07-12.20:13:31]

The mainline of Orthanc now carries on anonymization according to "PS 3.15-2017c Table E.1-1 Basic Profile" thanks to the [following changeset](https://hg.orthanc-server.com/orthanc/changeset/d19e716b79fa9e4e47e3ea8b9e70ea84e1d987e5). All the tags mentioned in this issue should now be correctly anonymized.
History
Date User Action Args
2026-07-29 15:51:20admincreate