|
451
|
1 .. _object-storage:
|
|
|
2
|
|
|
3
|
|
|
4 Cloud Object Storage plugins
|
|
|
5 ============================
|
|
|
6
|
|
|
7 .. contents::
|
|
|
8
|
|
|
9
|
|
|
10 Introduction
|
|
|
11 ------------
|
|
|
12
|
|
|
13 Osimis freely provides the `source code
|
|
|
14 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/>`__ of 3 plugins
|
|
|
15 to store the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__
|
|
|
16 at the 3 main providers: `AWS <https://aws.amazon.com/s3/>`__,
|
|
|
17 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ &
|
|
|
18 `Google Cloud <https://cloud.google.com/storage>`__
|
|
|
19
|
|
|
20 Storing Orthanc files in object storage and your index SQL in a
|
|
|
21 managed database allows you to have a stateless Orthanc that does
|
|
|
22 not store any data in its local file system which is highly recommended
|
|
|
23 when deploying an application in the cloud.
|
|
|
24
|
|
|
25
|
|
|
26 Compilation
|
|
|
27 -----------
|
|
|
28
|
|
|
29 .. highlight:: text
|
|
|
30
|
|
|
31 The procedure to compile the plugins is quite similar of that for the
|
|
|
32 :ref:`core of Orthanc <compiling>` although they usually require
|
|
|
33 some prerequisites. The documented procedure has been tested only
|
|
|
34 on a Debian Buster machine.
|
|
|
35
|
|
|
36 The compilation of each plugin produces a shared library that contains
|
|
|
37 the plugin.
|
|
|
38
|
|
|
39 Given thes plugins are used to interface with a commercial & proprietary
|
|
|
40 service, pre-compiled Windows/Docker binaries are available only for
|
|
|
41 companies who have subscribed for a `support contract <https://www.osimis.io/en/services.html#cloud-plugins>`__ at Osimis.
|
|
|
42
|
|
|
43
|
|
|
44 AWS S3 plugin
|
|
|
45 ^^^^^^^^^^^^^
|
|
|
46
|
|
|
47 Prerequisites: Compile the AWS C++ SDK::
|
|
|
48
|
|
|
49 $ mkdir ~/aws
|
|
|
50 $ cd ~/aws
|
|
|
51 $ git clone https://github.com/aws/aws-sdk-cpp.git
|
|
|
52 $
|
|
|
53 $ mkdir -p ~/aws/builds/aws-sdk-cpp
|
|
|
54 $ cd ~/aws/builds/aws-sdk-cpp
|
|
|
55 $ cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp
|
|
|
56 $ make -j 4
|
|
|
57 $ make install
|
|
|
58
|
|
|
59 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
|
|
|
60
|
|
|
61 $ ./vcpkg install cryptopp
|
|
|
62
|
|
|
63 Compile::
|
|
|
64
|
|
|
65 $ mkdir -p build/aws
|
|
|
66 $ cd build/aws
|
|
|
67 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws
|
|
|
68
|
|
|
69 Azure Blob Storage plugin
|
|
|
70 ^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
71
|
|
|
72 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
|
|
|
73
|
|
|
74 $ ./vcpkg install cpprestsdk
|
|
|
75
|
|
|
76
|
|
|
77 Compile::
|
|
|
78
|
|
|
79 $ mkdir -p build/azure
|
|
|
80 $ cd build/azure
|
|
|
81 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure
|
|
|
82
|
|
|
83 Google Storage plugin
|
|
|
84 ^^^^^^^^^^^^^^^^^^^^^
|
|
|
85
|
|
|
86 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
|
|
|
87
|
|
|
88 $ ./vcpkg install google-cloud-cpp
|
|
|
89 $ ./vcpkg install cryptopp
|
|
|
90
|
|
|
91 Compile::
|
|
|
92
|
|
|
93 $ mkdir -p build/google
|
|
|
94 $ cd build/google
|
|
|
95 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/google
|
|
|
96
|
|
|
97
|
|
|
98 Configuration
|
|
|
99 -------------
|
|
|
100
|
|
|
101 .. highlight:: json
|
|
|
102
|
|
|
103 AWS S3 plugin
|
|
|
104 ^^^^^^^^^^^^^
|
|
|
105
|
|
|
106 Sample configuration::
|
|
|
107
|
|
|
108 "AwsS3Storage" : {
|
|
|
109 "BucketName": "test-orthanc-s3-plugin",
|
|
|
110 "Region" : "eu-central-1",
|
|
|
111 "AccessKey" : "AKXXX",
|
|
|
112 "SecretKey" : "RhYYYY"
|
|
|
113 }
|
|
|
114
|
|
|
115 Azure Blob Storage plugin
|
|
|
116 ^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
117
|
|
|
118 Sample configuration::
|
|
|
119
|
|
|
120 "AzureBlobStorage" : {
|
|
|
121 "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=xxxxxxxxx;AccountKey=yyyyyyyy===;EndpointSuffix=core.windows.net",
|
|
|
122 "ContainerName" : "test-orthanc-storage-plugin"
|
|
|
123 }
|
|
|
124
|
|
|
125
|
|
|
126 Google Storage plugin
|
|
|
127 ^^^^^^^^^^^^^^^^^^^^^
|
|
|
128
|
|
|
129 Sample configuration::
|
|
|
130
|
|
|
131 "GoogleCloudStorage" : {
|
|
|
132 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json",
|
|
|
133 "BucketName": "test-orthanc-storage-plugin"
|
|
|
134 }
|
|
|
135
|
|
|
136
|
|
452
|
137 Sample setups
|
|
|
138 -------------
|
|
|
139
|
|
|
140 You'll find sample deployments and more info in the `Orthanc Setup Samples repository <https://bitbucket.org/osimis/orthanc-setup-samples/src/master/#markdown-header-for-osimisorthanc-pro-image-users>`__ .
|
|
|
141
|
|
|
142
|
|
451
|
143 Client-side encryption
|
|
|
144 ----------------------
|
|
|
145
|
|
|
146 Although all cloud providers already provide encryption at rest, the plugins provide
|
|
|
147 an optional layer of client-side encryption . It is very important that you understand
|
|
|
148 the scope and benefits of this additional layer of encryption.
|
|
|
149
|
|
|
150 Rationale
|
|
|
151 ^^^^^^^^^
|
|
|
152
|
|
|
153 Encryption at rest provided by cloud providers basically compares with a file-system disk encryption.
|
|
|
154 If someone has access to the disk, he won't have access to your data without the encryption key.
|
|
|
155
|
|
|
156 With cloud encryption at rest only, if someone has access to the "api-key" of your storage or if one
|
|
|
157 of your admin inadvertently make your storage public, `PHI <https://en.wikipedia.org/wiki/Protected_health_information>`__ will leak.
|
|
|
158
|
|
|
159 Once you use client-side encryption, you'll basically store packets of meaningless bytes on the cloud infrastructure.
|
|
|
160 So, if an "api-key" leaks or if the storage is misconfigured, packets of bytes will leak but not PHI since
|
|
|
161 no one will be able to decrypt them.
|
|
|
162
|
|
|
163 Another advantage is that these packets of bytes might eventually not be considered as PHI anymore and eventually
|
|
|
164 help you meet your local regulations (Please check your local regulations).
|
|
|
165
|
|
|
166 However, note that, if you're running entirely in a cloud environment, your decryption keys will still
|
|
|
167 be stored on the cloud infrastructure (VM disks - process RAM) and an attacker could still eventually gain access to this keys.
|
|
|
168 Furthermore, in the scope of the `Cloud Act <https://en.wikipedia.org/wiki/CLOUD_Act>`__ , the cloud provider might still have
|
|
|
169 the possibility to retrieve your data and encryption key (while it will still be more complex than with standard encryption at rest).
|
|
|
170
|
|
|
171 If Orthanc is running in your infrastructure with the Index DB on your infrastructure, and files are store in the cloud,
|
|
|
172 the master keys will remain on your infrastructure only and there's no way the data stored in the cloud could be decrypted outside your infrastructure.
|
|
|
173
|
|
|
174 Also note that, although the cloud providers also provide client-side encryption, we, as an open-source project,
|
|
|
175 wanted to provide our own implementation on which you'll have full control and extension capabilities.
|
|
|
176 This also allows us to implement the same logic on all cloud providers.
|
|
|
177
|
|
|
178 Our encryption is based on well-known standards (see below). Since it is documented and the source code is open-source,
|
|
|
179 feel-free to have your security expert review it before using it in a production environment.
|
|
|
180
|
|
|
181 Technical details
|
|
|
182 ^^^^^^^^^^^^^^^^^
|
|
|
183
|
|
|
184 Orthanc saves 2 kind of files: DICOM files and JSON summaries of DICOM files. Both files contain PHI.
|
|
|
185
|
|
452
|
186 When configuring the plugin, you'll have to provide a **Master Key** that we can also call the **Key Encryption Key (KEK)**.
|
|
451
|
187
|
|
452
|
188 For each file being saved, the plugin will generate a new **Data Encryption Key (DEK)**. This DEK, encrypted with the KEK will be pre-pended to the file.
|
|
451
|
189
|
|
|
190 If, at any point, your KEK leaks or you want to rotate your KEKs, you'll be able to use a new one to encrypt new files that are being added
|
|
|
191 and still use the old ones to decrypt data. You could then eventually start a side script to remove usages of the leaked/obsolete KEKs.
|
|
|
192
|
|
|
193 To summarize:
|
|
|
194
|
|
452
|
195 - We use `Crypto++ <https://www.cryptopp.com/>`__ to perform all encryptions.
|
|
451
|
196 - All keys (KEK and DEK) are AES-256 keys.
|
|
|
197 - DEKs and IVs are encrypted by KEK using CTR block cipher using a null IV.
|
|
|
198 - data is encrypted by DEK using GCM block cipher that will also perform integrity check on the whole file.
|
|
|
199
|
|
|
200 The format of data stored on disk is therefore the following:
|
|
|
201
|
|
|
202 - **VERSION HEADER**: 2 bytes: identify the structure of the following data currently `A1`
|
|
|
203 - **MASTER KEY ID**: 4 bytes: a numerical ID of the KEK that was used to encrypt the DEK
|
|
|
204 - **EIV**: 32 bytes: IV used by DEK for data encryption; encrypted by KEK
|
|
|
205 - **EDEK**: 32 bytes: the DEK encrypted by the KEK.
|
|
|
206 - **CIPHER TEXT**: variable length: the DICOM/JSON file encrypted by the DEK
|
|
|
207 - **TAG**: 16 bytes: integrity check performed on the whole encrypted file (including header, master key id, EIV and EDEK)
|
|
|
208
|
|
|
209 Configuration
|
|
|
210 ^^^^^^^^^^^^^
|
|
|
211
|
|
|
212 .. highlight:: text
|
|
|
213
|
|
|
214 AES Keys shall be 32 bytes long (256 bits) and encoded in base64. Here's a sample OpenSSL command to generate such a key::
|
|
|
215
|
|
|
216 openssl rand -base64 -out /tmp/test.key 32
|
|
|
217
|
|
|
218 Each key must have a unique id that is a uint32 number.
|
|
|
219
|
|
|
220 .. highlight:: json
|
|
|
221
|
|
|
222 Here's a sample configuration file of the `StorageEncryption` section of the plugins::
|
|
|
223
|
|
|
224 {
|
|
|
225 "StorageEncryption" : {
|
|
|
226 "Enable": true,
|
|
|
227 "MasterKey": [3, "/path/to/master.key"], // key id - path to the base64 encoded key
|
|
|
228 "PreviousMasterKeys" : [
|
|
|
229 [1, "/path/to/previous1.key"],
|
|
|
230 [2, "/path/to/previous2.key"]
|
|
|
231 ],
|
|
|
232 "MaxConcurrentInputSize" : 1024 // size in MB
|
|
|
233 }
|
|
|
234 }
|
|
|
235
|
|
|
236 **MaxConcurrentInputSize**: Since the memory used during encryption/decryption can grow up to a bit more
|
|
|
237 than 2 times the input, we want to limit the number of threads doing concurrent processing according
|
|
|
238 to the available memory instead of the number of concurrent threads. Therefore, if you're currently
|
|
|
239 ingesting small files, you can have a lot of thread working together while, if you're ingesting large
|
|
|
240 files, threads might have to wait before receiving a "slot" to access the encryption module.
|
