annotate Sphinx/source/plugins/object-storage.rst @ 500:4481882d9c83

typo
author Alain Mazy <alain@mazy.be>
date Tue, 01 Sep 2020 18:22:01 +0200
parents d255e02eb89d
children 4e426dec4fee
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
1 .. _object-storage:
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
2
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
3
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
4 Cloud Object Storage plugins
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
5 ============================
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
6
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
7 .. contents::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
8
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
9 Release notes
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
10 -------------
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
11
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
12 Release notes are available `here
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
13 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/NEWS>`__
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
14
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
15 Introduction
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
16 ------------
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
17
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
18 Osimis freely provides the `source code
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
19 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/>`__ of 3 plugins
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
20 to store the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
21 at the 3 main providers: `AWS <https://aws.amazon.com/s3/>`__,
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
22 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ &
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
23 `Google Cloud <https://cloud.google.com/storage>`__
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
24
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
25 Storing Orthanc files in object storage and your index SQL in a
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
26 managed database allows you to have a stateless Orthanc that does
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
27 not store any data in its local file system which is highly recommended
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
28 when deploying an application in the cloud.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
29
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
30
459
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
31 Pre-compiled binaries
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
32 ---------------------
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
33
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
34 These plugins are used to interface Orthanc with commercial and
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
35 proprietary cloud services that you accept to pay. As a consequence,
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
36 the Orthanc project doesn't freely provide pre-compiled binaries for
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
37 Docker, Windows, Linux or OS X. These pre-compiled binaries do exist,
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
38 but are reserved to the companies who have subscribed to a
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
39 `professional support contract
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
40 <https://www.osimis.io/en/services.html#cloud-plugins>`__ by
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
41 Osimis. Although you are obviously free to compile these plugins by
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
42 yourself (instructions are given below), purchasing such support
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
43 contracts makes the Orthanc project sustainable in the long term, to
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
44 the benefit of the worldwide community of medical imaging.
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
45
a4ed4e883337 highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 453
diff changeset
46
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
47 Compilation
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
48 -----------
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
49
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
50 .. highlight:: text
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
51
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
52 The procedure to compile the plugins is quite similar of that for the
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
53 :ref:`core of Orthanc <compiling>` although they usually require
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
54 some prerequisites. The documented procedure has been tested only
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
55 on a Debian Buster machine.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
56
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
57 The compilation of each plugin produces a shared library that contains
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
58 the plugin.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
59
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
60
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
61 AWS S3 plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
62 ^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
63
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
64 Prerequisites: Compile the AWS C++ SDK::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
65
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
66 $ mkdir ~/aws
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
67 $ cd ~/aws
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
68 $ git clone https://github.com/aws/aws-sdk-cpp.git
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
69 $
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
70 $ mkdir -p ~/aws/builds/aws-sdk-cpp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
71 $ cd ~/aws/builds/aws-sdk-cpp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
72 $ cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
73 $ make -j 4
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
74 $ make install
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
75
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
76 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
77
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
78 $ ./vcpkg install cryptopp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
79
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
80 Compile::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
81
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
82 $ mkdir -p build/aws
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
83 $ cd build/aws
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
84 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
85
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
86 Azure Blob Storage plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
87 ^^^^^^^^^^^^^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
88
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
89 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
90
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
91 $ ./vcpkg install cpprestsdk
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
92
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
93
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
94 Compile::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
95
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
96 $ mkdir -p build/azure
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
97 $ cd build/azure
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
98 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
99
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
100 Google Storage plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
101 ^^^^^^^^^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
102
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
103 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
104
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
105 $ ./vcpkg install google-cloud-cpp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
106 $ ./vcpkg install cryptopp
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
107
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
108 Compile::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
109
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
110 $ mkdir -p build/google
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
111 $ cd build/google
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
112 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/google
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
113
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
114
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
115 Configuration
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
116 -------------
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
117
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
118 .. highlight:: json
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
119
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
120 AWS S3 plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
121 ^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
122
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
123 Sample configuration::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
124
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
125 "AwsS3Storage" : {
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
126 "BucketName": "test-orthanc-s3-plugin",
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
127 "Region" : "eu-central-1",
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
128 "AccessKey" : "AKXXX",
463
fae16ea8b13c new AWS params
Alain Mazy <alain@mazy.be>
parents: 459
diff changeset
129 "SecretKey" : "RhYYYY",
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
130 "Endpoint": "", // custom endpoint
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
131 "ConnectionTimeout": 30, // connection timeout in seconds
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
132 "RequestTimeout": 1200, // request timeout in seconds (max time to upload/download a file)
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
133 "MigrationFromFileSystemEnabled": false, // see below
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
134 "StorageStructure": "flat" // see below
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
135 }
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
136
464
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
137 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway::
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
138
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
139 "AwsS3Storage" : {
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
140 "BucketName": "test-orthanc",
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
141 "Region": "fr-par",
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
142 "AccessKey": "XXX",
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
143 "SecretKey": "YYY",
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
144 "Endpoint": "s3.fr-par.scw.cloud"
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
145 },
5ea70331c0be AWS Endpoint doc
Alain Mazy <alain@mazy.be>
parents: 463
diff changeset
146
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
147 Azure Blob Storage plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
148 ^^^^^^^^^^^^^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
149
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
150 Sample configuration::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
151
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
152 "AzureBlobStorage" : {
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
153 "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=xxxxxxxxx;AccountKey=yyyyyyyy===;EndpointSuffix=core.windows.net",
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
154 "ContainerName" : "test-orthanc-storage-plugin",
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
155 "MigrationFromFileSystemEnabled": false, // see below
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
156 "StorageStructure": "flat" // see below
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
157 }
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
158
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
159
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
160 Google Storage plugin
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
161 ^^^^^^^^^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
162
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
163 Sample configuration::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
164
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
165 "GoogleCloudStorage" : {
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
166 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json",
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
167 "BucketName": "test-orthanc-storage-plugin",
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
168 "MigrationFromFileSystemEnabled": false, // see below
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
169 "StorageStructure": "flat" // see below
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
170 }
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
171
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
172
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
173 Migration & Storage structure
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
174 -----------------------------
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
175
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
176 The **StorageStructure** configuration allows you to select the way objects are organized
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
177 within the storage (``flat`` or ``legacy``).
500
Alain Mazy <alain@mazy.be>
parents: 499
diff changeset
178 Unlike the traditional file system in which Orthanc uses 2 levels
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
179 of folders, object storages usually have no limit on the number of files per folder and
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
180 therefore all objects are stored at the root level of the object storage. This is the
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
181 default ``flat`` behaviour. Note that, in the ``flat`` mode, an extension `.dcm` or `.json`
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
182 is added to the filename which is not the case in the legacy mode.
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
183
500
Alain Mazy <alain@mazy.be>
parents: 499
diff changeset
184 The ``legacy`` behaviour mimics the Orthanc File System convention. This is actually helpful
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
185 when migrating your data from a file system to an object storage since you can copy all the file
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
186 hierarchy as is.
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
187
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
188 Note that you can not change this configuration once you've uploaded the first files in Orthanc.
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
189
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
190 The **MigrationFromFileSystemEnabled** configuration has been for Orthanc to continue working
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
191 while you're migrating your data from the file system to the object storage. While this option is enabled,
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
192 Orthanc will store all new files into the object storage but will try to read/delete files
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
193 from both the file system and the object storage.
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
194
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
195 This option can be disabled as soon as all files have been copied from the file system to the
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
196 object storage. Note that Orthanc is not copying the files from one storage to the other; you'll
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
197 have to use a standard ``sync`` command from the object-storage provider.
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
198
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
199
452
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
200 Sample setups
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
201 -------------
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
202
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
203 You'll find sample deployments and more info in the `Orthanc Setup Samples repository <https://bitbucket.org/osimis/orthanc-setup-samples/src/master/#markdown-header-for-osimisorthanc-pro-image-users>`__ .
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
204
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
205
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
206 Client-side encryption
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
207 ----------------------
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
208
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
209 Although all cloud providers already provide encryption at rest, the plugins provide
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
210 an optional layer of client-side encryption . It is very important that you understand
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
211 the scope and benefits of this additional layer of encryption.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
212
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
213 Rationale
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
214 ^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
215
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
216 Encryption at rest provided by cloud providers basically compares with a file-system disk encryption.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
217 If someone has access to the disk, he won't have access to your data without the encryption key.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
218
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
219 With cloud encryption at rest only, if someone has access to the "api-key" of your storage or if one
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
220 of your admin inadvertently make your storage public, `PHI <https://en.wikipedia.org/wiki/Protected_health_information>`__ will leak.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
221
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
222 Once you use client-side encryption, you'll basically store packets of meaningless bytes on the cloud infrastructure.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
223 So, if an "api-key" leaks or if the storage is misconfigured, packets of bytes will leak but not PHI since
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
224 no one will be able to decrypt them.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
225
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
226 Another advantage is that these packets of bytes might eventually not be considered as PHI anymore and eventually
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
227 help you meet your local regulations (Please check your local regulations).
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
228
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
229 However, note that, if you're running entirely in a cloud environment, your decryption keys will still
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
230 be stored on the cloud infrastructure (VM disks - process RAM) and an attacker could still eventually gain access to this keys.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
231
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
232 If Orthanc is running in your infrastructure with the Index DB on your infrastructure, and files are store in the cloud,
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
233 the master keys will remain on your infrastructure only and there's no way the data stored in the cloud could be decrypted outside your infrastructure.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
234
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
235 Also note that, although the cloud providers also provide client-side encryption, we, as an open-source project,
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
236 wanted to provide our own implementation on which you'll have full control and extension capabilities.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
237 This also allows us to implement the same logic on all cloud providers.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
238
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
239 Our encryption is based on well-known standards (see below). Since it is documented and the source code is open-source,
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
240 feel-free to have your security expert review it before using it in a production environment.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
241
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
242 Technical details
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
243 ^^^^^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
244
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
245 Orthanc saves 2 kind of files: DICOM files and JSON summaries of DICOM files. Both files contain PHI.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
246
452
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
247 When configuring the plugin, you'll have to provide a **Master Key** that we can also call the **Key Encryption Key (KEK)**.
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
248
452
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
249 For each file being saved, the plugin will generate a new **Data Encryption Key (DEK)**. This DEK, encrypted with the KEK will be pre-pended to the file.
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
250
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
251 If, at any point, your KEK leaks or you want to rotate your KEKs, you'll be able to use a new one to encrypt new files that are being added
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
252 and still use the old ones to decrypt data. You could then eventually start a side script to remove usages of the leaked/obsolete KEKs.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
253
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
254 To summarize:
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
255
452
aef5c8b74381 fix + link to sample
Alain Mazy <alain@mazy.be>
parents: 451
diff changeset
256 - We use `Crypto++ <https://www.cryptopp.com/>`__ to perform all encryptions.
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
257 - All keys (KEK and DEK) are AES-256 keys.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
258 - DEKs and IVs are encrypted by KEK using CTR block cipher using a null IV.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
259 - data is encrypted by DEK using GCM block cipher that will also perform integrity check on the whole file.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
260
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
261 The format of data stored on disk is therefore the following:
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
262
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
263 - **VERSION HEADER**: 2 bytes: identify the structure of the following data currently `A1`
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
264 - **MASTER KEY ID**: 4 bytes: a numerical ID of the KEK that was used to encrypt the DEK
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
265 - **EIV**: 32 bytes: IV used by DEK for data encryption; encrypted by KEK
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
266 - **EDEK**: 32 bytes: the DEK encrypted by the KEK.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
267 - **CIPHER TEXT**: variable length: the DICOM/JSON file encrypted by the DEK
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
268 - **TAG**: 16 bytes: integrity check performed on the whole encrypted file (including header, master key id, EIV and EDEK)
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
269
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
270 Configuration
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
271 ^^^^^^^^^^^^^
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
272
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
273 .. highlight:: text
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
274
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
275 AES Keys shall be 32 bytes long (256 bits) and encoded in base64. Here's a sample OpenSSL command to generate such a key::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
276
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
277 openssl rand -base64 -out /tmp/test.key 32
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
278
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
279 Each key must have a unique id that is a uint32 number.
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
280
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
281 .. highlight:: json
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
282
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
283 Here's a sample configuration file of the `StorageEncryption` section of the plugins::
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
284
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
285 {
499
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
286 "GoogleCloudStorage" : {
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
287 "StorageEncryption" : {
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
288 "Enable": true,
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
289 "MasterKey": [3, "/path/to/master.key"], // key id - path to the base64 encoded key
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
290 "PreviousMasterKeys" : [
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
291 [1, "/path/to/previous1.key"],
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
292 [2, "/path/to/previous2.key"]
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
293 ],
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
294 "MaxConcurrentInputSize" : 1024 // size in MB
d255e02eb89d updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents: 464
diff changeset
295 }
451
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
296 }
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
297 }
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
298
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
299 **MaxConcurrentInputSize**: Since the memory used during encryption/decryption can grow up to a bit more
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
300 than 2 times the input, we want to limit the number of threads doing concurrent processing according
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
301 to the available memory instead of the number of concurrent threads. Therefore, if you're currently
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
302 ingesting small files, you can have a lot of thread working together while, if you're ingesting large
938206110483 added object storage
Alain Mazy <alain@mazy.be>
parents:
diff changeset
303 files, threads might have to wait before receiving a "slot" to access the encryption module.