Mercurial > hg > orthanc-book
annotate Sphinx/source/plugins/object-storage.rst @ 1113:a588960a72e5 default tip
spelling
| author | Alain Mazy <am@orthanc.team> |
|---|---|
| date | Mon, 28 Oct 2024 09:23:08 +0100 |
| parents | ceeab11899ee |
| children |
| rev | line source |
|---|---|
| 451 | 1 .. _object-storage: |
| 2 | |
| 3 | |
| 4 Cloud Object Storage plugins | |
| 5 ============================ | |
| 6 | |
| 7 .. contents:: | |
| 8 | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
9 Release notes |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
10 ------------- |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
11 |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
12 Release notes are available `here |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
13 <https://orthanc.uclouvain.be/hg/orthanc-object-storage/file/default/NEWS>`__. |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
14 |
| 451 | 15 Introduction |
| 16 ------------ | |
| 17 | |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
18 These 3 plugins enable storing the Orthanc files into `object storage |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
19 <https://en.wikipedia.org/wiki/Object_storage>`__ at the 3 public |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
20 cloud providers: `AWS <https://aws.amazon.com/s3/>`__, `Azure |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
21 <https://azure.microsoft.com/en-us/services/storage/blobs/>`__, and |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
22 `Google Cloud <https://cloud.google.com/storage>`__. |
| 451 | 23 |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
24 Storing Orthanc files in object storage and your index SQL in a |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
25 managed database allows you to have a stateless Orthanc that does not |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
26 store any data in its local file system, which is highly recommended |
| 451 | 27 when deploying an application in the cloud. |
| 28 | |
| 29 | |
|
459
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
30 Pre-compiled binaries |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
31 --------------------- |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
32 |
|
1084
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
33 Note about proprietary cloud storage |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
34 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 904 | 35 |
|
459
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
36 These plugins are used to interface Orthanc with commercial and |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
37 proprietary cloud services that you accept to pay. As a consequence, |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
38 the Orthanc project usually doesn't freely update them or fix them |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
39 unless the requester purchases a support contract, e.g., at `Orthanc |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
40 Team <https://orthanc.team>`__. |
| 880 | 41 |
| 42 Although you are obviously free to compile these plugins by | |
|
459
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
43 yourself (instructions are given below), purchasing such support |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
44 contracts makes the Orthanc project sustainable in the long term, to |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
45 the benefit of the worldwide community of medical imaging. |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
46 |
|
a4ed4e883337
highlighting the pre-compiled binaries for google, aws and azure
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
453
diff
changeset
|
47 |
|
1084
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
48 ``orthancteam/orthanc`` Docker image |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
49 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
50 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
51 All the 3 plugins are provided as part of the ``orthancteam/orthanc`` |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
52 :ref:`Docker images <docker-orthancteam>`. The AWS plugin is available |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
53 in the default Docker images, while the Azure and Google plugins are |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
54 available in the ``-full`` images. |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
55 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
56 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
57 AWS S3 plugin |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
58 ^^^^^^^^^^^^^ |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
59 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
60 Because the AWS S3 plugin is compatible with free and open-source |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
61 implementations of the S3 protocol (notably MinIO and Ceph), we |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
62 provide pre-compiled binaries of the AWS S3 plugin, which can be found |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
63 at the following locations: |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
64 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
65 * In the `official Microsoft Windows installers <https://orthanc.uclouvain.be/downloads/windows-64/installers/index.html>`__, starting with release 24.6.3. |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
66 * For `Microsoft Windows 32 <https://orthanc.uclouvain.be/downloads/windows-32/orthanc-aws-s3/index.html>`__, |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
67 * For `Microsoft Windows 64 <https://orthanc.uclouvain.be/downloads/windows-64/orthanc-aws-s3/index.html>`__, |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
68 * For `cross-distribution Linux <https://orthanc.uclouvain.be/downloads/linux-standard-base/orthanc-aws-s3/index.html>`__, |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
69 * In the :ref:`backport Debian repository <binaries>` (notably compatible with Ubuntu 22.04 and 24.04), |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
70 * In the :ref:`jodogne/orthanc Docker images <docker>`, |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
71 * In the :ref:`orthancteam/orthanc Docker images <docker-orthancteam>`. |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
72 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
73 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
74 Microsoft Azure plugin |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
75 ^^^^^^^^^^^^^^^^^^^^^^ |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
76 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
77 The Azure plugin is also available as part of the `Windows Installer |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
78 <https://orthanc.uclouvain.be/downloads/windows-64/installers/index.html>`__ |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
79 (only for 64bits platform). |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
80 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
81 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
82 |
|
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
83 |
| 451 | 84 Configuration |
| 85 ------------- | |
| 86 | |
| 87 .. highlight:: json | |
| 88 | |
| 89 AWS S3 plugin | |
| 90 ^^^^^^^^^^^^^ | |
| 91 | |
| 92 Sample configuration:: | |
| 93 | |
| 94 "AwsS3Storage" : { | |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
95 "BucketName": "test-orthanc-s3-plugin", |
| 451 | 96 "Region" : "eu-central-1", |
| 685 | 97 "AccessKey" : "AKXXX", // optional: if not specified, the plugin will use the default credentials manager (available from version 1.3.0) |
| 98 "SecretKey" : "RhYYYY", // optional: if not specified, the plugin will use the default credentials manager (available from version 1.3.0) | |
| 99 "Endpoint": "", // optional: custom endpoint | |
| 100 "ConnectionTimeout": 30, // optional: connection timeout in seconds | |
| 101 "RequestTimeout": 1200, // optional: request timeout in seconds (max time to upload/download a file) | |
| 102 "RootPath": "", // optional: see below | |
| 103 "MigrationFromFileSystemEnabled": false, // optional: see below | |
| 104 "StorageStructure": "flat", // optional: see below | |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
105 "EnableLegacyUnknownFiles": true, // optional: see below |
| 685 | 106 "VirtualAddressing": true, // optional: see the section related to MinIO |
| 880 | 107 "StorageEncryption" : {}, // optional: see the section related to encryption |
| 988 | 108 "HybridMode": "Disabled", // optional: see the section related to Hybrid storage |
| 990 | 109 "UseTransferManager": false, // optional: see below (available from version 2.3.0) |
| 110 "EnableAwsSdkLogs": false // optional: include AWS SDK logs in Orthanc logs | |
| 451 | 111 } |
| 112 | |
| 464 | 113 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway:: |
| 114 | |
| 115 "AwsS3Storage" : { | |
| 116 "BucketName": "test-orthanc", | |
| 117 "Region": "fr-par", | |
| 118 "AccessKey": "XXX", | |
| 119 "SecretKey": "YYY", | |
| 120 "Endpoint": "s3.fr-par.scw.cloud" | |
|
505
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
121 } |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
122 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
123 |
| 989 | 124 The **UseTransferManager** configuration is used to select the `Transfer Manager <https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/examples-s3-transfermanager.html>`__ mode in the AWS SDK client. |
| 125 This option was introduced in version 2.3.0. If set to false (default value), the default "object" mode is used. | |
| 988 | 126 |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
127 **Important:** On Microsoft Windows, it is recommended to set the |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
128 environment variable ``AWS_EC2_METADATA_DISABLED`` to ``true`` to |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
129 speed up the initialization of the plugin. The reasons are explained |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
130 in the `AWS official documentation |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
131 <https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-envvars.html>`__. |
|
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
132 |
| 988 | 133 |
| 746 | 134 .. _minio: |
| 135 | |
|
505
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
136 Emulation of AWS S3 using MinIO |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
137 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
138 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
139 .. highlight:: bash |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
140 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
141 The `MinIO project <https://min.io/>`__ can be used to emulate AWS S3 |
|
507
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
142 for local testing/prototyping. Here is a sample command to start a |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
143 MinIO server on your local computer using Docker (evidently, make sure |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
144 to set different credentials):: |
|
505
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
145 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
146 $ docker run -p 9000:9000 \ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
147 -e "MINIO_REGION=eu-west-1" \ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
148 -e "MINIO_ACCESS_KEY=AKIAIOSFODNN7EXAMPLE" \ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
149 -e "MINIO_SECRET_KEY=wJalrXUtnFEMI/K7MNG/bPxRfiCYEXAMPLEKEY" \ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
150 minio/minio server /data |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
151 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
152 .. highlight:: json |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
153 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
154 Note that the ``MINIO_REGION`` must be set to an arbitrary region that |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
155 is supported by AWS S3. |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
156 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
157 You can then open the URL `http://localhost:9000/ |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
158 <http://localhost:9000/>`__ with your Web browser to create a bucket, |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
159 say ``my-sample-bucket``. |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
160 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
161 Here is a corresponding full configuration for Orthanc:: |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
162 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
163 { |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
164 "Plugins" : [ <...> ], |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
165 "AwsS3Storage" : { |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
166 "BucketName": "my-sample-bucket", |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
167 "Region" : "eu-west-1", |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
168 "Endpoint": "http://localhost:9000/", |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
169 "AccessKey": "AKIAIOSFODNN7EXAMPLE", |
| 506 | 170 "SecretKey": "wJalrXUtnFEMI/K7MNG/bPxRfiCYEXAMPLEKEY", |
|
505
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
171 "VirtualAddressing" : false |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
172 } |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
173 } |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
174 |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
175 Note that the ``VirtualAddressing`` option must be set to ``false`` |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
176 for such a `local setup with MinIO to work |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
177 <https://github.com/aws/aws-sdk-cpp/issues/1425>`__. This option is |
|
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
178 **not** available in releases <= 1.1.0 of the AWS S3 plugin. |
|
507
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
179 |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
180 **Important:** If you get the cryptic error message |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
181 ``SignatureDoesNotMatch The request signature we calculated does not |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
182 match the signature you provided. Check your key and signing |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
183 method.``, this most probably indicates that your access key or your |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
184 secret key doesn't match the credentials that were used while starting |
|
a51542cfdfeb
warning about minio credentials
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
506
diff
changeset
|
185 the MinIO server. |
|
505
e4bea5b97890
Emulation of AWS S3 using MinIO
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
504
diff
changeset
|
186 |
| 464 | 187 |
| 451 | 188 Azure Blob Storage plugin |
| 189 ^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 190 | |
| 191 Sample configuration:: | |
| 192 | |
| 193 "AzureBlobStorage" : { | |
| 194 "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=xxxxxxxxx;AccountKey=yyyyyyyy===;EndpointSuffix=core.windows.net", | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
195 "ContainerName" : "test-orthanc-storage-plugin", |
| 647 | 196 "CreateContainerIfNotExists": true, // available from version 1.2.0 |
| 502 | 197 "RootPath": "", // see below |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
198 "MigrationFromFileSystemEnabled": false, // see below |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
199 "StorageStructure": "flat", // see below |
| 880 | 200 "EnableLegacyUnknownFiles": true, // optional: see below |
|
1106
ceeab11899ee
typo (bc s first commit)
Benoit Crickboom <bc@orthanc.team>
parents:
1084
diff
changeset
|
201 "StorageEncryption" : {}, // optional: see the section related to encryption |
| 880 | 202 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage |
| 451 | 203 } |
| 204 | |
| 205 | |
| 206 Google Storage plugin | |
| 207 ^^^^^^^^^^^^^^^^^^^^^ | |
| 208 | |
| 209 Sample configuration:: | |
| 210 | |
| 211 "GoogleCloudStorage" : { | |
| 212 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json", | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
213 "BucketName": "test-orthanc-storage-plugin", |
| 502 | 214 "RootPath": "", // see below |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
215 "MigrationFromFileSystemEnabled": false, // see below |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
216 "StorageStructure": "flat", // see below |
| 880 | 217 "EnableLegacyUnknownFiles": true, // optional: see below |
|
1106
ceeab11899ee
typo (bc s first commit)
Benoit Crickboom <bc@orthanc.team>
parents:
1084
diff
changeset
|
218 "StorageEncryption" : {}, // optional: see the section related to encryption |
| 880 | 219 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage |
| 451 | 220 } |
| 221 | |
| 222 | |
| 880 | 223 Migration & Hybrid mode Storage structure |
| 224 ----------------------------------------- | |
| 225 | |
| 226 Since version **2.1.0** of the plugins, an HybridMode as been introduced. | |
| 227 This mode allows reading/writing files from both/to the file system and the object-storage. | |
| 228 | |
| 229 By default, the ``HybridMode`` is ``Disabled``. This means that the plugins will access | |
| 230 only the object-storage. | |
| 231 | |
| 232 When the ``HybridMode`` is set to ``WriteToFileSystem``, it means that new files received | |
| 885 | 233 are stored on the file system. When accessing a file, it is first read from the file system |
| 880 | 234 and, if it is not found on the file system, it is read from the object-storage. |
| 235 | |
| 884 | 236 The ``WriteToFileSystem`` hybrid mode is useful for storing recent files on the file system for |
| 880 | 237 better performance and old files on the object-storage for lower cost and easier backups. |
| 238 | |
| 239 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it means that new files received | |
| 885 | 240 are stored on the object storage. When accessing a file, it is first read from the object storage |
| 880 | 241 and, if it is not found on the object-storage, it is read from the file system. |
| 242 | |
| 884 | 243 The ``WriteToObjectStorage`` hybrid mode is useful mainly during a migration from file system to |
| 880 | 244 object-storage, e.g, if you have deployed a VM in a cloud with local file system storage and want |
| 245 to move your files to object-storage without interrupting your service. | |
| 246 | |
| 247 Moving files between file-system and object-storage | |
| 248 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 249 | |
| 884 | 250 When the ``HybridMode`` is set to ``WriteToFileSystem``, it is sometimes useful to move old files |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
251 to the object-storage for long term archive or to "pre-fetch" files from object-storage to file |
| 880 | 252 system for improved performances e.g when before opening the study in a viewer. |
| 253 | |
| 884 | 254 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it is useful to move file from the |
| 880 | 255 file system to the object storage to perform a full data migration to object-storage. |
| 256 | |
| 257 To move files from one storage to the other, you should call the plugin Rest API:: | |
| 258 | |
| 259 $ curl -X POST http://localhost:8042/move-storage \ | |
| 260 --data '{ | |
| 261 "Resources": ["27f7126f-4f66fb14-03f4081b-f9341db2-53925988"], | |
| 262 "TargetStorage": "file-system", | |
| 263 "Asynchronous": true, | |
| 264 "Priority": 0 | |
| 265 }' | |
| 266 | |
| 267 This call creates a ``MoveStorageJob`` that can then be monitor to the ``/jobs`` route. | |
| 268 | |
| 269 The allowed values for ``TargetStorage`` are ``file-system`` or ``object-storage``. | |
| 270 | |
| 271 | |
| 272 Other configuration options | |
| 273 --------------------------- | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
274 |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
275 The **StorageStructure** configuration allows you to select the way objects are organized |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
276 within the storage (``flat`` or ``legacy``). |
| 500 | 277 Unlike the traditional file system in which Orthanc uses 2 levels |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
278 of folders, an object storage usually has no limit on the number of files per folder and |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
279 therefore all objects are stored at the root level of the object storage. This is the |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
280 default ``flat`` behavior. Note that, in the ``flat`` mode, an extension ``.dcm`` or ``.json`` |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
281 is added to the filename which is not the case in the legacy mode. |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
282 |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
283 The ``legacy`` behavior mimics the Orthanc File System convention. This is actually helpful |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
284 when migrating your data from a file system to an object storage since you can copy all the file |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
285 hierarchy as is. |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
286 |
| 1014 | 287 *Note* : With AWS S3, there might be `some limitations <https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html>`__ |
| 288 on the number of HTTP operations per *prefix*. Therefore, the ``legacy`` structure might | |
| 289 overcome these limitations. Note that, as of today, no one could confirm it had a real impact on Orthanc. | |
| 290 | |
| 502 | 291 The **RootPath** allows you to store the files in another folder as the root level of the |
| 520 | 292 object storage. Note: it shall not start with a ``/``. |
| 502 | 293 |
| 294 Note that you can not change these configurations once you've uploaded the first files in Orthanc. | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
295 |
| 880 | 296 The **MigrationFromFileSystemEnabled** configuration has been superseded by the **HybridMode** in v 2.1.0. |
| 508 | 297 |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
298 The **EnableLegacyUnknownFiles** configuration has been introduced to allow recent version of the plugins (from 1.3.3) |
|
1023
5d4701d8fe28
replaced osimis/orthanc by orthancteam/orthanc
Alain Mazy <am@osimis.io>
parents:
1014
diff
changeset
|
299 continue working with data that was saved with Orthanc version around 1.9.3 and plugins version around 1.2.0 (e.g. orthancteam/orthanc:21.5.1 docker images). |
|
766
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
300 With these specific versions, some ``.unk`` files were generated instead of ``.dcm.head`` files. With this configuration option enabled, |
|
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
301 when reading files, the plugin will try both file extensions. |
|
73ee8a489b24
object-storage: EnableLegacyUnknownFiles
Alain Mazy <am@osimis.io>
parents:
746
diff
changeset
|
302 If you have ``.unk`` files in your storage, you must enable this configuration. |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
303 |
| 452 | 304 Sample setups |
| 305 ------------- | |
| 306 | |
| 962 | 307 You'll find sample deployments and more info in the `Orthanc Setup Samples repository <https://github.com/orthanc-server/orthanc-setup-samples/tree/master/#markdown-header-for-software-integrators>`__ . |
| 452 | 308 |
|
511
5b574520a34c
performance tests for object-storage
Alain Mazy <alain@mazy.be>
parents:
509
diff
changeset
|
309 Performances |
|
5b574520a34c
performance tests for object-storage
Alain Mazy <alain@mazy.be>
parents:
509
diff
changeset
|
310 ------------ |
|
5b574520a34c
performance tests for object-storage
Alain Mazy <alain@mazy.be>
parents:
509
diff
changeset
|
311 |
| 962 | 312 You'll find some performance comparison between VM SSDs and object-storage `here <https://github.com/orthanc-server/orthanc-setup-samples/tree/master/docker/performance-tests/>`__ . |
|
511
5b574520a34c
performance tests for object-storage
Alain Mazy <alain@mazy.be>
parents:
509
diff
changeset
|
313 |
| 452 | 314 |
|
586
5f5519f1491a
securing the storage
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
543
diff
changeset
|
315 .. _client-side-encryption: |
|
5f5519f1491a
securing the storage
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
543
diff
changeset
|
316 |
| 451 | 317 Client-side encryption |
| 318 ---------------------- | |
| 319 | |
| 320 Although all cloud providers already provide encryption at rest, the plugins provide | |
| 321 an optional layer of client-side encryption . It is very important that you understand | |
| 322 the scope and benefits of this additional layer of encryption. | |
| 323 | |
| 324 Rationale | |
| 325 ^^^^^^^^^ | |
| 326 | |
| 327 Encryption at rest provided by cloud providers basically compares with a file-system disk encryption. | |
| 328 If someone has access to the disk, he won't have access to your data without the encryption key. | |
| 329 | |
| 330 With cloud encryption at rest only, if someone has access to the "api-key" of your storage or if one | |
| 331 of your admin inadvertently make your storage public, `PHI <https://en.wikipedia.org/wiki/Protected_health_information>`__ will leak. | |
| 332 | |
| 333 Once you use client-side encryption, you'll basically store packets of meaningless bytes on the cloud infrastructure. | |
| 334 So, if an "api-key" leaks or if the storage is misconfigured, packets of bytes will leak but not PHI since | |
| 335 no one will be able to decrypt them. | |
| 336 | |
| 884 | 337 Another advantage is that these packets of bytes might eventually not be considered as PHI anymore and potentially |
| 451 | 338 help you meet your local regulations (Please check your local regulations). |
| 339 | |
| 340 However, note that, if you're running entirely in a cloud environment, your decryption keys will still | |
| 341 be stored on the cloud infrastructure (VM disks - process RAM) and an attacker could still eventually gain access to this keys. | |
| 342 | |
| 884 | 343 If Orthanc is running in your infrastructure with the Index DB on your infrastructure, and files are stored in the cloud, |
| 451 | 344 the master keys will remain on your infrastructure only and there's no way the data stored in the cloud could be decrypted outside your infrastructure. |
| 345 | |
| 346 Also note that, although the cloud providers also provide client-side encryption, we, as an open-source project, | |
| 347 wanted to provide our own implementation on which you'll have full control and extension capabilities. | |
| 348 This also allows us to implement the same logic on all cloud providers. | |
| 349 | |
| 350 Our encryption is based on well-known standards (see below). Since it is documented and the source code is open-source, | |
| 351 feel-free to have your security expert review it before using it in a production environment. | |
| 352 | |
| 353 Technical details | |
| 354 ^^^^^^^^^^^^^^^^^ | |
| 355 | |
| 356 Orthanc saves 2 kind of files: DICOM files and JSON summaries of DICOM files. Both files contain PHI. | |
| 357 | |
| 452 | 358 When configuring the plugin, you'll have to provide a **Master Key** that we can also call the **Key Encryption Key (KEK)**. |
| 451 | 359 |
| 452 | 360 For each file being saved, the plugin will generate a new **Data Encryption Key (DEK)**. This DEK, encrypted with the KEK will be pre-pended to the file. |
| 451 | 361 |
| 362 If, at any point, your KEK leaks or you want to rotate your KEKs, you'll be able to use a new one to encrypt new files that are being added | |
| 363 and still use the old ones to decrypt data. You could then eventually start a side script to remove usages of the leaked/obsolete KEKs. | |
| 364 | |
| 365 To summarize: | |
| 366 | |
| 452 | 367 - We use `Crypto++ <https://www.cryptopp.com/>`__ to perform all encryptions. |
| 451 | 368 - All keys (KEK and DEK) are AES-256 keys. |
| 369 - DEKs and IVs are encrypted by KEK using CTR block cipher using a null IV. | |
| 370 - data is encrypted by DEK using GCM block cipher that will also perform integrity check on the whole file. | |
| 371 | |
| 372 The format of data stored on disk is therefore the following: | |
| 373 | |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
374 - **VERSION HEADER**: 2 bytes: identify the structure of the following data currently ``A1`` |
| 451 | 375 - **MASTER KEY ID**: 4 bytes: a numerical ID of the KEK that was used to encrypt the DEK |
| 376 - **EIV**: 32 bytes: IV used by DEK for data encryption; encrypted by KEK | |
| 377 - **EDEK**: 32 bytes: the DEK encrypted by the KEK. | |
| 378 - **CIPHER TEXT**: variable length: the DICOM/JSON file encrypted by the DEK | |
| 379 - **TAG**: 16 bytes: integrity check performed on the whole encrypted file (including header, master key id, EIV and EDEK) | |
| 380 | |
| 381 Configuration | |
| 382 ^^^^^^^^^^^^^ | |
| 383 | |
| 384 .. highlight:: text | |
| 385 | |
| 386 AES Keys shall be 32 bytes long (256 bits) and encoded in base64. Here's a sample OpenSSL command to generate such a key:: | |
| 387 | |
| 388 openssl rand -base64 -out /tmp/test.key 32 | |
| 389 | |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
390 Each key must have a unique id that is a ``uint32`` number. |
| 451 | 391 |
| 392 .. highlight:: json | |
| 393 | |
|
1083
e1f7f1e2d148
document AWS_EC2_METADATA_DISABLED
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1023
diff
changeset
|
394 Here's a sample configuration file of the ``StorageEncryption`` section of the plugins:: |
| 451 | 395 |
| 396 { | |
|
499
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
397 "GoogleCloudStorage" : { |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
398 "StorageEncryption" : { |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
399 "Enable": true, |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
400 "MasterKey": [3, "/path/to/master.key"], // key id - path to the base64 encoded key |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
401 "PreviousMasterKeys" : [ |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
402 [1, "/path/to/previous1.key"], |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
403 [2, "/path/to/previous2.key"] |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
404 ], |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
405 "MaxConcurrentInputSize" : 1024 // size in MB |
|
d255e02eb89d
updated object-storage doc for 1.0.0
Alain Mazy <alain@mazy.be>
parents:
464
diff
changeset
|
406 } |
| 451 | 407 } |
| 408 } | |
| 409 | |
| 410 **MaxConcurrentInputSize**: Since the memory used during encryption/decryption can grow up to a bit more | |
| 411 than 2 times the input, we want to limit the number of threads doing concurrent processing according | |
| 412 to the available memory instead of the number of concurrent threads. Therefore, if you're currently | |
| 413 ingesting small files, you can have a lot of thread working together while, if you're ingesting large | |
| 414 files, threads might have to wait before receiving a "slot" to access the encryption module. | |
| 988 | 415 |
| 416 | |
| 417 Compilation | |
| 418 ----------- | |
| 419 | |
| 420 .. highlight:: text | |
| 421 | |
| 422 The procedure to compile the plugins is quite similar of that for the | |
| 423 :ref:`core of Orthanc <compiling>` although they usually require | |
| 424 some prerequisites. The documented procedure has been tested only | |
| 425 on a Debian Buster machine. | |
| 426 | |
| 427 The compilation of each plugin produces a shared library that contains | |
| 428 the plugin. | |
| 429 | |
| 430 | |
| 431 AWS S3 plugin | |
| 432 ^^^^^^^^^^^^^ | |
| 433 | |
| 434 Prerequisites: Compile the AWS C++ SDK:: | |
| 435 | |
| 436 $ mkdir ~/aws | |
| 437 $ cd ~/aws | |
| 438 $ git clone https://github.com/aws/aws-sdk-cpp.git | |
| 439 $ | |
| 440 $ mkdir -p ~/aws/builds/aws-sdk-cpp | |
| 441 $ cd ~/aws/builds/aws-sdk-cpp | |
| 442 $ cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp | |
| 443 $ make -j 4 | |
| 444 $ make install | |
| 445 | |
| 446 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: | |
| 447 | |
| 448 $ ./vcpkg install cryptopp | |
| 449 | |
| 450 Compile:: | |
| 451 | |
| 452 $ mkdir -p build/aws | |
| 453 $ cd build/aws | |
| 454 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws | |
| 455 | |
| 456 | |
| 457 **NB:** If you don't want to use vcpkg, you can use the following | |
| 458 command (this syntax is not compatible with Ninja yet):: | |
| 459 | |
|
1084
337bbecf74c0
notes about precompiled aws s3 binaries
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
1083
diff
changeset
|
460 $ cmake -DCMAKE_BUILD_TYPE=Debug -DUSE_VCPKG_PACKAGES=OFF -DUSE_SYSTEM_GOOGLE_TEST=OFF -DUSE_SYSTEM_ORTHANC_SDK=OFF ../../orthanc-object-storage/Aws |
| 988 | 461 $ make |
| 462 | |
| 463 Crypto++ must be installed (on Ubuntu, run ``sudo apt install libcrypto++-dev``). | |
| 464 | |
| 465 | |
| 466 Azure Blob Storage plugin | |
| 467 ^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 468 | |
| 469 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: | |
| 470 | |
| 471 $ ./vcpkg install cryptopp | |
| 472 $ ./vcpkg install azure-storage-cpp | |
| 473 | |
| 474 | |
| 475 Compile:: | |
| 476 | |
| 477 $ mkdir -p build/azure | |
| 478 $ cd build/azure | |
| 479 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure | |
| 480 | |
| 481 Google Storage plugin | |
| 482 ^^^^^^^^^^^^^^^^^^^^^ | |
| 483 | |
| 484 Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: | |
| 485 | |
| 486 $ ./vcpkg install cryptopp | |
| 487 $ ./vcpkg install google-cloud-cpp | |
| 488 | |
| 489 Compile:: | |
| 490 | |
| 491 $ mkdir -p build/google | |
| 492 $ cd build/google | |
| 493 $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/google |