Mercurial > hg > orthanc-book
comparison Sphinx/source/plugins/object-storage.rst @ 880:ac9b677b73c3
object-storage 2.1.0
author | Alain Mazy <am@osimis.io> |
---|---|
date | Fri, 21 Oct 2022 14:57:05 +0200 |
parents | d7d3117e5186 |
children | d04096c1afd7 |
comparison
equal
deleted
inserted
replaced
879:f4b3e015d57f | 880:ac9b677b73c3 |
---|---|
13 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/NEWS>`__ | 13 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/NEWS>`__ |
14 | 14 |
15 Introduction | 15 Introduction |
16 ------------ | 16 ------------ |
17 | 17 |
18 Osimis freely provides the `source code | 18 These 3 plugins enable storing the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__ |
19 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/>`__ of 3 plugins | 19 at the 3 main Cloud providers: `AWS <https://aws.amazon.com/s3/>`__, |
20 to store the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__ | |
21 at the 3 main providers: `AWS <https://aws.amazon.com/s3/>`__, | |
22 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ & | 20 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ & |
23 `Google Cloud <https://cloud.google.com/storage>`__ | 21 `Google Cloud <https://cloud.google.com/storage>`__ |
24 | 22 |
25 Storing Orthanc files in object storage and your index SQL in a | 23 Storing Orthanc files in object storage and your index SQL in a |
26 managed database allows you to have a stateless Orthanc that does | 24 managed database allows you to have a stateless Orthanc that does |
29 | 27 |
30 | 28 |
31 Pre-compiled binaries | 29 Pre-compiled binaries |
32 --------------------- | 30 --------------------- |
33 | 31 |
32 These plugins are provided as part of the ``osimis/orthanc`` :ref:`Docker images <docker-osimis>`. | |
33 | |
34 These plugins are used to interface Orthanc with commercial and | 34 These plugins are used to interface Orthanc with commercial and |
35 proprietary cloud services that you accept to pay. As a consequence, | 35 proprietary cloud services that you accept to pay. As a consequence, |
36 the Orthanc project doesn't freely provide pre-compiled binaries for | 36 the Orthanc project usually doesn't freely update them or fix them unless |
37 Docker, Windows, Linux or OS X. These pre-compiled binaries do exist, | 37 the requester purchases a support contract e.g. at `Orthanc Team <https://orthanc.team>`__. |
38 but are reserved to the companies who have subscribed to a | 38 |
39 `professional support contract | 39 Although you are obviously free to compile these plugins by |
40 <https://osimis.io/en/orthanc-support-contract>`__ by | |
41 Osimis. Although you are obviously free to compile these plugins by | |
42 yourself (instructions are given below), purchasing such support | 40 yourself (instructions are given below), purchasing such support |
43 contracts makes the Orthanc project sustainable in the long term, to | 41 contracts makes the Orthanc project sustainable in the long term, to |
44 the benefit of the worldwide community of medical imaging. | 42 the benefit of the worldwide community of medical imaging. |
45 | 43 |
46 | 44 |
144 "RootPath": "", // optional: see below | 142 "RootPath": "", // optional: see below |
145 "MigrationFromFileSystemEnabled": false, // optional: see below | 143 "MigrationFromFileSystemEnabled": false, // optional: see below |
146 "StorageStructure": "flat", // optional: see below | 144 "StorageStructure": "flat", // optional: see below |
147 "EnableLegacyUnknownFiles": true, // optional: see below | 145 "EnableLegacyUnknownFiles": true, // optional: see below |
148 "VirtualAddressing": true, // optional: see the section related to MinIO | 146 "VirtualAddressing": true, // optional: see the section related to MinIO |
149 "StorageEncryption" : {} // optional: see the section related to encryption | 147 "StorageEncryption" : {}, // optional: see the section related to encryption |
148 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage | |
150 } | 149 } |
151 | 150 |
152 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway:: | 151 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway:: |
153 | 152 |
154 "AwsS3Storage" : { | 153 "AwsS3Storage" : { |
224 "ContainerName" : "test-orthanc-storage-plugin", | 223 "ContainerName" : "test-orthanc-storage-plugin", |
225 "CreateContainerIfNotExists": true, // available from version 1.2.0 | 224 "CreateContainerIfNotExists": true, // available from version 1.2.0 |
226 "RootPath": "", // see below | 225 "RootPath": "", // see below |
227 "MigrationFromFileSystemEnabled": false, // see below | 226 "MigrationFromFileSystemEnabled": false, // see below |
228 "StorageStructure": "flat", // see below | 227 "StorageStructure": "flat", // see below |
229 "EnableLegacyUnknownFiles": true // optional: see below | 228 "EnableLegacyUnknownFiles": true, // optional: see below |
229 "StorageEncryption" : {} // optional: see the section related to encryption | |
230 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage | |
230 } | 231 } |
231 | 232 |
232 | 233 |
233 Google Storage plugin | 234 Google Storage plugin |
234 ^^^^^^^^^^^^^^^^^^^^^ | 235 ^^^^^^^^^^^^^^^^^^^^^ |
239 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json", | 240 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json", |
240 "BucketName": "test-orthanc-storage-plugin", | 241 "BucketName": "test-orthanc-storage-plugin", |
241 "RootPath": "", // see below | 242 "RootPath": "", // see below |
242 "MigrationFromFileSystemEnabled": false, // see below | 243 "MigrationFromFileSystemEnabled": false, // see below |
243 "StorageStructure": "flat", // see below | 244 "StorageStructure": "flat", // see below |
244 "EnableLegacyUnknownFiles": true // optional: see below | 245 "EnableLegacyUnknownFiles": true, // optional: see below |
245 } | 246 "StorageEncryption" : {} // optional: see the section related to encryption |
246 | 247 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage |
247 | 248 } |
248 Migration & Storage structure | 249 |
249 ----------------------------- | 250 |
251 Migration & Hybrid mode Storage structure | |
252 ----------------------------------------- | |
253 | |
254 Since version **2.1.0** of the plugins, an HybridMode as been introduced. | |
255 This mode allows reading/writing files from both/to the file system and the object-storage. | |
256 | |
257 By default, the ``HybridMode`` is ``Disabled``. This means that the plugins will access | |
258 only the object-storage. | |
259 | |
260 When the ``HybridMode`` is set to ``WriteToFileSystem``, it means that new files received | |
261 are store on the file system. When accessing a file, it is first read from the file system | |
262 and, if it is not found on the file system, it is read from the object-storage. | |
263 | |
264 The ``WriteToFileSystem`` hybrid mode is usefull for storing recent files on the file system for | |
265 better performance and old files on the object-storage for lower cost and easier backups. | |
266 | |
267 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it means that new files received | |
268 are store on the object storage. When accessing a file, it is first read from the object storage | |
269 and, if it is not found on the object-storage, it is read from the file system. | |
270 | |
271 The ``WriteToObjectStorage`` hybrid mode is usefull mainly during a migration from file system to | |
272 object-storage, e.g, if you have deployed a VM in a cloud with local file system storage and want | |
273 to move your files to object-storage without interrupting your service. | |
274 | |
275 Moving files between file-system and object-storage | |
276 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
277 | |
278 When the ``HybridMode`` is set to ``WriteToFileSystem``, it is sometimes usefull to move old files | |
279 to the object-storage for long term archive or to `pre-fetch`` files from object-storage to file | |
280 system for improved performances e.g when before opening the study in a viewer. | |
281 | |
282 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it is usefull to move file from the | |
283 file system to the object storage to perform a full data migration to object-storage. | |
284 | |
285 To move files from one storage to the other, you should call the plugin Rest API:: | |
286 | |
287 $ curl -X POST http://localhost:8042/move-storage \ | |
288 --data '{ | |
289 "Resources": ["27f7126f-4f66fb14-03f4081b-f9341db2-53925988"], | |
290 "TargetStorage": "file-system", | |
291 "Asynchronous": true, | |
292 "Priority": 0 | |
293 }' | |
294 | |
295 This call creates a ``MoveStorageJob`` that can then be monitor to the ``/jobs`` route. | |
296 | |
297 The allowed values for ``TargetStorage`` are ``file-system`` or ``object-storage``. | |
298 | |
299 | |
300 Other configuration options | |
301 --------------------------- | |
250 | 302 |
251 The **StorageStructure** configuration allows you to select the way objects are organized | 303 The **StorageStructure** configuration allows you to select the way objects are organized |
252 within the storage (``flat`` or ``legacy``). | 304 within the storage (``flat`` or ``legacy``). |
253 Unlike the traditional file system in which Orthanc uses 2 levels | 305 Unlike the traditional file system in which Orthanc uses 2 levels |
254 of folders, object storages usually have no limit on the number of files per folder and | 306 of folders, object storages usually have no limit on the number of files per folder and |
263 The **RootPath** allows you to store the files in another folder as the root level of the | 315 The **RootPath** allows you to store the files in another folder as the root level of the |
264 object storage. Note: it shall not start with a ``/``. | 316 object storage. Note: it shall not start with a ``/``. |
265 | 317 |
266 Note that you can not change these configurations once you've uploaded the first files in Orthanc. | 318 Note that you can not change these configurations once you've uploaded the first files in Orthanc. |
267 | 319 |
268 The **MigrationFromFileSystemEnabled** configuration has been introduced for Orthanc to continue working | 320 The **MigrationFromFileSystemEnabled** configuration has been superseded by the **HybridMode** in v 2.1.0. |
269 while you're migrating your data from the file system to the object storage. While this option is enabled, | |
270 Orthanc will store all new files into the object storage but will try to read/delete files | |
271 from both the file system and the object storage. | |
272 | |
273 This option can be disabled as soon as all files have been copied from the file system to the | |
274 object storage. Note that Orthanc is not copying the files from one storage to the other; you'll | |
275 have to use a standard ``sync`` command from the object-storage provider. | |
276 | |
277 A migration script from File System to Azure Blob Storage is available courtesy of `Steve Hawes <https://github.com/jodogne/OrthancContributed/blob/master/Scripts/Migration/2020-09-08-TransferToAzure.sh>`__ . | |
278 | 321 |
279 The **EnableLegacyUnknownFiles** configuration has been introduced to allow recent version of the plugins (from 1.3.3) | 322 The **EnableLegacyUnknownFiles** configuration has been introduced to allow recent version of the plugins (from 1.3.3) |
280 continue working with data that was saved with Orthanc version around 1.9.3 and plugins version around 1.2.0 (e.g. osimis/orthanc:21.5.1 docker images). | 323 continue working with data that was saved with Orthanc version around 1.9.3 and plugins version around 1.2.0 (e.g. osimis/orthanc:21.5.1 docker images). |
281 With these specific versions, some ``.unk`` files were generated instead of ``.dcm.head`` files. With this configuration option enabled, | 324 With these specific versions, some ``.unk`` files were generated instead of ``.dcm.head`` files. With this configuration option enabled, |
282 when reading files, the plugin will try both file extensions. | 325 when reading files, the plugin will try both file extensions. |