Mercurial > hg > orthanc-book

comparison Sphinx/source/plugins/object-storage.rst @ 880:ac9b677b73c3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
object-storage 2.1.0
author Alain Mazy <am@osimis.io>
date Fri, 21 Oct 2022 14:57:05 +0200
parents d7d3117e5186
children d04096c1afd7
comparison
equal deleted inserted replaced
879:f4b3e015d57f 880:ac9b677b73c3
13 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/NEWS>`__ 13 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/NEWS>`__
14 14
15 Introduction 15 Introduction
16 ------------ 16 ------------
17 17
18 Osimis freely provides the `source code 18 These 3 plugins enable storing the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__
19 <https://hg.orthanc-server.com/orthanc-object-storage/file/default/>`__ of 3 plugins 19 at the 3 main Cloud providers: `AWS <https://aws.amazon.com/s3/>`__,
20 to store the Orthanc files in `Object Storage <https://en.wikipedia.org/wiki/Object_storage>`__
21 at the 3 main providers: `AWS <https://aws.amazon.com/s3/>`__,
22 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ & 20 `Azure <https://azure.microsoft.com/en-us/services/storage/blobs/>`__ &
23 `Google Cloud <https://cloud.google.com/storage>`__ 21 `Google Cloud <https://cloud.google.com/storage>`__
24 22
25 Storing Orthanc files in object storage and your index SQL in a 23 Storing Orthanc files in object storage and your index SQL in a
26 managed database allows you to have a stateless Orthanc that does 24 managed database allows you to have a stateless Orthanc that does
29 27
30 28
31 Pre-compiled binaries 29 Pre-compiled binaries
32 --------------------- 30 ---------------------
33 31
32 These plugins are provided as part of the ``osimis/orthanc`` :ref:`Docker images <docker-osimis>`.
33
34 These plugins are used to interface Orthanc with commercial and 34 These plugins are used to interface Orthanc with commercial and
35 proprietary cloud services that you accept to pay. As a consequence, 35 proprietary cloud services that you accept to pay. As a consequence,
36 the Orthanc project doesn't freely provide pre-compiled binaries for 36 the Orthanc project usually doesn't freely update them or fix them unless
37 Docker, Windows, Linux or OS X. These pre-compiled binaries do exist, 37 the requester purchases a support contract e.g. at `Orthanc Team <https://orthanc.team>`__.
38 but are reserved to the companies who have subscribed to a 38
39 `professional support contract 39 Although you are obviously free to compile these plugins by
40 <https://osimis.io/en/orthanc-support-contract>`__ by
41 Osimis. Although you are obviously free to compile these plugins by
42 yourself (instructions are given below), purchasing such support 40 yourself (instructions are given below), purchasing such support
43 contracts makes the Orthanc project sustainable in the long term, to 41 contracts makes the Orthanc project sustainable in the long term, to
44 the benefit of the worldwide community of medical imaging. 42 the benefit of the worldwide community of medical imaging.
45 43
46 44
144 "RootPath": "", // optional: see below 142 "RootPath": "", // optional: see below
145 "MigrationFromFileSystemEnabled": false, // optional: see below 143 "MigrationFromFileSystemEnabled": false, // optional: see below
146 "StorageStructure": "flat", // optional: see below 144 "StorageStructure": "flat", // optional: see below
147 "EnableLegacyUnknownFiles": true, // optional: see below 145 "EnableLegacyUnknownFiles": true, // optional: see below
148 "VirtualAddressing": true, // optional: see the section related to MinIO 146 "VirtualAddressing": true, // optional: see the section related to MinIO
149 "StorageEncryption" : {} // optional: see the section related to encryption 147 "StorageEncryption" : {}, // optional: see the section related to encryption
148 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage
150 } 149 }
151 150
152 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway:: 151 The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway::
153 152
154 "AwsS3Storage" : { 153 "AwsS3Storage" : {
224 "ContainerName" : "test-orthanc-storage-plugin", 223 "ContainerName" : "test-orthanc-storage-plugin",
225 "CreateContainerIfNotExists": true, // available from version 1.2.0 224 "CreateContainerIfNotExists": true, // available from version 1.2.0
226 "RootPath": "", // see below 225 "RootPath": "", // see below
227 "MigrationFromFileSystemEnabled": false, // see below 226 "MigrationFromFileSystemEnabled": false, // see below
228 "StorageStructure": "flat", // see below 227 "StorageStructure": "flat", // see below
229 "EnableLegacyUnknownFiles": true // optional: see below 228 "EnableLegacyUnknownFiles": true, // optional: see below
229 "StorageEncryption" : {} // optional: see the section related to encryption
230 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage
230 } 231 }
231 232
232 233
233 Google Storage plugin 234 Google Storage plugin
234 ^^^^^^^^^^^^^^^^^^^^^ 235 ^^^^^^^^^^^^^^^^^^^^^
239 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json", 240 "ServiceAccountFile": "/path/to/googleServiceAccountFile.json",
240 "BucketName": "test-orthanc-storage-plugin", 241 "BucketName": "test-orthanc-storage-plugin",
241 "RootPath": "", // see below 242 "RootPath": "", // see below
242 "MigrationFromFileSystemEnabled": false, // see below 243 "MigrationFromFileSystemEnabled": false, // see below
243 "StorageStructure": "flat", // see below 244 "StorageStructure": "flat", // see below
244 "EnableLegacyUnknownFiles": true // optional: see below 245 "EnableLegacyUnknownFiles": true, // optional: see below
245 } 246 "StorageEncryption" : {} // optional: see the section related to encryption
246 247 "HybridMode": "Disabled" // optional: see the section related to Hybrid storage
247 248 }
248 Migration & Storage structure 249
249 ----------------------------- 250
251 Migration & Hybrid mode Storage structure
252 -----------------------------------------
253
254 Since version **2.1.0** of the plugins, an HybridMode as been introduced.
255 This mode allows reading/writing files from both/to the file system and the object-storage.
256
257 By default, the ``HybridMode`` is ``Disabled``. This means that the plugins will access
258 only the object-storage.
259
260 When the ``HybridMode`` is set to ``WriteToFileSystem``, it means that new files received
261 are store on the file system. When accessing a file, it is first read from the file system
262 and, if it is not found on the file system, it is read from the object-storage.
263
264 The ``WriteToFileSystem`` hybrid mode is usefull for storing recent files on the file system for
265 better performance and old files on the object-storage for lower cost and easier backups.
266
267 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it means that new files received
268 are store on the object storage. When accessing a file, it is first read from the object storage
269 and, if it is not found on the object-storage, it is read from the file system.
270
271 The ``WriteToObjectStorage`` hybrid mode is usefull mainly during a migration from file system to
272 object-storage, e.g, if you have deployed a VM in a cloud with local file system storage and want
273 to move your files to object-storage without interrupting your service.
274
275 Moving files between file-system and object-storage
276 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
277
278 When the ``HybridMode`` is set to ``WriteToFileSystem``, it is sometimes usefull to move old files
279 to the object-storage for long term archive or to `pre-fetch`` files from object-storage to file
280 system for improved performances e.g when before opening the study in a viewer.
281
282 When the ``HybridMode`` is set to ``WriteToObjectStorage``, it is usefull to move file from the
283 file system to the object storage to perform a full data migration to object-storage.
284
285 To move files from one storage to the other, you should call the plugin Rest API::
286
287 $ curl -X POST http://localhost:8042/move-storage \
288 --data '{
289 "Resources": ["27f7126f-4f66fb14-03f4081b-f9341db2-53925988"],
290 "TargetStorage": "file-system",
291 "Asynchronous": true,
292 "Priority": 0
293 }'
294
295 This call creates a ``MoveStorageJob`` that can then be monitor to the ``/jobs`` route.
296
297 The allowed values for ``TargetStorage`` are ``file-system`` or ``object-storage``.
298
299
300 Other configuration options
301 ---------------------------
250 302
251 The **StorageStructure** configuration allows you to select the way objects are organized 303 The **StorageStructure** configuration allows you to select the way objects are organized
252 within the storage (``flat`` or ``legacy``). 304 within the storage (``flat`` or ``legacy``).
253 Unlike the traditional file system in which Orthanc uses 2 levels 305 Unlike the traditional file system in which Orthanc uses 2 levels
254 of folders, object storages usually have no limit on the number of files per folder and 306 of folders, object storages usually have no limit on the number of files per folder and
263 The **RootPath** allows you to store the files in another folder as the root level of the 315 The **RootPath** allows you to store the files in another folder as the root level of the
264 object storage. Note: it shall not start with a ``/``. 316 object storage. Note: it shall not start with a ``/``.
265 317
266 Note that you can not change these configurations once you've uploaded the first files in Orthanc. 318 Note that you can not change these configurations once you've uploaded the first files in Orthanc.
267 319
268 The **MigrationFromFileSystemEnabled** configuration has been introduced for Orthanc to continue working 320 The **MigrationFromFileSystemEnabled** configuration has been superseded by the **HybridMode** in v 2.1.0.
269 while you're migrating your data from the file system to the object storage. While this option is enabled,
270 Orthanc will store all new files into the object storage but will try to read/delete files
271 from both the file system and the object storage.
272
273 This option can be disabled as soon as all files have been copied from the file system to the
274 object storage. Note that Orthanc is not copying the files from one storage to the other; you'll
275 have to use a standard ``sync`` command from the object-storage provider.
276
277 A migration script from File System to Azure Blob Storage is available courtesy of `Steve Hawes <https://github.com/jodogne/OrthancContributed/blob/master/Scripts/Migration/2020-09-08-TransferToAzure.sh>`__ .
278 321
279 The **EnableLegacyUnknownFiles** configuration has been introduced to allow recent version of the plugins (from 1.3.3) 322 The **EnableLegacyUnknownFiles** configuration has been introduced to allow recent version of the plugins (from 1.3.3)
280 continue working with data that was saved with Orthanc version around 1.9.3 and plugins version around 1.2.0 (e.g. osimis/orthanc:21.5.1 docker images). 323 continue working with data that was saved with Orthanc version around 1.9.3 and plugins version around 1.2.0 (e.g. osimis/orthanc:21.5.1 docker images).
281 With these specific versions, some ``.unk`` files were generated instead of ``.dcm.head`` files. With this configuration option enabled, 324 With these specific versions, some ``.unk`` files were generated instead of ``.dcm.head`` files. With this configuration option enabled,
282 when reading files, the plugin will try both file extensions. 325 when reading files, the plugin will try both file extensions.