Mercurial > hg > orthanc-book
changeset 1167:84e0b7ec5a1b
advanced storage + pg 8
author | Alain Mazy <am@orthanc.team> |
---|---|
date | Mon, 16 Jun 2025 21:50:32 +0200 |
parents | 37664f3f523b |
children | 6a592d137f23 |
files | Sphinx/source/faq/orthanc-storage.rst Sphinx/source/plugins.rst Sphinx/source/plugins/advanced-storage.rst Sphinx/source/plugins/delayed-deletion-plugin.rst Sphinx/source/plugins/housekeeper.rst Sphinx/source/plugins/indexer.rst Sphinx/source/plugins/postgresql.rst |
diffstat | 7 files changed, 332 insertions(+), 2 deletions(-) [+] |
line wrap: on
line diff
--- a/Sphinx/source/faq/orthanc-storage.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/faq/orthanc-storage.rst Mon Jun 16 21:50:32 2025 +0200 @@ -50,6 +50,9 @@ first two hexadecimal characters of the UUID give the first-level folder, and the two next characters give the second-level folder. +Since Orthanc 1.12.8, a new :ref:`advanced storage <advanced-storage>` +plugin can be used to organize the storage in a more user friendly +structure. .. _orthanc-index:
--- a/Sphinx/source/plugins.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/plugins.rst Mon Jun 16 21:50:32 2025 +0200 @@ -103,6 +103,7 @@ .. toctree:: :maxdepth: 1 + plugins/advanced-storage.rst plugins/authorization.rst plugins/delayed-deletion-plugin.rst plugins/gdcm.rst
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Sphinx/source/plugins/advanced-storage.rst Mon Jun 16 21:50:32 2025 +0200 @@ -0,0 +1,296 @@ +.. _advanced-storage: + + +Advanced storage plugin +======================= + +.. contents:: + +This **official plugin** extends Orthanc with an advanced +storage mechanism: + +- It enables customization of the structure of the Orthanc storage. +- It supports multiple storages. +- It supports indexing an existing storage - replacing the :ref:`Indexer <indexer>` plugin. +- It supports delayed deletion - replacing the :ref:`Delayed deletion plugin <delayed-deletion-plugin>` plugin + +Restrictions +------------ + +.. warning:: + This plugin only works with either the default SQLite database (from Orthanc 1.12.8) + or the PostgreSQL Index plugin (from version 8.0). + + +How to get it ? +--------------- + +The source code is available on `Github <https://github.com/orthanc-server/orthanc-advanced-storage/>`__. + +Binaries are included in: + +- The `orthancteam/orthanc Docker image <https://hub.docker.com/r/orthancteam/orthanc>`__ +- The `Windows Installer <https://www.orthanc-server.com/download-windows.php>`__ +- The `MacOS packages <https://www.orthanc-server.com/static.php?page=download-mac>`__ + +Release notes are available `here <https://github.com/orthanc-server/orthanc-advanced-storage/blob/master/release-notes.md>`__. + +Compilation instructions are available below. + + +Usage +----- + +.. highlight:: json + +Once Orthanc is installed, you must change the :ref:`configuration file +<configuration>` to tell Orthanc where it can find the plugin: This is +done by properly modifying the ``Plugins`` option. You could for +instance use the following configuration file:: + + { + "Name" : "MyOrthanc", + [...] + "Plugins" : [ + "/home/user/OrthancAuthorization/Build/libAdvancedStorage.so" + ], + "AdvancedStorage" : { + "Enable": true, + + // .. see below "configuration section" + } + } + +Orthanc must of course be restarted after the modification of its +configuration file. + + +Configuration +------------- + +A full description of all configuration options is available directly +in the source code `default configuration file <https://github.com/orthanc-server/orthanc-advanced-storage/blob/master/Plugin/Configuration.json>`__. + + +Multiple storages +----------------- + +If your current storage gets full, you may configuration the plugin +to use another disk to store newly received DICOM files:: + + { + "StorageDirectory": "/var/lib/orthanc/db/", + ... + "AdvancedStorage": { + "Enable": true, + "MultipleStorages": { + "Storages": { + "ext1": "/mnt/disk1/orthanc", + "ext2": "/mnt/disk2/orthanc", + }, + "CurrentWriteStorage": "ext1" + } + } + +E.g., with the above configuration, Orthanc might have ingested DICOM +files a long time ago. These files are still stored in ``/var/lib/orthanc/db/`` +(the main ``StorageDirectory``). The advanced storage plugin will +still be able to read files from this directory. + +Then, the advanced storage plugin has been configured to use 2 extra disks +whose ids are ``ext1`` and ``ext2``. These identifiers can never change since +they are stored in DB for each file. + +The ``CurrentWriteStorage`` configuration defines where the new received +files are stored. + +Note that, if one of the disks gets full, you'll have to switch the ``CurrentWriteStorage`` +configuration manually. + + +Customizing paths +----------------- + +By default, in Orthanc, each file is +automatically associated with an `universally unique identifier (UUID) +<https://en.wikipedia.org/wiki/Universally_unique_identifier>`__ and files are +stored in a 3-level hierarchy of directories; the +first two hexadecimal characters of the UUID give the first-level +folder, and the two next characters give the second-level folder (e.g.: ``/var/lib/orthanc/db/5f/39/5f3936ea-95b6-4ad9-b0f1-4075be3e52d0``). +This structure is machine friendly but is not convenient to browse for a human. + +One of the great feature of the advanced storage plugin is its ability to +customize the storage structure. The ``NamingScheme`` configuration enables full +customization of the storage structure by using :ref:`DICOM Tags <main-dicom-tags>` or :ref:`Orthanc identifiers +<orthanc-ids>` from the DICOM instance. + +E.g., a ``NamingScheme`` of ``{PatientID} - {PatientName}/{StudyDate} - {StudyDescription}/{SeriesNumber}/{pad6(InstanceNumber)}-{UUID}{.ext}`` +will produce paths like ``1234 - WHO^JOHN/20241102 - HEAD SCAN/100/000007-5f3936ea-95b6-4ad9-b0f1-4075be3e52d0.dcm``. + +Check the `configuration file <https://github.com/orthanc-server/orthanc-advanced-storage/blob/master/Plugin/Configuration.json>`__ for +a full description of the keywords that can be used in the ``NamingScheme``. + +To prevent files from being overwritten, it is very important that their path is unique ! +Therefore, your NamingScheme must always include: + +- either the file ``{UUID}`` +- or, if you have not set ``"OverwriteInstances"`` to true, at least: + + - a patient identifier ``{PatientID}`` or ``{OrthancPatientID}``, + - a study identifier ``{StudyInstanceUID}`` or ``{OrthancStudyID}``, + - a series identifier ``{SeriesInstanceUID}`` or ``{OrthancSeriesID}``, + - an instance identifier ``{SOPInstanceUID}`` or ``{OrthancInstanceID}`` + +The ``NamingScheme`` defines a **relative** path to either the ``"StorageDirectory"`` of Orthanc or one of +the ``"MultipleStorages"`` of this plugin. + +The relative path generated from the ``NamingScheme`` is stored in the SQL DB. Therefore, you may change the +``NamingScheme`` at any time and you'll still be able to access previously saved files. + + +Indexer mode +------------ + +This plugin can actually also replace the :ref:`Folder Indexer plugin<indexer>`. The +Indexer plugin needed a separate SQLite DB which made it impossible to use with multiple +Orthanc instances or uncomfortable to use together with the PostgreSQL plugin. + +The advanced-storage implements the same features as the Indexer plugin without requiring +a separate DB. Everything is stored in the Orthanc main Database. + +The ``Indexer mode`` has its own configuration:: + + { + "StorageDirectory": "/var/lib/orthanc/db/", + ... + "AdvancedStorage": { + "Enable": true, + "Indexer": { + "Enable": true, + "Folders": [ "/tmp/dicom-files" ], + ... + "TakeOwnership": false + } + } + +Check the `configuration file <https://github.com/orthanc-server/orthanc-advanced-storage/blob/master/Plugin/Configuration.json>`__ for +all the ``Indexer mode`` configurations. + +If you set ``TakeOwnership`` to false (default), the ``Indexer mode`` will have the +exact same behavior as the Indexer plugin. Orthanc will not own the indexed files +and will therefore not delete the files if you delete the related resources in Orthanc. + +If you set ``TakeOwnership`` to true, all indexed files will belong to Orthanc +and will therefore delete the files if you delete the related resources in Orthanc. + +Setting ``TakeOwnership`` to true is useful e.g. when you have been using Orthanc with +the default SQLite DB and you wish to switch to PostgreSQL. Orthanc will then be able +to *adopt* the DICOM files from the previous Orthanc installation. TODO: check this sample setup TODO + + +Delayed deletion mode +--------------------- + +This plugin can actually also replace the :ref:`Delayed deletion plugin<delayed-deletion-plugin>`. The +Delayed deletion plugin needed a separate SQLite DB which made it impossible to use with multiple +Orthanc instances or uncomfortable to use together with the PostgreSQL plugin. + +The advanced-storage implements the same features as the Delayed deletion plugin without requiring +a separate DB. Everything is stored in the Orthanc main Database. + +The ``Delayed deletion mode`` has its own configuration:: + + { + "StorageDirectory": "/var/lib/orthanc/db/", + ... + "AdvancedStorage": { + "Enable": true, + "DelayedDeletion": { + "Enable": true, + ... + } + } + +Check the `configuration file <https://github.com/orthanc-server/orthanc-advanced-storage/blob/master/Plugin/Configuration.json>`__ for +all the ``Delayed deletion mode`` configurations. + + + +REST API extensions +------------------- + +This plugin brings in a few new API routes: + +**adopt-instance** to adopt an instance that is outside the storage. This is equivalent to the +Indexer mode adopting an instance:: + + $ curl http://localhost:8042/plugins/advances-storage/adopt-instance -d @- << EOF + { + "Path" : "/tmp/my-dicom-file.dcm", + "TakeOwnership" : false + } + EOF + +**abandon-instance** to remove an adopted instance (if Orthanc is not the owner of the instance). +This is equivalent to the Indexer mode abandoning an instance e.g. the indexed file has been deleted:: + + $ curl http://localhost:8042/plugins/advances-storage/abandon-instance -d @- << EOF + { + "Path" : "/tmp/my-dicom-file.dcm" + } + EOF + +**move-storage** to move a resource from a storage to another one. Note: it does not recompute the relative path +but only change the base path (aka ``StorageId``):: + + $ curl http://localhost:8044/plugins/advanced-storage/move-storage -d @- << EOF + { + "Resources": ["ca58b590-8a115ed5-906f7f21-c7af8058-2637f722"], + "TargetStorageId": "ext2" + } + EOF + + +The plugin now provides extra information in the **../attachments/info** routes. E.g.:: + + $ curl http://localhost:8042/instances/ca58b590-8a115ed5-906f7f21-c7af8058-2637f722/attachments/dicom/info + + will return these new fields + + { + ... + "IsOwnedByOrthanc" : true, + "Path" : "1234 - WHO^JOHN/20241102 - HEAD SCAN/100/000007-5f3936ea-95b6-4ad9-b0f1-4075be3e52d0.dcm", + "StorageId" : "ext1" + } + + +The plugin also provides its status in this route **/plugins/advanced-storage/status**. E.g.:: + + $ curl http://localhost:8042/plugins/advanced-storage/status + + will return + + { + "DelayedDeletionIsActive": true, + "FilesPendingDeletion": 123, + "IndexerIsActive": true + } + + +Compilation +----------- + +.. highlight:: bash + +The procedure to compile this plugin is similar of that for the +:ref:`core of Orthanc <binaries>`. The following commands should work +for most UNIX-like distribution (including GNU/Linux):: + + $ mkdir Build + $ cd Build + $ cmake .. -DSTATIC_BUILD=ON -DCMAKE_BUILD_TYPE=Release + $ make + +The compilation will produce a shared library ``AdvancedStorage`` +that contains the plugin.
--- a/Sphinx/source/plugins/delayed-deletion-plugin.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/plugins/delayed-deletion-plugin.rst Mon Jun 16 21:50:32 2025 +0200 @@ -18,6 +18,11 @@ This queue is stored in a SQLite DB stored on disk which allows the plugin to resume deletions if Orthanc is stopped/restarted while deleting files. +.. note:: + This plugin now has an alternative implementation as part of the + :ref:`Advanced storage <advanced-storage>` plugin. + + Configuration -------------
--- a/Sphinx/source/plugins/housekeeper.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/plugins/housekeeper.rst Mon Jun 16 21:50:32 2025 +0200 @@ -50,6 +50,13 @@ // any changes in configuration "Force": false, + // New in 1.12.9 + // If "Force" is set to true, forces the "ReconstructFiles" + // option when reconstructing resources even if the plugin + // did not detect any changes in the configuration that + // should trigger a Reconstruct. + "ForceReconstructFiles": false, + // Delay (in seconds) between reconstruction of 2 studies // This avoids overloading Orthanc with the housekeeping // process and leaves room for other operations.
--- a/Sphinx/source/plugins/indexer.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/plugins/indexer.rst Mon Jun 16 21:50:32 2025 +0200 @@ -18,6 +18,10 @@ clients. The DICOM files are **not** copied, so this solution has a very small footprint in terms of storage requirements. +.. note:: + This plugin now has an alternative implementation as part of the + :ref:`Advanced storage <advanced-storage>` plugin. + Compilation -----------
--- a/Sphinx/source/plugins/postgresql.rst Sat Jun 14 17:53:24 2025 +0200 +++ b/Sphinx/source/plugins/postgresql.rst Mon Jun 16 21:50:32 2025 +0200 @@ -308,16 +308,23 @@ +---------------------------+-------------------------------------------+ | 7.0 - 7.1 | 3 | +---------------------------+-------------------------------------------+ -| from 7.2 | 4 | +| 7.2 | 4 | ++---------------------------+-------------------------------------------+ +| from 8.0 | 5 | +---------------------------+-------------------------------------------+ Upgrades from one revision to the other is always automatic. Furthermore, if you are upgrading -from e.g plugin 3.3 to 7.2, Orthanc will apply all migration steps autonomously. +from e.g plugin 3.3 to 8.0, Orthanc will apply all migration steps autonomously. However, if, for some reasons, you would like to reinstall a previous plugin version, the older plugin might refuse to start because the revision is newer and unknown to it. +To downgrade from revision 5 to revision 4, one might run this procedure:: + + $ wget https://orthanc.uclouvain.be/hg/orthanc-databases/raw-file/default/PostgreSQL/Plugins/SQL/Downgrades/Rev5ToRev4.sql + $ psql -U postgres -f Rev5ToRev4.sql + To downgrade from revision 4 to revision 3, one might run this procedure:: $ wget https://orthanc.uclouvain.be/hg/orthanc-databases/raw-file/default/PostgreSQL/Plugins/SQL/Downgrades/Rev4ToRev3.sql @@ -373,6 +380,13 @@ $ psql -U postgres -f Rev3ToRev4.sql $ psql -U postgres -f PrepareIndex.sql +To upgrade manually from revision 4 to revision 5:: + + $ wget https://orthanc.uclouvain.be/hg/orthanc-databases/raw-file/default/PostgreSQL/Plugins/SQL/Upgrades/Rev4ToRev5.sql + $ wget https://orthanc.uclouvain.be/hg/orthanc-databases/raw-file/default/PostgreSQL/Plugins/SQL/PrepareIndex.sql + $ psql -U postgres -f Rev4ToRev5.sql + $ psql -U postgres -f PrepareIndex.sql + These procedures are identical to the one performed automatically by Orthanc when it detects that an upgraded is required.