Mercurial > hg > orthanc-book
changeset 1140:e3142f727541
clarified storage/index
author | Alain Mazy <am@orthanc.team> |
---|---|
date | Mon, 10 Feb 2025 09:57:26 +0100 |
parents | f736ab8fcac3 |
children | 6512a953c17b |
files | Sphinx/source/faq/orthanc-storage.rst |
diffstat | 1 files changed, 69 insertions(+), 24 deletions(-) [+] |
line wrap: on
line diff
--- a/Sphinx/source/faq/orthanc-storage.rst Tue Feb 04 13:07:21 2025 +0100 +++ b/Sphinx/source/faq/orthanc-storage.rst Mon Feb 10 09:57:26 2025 +0100 @@ -3,19 +3,41 @@ How does Orthanc store its database? ==================================== +Orthanc actually uses 2 different places to store its data: + +* the files are saved in the :ref:`Storage Area <orthanc-storage-area>`, + usually, a standard file-system but it can also be replaced by a cloud + :ref:`object storage <object-storage>` like AWS S3, Azure blob storage or Google cloud. +* a summary of all resources is saved in the :ref:`Index <orthanc-index>` + that is a SQL database. + +Orthanc always needs both the ``Storage`` and the ``Index`` and these 2 components +must always remain synchronized. + + .. _orthanc-storage-area: Storage area ------------ -**By default** (i.e. if no database plugin such as :ref:`PostgreSQL -<postgresql>` or :ref:`MySQL <mysql>` is used), Orthanc stores all the +**By default**, Orthanc stores all the DICOM files it receives in a folder called ``OrthancStorage`` on the -filesystem. +filesystem (defined in the ``StorageDirectory`` configuration in the +:ref:`configuration file <configuration>`). + +The default storage can also be replaced by a plugin to store these +files in an :ref:`object storage <object-storage>` like AWS S3, Azure +blob storage or Google cloud. -More precisely, the ``OrthancStorage`` folder contains a set of +Alternatively, the file storage can also be implemented inside a +:ref:`PostgreSQL <postgresql>` or :ref:`MySQL <mysql>` Database but +this is actually quite uncommon. + +More precisely, the ``Storage`` contains a set of so-called **attachments**, that may correspond to either a DICOM file, -a JSON file, or any user-defined file. Internally, each attachment is +a JSON file, or any user-defined file. + +Internally, each attachment is automatically associated with an `universally unique identifier (UUID) <https://en.wikipedia.org/wiki/Universally_unique_identifier>`__. Orthanc can be configured to compress these files on-the-fly in order @@ -29,21 +51,41 @@ folder, and the two next characters give the second-level folder. -SQLite index ------------- + +.. _orthanc-index: + +Orthanc Index +------------- + +Orthanc also maintains a summary of all the DICOM resources in a SQL +database in the so called ``Index``. This ``Index`` is mandatory to +rapidly provide information when browsing and accessing the resources +either through the :ref:`REST API of Orthanc <rest>` or through the +:ref:`DICOM protocol <dicom-guide>`. -Inside the same ``OrthancStorage`` folder, Orthanc maintains a `SQLite -database <https://en.wikipedia.org/wiki/SQLite>`__ called ``index`` -that **indexes** all these attachments. The database records, for each -attachment, its compression method, and its MD5 hashes before and +**By default**, this index is implemented in a `SQLite +database <https://en.wikipedia.org/wiki/SQLite>`__ that is stored +in the same folder as the files (if you are using a file-system). +This folder is defined by the ``IndexDirectory`` in the :ref:`configuration +option <configuration>`) + +The default ``Index`` can also be replaced by a plugin to store the +index in a :ref:`PostgreSQL <postgresql>`, :ref:`MySQL <mysql>` or +:ref:`ODBC <odbc>` Database. + + +Index content +------------- + +The ``Index`` database **indexes** all the attachments stored in the ``Storage``. +The database records, for each attachment, its compression method, and its MD5 hashes before and after compression in order to detect disk corruption (cf. the ``StoreMD5ForAttachments`` :ref:`configuration option <configuration>`). One attachment must be associated with one :ref:`DICOM resource <model-world>` (patient, study, series, or instance). Incoming DICOM -files and associated JSON summary are associated with one -instance-level resource, but user-defined attachments can be +files are associated with one instance-level resource, but user-defined attachments can be associated with any kind of resource. Given one DICOM resource, all of its child attachments are identified @@ -55,9 +97,10 @@ information for each DICOM resource, notably the :ref:`metadata <metadata>`, the :ref:`history of changes <changes>`, and an associative map that stores the so-called "main" DICOM tags (to avoid -accessing the storage folder are when this is not needed). The SQLite -database schema is kept as simple as possible, and can be found in the -following two files of the source code of Orthanc: +accessing the storage folder are when this is not needed). + +The database schema is kept as simple as possible, e.g, for SQLite, +the schema can be found in the following two files of the source code of Orthanc: `PrepareDatabase.sql <https://orthanc.uclouvain.be/hg/orthanc/file/Orthanc-1.12.6/OrthancServer/Sources/Database/PrepareDatabase.sql>`__ and `InstallTrackAttachmentsSize.sql @@ -67,20 +110,22 @@ Direct access ------------- -Directly accessing the content of the ``OrthancStorage`` folder and -the content of the SQLite/MySQL/PostgreSQL database is strongly +Directly accessing the content of the ``Storage`` folder and +the content of the SQLite/MySQL/PostgreSQL ``Index`` database is strongly discouraged for several reasons: -* The internal organization outlined above is only true when no +* The ``Storage`` internal organization outlined above is only true when no database plugin is used (e.g. the :ref:`PostgreSQL <postgresql>` and :ref:`MySQL <mysql>` plugins can be configured to store the attachments inside a database). * Orthanc can be configured to compress the attachments before writing - them on the disk (cf. the ``StorageCompression`` option). -* By directly reading the content of ``OrthancStorage``, you bypass + them on the disk (cf. the ``StorageCompression`` option) making them + less easily readable by an external tool (check the ``OrthancRecoverCompressedFile`` + executable in the Orthanc distribution). +* By directly reading/writing the content of the ``Storage``, you bypass all the locking mechanisms used by Orthanc, which might result in data corruption. -* One SQLite database should be accessed by at most one process at any +* If you are using SQLite for the ``Index``, one SQLite database should be accessed by at most one process at any time to avoid any problem (e.g. with NFS filesystems), for reasons that are `explained in the SQLite FAQ <https://www.sqlite.org/faq.html#q5>`__. Orthanc will stop if it @@ -89,10 +134,10 @@ successive versions of Orthanc or of the database plugins. As a consequence, it is **HIGHLY recommended NOT to directly access** -the ``OrthancStorage`` folder and the SQLite/MySQL/PostgreSQL +the ``Storage`` and the SQLite/MySQL/PostgreSQL ``Index`` database. Use the :ref:`REST API <rest>` instead, which contains primitives to access the attachments (cf. the ``.../attachments/...`` -URIs). +URIs) and all other resources. The only exception to this rule is for **read-only access when Orthanc is stopped**, e.g. as a part of a :ref:`backup <backup>` or