changeset 1140:e3142f727541

clarified storage/index
author Alain Mazy <am@orthanc.team>
date Mon, 10 Feb 2025 09:57:26 +0100
parents f736ab8fcac3
children 6512a953c17b
files Sphinx/source/faq/orthanc-storage.rst
diffstat 1 files changed, 69 insertions(+), 24 deletions(-) [+]
line wrap: on
line diff
--- a/Sphinx/source/faq/orthanc-storage.rst	Tue Feb 04 13:07:21 2025 +0100
+++ b/Sphinx/source/faq/orthanc-storage.rst	Mon Feb 10 09:57:26 2025 +0100
@@ -3,19 +3,41 @@
 How does Orthanc store its database?
 ====================================
 
+Orthanc actually uses 2 different places to store its data:
+
+* the files are saved in the :ref:`Storage Area <orthanc-storage-area>`, 
+  usually, a standard file-system but it can also be replaced by a cloud 
+  :ref:`object storage <object-storage>` like AWS S3, Azure blob storage or Google cloud.
+* a summary of all resources is saved in the :ref:`Index <orthanc-index>`
+  that is a SQL database.
+
+Orthanc always needs both the ``Storage`` and the ``Index`` and these 2 components
+must always remain synchronized.
+
+
 .. _orthanc-storage-area:
 
 Storage area
 ------------
 
-**By default** (i.e. if no database plugin such as :ref:`PostgreSQL
-<postgresql>` or :ref:`MySQL <mysql>` is used), Orthanc stores all the
+**By default**, Orthanc stores all the
 DICOM files it receives in a folder called ``OrthancStorage`` on the
-filesystem.
+filesystem (defined in the ``StorageDirectory`` configuration in the 
+:ref:`configuration file <configuration>`).
+
+The default storage can also be replaced by a plugin to store these 
+files in an :ref:`object storage <object-storage>` like AWS S3, Azure 
+blob storage or Google cloud.  
 
-More precisely, the ``OrthancStorage`` folder contains a set of
+Alternatively, the file storage can also be implemented inside a 
+:ref:`PostgreSQL <postgresql>` or :ref:`MySQL <mysql>` Database but 
+this is actually quite uncommon.
+
+More precisely, the ``Storage`` contains a set of
 so-called **attachments**, that may correspond to either a DICOM file,
-a JSON file, or any user-defined file. Internally, each attachment is
+a JSON file, or any user-defined file. 
+
+Internally, each attachment is
 automatically associated with an `universally unique identifier (UUID)
 <https://en.wikipedia.org/wiki/Universally_unique_identifier>`__.
 Orthanc can be configured to compress these files on-the-fly in order
@@ -29,21 +51,41 @@
 folder, and the two next characters give the second-level folder.
 
 
-SQLite index
-------------
+
+.. _orthanc-index:
+
+Orthanc Index
+-------------
+
+Orthanc also maintains a summary of all the DICOM resources in a SQL
+database in the so called ``Index``.  This ``Index`` is mandatory to
+rapidly provide information when browsing and accessing the resources
+either through the :ref:`REST API of Orthanc <rest>` or through the
+:ref:`DICOM protocol <dicom-guide>`.
 
-Inside the same ``OrthancStorage`` folder, Orthanc maintains a `SQLite
-database <https://en.wikipedia.org/wiki/SQLite>`__ called ``index``
-that **indexes** all these attachments. The database records, for each
-attachment, its compression method, and its MD5 hashes before and
+**By default**, this index is implemented in a `SQLite
+database <https://en.wikipedia.org/wiki/SQLite>`__ that is stored
+in the same folder as the files (if you are using a file-system).
+This folder is defined by the ``IndexDirectory`` in the :ref:`configuration
+option <configuration>`)
+
+The default ``Index`` can also be replaced by a plugin to store the 
+index in a :ref:`PostgreSQL <postgresql>`, :ref:`MySQL <mysql>` or 
+:ref:`ODBC <odbc>` Database.
+
+
+Index content
+-------------
+
+The ``Index`` database **indexes** all the attachments stored in the ``Storage``. 
+The database records, for each attachment, its compression method, and its MD5 hashes before and
 after compression in order to detect disk corruption (cf. the
 ``StoreMD5ForAttachments`` :ref:`configuration option
 <configuration>`).
 
 One attachment must be associated with one :ref:`DICOM resource
 <model-world>` (patient, study, series, or instance). Incoming DICOM
-files and associated JSON summary are associated with one
-instance-level resource, but user-defined attachments can be
+files are associated with one instance-level resource, but user-defined attachments can be
 associated with any kind of resource. 
 
 Given one DICOM resource, all of its child attachments are identified
@@ -55,9 +97,10 @@
 information for each DICOM resource, notably the :ref:`metadata
 <metadata>`, the :ref:`history of changes <changes>`, and an
 associative map that stores the so-called "main" DICOM tags (to avoid
-accessing the storage folder are when this is not needed). The SQLite
-database schema is kept as simple as possible, and can be found in the
-following two files of the source code of Orthanc:
+accessing the storage folder are when this is not needed). 
+
+The database schema is kept as simple as possible, e.g, for SQLite,
+the schema can be found in the following two files of the source code of Orthanc:
 `PrepareDatabase.sql
 <https://orthanc.uclouvain.be/hg/orthanc/file/Orthanc-1.12.6/OrthancServer/Sources/Database/PrepareDatabase.sql>`__
 and `InstallTrackAttachmentsSize.sql
@@ -67,20 +110,22 @@
 Direct access
 -------------
 
-Directly accessing the content of the ``OrthancStorage`` folder and
-the content of the SQLite/MySQL/PostgreSQL database is strongly
+Directly accessing the content of the ``Storage`` folder and
+the content of the SQLite/MySQL/PostgreSQL ``Index`` database is strongly
 discouraged for several reasons:
 
-* The internal organization outlined above is only true when no
+* The ``Storage`` internal organization outlined above is only true when no
   database plugin is used (e.g. the :ref:`PostgreSQL <postgresql>` and
   :ref:`MySQL <mysql>` plugins can be configured to store the
   attachments inside a database).
 * Orthanc can be configured to compress the attachments before writing
-  them on the disk (cf. the ``StorageCompression`` option).
-* By directly reading the content of ``OrthancStorage``, you bypass
+  them on the disk (cf. the ``StorageCompression`` option) making them
+  less easily readable by an external tool (check the ``OrthancRecoverCompressedFile``
+  executable in the Orthanc distribution).  
+* By directly reading/writing the content of the ``Storage``, you bypass
   all the locking mechanisms used by Orthanc, which might result in
   data corruption.
-* One SQLite database should be accessed by at most one process at any
+* If you are using SQLite for the ``Index``, one SQLite database should be accessed by at most one process at any
   time to avoid any problem (e.g. with NFS filesystems), for reasons
   that are `explained in the SQLite FAQ
   <https://www.sqlite.org/faq.html#q5>`__. Orthanc will stop if it
@@ -89,10 +134,10 @@
   successive versions of Orthanc or of the database plugins.
   
 As a consequence, it is **HIGHLY recommended NOT to directly access**
-the ``OrthancStorage`` folder and the SQLite/MySQL/PostgreSQL
+the ``Storage`` and the SQLite/MySQL/PostgreSQL ``Index``
 database. Use the :ref:`REST API <rest>` instead, which contains
 primitives to access the attachments (cf. the ``.../attachments/...``
-URIs).
+URIs) and all other resources.
 
 The only exception to this rule is for **read-only access when Orthanc
 is stopped**, e.g. as a part of a :ref:`backup <backup>` or