Mercurial > hg > orthanc-book

annotate Sphinx/source/faq/orthanc-storage.rst @ 618:cfeb018b9150 Orthanc-1.9.1

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Orthanc 1.9.1
author Sebastien Jodogne <s.jodogne@gmail.com>
date Thu, 25 Feb 2021 19:06:06 +0100
parents eaa6cdfa7ba6
children debcf6b6d070
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
46
12b204ee328d remark about replication
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 37
diff changeset
1 .. _orthanc-storage:
12b204ee328d remark about replication
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 37
diff changeset
2
37
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 36
diff changeset
3 How does Orthanc store its database?
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 36
diff changeset
4 ====================================
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
5
318
83d822f11e78 SeriesMetadata and StudiesMetadata in DICOMweb
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 290
diff changeset
6 Storage area
83d822f11e78 SeriesMetadata and StudiesMetadata in DICOMweb
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 290
diff changeset
7 ------------
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
8
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
9 **By default** (i.e. if no database plugin such as :ref:`PostgreSQL
154
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 46
diff changeset
10 <postgresql>` or :ref:`MySQL <mysql>` is used), Orthanc stores all the
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 46
diff changeset
11 DICOM files it receives in a folder called ``OrthancStorage`` on the
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 46
diff changeset
12 filesystem. Orthanc also associates each incoming DICOM file with a
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 46
diff changeset
13 JSON file that summarizes all its DICOM tags, which speeds up
577
7d27154e70f9 cheatsheet and full openapi
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 568
diff changeset
14 subsequent processing by avoiding a costly DICOM parsing (these
7d27154e70f9 cheatsheet and full openapi
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 568
diff changeset
15 summaries are refered to as ``DICOM-as-JSON`` :ref:`attachments
7d27154e70f9 cheatsheet and full openapi
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 568
diff changeset
16 <metadata>`).
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
17
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
18 More generally, the ``OrthancStorage`` folder contains a set of
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
19 so-called **attachments**, that may correspond to either a DICOM file,
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
20 a JSON file, or any user-defined file. Internally, each attachment is
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
21 automatically associated with an `universally unique identifier (UUID)
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
22 <https://en.wikipedia.org/wiki/Universally_unique_identifier>`__.
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
23 Orthanc can be configured to compress these files on-the-fly in order
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
24 to save disk space (cf. the ``StorageCompression`` :ref:`configuration
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
25 option <configuration>`).
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
26
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
27 To reduce the number of files in a single directory (which is
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
28 something that some operating systems might not like), a 3-level
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
29 hierarchy of directories is created to store the attachments: The
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
30 first two hexadecimal characters of the UUID give the first-level
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
31 folder, and the two next characters give the second-level folder.
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
32
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
33
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
34 SQLite index
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
35 ------------
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
36
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
37 Inside the same ``OrthancStorage`` folder, Orthanc maintains a `SQLite
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
38 database <https://en.wikipedia.org/wiki/SQLite>`__ called ``index``
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
39 that **indexes** all these attachments. The database records, for each
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
40 attachment, its compression method, and its MD5 hashes before and
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
41 after compression in order to detect disk corruption (cf. the
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
42 ``StoreMD5ForAttachments`` :ref:`configuration option
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
43 <configuration>`).
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
44
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
45 One attachment must be associated with one :ref:`DICOM resource
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
46 <model-world>` (patient, study, series, or instance). Incoming DICOM
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
47 files and associated JSON summary are associated with one
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
48 instance-level resource, but user-defined attachments can be
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
49 associated with any kind of resource.
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
50
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
51 Given one DICOM resource, all of its child attachments are identified
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
52 by a number between 0 and 65535. Identifiers <= 1023 are reserved for
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
53 the Orthanc core, whereas identifiers >= 1024 can be user-defined for
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
54 external applications.
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
55
272
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
56 Besides the attachments, the database index maintains other
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
57 information for each DICOM resource, notably the :ref:`metadata
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
58 <metadata>`, the :ref:`history of changes <changes>`, and an
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
59 associative map that stores the so-called "main" DICOM tags (to avoid
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
60 accessing the storage folder are when this is not needed). The SQLite
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
61 database schema is kept as simple as possible, and can be found in the
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
62 following two files of the source code of Orthanc:
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
63 `PrepareDatabase.sql
618
cfeb018b9150 Orthanc 1.9.1
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 607
diff changeset
64 <https://hg.orthanc-server.com/orthanc/file/Orthanc-1.9.1/OrthancServer/Sources/Database/PrepareDatabase.sql>`__
272
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
65 and `InstallTrackAttachmentsSize.sql
618
cfeb018b9150 Orthanc 1.9.1
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 607
diff changeset
66 <https://hg.orthanc-server.com/orthanc/file/Orthanc-1.9.1/OrthancServer/Sources/Database/InstallTrackAttachmentsSize.sql>`__.
272
07f109e3f3f7 database schema
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 154
diff changeset
67
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
68
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
69 Direct access
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
70 -------------
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
71
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
72 Directly accessing the content of the ``OrthancStorage`` folder and
409
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
73 the content of the SQLite/MySQL/PostgreSQL database is strongly
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
74 discouraged for several reasons:
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
75
409
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
76 * The internal organization outlined above is only true when no
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
77 database plugin is used (e.g. the :ref:`PostgreSQL <postgresql>` and
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
78 :ref:`MySQL <mysql>` plugins can be configured to store the
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
79 attachments inside a database).
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
80 * Orthanc can be configured to compress the attachments before writing
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
81 them on the disk (cf. the ``StorageCompression`` option).
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
82 * By directly reading the content of ``OrthancStorage``, you bypass
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
83 all the locking mechanisms used by Orthanc, which might result in
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
84 data corruption.
36
6474af38dcbd SQLITE_BUSY
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 35
diff changeset
85 * One SQLite database should be accessed by at most one process at any
6474af38dcbd SQLITE_BUSY
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 35
diff changeset
86 time to avoid any problem (e.g. with NFS filesystems), for reasons
6474af38dcbd SQLITE_BUSY
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 35
diff changeset
87 that are `explained in the SQLite FAQ
6474af38dcbd SQLITE_BUSY
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 35
diff changeset
88 <https://www.sqlite.org/faq.html#q5>`__. Orthanc will stop if it
6474af38dcbd SQLITE_BUSY
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 35
diff changeset
89 receives the ``SQLITE_BUSY`` status.
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
90
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
91 As a consequence, it is **HIGHLY recommended NOT to directly access**
409
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
92 the ``OrthancStorage`` folder and the SQLite/MySQL/PostgreSQL
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
93 database. Use the :ref:`REST API <rest>` instead, which contains
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
94 primitives to access the attachments (cf. the ``.../attachments/...``
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 392
diff changeset
95 URIs).
35
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
96
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
97 The only exception to this rule is for **read-only access when Orthanc
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
98 is stopped**, e.g. as a part of a :ref:`backup <backup>` or
5737f51ff94e How does Orthanc stores its database
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
99 :ref:`upgrade/replication <replication>` process.