Mercurial > hg > orthanc-book
annotate Sphinx/source/plugins/transfers.rst @ 916:dafff7d1c975
fix
| author | Alain Mazy <am@osimis.io> |
|---|---|
| date | Tue, 07 Feb 2023 15:26:12 +0100 |
| parents | 1af054b77e9c |
| children | 1316bc62b5d5 |
| rev | line source |
|---|---|
| 164 | 1 .. _transfers: |
| 2 | |
| 3 | |
| 4 Transfers accelerator plugin | |
| 5 ============================ | |
| 6 | |
| 7 .. contents:: | |
| 8 | |
| 9 Osimis provides a `transfers accelerator plugin | |
| 449 | 10 <https://hg.orthanc-server.com/orthanc-transfers/file/default>`__ whose |
| 164 | 11 purpose is to speed up the transfers of DICOM instances over networks |
| 12 (with respect to the native DICOM protocol or to the built-in | |
| 13 :ref:`Orthanc peers <peers>` mechanism). | |
| 14 | |
| 15 | |
| 16 Description | |
| 17 ----------- | |
| 18 | |
| 19 This plugin can be used to **send** local images to remote Orthanc | |
| 20 peers, or to locally **retrieve** images stored on remote Orthanc | |
| 21 peers. | |
| 22 | |
| 23 The plugin also implements **storage commitment**, i.e. the peer that | |
| 24 initiates the transfer is informed whether *all* the DICOM instances | |
| 25 have been properly received by the other peer. The DICOM instances are | |
| 26 individually validated using their MD5 checksum. In other words, this | |
| 27 plugin provides atomicity in the transfers (i.e. a study/series is | |
| 28 considered as a whole, and partial transfers are prevented). | |
| 29 | |
| 30 Note that the protocol is **entirely built over HTTP/HTTPS** (and not | |
| 31 directly over TCP), making it friendly with network firewalls and Web | |
| 32 caches. Also, the plugin takes advantage of the **jobs engine** of | |
| 33 Orthanc, so that transfers can be easily paused/canceled/resubmitted. | |
| 34 | |
| 35 Technically, this plugin extends the REST API of Orthanc with | |
| 36 endpoints that optimize the use of the network bandwidth over the HTTP | |
| 37 and HTTPS protocols, through the combination of the following | |
| 38 mechanisms: | |
| 39 | |
| 40 * Small DICOM instances are grouped together to form so-called | |
| 41 "buckets" of some megabytes in order to reduce the number of HTTP | |
| 42 handshakes. | |
| 43 | |
| 44 * Large DICOM instances are split as a set of smaller buckets in | |
| 45 order to bypass nasty effects of TCP congestion control on | |
| 46 low-quality network links. | |
| 47 | |
| 48 * Buckets are downloaded/uploaded concurrently by several threads. | |
| 49 | |
| 50 * Buckets can be individually compressed using the gzip algorithm, | |
| 51 hereby reducing the network usage. On a typical medical image, this | |
| 52 can divide the volume of the transmission by a factor 2 to 3, at | |
| 53 the price of a larger CPU usage. | |
| 54 | |
| 55 * Sending images to remote Orthanc peers can either be done with HTTP | |
| 56 ``PUT`` requests (so-called "push mode"), or with HTTP ``GET`` | |
| 57 requests if the local Orthanc server has a public IP address | |
| 58 (so-called "pull mode"). | |
| 59 | |
| 60 | |
| 61 | |
| 62 Compilation | |
| 63 ----------- | |
| 64 | |
| 65 Static linking | |
| 66 ^^^^^^^^^^^^^^ | |
| 67 | |
| 68 .. highlight:: text | |
| 69 | |
| 70 The procedure to compile the plugin is similar to that for the | |
| 71 :ref:`core of Orthanc <compiling>`. The following commands should work | |
| 72 for most UNIX-like distribution (including GNU/Linux):: | |
| 73 | |
| 74 $ mkdir BuildTransfers | |
| 75 $ cd BuildTransfers | |
| 76 $ cmake .. -DSTATIC_BUILD=ON -DCMAKE_BUILD_TYPE=Release | |
| 77 $ make | |
| 78 | |
| 79 The compilation will produce the shared library ``OrthancTransfers`` | |
| 80 that can be loaded as a plugin by Orthanc. | |
| 81 | |
| 82 | |
| 83 Microsoft Windows | |
| 84 ^^^^^^^^^^^^^^^^^ | |
| 85 | |
| 86 Pre-compiled binaries for Microsoft Windows `are available | |
|
358
011b01ccf52d
fixing external hyperlinks
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
221
diff
changeset
|
87 <https://www.orthanc-server.com/browse.php?path=/plugin-transfers>`__. |
| 164 | 88 |
| 89 | |
| 90 Dynamic linking on Ubuntu 16.04 | |
| 91 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | |
| 92 | |
| 93 .. highlight:: text | |
| 94 | |
| 95 If static linking is not desired, here are build instructions for | |
| 96 Ubuntu 16.04 (provided build dependencies for the :ref:`core of | |
| 97 Orthanc <compiling>` have already been installed):: | |
| 98 | |
| 99 $ mkdir BuildTransfers | |
| 100 $ cd BuildTransfers | |
| 101 $ cmake .. -DCMAKE_BUILD_TYPE=Release \ | |
| 102 -DALLOW_DOWNLOADS=ON \ | |
| 103 -DUSE_SYSTEM_GOOGLE_TEST=OFF \ | |
| 104 -DUSE_SYSTEM_ORTHANC_SDK=OFF | |
| 105 $ make | |
| 106 | |
| 107 | |
| 108 Basic usage: Sending images | |
| 109 --------------------------- | |
| 110 | |
| 111 .. highlight:: json | |
| 112 | |
| 113 You of course first have to :ref:`install Orthanc <binaries>`, with a | |
| 114 version above 1.4.2. Secondly, you have to load the plugin and to | |
| 115 **declare the remote Orthanc peers** in the :ref:`configuration file | |
| 116 <configuration>`. Here is a minimal example (obviously, adapt the | |
| 117 parameters):: | |
| 118 | |
| 119 { | |
| 120 "Name" : "MyOrthanc", | |
| 121 "Plugins" : [ | |
| 122 "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so" | |
| 123 ], | |
| 124 "OrthancPeers" : { | |
| 125 "remote" : [ "http://1.2.3.4:8042/" ] | |
| 126 } | |
| 127 } | |
| 128 | |
| 129 Once Orthanc is running, when you open a patient, a study, or a series | |
| 130 in :ref:`Orthanc Explorer <orthanc-explorer>`, you will see a new | |
| 131 yellow button entitled ``Transfers accelerator``. By clicking on this | |
| 132 button, you will be able to send the local patient/study/series to one | |
| 133 of the remote Orthanc peers (provided they are also equipped with the | |
| 134 transfers accelerator plugin). | |
| 135 | |
| 136 | |
| 137 REST API | |
| 138 -------- | |
| 139 | |
| 140 .. highlight:: bash | |
| 141 | |
| 142 Here is a sample command line to **receive** a patient from the remote | |
| 143 peer called ``remote``:: | |
| 144 | |
| 890 | 145 $ curl -v -X POST http://localhost:8042/transfers/pull \ |
| 146 --data '{ | |
| 147 "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}], | |
| 148 "Compression" : "gzip", | |
| 149 "Peer" : "remote" | |
| 150 }' | |
| 151 | |
| 152 | |
| 164 | 153 |
| 154 Note that several resources from different levels (patient, study, | |
| 155 series or instances) can be retrieved at once. | |
| 156 | |
| 157 Conversely, here is a sample command line to **send** the same patient | |
| 158 to the remote peer ``remote``:: | |
| 159 | |
| 890 | 160 $ curl -v -X POST http://localhost:8042/transfers/send \ |
| 161 --data '{ | |
| 162 "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}], | |
| 163 "Compression" : "gzip", | |
| 164 "Peer" : "remote" | |
| 165 }' | |
| 164 | 166 |
| 167 The command above is the one that is issued by Orthanc Explorer under | |
| 168 the hood (see section above). | |
| 169 | |
| 170 | |
| 171 | |
| 172 Sending in pull vs. push mode | |
| 173 ----------------------------- | |
| 174 | |
| 175 In the case DICOM instances are being **sent** to a remote peer, the | |
| 176 plugin can work in two different modes: | |
| 177 | |
| 178 * In **"pull" mode**, the plugin will transfer images by using as many | |
| 179 HTTP ``GET`` requests as possible. | |
| 180 | |
| 181 * In **"push" mode**, it will use a sequence of HTTP ``PUT`` requests. | |
| 182 | |
| 183 Push mode is easier to setup, but pull mode should be favored, as it | |
| 184 might lead to improved performance in the presence of Web caches. For | |
| 185 pull mode to work, the remote peer must be able to make calls to the | |
| 186 REST API of the local peer. This often means that the local peer has a | |
| 187 public IP address. | |
| 188 | |
| 189 In order to enable pull mode to send image from Orthanc peer "A" to | |
| 190 another Orthanc peer "B", 2 actions must be taken: | |
| 191 | |
| 192 1. "B" must have "A" defined as one of its peers, by adding "A" to its | |
| 193 ``OrthancPeers`` configuration section. | |
| 194 | |
| 195 2. "A" must also have "B" defined as one of its peers, and the | |
| 196 ``RemoteSelf`` property must be provided for peer "B". This option | |
| 197 specifies the symbolic name under which "B" is known to "A". | |
| 198 | |
| 199 .. highlight:: json | |
| 200 | |
| 201 Here is a sample configuration for "A":: | |
| 202 | |
| 203 { | |
| 204 "Name" : "A", | |
| 205 "Plugins" : [ | |
| 206 "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so" | |
| 207 ], | |
| 208 "OrthancPeers" : { | |
| 209 "B" : { | |
| 210 "Url" : "http://b.myinstitution.com:8042/", | |
| 211 "RemoteSelf" : "A" | |
| 212 } | |
| 213 } | |
| 214 } | |
| 215 | |
| 216 And here is a sample configuration for "B":: | |
| 217 | |
| 218 { | |
| 219 "Name" : "B", | |
| 220 "Plugins" : [ | |
| 221 "/home/user/orthanc-transfers/BuildTransfers/libOrthancTransfers.so" | |
| 222 ], | |
| 223 "OrthancPeers" : { | |
| 224 "A" : { | |
| 225 "Url" : "http://a.myinstitution.com:8042/" | |
| 226 } | |
| 227 } | |
| 228 } | |
| 229 | |
| 230 | |
| 231 | |
| 232 NB: **Receiving** images is always done in pull mode. | |
| 233 | |
| 234 | |
| 235 | |
| 236 Advanced options | |
| 237 ---------------- | |
| 238 | |
| 239 Besides the ``OrthancPeers`` configuration option, several advanced | |
| 240 options are available to fine-tune the configuration of the | |
| 241 plugin. They are listed below:: | |
| 242 | |
| 243 { | |
| 244 ... | |
| 245 "HttpTimeout" : 120, // Can be increased on slow networks | |
| 246 "Transfers" : { | |
| 247 "Threads" : 6, // Number of worker threads for one transfer | |
| 248 "BucketSize" : 4096, // Optimal size for a bucket (in KB) | |
| 249 "CacheSize" : 128, // Size of the memory cache to process DICOM files (in MB) | |
|
221
26441131d725
option MaxHttpRetries in transfers accelerator
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
164
diff
changeset
|
250 "MaxPushTransactions" : 4, // Maximum number of simultaneous receptions in push mode |
| 916 | 251 "MaxHttpRetries" : 0, // Maximum number of HTTP retries for one bucket |
| 252 "PeerConnectivityTimeout": 2 // HTTP Timeout (in seconds) used when checking if a remote peer has the transfer plugin enabled in /transfers/peers GET route | |
| 164 | 253 } |
| 254 } | |
| 890 | 255 |
| 256 | |
| 257 Working with load-balancers | |
| 258 --------------------------- | |
| 259 | |
| 260 .. highlight:: bash | |
| 261 | |
| 262 If the receiving Orthanc instance is implemented by a cluster of Orthanc instances | |
| 263 behind a load-balancer, it is very important that all requests relating to a single | |
| 264 **"push"** transfer target the same Orthanc instance. | |
| 265 | |
| 266 In order to achieve this, in your load-balancer, you may use the ``sender-transfer-id`` | |
| 267 HTTP header to route the requests. This header is populated in every outgoing HTTP request. | |
| 268 By default, its value is a random uuid. If required, you may force the value of this | |
| 269 HTTP header by adding a ``SenderTransferID`` field in the payload when creating | |
| 270 the transfer:: | |
| 271 | |
| 272 $ curl -v -X POST http://localhost:8042/transfers/send \ | |
| 273 --data '{ | |
| 274 "Resources" : [{"Level":"Patient","ID":"16738bc3-e47ed42a-43ce044c-a3414a45-cb069bd0"}], | |
| 275 "Compression" : "gzip", | |
| 276 "Peer" : "remote", | |
| 277 "SenderTransferID" : "my-transfer-id" | |
| 278 }' |