[BitBucket user: Thibault Nélis] [BitBucket date: 2019-12-24.10:43:39] It seems a GET request for the DICOMweb Retrieve Transaction Study Metadata resource has Orthanc read all instances internally, resulting in very low performance. We infer this by looking at verbose log output: ``` I1224 09:46:15.394979 HttpServer.cpp:822] GET /dicom-web/studies/2.16.840.1.113669.632.20.1211.10000315526/metadata I1224 09:46:15.395010 OrthancPlugins.cpp:1628] Delegating HTTP request to plugin for URI: /dicom-web/studies/2.16.840.1.113669.632.20.1211.10000315526/metadata I1224 09:46:15.395289 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /studies/0dff9280-2375f586-1297870c-4cbd6d1c-527e36e5/instances (built-in API) I1224 09:46:15.395973 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/e16accdd-561117f2-c5d26638-80a58c32-2cdd6dcc/tags?short (built-in API) I1224 09:46:15.396002 FilesystemStorage.cpp:155] Reading attachment "4fce8a7c-a627-4e53-94dd-21acdf2541c9" of "JSON summary of DICOM" content type I1224 09:46:15.396947 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/e16accdd-561117f2-c5d26638-80a58c32-2cdd6dcc/file (built-in API) I1224 09:46:15.397000 FilesystemStorage.cpp:155] Reading attachment "64cc1c97-524c-4e19-b957-c717b5a563b5" of " I1224 09:46:15.398724 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/7c92ce8e-bbf67ed2-ffa3b8c1-a3b35d94-7ff3ae26/tags?short (built-in API) I1224 09:46:15.398776 FilesystemStorage.cpp:155] Reading attachment "55069d3d-07e3-4c17-9215-281dddd36ea7" of " I1224 09:46:15.399701 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/7c92ce8e-bbf67ed2-ffa3b8c1-a3b35d94-7ff3ae26/file (built-in API) I1224 09:46:15.399751 FilesystemStorage.cpp:155] Reading attachment "b054f630-bc89-4444-8995-37849f3bcc03" of "DICOM" content type I1224 09:46:15.401009 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/48386f50-86385e69-183c81a0-21e4720b-67a10f8f/tags?short (built-in API) I1224 09:46:15.401076 FilesystemStorage.cpp:155] Reading attachment "ecc0ea51-46cb-4c32-8655-812ca17da357" of " I1224 09:46:15.401965 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/48386f50-86385e69-183c81a0-21e4720b-67a10f8f/file (built-in API) I1224 09:46:15.402011 FilesystemStorage.cpp:155] Reading attachment "20a26895-e53a-437e-88f3-c643e08704dd" of "DICOM" content type I1224 09:46:15.403256 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/636bd1e4-d1e9b806-2e91d24a-d298ecd5-18fbfa3e/tags?short (built-in API) I1224 09:46:15.403306 FilesystemStorage.cpp:155] Reading attachment "ad741d45-ad77-4edf-8825-ff7fd1c63605" of " I1224 09:46:15.404010 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/636bd1e4-d1e9b806-2e91d24a-d298ecd5-18fbfa3e/file (built-in API) I1224 09:46:15.404055 FilesystemStorage.cpp:155] Reading attachment "6e7aa29c-f92d-422f-a52b-4cc131ea6c50" of "DICOM" content type I1224 09:46:15.405271 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/dc1dc991-d095027b-988cbe72-06848edc-8c42ef38/tags?short (built-in API) I1224 09:46:15.405337 FilesystemStorage.cpp:155] Reading attachment "8dae93e5-6672-40e4-ba8e-253140f5eb1e" of " I1224 09:46:15.406019 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/dc1dc991-d095027b-988cbe72-06848edc-8c42ef38/file (built-in API) I1224 09:46:15.406065 FilesystemStorage.cpp:155] Reading attachment "a1ed2a63-27a2-4a62-8a2f-81413919c8c3" of " I1224 09:46:15.407308 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/619213d6-7115f549-b0c9a32c-916e340b-45e8fa0e/tags?short (built-in API) I1224 09:46:15.407338 FilesystemStorage.cpp:155] Reading attachment "787db8be-a307-42e1-bd14-cf57bac474c3" of "JSON summary of DICOM" content type I1224 09:46:15.407965 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/619213d6-7115f549-b0c9a32c-916e340b-45e8fa0e/file (built-in API) I1224 09:46:15.407990 FilesystemStorage.cpp:155] Reading attachment "08c0d021-a2d8-4a29-832b-3c4ead4c75be" of "DICOM" content type I1224 09:46:15.409194 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/97b61d9a-edaa82fe-78d24795-c43edab4-3bc917f8/tags?short (built-in API) I1224 09:46:15.409224 FilesystemStorage.cpp:155] Reading attachment "8a87aa21-febc-4c93-8e4b-ab57e27994af" of "JSON summary of DICOM" content type I1224 09:46:15.409915 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/97b61d9a-edaa82fe-78d24795-c43edab4-3bc917f8/file (built-in API) I1224 09:46:15.409941 FilesystemStorage.cpp:155] Reading attachment "38653a2b-ab08-40e0-9a6c-8f0e96837c25" of "DICOM" content type I1224 09:46:15.411284 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/3ca86d6a-2753d0c8-7c6e6cff-45d259dc-0b49c9d8/tags?short (built-in API) I1224 09:46:15.411314 FilesystemStorage.cpp:155] Reading attachment "cf84fe07-0abc-4536-b205-658267afd086" of "JSON summary of DICOM" content type I1224 09:46:15.411948 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/3ca86d6a-2753d0c8-7c6e6cff-45d259dc-0b49c9d8/file (built-in API) I1224 09:46:15.411976 FilesystemStorage.cpp:155] Reading attachment "0ff69527-900a-46eb-901e-e4aa65b3d971" of "DICOM" content type I1224 09:46:15.413172 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/ff7b4ae9-ed3465f0-e5b41f04-0620305c-a04fb1be/tags?short (built-in API) I1224 09:46:15.413201 FilesystemStorage.cpp:155] Reading attachment "5dab9e56-08c0-4b4e-be7b-0c313ad98a98" of "JSON summary of DICOM" content type I1224 09:46:15.413824 OrthancPlugins.cpp:2048] Plugin making REST GET call on URI /instances/ff7b4ae9-ed3465f0-e5b41f04-0620305c-a04fb1be/file (built-in API) I1224 09:46:15.413850 FilesystemStorage.cpp:155] Reading attachment "399fc9c5-0464-45c4-89f6-0000be078bf5" of "DICOM" content type ``` \(See entries like “`Reading attachment "399fc9c5-0464-45c4-89f6-0000be078bf5" of "DICOM" content type`“ for events evidently triggered by internal calls to corresponding `file` instance subresources.\) * For systems holding either large studies with many small instances or studies with large instances, the system slows down to a crawl \(upwards of 10 minutes for a single call for a large study of more than a thousand instances on a production system\). * Although we do expect ~linear time correlated with the number of instances, current performance does not match expectations for a modern system. We believe that leveraging the index database to service these requests could, as an example. * Instance data size should not impact metadata retrieval time. Observation for that last point: ``` $ sudo tee /proc/sys/vm/drop_caches <<<1 &>/dev/null $ time http $orthanc/dicom-web/studies/2.16.840.1.113669.632.20.1211.10000315526/metadata >/dev/null real 0m0.411s user 0m0.243s sys 0m0.036s $ sudo tee /proc/sys/vm/drop_caches <<<1 &>/dev/null $ time http $orthanc/dicom-web/studies/1.3.51.0.1.1.89.147.161.148.5354976.4875515/metadata >/dev/null real 0m2.724s user 0m0.250s sys 0m0.031s $ http $orthanc/studies/0dff9280-2375f586-1297870c-4cbd6d1c-527e36e5 | jq -r '.MainDicomTags.StudyInstanceUID' 2.16.840.1.113669.632.20.1211.10000315526 $ http $orthanc/studies/0dff9280-2375f586-1297870c-4cbd6d1c-527e36e5/statistics | jq .CountInstances 9 $ http $orthanc/studies/efe74f24-d89b716e-5e3d93cd-8081e0c1-7abfdd42 | jq -r '.MainDicomTags.StudyInstanceUID' 1.3.51.0.1.1.89.147.161.148.5354976.4875515 $ http $orthanc/studies/efe74f24-d89b716e-5e3d93cd-8081e0c1-7abfdd42/statistics | jq .CountInstances 1 ``` As we can see, metadata retrieval for the second study here is dramatically slower despite it containing less instances \(we assume here that metadata size is similar\). When compounded \(1000\+ instances\) it’s easy to imagine dramatic differences in performance profiles.
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2019-12-24.11:07:39] This issue is under active discussion on the forum: https://groups.google.com/d/msg/orthanc-users/y1N5zOFVk0M/wal0jtwpBgAJ
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2020-02-17.15:36:25] This is fixed by the following changeset: https://hg.orthanc-server.com/orthanc-dicomweb/changeset/27001924c4564a517f1a4c6dacc3453a629aff04 This is documented in the Orthanc Book: https://book.orthanc-server.com/plugins/dicomweb.html#server-related-options
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2020-02-17.15:51:43] Besides the new options “StudiesMetadata“ and “SeriesMetadata”, the forthcoming DICOMweb plugin 1.1 does not access the “DICOM-as-JSON” attachments anymore. This divides by two the number of access to the storage area, even if the new options are set to “Full”. Here is a status in the number of accesses to the storage area in the various configurations: ```plaintext Test command (this is the VIX dataset): $ curl http://localhost:8042/dicom-web/studies/2.16.840.1.113669.632.20.1211.10000315526/metadata DICOMweb plugin 1.0: $ ./Orthanc Configuration.json --verbose 2>&1 | grep 'Reading attachment' | wc -l 500 DICOMweb plugin mainline, with "StudiesMetadata" equal to "Full": $ ./Orthanc Configuration.json --verbose 2>&1 | grep 'Reading attachment' | wc -l 250 DICOMweb plugin mainline, with "StudiesMetadata" equal to "MainDicomTags" (only uses the Orthanc index): $ ./Orthanc Configuration.json --verbose 2>&1 | grep 'Reading attachment' | wc -l 0 ```