Mercurial > hg > orthanc-dicomweb
view TODO @ 568:2c494f872f70 am-experimental
wip: timing + one large query/vs individual queries to retrieve instance tags
author | Alain Mazy <am@osimis.io> |
---|---|
date | Tue, 20 Jun 2023 09:42:00 +0200 |
parents | de4a56ecd2bc |
children |
line wrap: on
line source
* Implement capabilities: https://www.dicomstandard.org/using/dicomweb/capabilities/ from https://groups.google.com/d/msgid/orthanc-users/c60227f2-c6da-4fd9-9b03-3ce9bf7d1af5n%40googlegroups.com?utm_medium=email&utm_source=footer * /rendered at study level shall return all instances, not only one (https://groups.google.com/g/orthanc-users/c/uFWanYhV8Fs/m/ezi1iXCXCAAJ) Check /rendered at series level too. * Implement serialization of DicomWeb jobs * Add support for application/zip in /dicom-web/studies/ (aka sup 211: https://www.dicomstandard.org/docs/librariesprovider2/dicomdocuments/news/ftsup/docs/sups/sup211.pdf?sfvrsn=9fe9edae_2) * Based on this discussion: https://discourse.orthanc-server.org/t/series-metadata-retrieval-is-very-long-even-with-configuration-optimization/3389 optimize studies/.../series/.../metadata route when "SeriesMetadata" is set to "MainDicomTags" and "ExtraMainDicomTags" are configured according to recommandation (from this setup: https://bitbucket.org/osimis/orthanc-setup-samples/src/master/docker/stone-viewer/docker-compose.yml). with a 600 instance series with SQLite - all timings are performed without verbose logs !!!!: - time curl http://localhost:8043/dicom-web/studies/1.2.276.0.7230010.3.1.2.1215942821.4756.1664826045.3529/series/1.2.276.0.7230010.3.1.3.1215942821.4756.1664833048.11984/metadata > /dev/null -> 0.750 second because the plugin makes 600 calls to /instances/...?full to retrieve all main dicom tags from the DB (but this also retrieves other data like attachment info + instances metadata) - In "Experimental" mode, we are calling this route: time curl "http://localhost:8043/series/fdd31453-fa62d8cb-b681823c-f294d34c-182bca4b/instances?full&requestedTags=0008,0005;0008,0008;0008,0012;0008,0013;0008,0016;0008,0018;0008,0023;0008,0033;0018,0050;0020,0012;0020,0013;0020,0032;0020,0037;0020,0052;0020,0100;0020,4000;0028,0004;0028,0008;0028,0010;0028,0011;0028,0030;0028,0100;0028,0101;0028,0103;0028,1050;0028,1051;0028,1052;0028,1053;0054,1330" > /dev/null -> 0.175 second BUT the total time required by the /metadata route is still around 0.9 second because we spend 600-700ms formating the output !!! (see TODO_OPTI in WadoRs.cpp) -> 0.835 second in "Experimental" mode (10% slower if tags are actually in DB of course !) -> 0.700 second in "MainDicomTags" mode (10% faster) - in the am-experimental branch of Orthanc, we have implemented a new dicomWeb JSON format and 'include' get arguments: time curl "http://localhost:8043/series/fdd31453-fa62d8cb-b681823c-f294d34c-182bca4b/instances?dicomWeb&include=RequestedTags&requestedTags=0008,0005;0008,0008;0008,0012;0008,0013;0008,0016;0008,0018;0008,0023;0008,0033;0018,0050;0020,0012;0020,0013;0020,0032;0020,0037;0020,0052;0020,0100;0020,4000;0028,0004;0028,0008;0028,0010;0028,0011;0028,0030;0028,0100;0028,0101;0028,0103;0028,1050;0028,1051;0028,1052;0028,1053;0054,1330" > /dev/null -> 0.212 second (with the output in DicomWeb JSON !!) total time for the /metadata route is now -> 0.459 second in "Experimental" mode. -> 0.448 second in "MainDicomTags" mode. There is still a high cost for serialization (which is now simply copying and merging json nodes) // The "Full" mode optimized with reading only "file-until-pixel-data" and with a single call to get all instances tags is now -> 0.450 s // after reading 4 files together (still only one DicomWebToJson serializer) -> 0.450 s // after reading 4 files together, with 4 DicomWebToJson serializers -> 0.230 s 4 threads (after removing the ReadJson in debug !): W0614 15:49:38.925856 PluginsManager.cpp:157] Get instances tags 90 W0614 15:49:39.251882 PluginsManager.cpp:157] before send 416 W0614 15:49:39.255049 PluginsManager.cpp:157] after send 420 single thread: W0614 15:52:04.881710 PluginsManager.cpp:157] Get instances tags 92 W0614 15:52:05.228800 PluginsManager.cpp:157] before send 439 W0614 15:52:05.231285 PluginsManager.cpp:157] after send 441 old-single threaded code (no log during measures): - total time: ~553us - get file-until-pixel-data from plugin SDK: ~100us - writer.AddDicom: ~450us - format DicomWeb JSON: ~450us - AddChunk: 2us after removing the ReadJson in debug ! multi-threaded code with a single thread (measured with no logs !): - get file-until-pixel-data from plugin SDK: ~91us - format DicomWeb JSON: ~420us - add to writer: ~4us - TOTAL TIME: 413ms (85ms to get the instances tags) multi-threaded code with 2 threads (measured with no logs !): - get file-until-pixel-data from plugin SDK: ~110us - format DicomWeb JSON: ~450us - add to writer: ~4us - TOTAL TIME: 275ms (85ms to get the instances tags) multi-threaded code with 4 threads (measured with no logs !): - get file-until-pixel-data from plugin SDK: ~120us - format DicomWeb JSON: ~490us - add to writer: ~4us - TOTAL TIME: 194ms (85ms to get the instances tags) multi-threaded code with 4 threads (measured with no logs !) + !oneLargeQuery mode -> get the instance SOPInstanceUID in the thread: - get file-until-pixel-data + SOP InstanceUID from plugin SDK: ~420us - format DicomWeb JSON: ~490us - add to writer: ~4us - TOTAL TIME: 158ms - note that all measurements have been performed on a DB with a single series ! We should repeat that with a more realistic DB