Mercurial > hg > orthanc-book
changeset 988:afea429661c2
S3 transfer mode
author | Alain Mazy <am@osimis.io> |
---|---|
date | Thu, 16 Nov 2023 08:37:28 +0100 |
parents | 759f7d5aca94 |
children | 38909a00964c |
files | Sphinx/source/plugins/object-storage.rst |
diffstat | 1 files changed, 86 insertions(+), 80 deletions(-) [+] |
line wrap: on
line diff
--- a/Sphinx/source/plugins/object-storage.rst Wed Oct 18 15:43:07 2023 +0200 +++ b/Sphinx/source/plugins/object-storage.rst Thu Nov 16 08:37:28 2023 +0100 @@ -48,85 +48,6 @@ the benefit of the worldwide community of medical imaging. -Compilation ------------ - -.. highlight:: text - -The procedure to compile the plugins is quite similar of that for the -:ref:`core of Orthanc <compiling>` although they usually require -some prerequisites. The documented procedure has been tested only -on a Debian Buster machine. - -The compilation of each plugin produces a shared library that contains -the plugin. - - -AWS S3 plugin -^^^^^^^^^^^^^ - -Prerequisites: Compile the AWS C++ SDK:: - - $ mkdir ~/aws - $ cd ~/aws - $ git clone https://github.com/aws/aws-sdk-cpp.git - $ - $ mkdir -p ~/aws/builds/aws-sdk-cpp - $ cd ~/aws/builds/aws-sdk-cpp - $ cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp - $ make -j 4 - $ make install - -Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: - - $ ./vcpkg install cryptopp - -Compile:: - - $ mkdir -p build/aws - $ cd build/aws - $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws - - -**NB:** If you don't want to use vcpkg, you can use the following -command (this syntax is not compatible with Ninja yet):: - - $ cmake -DCMAKE_BUILD_TYPE=Debug -DUSE_VCPKG_PACKAGES=OFF -DUSE_SYSTEM_GOOGLE_TEST=OFF ../../orthanc-object-storage/Aws - $ make - -Crypto++ must be installed (on Ubuntu, run ``sudo apt install libcrypto++-dev``). - - -Azure Blob Storage plugin -^^^^^^^^^^^^^^^^^^^^^^^^^ - -Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: - -$ ./vcpkg install cryptopp -$ ./vcpkg install azure-storage-cpp - - -Compile:: - - $ mkdir -p build/azure - $ cd build/azure - $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure - -Google Storage plugin -^^^^^^^^^^^^^^^^^^^^^ - -Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: - -$ ./vcpkg install cryptopp -$ ./vcpkg install google-cloud-cpp - -Compile:: - - $ mkdir -p build/google - $ cd build/google - $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/google - - Configuration ------------- @@ -151,7 +72,8 @@ "EnableLegacyUnknownFiles": true, // optional: see below "VirtualAddressing": true, // optional: see the section related to MinIO "StorageEncryption" : {}, // optional: see the section related to encryption - "HybridMode": "Disabled" // optional: see the section related to Hybrid storage + "HybridMode": "Disabled", // optional: see the section related to Hybrid storage + "UseTransferMode": true // optional: see below (available from version 2.3.0) } The **EndPoint** configuration is used when accessing an S3 compatible cloud provider. I.e. here is a configuration to store data on Scaleway:: @@ -165,6 +87,11 @@ } +The **UseTransferMode** configuration is used to select the `Transfer Manager <https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/examples-s3-transfermanager.html>`__ mode in the AWS SDK client. +This option was introduced in version 2.3.0 and is the default one. If set to false, the default mode +is used. + + .. _minio: Emulation of AWS S3 using MinIO @@ -442,3 +369,82 @@ to the available memory instead of the number of concurrent threads. Therefore, if you're currently ingesting small files, you can have a lot of thread working together while, if you're ingesting large files, threads might have to wait before receiving a "slot" to access the encryption module. + + +Compilation +----------- + +.. highlight:: text + +The procedure to compile the plugins is quite similar of that for the +:ref:`core of Orthanc <compiling>` although they usually require +some prerequisites. The documented procedure has been tested only +on a Debian Buster machine. + +The compilation of each plugin produces a shared library that contains +the plugin. + + +AWS S3 plugin +^^^^^^^^^^^^^ + +Prerequisites: Compile the AWS C++ SDK:: + + $ mkdir ~/aws + $ cd ~/aws + $ git clone https://github.com/aws/aws-sdk-cpp.git + $ + $ mkdir -p ~/aws/builds/aws-sdk-cpp + $ cd ~/aws/builds/aws-sdk-cpp + $ cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp + $ make -j 4 + $ make install + +Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: + + $ ./vcpkg install cryptopp + +Compile:: + + $ mkdir -p build/aws + $ cd build/aws + $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws + + +**NB:** If you don't want to use vcpkg, you can use the following +command (this syntax is not compatible with Ninja yet):: + + $ cmake -DCMAKE_BUILD_TYPE=Debug -DUSE_VCPKG_PACKAGES=OFF -DUSE_SYSTEM_GOOGLE_TEST=OFF ../../orthanc-object-storage/Aws + $ make + +Crypto++ must be installed (on Ubuntu, run ``sudo apt install libcrypto++-dev``). + + +Azure Blob Storage plugin +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: + +$ ./vcpkg install cryptopp +$ ./vcpkg install azure-storage-cpp + + +Compile:: + + $ mkdir -p build/azure + $ cd build/azure + $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure + +Google Storage plugin +^^^^^^^^^^^^^^^^^^^^^ + +Prerequisites: Install `vcpkg <https://github.com/Microsoft/vcpkg>`__ dependencies:: + +$ ./vcpkg install cryptopp +$ ./vcpkg install google-cloud-cpp + +Compile:: + + $ mkdir -p build/google + $ cd build/google + $ cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/google