Mercurial > hg > orthanc-object-storage
annotate README.md @ 99:f4e05641c108
rewrote using the latest azure C++ sdk
| author | Alain Mazy <am@osimis.io> |
|---|---|
| date | Thu, 20 Jul 2023 15:09:57 +0200 |
| parents | d65580d2e9ca |
| children | 57be7094d6c5 |
| rev | line source |
|---|---|
| 1 | 1 # README # |
| 2 | |
| 3 Orthanc object-storages plugin for main cloud providers (Google/Azure/AWS) | |
| 4 | |
| 4 | 5 Check the [Orthanc book](https://book.orthanc-server.com/plugins/object-storage.html) for complete documentation. |
| 1 | 6 |
| 5 | 7 ## info for developers ## |
| 8 | |
| 9 Here's a sample configuration file of the `StorageEncryption` section of the plugins: | |
| 10 | |
| 11 ``` | |
| 12 { | |
| 15 | 13 "GoogleCloudStorage" : { |
| 14 "StorageEncryption" : { | |
| 15 "Enable": true, | |
| 16 "MasterKey": [3, "/path/to/master.key"], // key id - path to the base64 encoded key | |
| 17 "PreviousMasterKeys" : [ | |
| 18 [ 1, "/path/to/previous1.key"], | |
| 19 [ 2, "/path/to/previous2.key"] | |
| 20 ], | |
| 21 "MaxConcurrentInputSize" : 1024 // size in MB | |
| 22 } | |
| 5 | 23 } |
| 24 } | |
| 25 ``` | |
| 26 | |
| 27 ### Compile Google plugin ### | |
| 28 | |
| 29 * `./vcpkg install google-cloud-cpp` | |
| 30 * `./vcpkg install cryptopp` | |
| 31 * `hg clone ...` | |
| 32 * `mkdir -p build/google` | |
| 33 * `cd build/google` | |
| 15 | 34 * `cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Google` |
| 5 | 35 |
| 36 ### Google plugin configuration ### | |
| 37 | |
| 38 ``` | |
| 39 "GoogleCloudStorage" : { | |
| 15 | 40 "ServiceAccountFile" : "/.../googleServiceAccountFile.json", |
| 41 "BucketName": "test-orthanc-storage-plugin", | |
| 20 | 42 "RootPath": "", // optional: folder in which files are stored (ex: my/path/to/myfolder) |
| 15 | 43 "StorageEncryption" : {...}, |
| 44 "StorageStructure" : "flat", | |
| 45 "MigrationFromFileSystemEnabled" : false | |
| 5 | 46 } |
| 47 | |
| 48 ``` | |
| 49 | |
| 50 ## Azure Blob Storage plugin ## | |
| 51 | |
| 52 ### Prerequisites ### | |
| 53 | |
| 54 * Install [vcpkg](https://github.com/Microsoft/vcpkg) | |
| 55 | |
| 56 ### Compile Azure plugin ### | |
| 57 | |
| 90 | 58 On Linux, with vcpkg version `2022.11.14`: |
| 34 | 59 |
| 5 | 60 * `./vcpkg install cpprestsdk` |
| 34 | 61 * `./vcpkg install cryptopp` |
| 5 | 62 * `hg clone ...` |
| 63 * `mkdir -p build/azure` | |
| 64 * `cd build/azure` | |
| 65 * `cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Azure` | |
| 66 | |
| 90 | 67 On Windows, with vcpkg version `2022.11.14` : |
| 34 | 68 |
| 69 * `.\vcpkg.exe install cpprestsdk:x64-windows-static` | |
| 70 * `.\vcpkg.exe install azure-storage-cpp:x64-windows-static` | |
| 71 * `.\vcpkg.exe install cryptopp:x64-windows-static` | |
| 72 * `hg clone ...` | |
| 73 * `mkdir -p build/azure` | |
| 74 * `cd build/azure` | |
| 75 * `cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake -DVCPKG_TARGET_TRIPLET=x64-windows-static -DSTATIC_BUILD=ON -DCMAKE_BUILD_TYPE="Release" ../../orthanc-object-storage/Azure` | |
| 76 * `cmake --build . --config Release` | |
| 77 | |
| 78 | |
| 5 | 79 ### Azure plugin configuration ### |
| 80 | |
| 81 ``` | |
| 82 "AzureBlobStorage" : { | |
| 83 "ConnectionString": "DefaultEndpointsProtocol=https;AccountName=xxxxxxxxx;AccountKey=yyyyyyyy===;EndpointSuffix=core.windows.net", | |
| 15 | 84 "ContainerName" : "test-orthanc-storage-plugin", |
| 50 | 85 "CreateContainerIfNotExists": true, // available from version 1.2.0 |
| 20 | 86 "RootPath": "", // optional: folder in which files are stored (ex: my/path/to/myfolder) |
| 15 | 87 "StorageEncryption" : {...}, |
| 88 "StorageStructure" : "flat", | |
| 89 "MigrationFromFileSystemEnabled" : false | |
| 5 | 90 } |
| 91 ``` | |
| 92 | |
| 93 ## AWS S3 Storage plugin ## | |
| 94 | |
| 95 ### Prerequisites ### | |
| 96 | |
| 97 * Install [vcpkg](https://github.com/Microsoft/vcpkg) | |
| 98 | |
| 99 * compile the AWS C++ SDK | |
| 100 | |
| 101 ``` | |
| 102 | |
| 103 mkdir ~/aws | |
| 104 cd ~/aws | |
| 105 git clone https://github.com/aws/aws-sdk-cpp.git | |
| 106 | |
| 107 mkdir -p ~/aws/builds/aws-sdk-cpp | |
| 108 cd ~/aws/builds/aws-sdk-cpp | |
| 109 cmake -DBUILD_ONLY="s3;transfer" ~/aws/aws-sdk-cpp | |
| 110 make -j 4 | |
| 111 make install | |
| 112 ``` | |
| 113 | |
| 114 ### Compile AWS S3 plugin ### | |
| 115 | |
| 116 * `./vcpkg install cryptopp` | |
| 117 * `hg clone ...` | |
| 118 * `mkdir -p build/aws` | |
| 119 * `cd build/aws` | |
| 120 * `cmake -DCMAKE_TOOLCHAIN_FILE=[vcpkg root]\scripts\buildsystems\vcpkg.cmake ../../orthanc-object-storage/Aws` | |
| 121 | |
| 54 | 122 ### AWS S3 plugin configuration ### |
| 5 | 123 |
| 124 ``` | |
| 125 "AwsS3Storage" : { | |
| 126 "BucketName": "test-orthanc-s3-plugin", | |
| 127 "Region" : "eu-central-1", | |
| 54 | 128 "AccessKey" : "AKXXX", // optional: if not specified, the plugin will use the default credentials manager (from version 1.3.0) |
| 129 "SecretKey" : "RhYYYY", // optional: if not specified, the plugin will use the default credentials manager (from version 1.3.0) | |
|
6
393fcf337462
AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout
Alain Mazy
parents:
5
diff
changeset
|
130 "Endpoint": "", // optional: custom endpoint |
|
393fcf337462
AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout
Alain Mazy
parents:
5
diff
changeset
|
131 "ConnectionTimeout": 30, // optional: connection timeout in seconds |
| 15 | 132 "RequestTimeout": 1200, // optional: request timeout in seconds (max time to upload/download a file) |
| 20 | 133 "RootPath": "", // optional: folder in which files are stored (ex: my/path/to/myfolder) |
| 134 "StorageEncryption" : {...}, // optional | |
| 135 "StorageStructure" : "flat", // optional | |
| 77 | 136 "MigrationFromFileSystemEnabled" : false, // optional (deprecated, is now equivalent to "HybridMode": "WriteToObjectStorage") |
| 137 "HybridMode": "WriteToDisk" // "WriteToDisk", "WriteToObjectStorage", "Disabled" | |
| 5 | 138 } |
| 77 | 139 ``` |
| 140 | |
| 141 ### Testing the S3 plugin with minio | |
| 142 | |
| 143 ``` | |
| 144 docker run -p 9000:9000 -p 9001:9001 -e MINIO_REGION=eu-west-1 -e MINIO_ROOT_USER=minio -e MINIO_ROOT_PASSWORD=miniopwd minio/minio server /data --console-address ":9001" | |
| 145 ``` | |
| 146 | |
| 147 config file: | |
| 148 ``` | |
| 149 "AwsS3Storage" : { | |
| 150 "BucketName": "orthanc", | |
| 151 "Region": "eu-west-1", | |
| 152 "Endpoint": "http://127.0.0.1:9000/", | |
| 153 "AccessKey": "minio", | |
| 154 "SecretKey": "miniopwd", | |
| 155 "VirtualAddressing": false | |
| 156 | |
| 157 // "StorageEncryption" : { | |
| 158 // "Enable": true, | |
| 159 // "MasterKey": [1, "/home/test/encryption/test.key"], | |
| 160 // "MaxConcurrentInputSize": 1024, | |
| 161 // "Verbose": true | |
| 162 // } // optional: see the section related to encryption | |
| 163 } | |
| 164 | |
| 165 ``` | |
| 166 | |
| 167 Test the hybrid mode | |
| 168 - start in "HybridMode": "WriteToFileSystem", | |
| 169 - upload instances 1 & 2 | |
| 170 - restart in "HybridMode": "WriteToObjectStorage", | |
| 171 - check that you can read instance 1 and that you can delete it | |
| 172 - upload instances 3 & 4 | |
| 173 - restart in "HybridMode": "WriteToFileSystem", | |
| 174 - check that you can read instance 3 and that you can delete it | |
| 175 - final check: | |
| 176 - there should be only one file in the disk storage | |
| 177 - there should be only one file in the S3 bucket | |
| 178 | |
|
83
431ab61b5760
/move-storage when HybridMode is enabled
Alain Mazy <am@osimis.io>
parents:
77
diff
changeset
|
179 test moving a study to file-system storage |
|
431ab61b5760
/move-storage when HybridMode is enabled
Alain Mazy <am@osimis.io>
parents:
77
diff
changeset
|
180 curl http://localhost:8043/move-storage -d '{"Resources": ["737c0c8d-ea890b4d-e36a43bb-fb8c8d41-aa0ed0a8"], "TargetStorage" : "file-system"}' |
|
431ab61b5760
/move-storage when HybridMode is enabled
Alain Mazy <am@osimis.io>
parents:
77
diff
changeset
|
181 curl http://localhost:8043/move-storage -d '{"Resources": ["737c0c8d-ea890b4d-e36a43bb-fb8c8d41-aa0ed0a8"], "TargetStorage" : "object-storage"}' |