orthanc-object-storage: Aws/AwsS3StoragePlugin.cpp annotate

annotate Aws/AwsS3StoragePlugin.cpp @ 110:37a4b8e2577f

sync orthanc + SDK 1.12.1

author	Alain Mazy <am@osimis.io>
date	Tue, 10 Oct 2023 15:40:36 +0200
parents	87972cfe7ac5
children	407bd022b0cf

rev	line source
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	1 /**
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	2 * Cloud storage plugins for Orthanc
37 f55b2afdf53d upgrade to year 2021 Sebastien Jodogne <s.jodogne@gmail.com> parents: 33 diff changeset	3 * Copyright (C) 2020-2021 Osimis S.A., Belgium
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	4 *
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	5 * This program is free software: you can redistribute it and/or
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	6 * modify it under the terms of the GNU Affero General Public License
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	7 * as published by the Free Software Foundation, either version 3 of
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	8 * the License, or (at your option) any later version.
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	9 *
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	10 * This program is distributed in the hope that it will be useful, but
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	11 * WITHOUT ANY WARRANTY; without even the implied warranty of
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	12 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	13 * Affero General Public License for more details.
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	14 *
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	15 * You should have received a copy of the GNU Affero General Public License
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	16 * along with this program. If not, see <http://www.gnu.org/licenses/>.
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	17 **/
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	18
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	19 #include "AwsS3StoragePlugin.h"
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	20
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	21 #include <aws/core/Aws.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	22 #include <aws/s3/S3Client.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	23 #include <aws/s3/model/PutObjectRequest.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	24 #include <aws/s3/model/GetObjectRequest.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	25 #include <aws/s3/model/ListObjectsRequest.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	26 #include <aws/s3/model/DeleteObjectRequest.h>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	27 #include <aws/core/auth/AWSCredentialsProvider.h>
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	28 #include <aws/core/utils/HashingUtils.h>
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	29 #include <aws/core/utils/memory/stl/AWSStreamFwd.h>
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	30 #include <aws/core/utils/memory/stl/AWSStringStream.h>
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	31 #include <aws/core/utils/memory/AWSMemory.h>
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	32 #include <aws/core/utils/stream/PreallocatedStreamBuf.h>
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	33 #include <aws/core/utils/StringUtils.h>
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	34 #include <aws/transfer/TransferManager.h>
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	35 #include <aws/crt/Api.h>
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	36 #include <fstream>
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	37
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	38 #include <boost/lexical_cast.hpp>
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	39 #include <boost/interprocess/streams/bufferstream.hpp>
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	40 #include <iostream>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	41 #include <fstream>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	42
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	43 const char* ALLOCATION_TAG = "OrthancS3";
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	44
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	45 class AwsS3StoragePlugin : public BaseStorage
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	46 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	47 public:
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	48
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	49 std::string bucketName_;
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	50 bool storageContainsUnknownFiles_;
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	51 bool useTransferManager_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	52 std::shared_ptr<Aws::S3::S3Client> client_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	53 std::shared_ptr<Aws::Utils::Threading::Executor> executor_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	54 std::shared_ptr<Aws::Transfer::TransferManager> transferManager_;
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	55
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	56 public:
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	57
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	58 AwsS3StoragePlugin(const std::string& nameForLogs, std::shared_ptr<Aws::S3::S3Client> client, const std::string& bucketName, bool enableLegacyStorageStructure, bool storageContainsUnknownFiles, bool useTransferManager);
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	59
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	60 virtual ~AwsS3StoragePlugin();
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	61
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	62 virtual IWriter* GetWriterForObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	63 virtual IReader* GetReaderForObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	64 virtual void DeleteObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	65 };
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	66
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	67
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	68 class DirectWriter : public IStorage::IWriter
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	69 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	70 std::string path_;
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	71 std::shared_ptr<Aws::S3::S3Client> client_;
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	72 std::string bucketName_;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	73
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	74 public:
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	75 DirectWriter(std::shared_ptr<Aws::S3::S3Client> client, const std::string& bucketName, const std::string& path)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	76 : path_(path),
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	77 client_(client),
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	78 bucketName_(bucketName)
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	79 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	80 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	81
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	82 virtual ~DirectWriter()
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	83 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	84 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	85
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	86 virtual void Write(const char* data, size_t size)
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	87 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	88 Aws::S3::Model::PutObjectRequest putObjectRequest;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	89
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	90 putObjectRequest.SetBucket(bucketName_.c_str());
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	91 putObjectRequest.SetKey(path_.c_str());
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	92
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	93 std::shared_ptr<Aws::StringStream> stream = Aws::MakeShared<Aws::StringStream>(ALLOCATION_TAG, std::ios_base::in \| std::ios_base::binary);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	94
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	95 stream->rdbuf()->pubsetbuf(const_cast<char*>(data), size);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	96 stream->rdbuf()->pubseekpos(size);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	97 stream->seekg(0);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	98
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	99 putObjectRequest.SetBody(stream);
74 a25b4140e7e4 AWS: added the content MD5 in the request when writing Alain Mazy <am@osimis.io> parents: 70 diff changeset	100 putObjectRequest.SetContentMD5(Aws::Utils::HashingUtils::Base64Encode(Aws::Utils::HashingUtils::CalculateMD5(*stream)));
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	101
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	102 auto result = client_->PutObject(putObjectRequest);
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	103
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	104 if (!result.IsSuccess())
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	105 {
62 d10696f94959 more detailed AWS errors Alain Mazy <am@osimis.io> parents: 57 diff changeset	106 throw StoragePluginException(std::string("error while writing file ") + path_ + ": response code = " + boost::lexical_cast<std::string>((int)result.GetError().GetResponseCode()) + " " + result.GetError().GetExceptionName().c_str() + " " + result.GetError().GetMessage().c_str());
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	107 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	108 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	109 };
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	110
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	111
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	112 class DirectReader : public IStorage::IReader
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	113 {
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	114 protected:
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	115 std::shared_ptr<Aws::S3::S3Client> client_;
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	116 std::string bucketName_;
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	117 std::list<std::string> paths_;
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	118 std::string uuid_;
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	119
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	120 public:
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	121 DirectReader(std::shared_ptr<Aws::S3::S3Client> client, const std::string& bucketName, const std::list<std::string>& paths, const char* uuid)
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	122 : client_(client),
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	123 bucketName_(bucketName),
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	124 paths_(paths),
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	125 uuid_(uuid)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	126 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	127 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	128
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	129 virtual ~DirectReader()
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	130 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	131
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	132 }
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	133
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	134 virtual size_t GetSize()
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	135 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	136 std::string firstExceptionMessage;
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	137
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	138 for (auto& path: paths_)
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	139 {
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	140 try
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	141 {
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	142 return _GetSize(path);
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	143 }
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	144 catch (StoragePluginException& ex)
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	145 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	146 if (firstExceptionMessage.empty())
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	147 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	148 firstExceptionMessage = ex.what();
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	149 }
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	150 //ignore to retry
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	151 }
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	152 }
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	153 throw StoragePluginException(firstExceptionMessage);
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	154 }
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	155
41 d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	156 virtual void ReadWhole(char* data, size_t size)
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	157 {
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	158 _Read(data, size, 0, false);
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	159 }
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	160
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	161 virtual void ReadRange(char* data, size_t size, size_t fromOffset)
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	162 {
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	163 _Read(data, size, fromOffset, true);
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	164 }
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	165
70 16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	166 private:
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	167
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	168 size_t _GetSize(const std::string& path)
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	169 {
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	170 Aws::S3::Model::ListObjectsRequest listObjectRequest;
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	171 listObjectRequest.SetBucket(bucketName_.c_str());
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	172 listObjectRequest.SetPrefix(path.c_str());
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	173
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	174 auto result = client_->ListObjects(listObjectRequest);
70 16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	175
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	176 if (result.IsSuccess())
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	177 {
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	178 Aws::Vector<Aws::S3::Model::Object> objectList =
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	179 result.GetResult().GetContents();
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	180
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	181 if (objectList.size() == 1)
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	182 {
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	183 return objectList[0].GetSize();
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	184 }
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	185 else if (objectList.size() > 1)
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	186 {
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	187 throw StoragePluginException(std::string("error while reading file ") + path + ": multiple objet with same name !");
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	188 }
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	189 throw StoragePluginException(std::string("error while reading file ") + path + ": object not found !");
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	190 }
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	191 else
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	192 {
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	193 throw StoragePluginException(std::string("error while reading file ") + path + ": " + result.GetError().GetExceptionName().c_str() + " " + result.GetError().GetMessage().c_str());
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	194 }
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	195 }
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	196
16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	197
41 d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	198 void _Read(char* data, size_t size, size_t fromOffset, bool useRange)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	199 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	200 std::string firstExceptionMessage;
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	201
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	202 for (auto& path: paths_)
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	203 {
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	204 try
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	205 {
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	206 return __Read(path, data, size, fromOffset, useRange);
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	207 }
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	208 catch (StoragePluginException& ex)
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	209 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	210 if (firstExceptionMessage.empty())
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	211 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	212 firstExceptionMessage = ex.what();
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	213 }
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	214 //ignore to retry
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	215 }
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	216 }
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	217 throw StoragePluginException(firstExceptionMessage);
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	218 }
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	219
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	220 void __Read(const std::string& path, char* data, size_t size, size_t fromOffset, bool useRange)
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	221 {
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	222 Aws::S3::Model::GetObjectRequest getObjectRequest;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	223 getObjectRequest.SetBucket(bucketName_.c_str());
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	224 getObjectRequest.SetKey(path.c_str());
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	225
41 d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	226 if (useRange)
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	227 {
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	228 // https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	229 std::string range = std::string("bytes=") + boost::lexical_cast<std::string>(fromOffset) + "-" + boost::lexical_cast<std::string>(fromOffset + size -1);
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	230 getObjectRequest.SetRange(range.c_str());
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	231 }
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	232
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	233 getObjectRequest.SetResponseStreamFactory(
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	234 [data, size]()
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	235 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	236 std::unique_ptr<Aws::StringStream>
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	237 istream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	238
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	239 istream->rdbuf()->pubsetbuf(static_cast<char*>(data),
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	240 size);
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	241
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	242 return istream.release();
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	243 });
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	244
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	245 // Get the object
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	246 auto result = client_->GetObject(getObjectRequest);
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	247 if (result.IsSuccess())
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	248 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	249 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	250 else
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	251 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	252 throw StoragePluginException(std::string("error while reading file ") + path + ": response code = " + boost::lexical_cast<std::string>((int)result.GetError().GetResponseCode()) + " " + result.GetError().GetExceptionName().c_str() + " " + result.GetError().GetMessage().c_str());
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	253 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	254 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	255
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	256 };
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	257
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	258
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	259
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	260 class TransferWriter : public IStorage::IWriter
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	261 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	262 std::string path_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	263 std::shared_ptr<Aws::Transfer::TransferManager> transferManager_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	264 std::string bucketName_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	265
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	266 public:
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	267 TransferWriter(std::shared_ptr<Aws::Transfer::TransferManager> transferManager, const std::string& bucketName, const std::string& path)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	268 : path_(path),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	269 transferManager_(transferManager),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	270 bucketName_(bucketName)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	271 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	272 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	273
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	274 virtual ~TransferWriter()
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	275 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	276 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	277
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	278 virtual void Write(const char* data, size_t size)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	279 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	280 boost::interprocess::bufferstream buffer(const_cast<char>(static_cast<const char>(data)), static_cast<size_t>(size));
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	281 std::shared_ptr<Aws::IOStream> body = Aws::MakeShared<Aws::IOStream>(ALLOCATION_TAG, buffer.rdbuf());
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	282
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	283 std::shared_ptr<Aws::Transfer::TransferHandle> transferHandle = transferManager_->UploadFile(body, bucketName_, path_.c_str(), "application/binary", Aws::Map<Aws::String, Aws::String>());
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	284 transferHandle->WaitUntilFinished();
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	285
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	286 if (transferHandle->GetStatus() != Aws::Transfer::TransferStatus::COMPLETED)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	287 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	288 throw StoragePluginException(std::string("error while writing file ") + path_ + ": response code = " + boost::lexical_cast<std::string>(static_cast<int>(transferHandle->GetLastError().GetResponseCode())) + " " + transferHandle->GetLastError().GetMessage());
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	289 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	290 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	291 };
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	292
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	293
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	294 class TransferReader : public DirectReader
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	295 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	296 std::shared_ptr<Aws::Transfer::TransferManager> transferManager_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	297
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	298 public:
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	299 TransferReader(std::shared_ptr<Aws::Transfer::TransferManager> transferManager, std::shared_ptr<Aws::S3::S3Client> client, const std::string& bucketName, const std::list<std::string>& paths, const char* uuid)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	300 : DirectReader(client, bucketName, paths, uuid),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	301 transferManager_(transferManager)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	302 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	303 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	304
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	305 virtual ~TransferReader()
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	306 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	307
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	308 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	309
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	310 virtual void ReadWhole(char* data, size_t size)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	311 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	312 std::string firstExceptionMessage;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	313
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	314 for (auto& path: paths_)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	315 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	316 try
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	317 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	318 // The local variable 'streamBuffer' is captured by reference in a lambda.
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	319 // It must persist until all downloading by the 'transfer_manager' is complete.
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	320 Aws::Utils::Stream::PreallocatedStreamBuf streamBuffer(reinterpret_cast<unsigned char*>(data), size);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	321
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	322 std::shared_ptr<Aws::Transfer::TransferHandle> downloadHandler = transferManager_->DownloadFile(bucketName_, path, [&]() { //Define a lambda expression for the callback method parameter to stream back the data.
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	323 return Aws::New<Aws::IOStream>(ALLOCATION_TAG, &streamBuffer);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	324 });
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	325
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	326 downloadHandler->WaitUntilFinished();
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	327
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	328 if (downloadHandler->GetStatus() == Aws::Transfer::TransferStatus::COMPLETED)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	329 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	330 return;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	331 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	332 else if (firstExceptionMessage.empty())
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	333 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	334 firstExceptionMessage = downloadHandler->GetLastError().GetMessage();
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	335 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	336 // getObjectRequest.SetResponseStreamFactory(
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	337 // [data, size]()
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	338 // {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	339 // std::unique_ptr<Aws::StringStream>
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	340 // istream(Aws::New<Aws::StringStream>(ALLOCATION_TAG));
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	341
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	342 // istream->rdbuf()->pubsetbuf(static_cast<char*>(data),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	343 // size);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	344
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	345 // return istream.release();
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	346 // });
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	347
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	348
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	349 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	350 catch (StoragePluginException& ex)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	351 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	352 if (firstExceptionMessage.empty())
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	353 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	354 firstExceptionMessage = ex.what();
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	355 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	356 //ignore to retry
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	357 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	358 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	359 throw StoragePluginException(firstExceptionMessage);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	360 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	361
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	362 };
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	363
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	364
41 d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	365
d99afdf6d872 ReadRange for AWS Alain Mazy <am@osimis.io> parents: 37 diff changeset	366
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	367 const char* AwsS3StoragePluginFactory::GetStoragePluginName()
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	368 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	369 return "AWS S3 Storage";
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	370 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	371
94 1bc055199cd2 Added a description for all plugins Alain Mazy <am@osimis.io> parents: 78 diff changeset	372 const char* AwsS3StoragePluginFactory::GetStorageDescription()
1bc055199cd2 Added a description for all plugins Alain Mazy <am@osimis.io> parents: 78 diff changeset	373 {
1bc055199cd2 Added a description for all plugins Alain Mazy <am@osimis.io> parents: 78 diff changeset	374 return "Stores the Orthanc storage area in AWS S3";
1bc055199cd2 Added a description for all plugins Alain Mazy <am@osimis.io> parents: 78 diff changeset	375 }
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	376
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	377 static std::unique_ptr<Aws::Crt::ApiHandle> api_;
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	378 static std::unique_ptr<Aws::SDKOptions> sdkOptions_;
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	379
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	380
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	381 IStorage* AwsS3StoragePluginFactory::CreateStorage(const std::string& nameForLogs, const OrthancPlugins::OrthancConfiguration& orthancConfig)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	382 {
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	383 if (sdkOptions_.get() != NULL)
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	384 {
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	385 throw Orthanc::OrthancException(Orthanc::ErrorCode_BadSequenceOfCalls, "Cannot initialize twice");
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	386 }
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	387
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	388 api_.reset(new Aws::Crt::ApiHandle);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	389
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	390 sdkOptions_.reset(new Aws::SDKOptions);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	391 sdkOptions_->cryptoOptions.initAndCleanupOpenSSL = false; // Done by the Orthanc framework
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	392 sdkOptions_->httpOptions.initAndCleanupCurl = false; // Done by the Orthanc framework
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	393
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	394 Aws::InitAPI(*sdkOptions_);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	395
15 2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	396 bool enableLegacyStorageStructure;
70 16e419fe80c5 Google & Azure: Added the "EnableLegacyUnknownFiles" configuration option Alain Mazy <am@osimis.io> parents: 68 diff changeset	397 bool storageContainsUnknownFiles;
15 2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	398
77 80792bb9600e new HybridMode Alain Mazy <am@osimis.io> parents: 74 diff changeset	399 if (!orthancConfig.IsSection(GetConfigurationSectionName()))
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	400 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	401 OrthancPlugins::LogWarning(std::string(GetStoragePluginName()) + " plugin, section missing. Plugin is not enabled.");
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	402 return nullptr;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	403 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	404
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	405 OrthancPlugins::OrthancConfiguration pluginSection;
77 80792bb9600e new HybridMode Alain Mazy <am@osimis.io> parents: 74 diff changeset	406 orthancConfig.GetSection(pluginSection, GetConfigurationSectionName());
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	407
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	408 if (!BaseStorage::ReadCommonConfiguration(enableLegacyStorageStructure, storageContainsUnknownFiles, pluginSection))
15 2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	409 {
2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	410 return nullptr;
2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	411 }
2a02b21f0a19 migration + storage structure Alain Mazy parents: 6 diff changeset	412
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	413 std::string bucketName;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	414 std::string region;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	415 std::string accessKey;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	416 std::string secretKey;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	417
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	418 if (!pluginSection.LookupStringValue(bucketName, "BucketName"))
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	419 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	420 OrthancPlugins::LogError("AwsS3Storage/BucketName configuration missing. Unable to initialize plugin");
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	421 return nullptr;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	422 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	423
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	424 if (!pluginSection.LookupStringValue(region, "Region"))
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	425 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	426 OrthancPlugins::LogError("AwsS3Storage/Region configuration missing. Unable to initialize plugin");
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	427 return nullptr;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	428 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	429
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	430 const std::string endpoint = pluginSection.GetStringValue("Endpoint", "");
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	431 const unsigned int connectTimeout = pluginSection.GetUnsignedIntegerValue("ConnectTimeout", 30);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	432 const unsigned int requestTimeout = pluginSection.GetUnsignedIntegerValue("RequestTimeout", 1200);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	433 const bool virtualAddressing = pluginSection.GetBooleanValue("VirtualAddressing", true);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	434 const std::string caFile = orthancConfig.GetStringValue("HttpsCACertificates", "");
27 e1f52b851827 Added "VirtualAddressing" configuration option in the AWS S3 plugin (for compatibility with minio) Sebastien Jodogne <s.jodogne@gmail.com> parents: 15 diff changeset	435
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	436 try
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	437 {
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	438 Aws::Client::ClientConfiguration configuration;
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	439
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	440 configuration.region = region.c_str();
6 393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	441 configuration.scheme = Aws::Http::Scheme::HTTPS;
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	442 configuration.connectTimeoutMs = connectTimeout * 1000;
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	443 configuration.requestTimeoutMs = requestTimeout * 1000;
27 e1f52b851827 Added "VirtualAddressing" configuration option in the AWS S3 plugin (for compatibility with minio) Sebastien Jodogne <s.jodogne@gmail.com> parents: 15 diff changeset	444 configuration.httpRequestTimeoutMs = requestTimeout * 1000;
6 393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	445
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	446 if (!endpoint.empty())
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	447 {
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	448 configuration.endpointOverride = endpoint.c_str();
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	449 }
393fcf337462 AWS: added 3 configurations: Endpoint, ConnectionTimeout, RequestTimeout Alain Mazy parents: 1 diff changeset	450
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	451 if (!caFile.empty())
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	452 {
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	453 configuration.caFile = caFile;
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	454 }
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	455
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	456 bool useTransferManager = true; // new in v 2.3.0
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	457 pluginSection.LookupBooleanValue(useTransferManager, "UseTransferManager");
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	458
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	459 if (pluginSection.LookupStringValue(accessKey, "AccessKey") && pluginSection.LookupStringValue(secretKey, "SecretKey"))
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	460 {
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	461 OrthancPlugins::LogInfo("AWS S3 Storage: using credentials from the configuration file");
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	462 Aws::Auth::AWSCredentials credentials(accessKey.c_str(), secretKey.c_str());
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	463
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	464 std::shared_ptr<Aws::S3::S3Client> client = Aws::MakeShared<Aws::S3::S3Client>(ALLOCATION_TAG, credentials, configuration, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, virtualAddressing);
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	465
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	466 OrthancPlugins::LogInfo("AWS S3 storage initialized");
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	467
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	468 return new AwsS3StoragePlugin(nameForLogs, client, bucketName, enableLegacyStorageStructure, storageContainsUnknownFiles, useTransferManager);
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	469 }
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	470 else
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	471 {
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	472 // when using default credentials, credentials are not checked at startup but only the first time you try to access the bucket !
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	473 OrthancPlugins::LogInfo("AWS S3 Storage: using default credentials provider");
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	474 std::shared_ptr<Aws::S3::S3Client> client = Aws::MakeShared<Aws::S3::S3Client>(ALLOCATION_TAG, configuration, Aws::Client::AWSAuthV4Signer::PayloadSigningPolicy::Never, virtualAddressing);
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	475
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	476 OrthancPlugins::LogInfo("AWS S3 storage initialized");
1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	477
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	478 return new AwsS3StoragePlugin(nameForLogs, client, bucketName, enableLegacyStorageStructure, storageContainsUnknownFiles, useTransferManager);
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	479 }
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	480 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	481 catch (const std::exception& e)
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	482 {
53 1c3e34f5c5c6 AWS S3: if no access & secret keys are provided, now getting the credentials from the default credentials manager Alain Mazy <am@osimis.io> parents: 41 diff changeset	483 OrthancPlugins::LogError(std::string("AWS S3 Storage plugin: failed to initialize plugin: ") + e.what());
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	484 return nullptr;
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	485 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	486
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	487 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	488
57 ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	489
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	490 AwsS3StoragePlugin::~AwsS3StoragePlugin()
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	491 {
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	492 assert(sdkOptions_.get() != NULL);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	493 Aws::ShutdownAPI(*sdkOptions_);
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	494 api_.reset();
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	495 }
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	496
ba1be668e475 fix initialization of the aws static library Sebastien Jodogne <s.jodogne@gmail.com> parents: 53 diff changeset	497
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	498 AwsS3StoragePlugin::AwsS3StoragePlugin(const std::string& nameForLogs, std::shared_ptr<Aws::S3::S3Client> client, const std::string& bucketName, bool enableLegacyStorageStructure, bool storageContainsUnknownFiles, bool useTransferManager)
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	499 : BaseStorage(nameForLogs, enableLegacyStorageStructure),
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	500 bucketName_(bucketName),
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	501 storageContainsUnknownFiles_(storageContainsUnknownFiles),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	502 useTransferManager_(useTransferManager),
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	503 client_(client)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	504 {
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	505 if (useTransferManager_)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	506 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	507 executor_ = Aws::MakeShared<Aws::Utils::Threading::PooledThreadExecutor>(ALLOCATION_TAG, 10);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	508 Aws::Transfer::TransferManagerConfiguration transferConfig(executor_.get());
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	509 transferConfig.s3Client = client_;
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	510
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	511 transferManager_ = Aws::Transfer::TransferManager::Create(transferConfig);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	512 }
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	513 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	514
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	515 IStorage::IWriter* AwsS3StoragePlugin::GetWriterForObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	516 {
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	517 if (useTransferManager_)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	518 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	519 return new TransferWriter(transferManager_, bucketName_, GetPath(uuid, type, encryptionEnabled));
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	520 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	521 else
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	522 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	523 return new DirectWriter(client_, bucketName_, GetPath(uuid, type, encryptionEnabled));
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	524 }
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	525 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	526
78 d7295e8678d7 renames Alain Mazy <am@osimis.io> parents: 77 diff changeset	527 IStorage::IReader* AwsS3StoragePlugin::GetReaderForObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	528 {
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	529 std::list<std::string> paths;
c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	530 paths.push_back(GetPath(uuid, type, encryptionEnabled, false));
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	531 if (storageContainsUnknownFiles_)
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	532 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	533 paths.push_back(GetPath(uuid, type, encryptionEnabled, true));
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	534 }
64 c4f56973a279 Fix reading/deleting DCM header files that were saved with plugin v 1.2.0 and Orthanc 1.9.3 which had a .unk extension Alain Mazy <am@osimis.io> parents: 62 diff changeset	535
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	536 if (useTransferManager_)
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	537 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	538 return new TransferReader(transferManager_, client_, bucketName_, paths, uuid);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	539 }
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	540 else
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	541 {
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	542 return new DirectReader(client_, bucketName_, paths, uuid);
c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	543 }
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	544 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	545
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	546 void AwsS3StoragePlugin::DeleteObject(const char* uuid, OrthancPluginContentType type, bool encryptionEnabled)
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	547 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	548 std::string firstExceptionMessage;
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	549
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	550 std::list<std::string> paths;
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	551 paths.push_back(GetPath(uuid, type, encryptionEnabled, false));
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	552 if (storageContainsUnknownFiles_)
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	553 {
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	554 paths.push_back(GetPath(uuid, type, encryptionEnabled, true));
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	555 }
fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	556
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	557 // DeleteObject succeeds even if the file does not exist -> we need to try to delete every path
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	558 for (auto& path: paths)
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	559 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	560 Aws::S3::Model::DeleteObjectRequest deleteObjectRequest;
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	561 deleteObjectRequest.SetBucket(bucketName_.c_str());
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	562 deleteObjectRequest.SetKey(path.c_str());
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	563
106 c9356e42af99 added TransferMode for S3 (currently affected by https://github.com/aws/aws-sdk-cpp/issues/2319 since we are using version 1.9.45) Alain Mazy <am@osimis.io> parents: 94 diff changeset	564 auto result = client_->DeleteObject(deleteObjectRequest);
68 58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	565
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	566 if (!result.IsSuccess() && firstExceptionMessage.empty())
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	567 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	568 firstExceptionMessage = std::string("error while deleting file ") + path + ": response code = " + boost::lexical_cast<std::string>((int)result.GetError().GetResponseCode()) + " " + result.GetError().GetExceptionName().c_str() + " " + result.GetError().GetMessage().c_str();
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	569 }
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	570 }
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	571
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	572 if (!firstExceptionMessage.empty())
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	573 {
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	574 throw StoragePluginException(firstExceptionMessage);
58a03fce4897 new option for AWS: EnableLegacyUnknownFiles Alain Mazy <am@osimis.io> parents: 64 diff changeset	575 }
1 fc26a8fc54d5 initial release Alain Mazy <alain@mazy.be> parents: diff changeset	576 }

Mercurial > hg > orthanc-object-storage

annotate Aws/AwsS3StoragePlugin.cpp @ 110:37a4b8e2577f