# HG changeset patch # User Alain Mazy # Date 1656501320 -7200 # Node ID 22966345eaba5af029b13b5b67f1292cfbf5341e # Parent 207f259c41c57ca4ce06ba13bbdf04904e692b19 skip StorageCache when verifying md5 or when re-compressing attachments diff -r 207f259c41c5 -r 22966345eaba OrthancFramework/Sources/FileStorage/StorageAccessor.cpp --- a/OrthancFramework/Sources/FileStorage/StorageAccessor.cpp Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancFramework/Sources/FileStorage/StorageAccessor.cpp Wed Jun 29 13:15:20 2022 +0200 @@ -62,7 +62,7 @@ }; - StorageAccessor::StorageAccessor(IStorageArea &area, StorageCache& cache) : + StorageAccessor::StorageAccessor(IStorageArea &area, StorageCache* cache) : area_(area), cache_(cache), metrics_(NULL) @@ -70,7 +70,7 @@ } StorageAccessor::StorageAccessor(IStorageArea &area, - StorageCache& cache, + StorageCache* cache, MetricsRegistry &metrics) : area_(area), cache_(cache), @@ -101,7 +101,11 @@ MetricsTimer timer(*this, METRICS_CREATE); area_.Create(uuid, data, size, type); - cache_.Add(uuid, type, data, size); + + if (cache_ != NULL) + { + cache_->Add(uuid, type, data, size); + } return FileInfo(uuid, type, size, md5); } @@ -133,7 +137,11 @@ } } - cache_.Add(uuid, type, data, size); // always add uncompressed data to cache + if (cache_ != NULL) + { + cache_->Add(uuid, type, data, size); // always add uncompressed data to cache + } + return FileInfo(uuid, type, size, md5, CompressionType_ZlibWithSize, compressed.size(), compressedMD5); } @@ -156,7 +164,8 @@ void StorageAccessor::Read(std::string& content, const FileInfo& info) { - if (!cache_.Fetch(content, info.GetUuid(), info.GetContentType())) + if (cache_ == NULL || + !cache_->Fetch(content, info.GetUuid(), info.GetContentType())) { switch (info.GetCompressionType()) { @@ -192,7 +201,10 @@ } // always store the uncompressed data in cache - cache_.Add(info.GetUuid(), info.GetContentType(), content); + if (cache_ != NULL) + { + cache_->Add(info.GetUuid(), info.GetContentType(), content); + } } // TODO Check the validity of the uncompressed MD5? @@ -202,7 +214,7 @@ void StorageAccessor::ReadRaw(std::string& content, const FileInfo& info) { - if (!cache_.Fetch(content, info.GetUuid(), info.GetContentType())) + if (cache_ == NULL || !cache_->Fetch(content, info.GetUuid(), info.GetContentType())) { MetricsTimer timer(*this, METRICS_READ); std::unique_ptr buffer(area_.Read(info.GetUuid(), info.GetContentType())); @@ -214,7 +226,10 @@ void StorageAccessor::Remove(const std::string& fileUuid, FileContentType type) { - cache_.Invalidate(fileUuid, type); + if (cache_ != NULL) + { + cache_->Invalidate(fileUuid, type); + } { MetricsTimer timer(*this, METRICS_REMOVE); @@ -234,14 +249,17 @@ FileContentType contentType, uint64_t end /* exclusive */) { - if (!cache_.FetchStartRange(target, fileUuid, contentType, end)) + if (cache_ == NULL || !cache_->FetchStartRange(target, fileUuid, contentType, end)) { MetricsTimer timer(*this, METRICS_READ); std::unique_ptr buffer(area_.ReadRange(fileUuid, contentType, 0, end)); assert(buffer->GetSize() == end); buffer->MoveToString(target); - cache_.AddStartRange(fileUuid, contentType, target); + if (cache_ != NULL) + { + cache_->AddStartRange(fileUuid, contentType, target); + } } } @@ -251,13 +269,16 @@ const FileInfo& info, const std::string& mime) { - if (!cache_.Fetch(sender.GetBuffer(), info.GetUuid(), info.GetContentType())) + if (cache_ == NULL || !cache_->Fetch(sender.GetBuffer(), info.GetUuid(), info.GetContentType())) { MetricsTimer timer(*this, METRICS_READ); std::unique_ptr buffer(area_.Read(info.GetUuid(), info.GetContentType())); buffer->MoveToString(sender.GetBuffer()); - cache_.Add(info.GetUuid(), info.GetContentType(), sender.GetBuffer()); + if (cache_ != NULL) + { + cache_->Add(info.GetUuid(), info.GetContentType(), sender.GetBuffer()); + } } sender.SetContentType(mime); diff -r 207f259c41c5 -r 22966345eaba OrthancFramework/Sources/FileStorage/StorageAccessor.h --- a/OrthancFramework/Sources/FileStorage/StorageAccessor.h Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancFramework/Sources/FileStorage/StorageAccessor.h Wed Jun 29 13:15:20 2022 +0200 @@ -68,7 +68,7 @@ class MetricsTimer; IStorageArea& area_; - StorageCache& cache_; + StorageCache* cache_; MetricsRegistry* metrics_; #if ORTHANC_ENABLE_CIVETWEB == 1 || ORTHANC_ENABLE_MONGOOSE == 1 @@ -79,10 +79,10 @@ public: explicit StorageAccessor(IStorageArea& area, - StorageCache& cache); + StorageCache* cache); StorageAccessor(IStorageArea& area, - StorageCache& cache, + StorageCache* cache, MetricsRegistry& metrics); FileInfo Write(const void* data, diff -r 207f259c41c5 -r 22966345eaba OrthancFramework/UnitTestsSources/FileStorageTests.cpp --- a/OrthancFramework/UnitTestsSources/FileStorageTests.cpp Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancFramework/UnitTestsSources/FileStorageTests.cpp Wed Jun 29 13:15:20 2022 +0200 @@ -127,7 +127,7 @@ { FilesystemStorage s("UnitTestsStorage"); StorageCache cache; - StorageAccessor accessor(s, cache); + StorageAccessor accessor(s, &cache); std::string data = "Hello world"; FileInfo info = accessor.Write(data, FileContentType_Dicom, CompressionType_None, true); @@ -149,7 +149,7 @@ { FilesystemStorage s("UnitTestsStorage"); StorageCache cache; - StorageAccessor accessor(s, cache); + StorageAccessor accessor(s, &cache); std::string data = "Hello world"; FileInfo info = accessor.Write(data, FileContentType_Dicom, CompressionType_ZlibWithSize, true); @@ -170,7 +170,7 @@ { FilesystemStorage s("UnitTestsStorage"); StorageCache cache; - StorageAccessor accessor(s, cache); + StorageAccessor accessor(s, &cache); std::string r; std::string compressedData = "Hello"; diff -r 207f259c41c5 -r 22966345eaba OrthancServer/Sources/OrthancRestApi/OrthancRestResources.cpp --- a/OrthancServer/Sources/OrthancRestApi/OrthancRestResources.cpp Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancServer/Sources/OrthancRestApi/OrthancRestResources.cpp Wed Jun 29 13:15:20 2022 +0200 @@ -2132,7 +2132,7 @@ // Return the raw data (possibly compressed), as stored on the filesystem std::string content; int64_t revision; - context.ReadAttachment(content, revision, publicId, type, false); + context.ReadAttachment(content, revision, publicId, type, false, true /* skipCache when you absolutely need the compressed data */); int64_t userRevision; std::string userMD5; @@ -2314,7 +2314,7 @@ // First check whether the compressed data is correctly stored in the disk std::string data; - context.ReadAttachment(data, revision, publicId, StringToContentType(name), false); + context.ReadAttachment(data, revision, publicId, StringToContentType(name), false, true /* skipCache when you absolutely need the compressed data */); std::string actualMD5; Toolbox::ComputeMD5(actualMD5, data); @@ -2329,7 +2329,7 @@ } else { - context.ReadAttachment(data, revision, publicId, StringToContentType(name), true); + context.ReadAttachment(data, revision, publicId, StringToContentType(name), true, true /* skipCache when you absolutely need the compressed data */); Toolbox::ComputeMD5(actualMD5, data); ok = (actualMD5 == info.GetUncompressedMD5()); } diff -r 207f259c41c5 -r 22966345eaba OrthancServer/Sources/ServerContext.cpp --- a/OrthancServer/Sources/ServerContext.cpp Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancServer/Sources/ServerContext.cpp Wed Jun 29 13:15:20 2022 +0200 @@ -488,7 +488,7 @@ void ServerContext::RemoveFile(const std::string& fileUuid, FileContentType type) { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.Remove(fileUuid, type); } @@ -533,7 +533,7 @@ try { MetricsRegistry::Timer timer(GetMetricsRegistry(), "orthanc_store_dicom_duration_ms"); - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); DicomInstanceHasher hasher(summary); resultPublicId = hasher.HashInstance(); @@ -851,7 +851,7 @@ } else { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.AnswerFile(output, attachment, GetFileContentMime(content)); } } @@ -881,7 +881,7 @@ std::string content; - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.Read(content, attachment); FileInfo modified = accessor.Write(content.empty() ? NULL : content.c_str(), @@ -937,7 +937,7 @@ std::string dicom; { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.Read(dicom, attachment); } @@ -1003,7 +1003,7 @@ std::string dicom; { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.ReadStartRange(dicom, attachment.GetUuid(), FileContentType_Dicom, pixelDataOffset); } @@ -1026,7 +1026,7 @@ std::string dicomAsJson; { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.Read(dicomAsJson, attachment); } @@ -1132,7 +1132,7 @@ { uint64_t pixelDataOffset = boost::lexical_cast(s); - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); accessor.ReadStartRange(dicom, attachment.GetUuid(), attachment.GetContentType(), pixelDataOffset); assert(dicom.size() == pixelDataOffset); @@ -1153,7 +1153,8 @@ int64_t& revision, const std::string& instancePublicId, FileContentType content, - bool uncompressIfNeeded) + bool uncompressIfNeeded, + bool skipCache) { FileInfo attachment; if (!index_.LookupAttachment(attachment, revision, instancePublicId, content)) @@ -1166,7 +1167,14 @@ assert(attachment.GetContentType() == content); { - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageCache* cache = NULL; + + if (!skipCache) + { + cache = &storageCache_; + } + + StorageAccessor accessor(area_, cache, GetMetricsRegistry()); if (uncompressIfNeeded) { @@ -1266,7 +1274,7 @@ // TODO Should we use "gzip" instead? CompressionType compression = (compressionEnabled_ ? CompressionType_ZlibWithSize : CompressionType_None); - StorageAccessor accessor(area_, storageCache_, GetMetricsRegistry()); + StorageAccessor accessor(area_, &storageCache_, GetMetricsRegistry()); FileInfo attachment = accessor.Write(data, size, attachmentType, compression, storeMD5_); try diff -r 207f259c41c5 -r 22966345eaba OrthancServer/Sources/ServerContext.h --- a/OrthancServer/Sources/ServerContext.h Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancServer/Sources/ServerContext.h Wed Jun 29 13:15:20 2022 +0200 @@ -370,7 +370,8 @@ int64_t& revision, const std::string& instancePublicId, FileContentType content, - bool uncompressIfNeeded); + bool uncompressIfNeeded, + bool skipCache = false); void SetStoreMD5ForAttachments(bool storeMD5); diff -r 207f259c41c5 -r 22966345eaba OrthancServer/Sources/ServerToolbox.cpp --- a/OrthancServer/Sources/ServerToolbox.cpp Wed Jun 29 09:43:35 2022 +0200 +++ b/OrthancServer/Sources/ServerToolbox.cpp Wed Jun 29 13:15:20 2022 +0200 @@ -144,8 +144,7 @@ try { // Read and parse the content of the DICOM file - StorageCache cache; // we create a temporary cache for this operation (required by the StorageAccessor) - StorageAccessor accessor(storageArea, cache); + StorageAccessor accessor(storageArea, NULL); // no cache std::string content; accessor.Read(content, attachment);