Mercurial > hg > orthanc

annotate OrthancFramework/Sources/FileStorage/StorageCache.cpp @ 4906:f656fb878b50

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
reintroduced storage cache for StartRange files
author Alain Mazy <am@osimis.io>
date Mon, 21 Feb 2022 10:11:38 +0100
parents ea5f1c6ed07e
children 0ea402b4d901
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
1 /**
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
2 * Orthanc - A Lightweight, RESTful DICOM Store
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
3 * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
4 * Department, University Hospital of Liege, Belgium
4870
43e613a7756b upgrade to year 2022
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4831
diff changeset
5 * Copyright (C) 2017-2022 Osimis S.A., Belgium
43e613a7756b upgrade to year 2022
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4831
diff changeset
6 * Copyright (C) 2021-2022 Sebastien Jodogne, ICTEAM UCLouvain, Belgium
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
7 *
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
8 * This program is free software: you can redistribute it and/or
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
9 * modify it under the terms of the GNU Lesser General Public License
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
10 * as published by the Free Software Foundation, either version 3 of
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
11 * the License, or (at your option) any later version.
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
12 *
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
13 * This program is distributed in the hope that it will be useful, but
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
14 * WITHOUT ANY WARRANTY; without even the implied warranty of
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
15 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
16 * Lesser General Public License for more details.
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
17 *
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
18 * You should have received a copy of the GNU Lesser General Public
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
19 * License along with this program. If not, see
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
20 * <http://www.gnu.org/licenses/>.
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
21 **/
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
22
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
23
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
24 #include "../PrecompiledHeaders.h"
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
25 #include "StorageCache.h"
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
26
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
27 #include "../Compatibility.h"
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
28 #include "../Logging.h"
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
29 #include "../OrthancException.h"
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
30
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
31 #include <boost/lexical_cast.hpp>
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
32
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
33
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
34 namespace Orthanc
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
35 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
36 static std::string GetCacheKeyFullFile(const std::string& uuid,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
37 FileContentType contentType)
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
38 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
39 return uuid + ":" + boost::lexical_cast<std::string>(contentType) + ":1";
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
40 }
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
41
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
42 static std::string GetCacheKeyStartRange(const std::string& uuid,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
43 FileContentType contentType)
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
44 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
45 return uuid + ":" + boost::lexical_cast<std::string>(contentType) + ":0";
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
46 }
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
47
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
48 void StorageCache::SetMaximumSize(size_t size)
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
49 {
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
50 cache_.SetMaximumSize(size);
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
51 }
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
52
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
53
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
54 void StorageCache::Add(const std::string& uuid,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
55 FileContentType contentType,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
56 const std::string& value)
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
57 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
58 const std::string key = GetCacheKeyFullFile(uuid, contentType);
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
59 cache_.Add(key, value);
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
60 }
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
61
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
62
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
63 void StorageCache::Add(const std::string& uuid,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
64 FileContentType contentType,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
65 const void* buffer,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
66 size_t size)
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
67 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
68 const std::string key = GetCacheKeyFullFile(uuid, contentType);
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
69 cache_.Add(key, buffer, size);
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
70 }
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
71
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
72
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
73 void StorageCache::AddStartRange(const std::string& uuid,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
74 FileContentType contentType,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
75 const std::string& value)
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
76 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
77 const std::string key = GetCacheKeyStartRange(uuid, contentType);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
78 cache_.Add(key, value);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
79 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
80
4826
6c276fac0cc0 fix storage cache for user attachment -> the cache is ignoring them
Alain Mazy <am@osimis.io>
parents: 4792
diff changeset
81
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
82 void StorageCache::Invalidate(const std::string& uuid,
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
83 FileContentType contentType)
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
84 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
85 // invalidate both full file + start range file
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
86 const std::string keyFullFile = GetCacheKeyFullFile(uuid, contentType);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
87 cache_.Invalidate(keyFullFile);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
88
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
89 const std::string keyPartialFile = GetCacheKeyStartRange(uuid, contentType);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
90 cache_.Invalidate(keyPartialFile);
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
91 }
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
92
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
93
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
94 bool StorageCache::Fetch(std::string& value,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
95 const std::string& uuid,
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
96 FileContentType contentType)
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
97 {
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
98 const std::string key = GetCacheKeyFullFile(uuid, contentType);
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
99 if (cache_.Fetch(value, key))
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
100 {
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
101 LOG(INFO) << "Read attachment \"" << uuid << "\" with content type "
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
102 << boost::lexical_cast<std::string>(contentType) << " from cache";
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
103 return true;
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
104 }
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
105 else
4792
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
106 {
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
107 return false;
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
108 }
434843934307 Added a StorageCache in the StorageAccessor
Alain Mazy <am@osimis.io>
parents:
diff changeset
109 }
4906
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
110
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
111 bool StorageCache::FetchStartRange(std::string& value,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
112 const std::string& uuid,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
113 FileContentType contentType,
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
114 uint64_t end)
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
115 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
116 // first try to get the start of file only from cache
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
117 const std::string keyPartialFile = GetCacheKeyStartRange(uuid, contentType);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
118 if (cache_.Fetch(value, keyPartialFile) && value.size() >= end)
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
119 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
120 if (value.size() > end) // the start range that has been cached is larger than the requested value
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
121 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
122 value.resize(end);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
123 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
124
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
125 LOG(INFO) << "Read start of attachment \"" << uuid << "\" with content type "
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
126 << boost::lexical_cast<std::string>(contentType) << " from cache";
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
127 return true;
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
128 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
129 else
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
130 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
131 // try to get the full file from cache
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
132 if (Fetch(value, uuid, contentType))
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
133 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
134 if (value.size() < end)
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
135 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
136 throw OrthancException(ErrorCode_CorruptedFile);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
137 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
138
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
139 value.resize(end);
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
140 return true;
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
141 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
142 else
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
143 {
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
144 return false;
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
145 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
146 }
f656fb878b50 reintroduced storage cache for StartRange files
Alain Mazy <am@osimis.io>
parents: 4900
diff changeset
147 }
4900
ea5f1c6ed07e fix cache of storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 4870
diff changeset
148 }