Mercurial > hg > orthanc
changeset 5346:566e8d32bd3a
Reduce the frequency of memory trimming from 100ms to 30s to avoid high idle CPU load
author | Alain Mazy <am@osimis.io> |
---|---|
date | Thu, 29 Jun 2023 09:43:20 +0200 |
parents | 65d55cc86a41 |
children | 4f686f6150c7 |
files | NEWS OrthancServer/Resources/ImplementationNotes/memory_consumption.txt OrthancServer/Sources/ServerContext.cpp OrthancServer/Sources/ServerContext.h |
diffstat | 4 files changed, 42 insertions(+), 10 deletions(-) [+] |
line wrap: on
line diff
--- a/NEWS Wed Jun 28 11:54:41 2023 +0200 +++ b/NEWS Thu Jun 29 09:43:20 2023 +0200 @@ -5,7 +5,10 @@ ------- * Orthanc now anonymizes according to Basic Profile of PS 3.15-2023b Table E.1-1 -* Added metrics "orthanc_storage_read_bytes" and "orthanc_storage_written_bytes" +* Added metrics: + - "orthanc_storage_read_bytes" + - "orthanc_storage_written_bytes" + - "orthanc_memory_trimming_duration_ms" REST API -------- @@ -34,6 +37,8 @@ * Metrics can be stored either as floating-point numbers, or as integers * Upgraded dependencies for static builds: - boost 1.82.0 +* Reduce the frequency of memory trimming from 100ms to 30s to avoid high idle + CPU load (https://discourse.orthanc-server.org/t/onchange-callbacks-and-cpu-loads/3534). Version 1.12.0 (2023-04-14)
--- a/OrthancServer/Resources/ImplementationNotes/memory_consumption.txt Wed Jun 28 11:54:41 2023 +0200 +++ b/OrthancServer/Resources/ImplementationNotes/memory_consumption.txt Thu Jun 29 09:43:20 2023 +0200 @@ -1,6 +1,7 @@ In Orthanc 1.11.3, we have introduced a Housekeeper thread that tries to give back unused memory back to the system. This is implemented -by calling malloc_trim every 100ms. +by calling malloc_trim every 30s (note: on 1.11.3 and 1.12.0, the interval +was 100ms which caused high idle CPU load). Here is how we validated the effect of this new feature: @@ -133,3 +134,16 @@ specifies the minimum size that is released. So, even without malloc_trim, Orthanc is able to give back memory to the system. - free() never gives back block allocated by mmap() to the system, only malloc_trim() does ! + +UPDATE on June 2023: +------------------- + +Given this discussion: https://discourse.orthanc-server.org/t/onchange-callbacks-and-cpu-loads/3534, +changed the interval from 100ms to 30s. +We also added a metrics to monitor the duration: orthanc_memory_trimming_duration_ms + +Good reference article: +https://www.algolia.com/blog/engineering/when-allocators-are-hoarding-your-precious-memory/ + + +
--- a/OrthancServer/Sources/ServerContext.cpp Wed Jun 28 11:54:41 2023 +0200 +++ b/OrthancServer/Sources/ServerContext.cpp Thu Jun 29 09:43:20 2023 +0200 @@ -110,16 +110,28 @@ #if HAVE_MALLOC_TRIM == 1 void ServerContext::MemoryTrimmingThread(ServerContext* that, - unsigned int sleepDelay) + unsigned int intervalInSeconds) { + boost::posix_time::ptime lastExecution = boost::posix_time::second_clock::universal_time(); + // This thread is started only if malloc_trim is defined while (!that->done_) { - boost::this_thread::sleep(boost::posix_time::milliseconds(sleepDelay)); - - // If possible, gives memory back to the system - // (see OrthancServer/Resources/ImplementationNotes/memory_consumption.txt) - malloc_trim(128*1024); + boost::posix_time::ptime now = boost::posix_time::second_clock::universal_time(); + boost::posix_time::time_duration elapsed = now - lastExecution; + + if (elapsed.total_seconds() > intervalInSeconds) + { + // If possible, gives memory back to the system + // (see OrthancServer/Resources/ImplementationNotes/memory_consumption.txt) + { + MetricsRegistry::Timer timer(that->GetMetricsRegistry(), "orthanc_memory_trimming_duration_ms"); + malloc_trim(128*1024); + } + lastExecution = boost::posix_time::second_clock::universal_time(); + } + + boost::this_thread::sleep(boost::posix_time::milliseconds(100)); } } #endif @@ -439,7 +451,8 @@ changeThread_ = boost::thread(ChangeThread, this, (unitTesting ? 20 : 100)); #if HAVE_MALLOC_TRIM == 1 - memoryTrimmingThread_ = boost::thread(MemoryTrimmingThread, this, 100); + LOG(INFO) << "Starting memory trimming thread at 30 seconds interval"; + memoryTrimmingThread_ = boost::thread(MemoryTrimmingThread, this, 30); #else LOG(INFO) << "Your platform does not support malloc_trim(), not starting the memory trimming thread"; #endif
--- a/OrthancServer/Sources/ServerContext.h Wed Jun 28 11:54:41 2023 +0200 +++ b/OrthancServer/Sources/ServerContext.h Thu Jun 29 09:43:20 2023 +0200 @@ -189,7 +189,7 @@ #if HAVE_MALLOC_TRIM == 1 static void MemoryTrimmingThread(ServerContext* that, - unsigned int sleepDelay); + unsigned int intervalInSeconds); #endif void SaveJobsEngine();