# HG changeset patch # User Sebastien Jodogne # Date 1531814799 -7200 # Node ID fb8d4cd2f6181088c4e2f287b87e61f8347a145c # Parent 442102e149337d920efb05fb267abbb40b8b96c9 fix applications diff -r 442102e14933 -r fb8d4cd2f618 Applications/ApplicationToolbox.cpp --- a/Applications/ApplicationToolbox.cpp Tue Jul 17 09:55:24 2018 +0200 +++ b/Applications/ApplicationToolbox.cpp Tue Jul 17 10:06:39 2018 +0200 @@ -22,13 +22,14 @@ #include "ApplicationToolbox.h" #include "../Framework/Inputs/OpenSlideLibrary.h" +#include "../Framework/MultiThreading/BagOfTasksProcessor.h" #include #include #include -#include #include #include +#include #include #include @@ -63,7 +64,7 @@ void GlobalInitialize() { Orthanc::Logging::Initialize(); - Orthanc::HttpClient::InitializeOpenSsl(); + Orthanc::Toolbox::InitializeOpenSsl(); Orthanc::HttpClient::GlobalInitialize(); Orthanc::FromDcmtkBridge::InitializeDictionary(false /* don't load private dictionary */); assert(DisplayPerformanceWarning()); @@ -74,7 +75,7 @@ { OrthancWSI::OpenSlideLibrary::Finalize(); Orthanc::HttpClient::GlobalFinalize(); - Orthanc::HttpClient::FinalizeOpenSsl(); + Orthanc::Toolbox::FinalizeOpenSsl(); } diff -r 442102e14933 -r fb8d4cd2f618 Applications/ApplicationToolbox.h --- a/Applications/ApplicationToolbox.h Tue Jul 17 09:55:24 2018 +0200 +++ b/Applications/ApplicationToolbox.h Tue Jul 17 10:06:39 2018 +0200 @@ -21,7 +21,8 @@ #pragma once -#include +#include "../Framework/MultiThreading/BagOfTasks.h" + #include #include diff -r 442102e14933 -r fb8d4cd2f618 Applications/CMakeLists.txt --- a/Applications/CMakeLists.txt Tue Jul 17 09:55:24 2018 +0200 +++ b/Applications/CMakeLists.txt Tue Jul 17 10:06:39 2018 +0200 @@ -100,14 +100,15 @@ ${ORTHANC_WSI_DIR}/Framework/Inputs/TiledPyramidStatistics.cpp ${ORTHANC_WSI_DIR}/Framework/Jpeg2000Reader.cpp ${ORTHANC_WSI_DIR}/Framework/Jpeg2000Writer.cpp - ${ORTHANC_WSI_DIR}/Framework/Targets/FolderTarget.cpp - ${ORTHANC_WSI_DIR}/Framework/Targets/OrthancTarget.cpp + ${ORTHANC_WSI_DIR}/Framework/MultiThreading/BagOfTasksProcessor.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/DicomPyramidWriter.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/HierarchicalTiffWriter.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/InMemoryTiledImage.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/MultiframeDicomWriter.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/PyramidWriterBase.cpp ${ORTHANC_WSI_DIR}/Framework/Outputs/TruncatedPyramidWriter.cpp + ${ORTHANC_WSI_DIR}/Framework/Targets/FolderTarget.cpp + ${ORTHANC_WSI_DIR}/Framework/Targets/OrthancTarget.cpp ) EmbedResources( diff -r 442102e14933 -r fb8d4cd2f618 Applications/Dicomizer.cpp --- a/Applications/Dicomizer.cpp Tue Jul 17 09:55:24 2018 +0200 +++ b/Applications/Dicomizer.cpp Tue Jul 17 10:06:39 2018 +0200 @@ -29,12 +29,12 @@ #include "../Framework/Inputs/TiledJpegImage.h" #include "../Framework/Inputs/TiledPngImage.h" #include "../Framework/Inputs/TiledPyramidStatistics.h" +#include "../Framework/MultiThreading/BagOfTasksProcessor.h" #include "../Framework/Outputs/DicomPyramidWriter.h" #include "../Framework/Outputs/TruncatedPyramidWriter.h" #include #include -#include #include #include diff -r 442102e14933 -r fb8d4cd2f618 Framework/Algorithms/ReconstructPyramidCommand.h --- a/Framework/Algorithms/ReconstructPyramidCommand.h Tue Jul 17 09:55:24 2018 +0200 +++ b/Framework/Algorithms/ReconstructPyramidCommand.h Tue Jul 17 10:06:39 2018 +0200 @@ -23,7 +23,7 @@ #include "PyramidReader.h" #include "../Outputs/IPyramidWriter.h" -#include +#include "../MultiThreading/BagOfTasks.h" namespace OrthancWSI diff -r 442102e14933 -r fb8d4cd2f618 Framework/Algorithms/TranscodeTileCommand.h --- a/Framework/Algorithms/TranscodeTileCommand.h Tue Jul 17 09:55:24 2018 +0200 +++ b/Framework/Algorithms/TranscodeTileCommand.h Tue Jul 17 10:06:39 2018 +0200 @@ -23,8 +23,7 @@ #include "PyramidReader.h" #include "../Outputs/IPyramidWriter.h" - -#include +#include "../MultiThreading/BagOfTasks.h" namespace OrthancWSI { diff -r 442102e14933 -r fb8d4cd2f618 Framework/MultiThreading/BagOfTasks.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Framework/MultiThreading/BagOfTasks.h Tue Jul 17 10:06:39 2018 +0200 @@ -0,0 +1,72 @@ +/** + * Orthanc - A Lightweight, RESTful DICOM Store + * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics + * Department, University Hospital of Liege, Belgium + * Copyright (C) 2017-2018 Osimis S.A., Belgium + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Affero General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + **/ + + +#pragma once + +#include + +#include +#include + +namespace Orthanc +{ + class BagOfTasks : public boost::noncopyable + { + private: + typedef std::list Tasks; + + Tasks tasks_; + + public: + ~BagOfTasks() + { + for (Tasks::iterator it = tasks_.begin(); it != tasks_.end(); ++it) + { + delete *it; + } + } + + ICommand* Pop() + { + ICommand* task = tasks_.front(); + tasks_.pop_front(); + return task; + } + + void Push(ICommand* task) // Takes ownership + { + if (task != NULL) + { + tasks_.push_back(task); + } + } + + size_t GetSize() const + { + return tasks_.size(); + } + + bool IsEmpty() const + { + return tasks_.empty(); + } + }; +} diff -r 442102e14933 -r fb8d4cd2f618 Framework/MultiThreading/BagOfTasksProcessor.cpp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Framework/MultiThreading/BagOfTasksProcessor.cpp Tue Jul 17 10:06:39 2018 +0200 @@ -0,0 +1,264 @@ +/** + * Orthanc - A Lightweight, RESTful DICOM Store + * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics + * Department, University Hospital of Liege, Belgium + * Copyright (C) 2017-2018 Osimis S.A., Belgium + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Affero General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + **/ + + +#include "BagOfTasksProcessor.h" + +#include +#include + +#include + +namespace Orthanc +{ + class BagOfTasksProcessor::Task : public IDynamicObject + { + private: + uint64_t bag_; + std::auto_ptr command_; + + public: + Task(uint64_t bag, + ICommand* command) : + bag_(bag), + command_(command) + { + } + + bool Execute() + { + try + { + return command_->Execute(); + } + catch (OrthancException& e) + { + LOG(ERROR) << "Exception while processing a bag of tasks: " << e.What(); + return false; + } + catch (std::runtime_error& e) + { + LOG(ERROR) << "Runtime exception while processing a bag of tasks: " << e.what(); + return false; + } + catch (...) + { + LOG(ERROR) << "Native exception while processing a bag of tasks"; + return false; + } + } + + uint64_t GetBag() + { + return bag_; + } + }; + + + void BagOfTasksProcessor::SignalProgress(Task& task, + Bag& bag) + { + assert(bag.done_ < bag.size_); + + bag.done_ += 1; + + if (bag.done_ == bag.size_) + { + exitStatus_[task.GetBag()] = (bag.status_ == BagStatus_Running); + bagFinished_.notify_all(); + } + } + + void BagOfTasksProcessor::Worker(BagOfTasksProcessor* that) + { + while (that->continue_) + { + std::auto_ptr obj(that->queue_.Dequeue(100)); + if (obj.get() != NULL) + { + Task& task = *dynamic_cast(obj.get()); + + { + boost::mutex::scoped_lock lock(that->mutex_); + + Bags::iterator bag = that->bags_.find(task.GetBag()); + assert(bag != that->bags_.end()); + assert(bag->second.done_ < bag->second.size_); + + if (bag->second.status_ != BagStatus_Running) + { + // Do not execute this task, as its parent bag of tasks + // has failed or is tagged as canceled + that->SignalProgress(task, bag->second); + continue; + } + } + + bool success = task.Execute(); + + { + boost::mutex::scoped_lock lock(that->mutex_); + + Bags::iterator bag = that->bags_.find(task.GetBag()); + assert(bag != that->bags_.end()); + + if (!success) + { + bag->second.status_ = BagStatus_Failed; + } + + that->SignalProgress(task, bag->second); + } + } + } + } + + + void BagOfTasksProcessor::Cancel(int64_t bag) + { + boost::mutex::scoped_lock lock(mutex_); + + Bags::iterator it = bags_.find(bag); + if (it != bags_.end()) + { + it->second.status_ = BagStatus_Canceled; + } + } + + + bool BagOfTasksProcessor::Join(int64_t bag) + { + boost::mutex::scoped_lock lock(mutex_); + + while (continue_) + { + ExitStatus::iterator it = exitStatus_.find(bag); + if (it == exitStatus_.end()) // The bag is still running + { + bagFinished_.wait(lock); + } + else + { + bool status = it->second; + exitStatus_.erase(it); + return status; + } + } + + return false; // The processor is stopping + } + + + float BagOfTasksProcessor::GetProgress(int64_t bag) + { + boost::mutex::scoped_lock lock(mutex_); + + Bags::const_iterator it = bags_.find(bag); + if (it == bags_.end()) + { + // The bag of tasks has finished + return 1.0f; + } + else + { + return (static_cast(it->second.done_) / + static_cast(it->second.size_)); + } + } + + + bool BagOfTasksProcessor::Handle::Join() + { + if (hasJoined_) + { + return status_; + } + else + { + status_ = that_.Join(bag_); + hasJoined_ = true; + return status_; + } + } + + + BagOfTasksProcessor::BagOfTasksProcessor(size_t countThreads) : + countBags_(0), + continue_(true) + { + if (countThreads == 0) + { + throw OrthancException(ErrorCode_ParameterOutOfRange); + } + + threads_.resize(countThreads); + + for (size_t i = 0; i < threads_.size(); i++) + { + threads_[i] = new boost::thread(Worker, this); + } + } + + + BagOfTasksProcessor::~BagOfTasksProcessor() + { + continue_ = false; + + bagFinished_.notify_all(); // Wakes up all the pending "Join()" + + for (size_t i = 0; i < threads_.size(); i++) + { + if (threads_[i]) + { + if (threads_[i]->joinable()) + { + threads_[i]->join(); + } + + delete threads_[i]; + threads_[i] = NULL; + } + } + } + + + BagOfTasksProcessor::Handle* BagOfTasksProcessor::Submit(BagOfTasks& tasks) + { + if (tasks.GetSize() == 0) + { + return new Handle(*this, 0, true); + } + + boost::mutex::scoped_lock lock(mutex_); + + uint64_t id = countBags_; + countBags_ += 1; + + Bag bag(tasks.GetSize()); + bags_[id] = bag; + + while (!tasks.IsEmpty()) + { + queue_.Enqueue(new Task(id, tasks.Pop())); + } + + return new Handle(*this, id, false); + } +} diff -r 442102e14933 -r fb8d4cd2f618 Framework/MultiThreading/BagOfTasksProcessor.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Framework/MultiThreading/BagOfTasksProcessor.h Tue Jul 17 10:06:39 2018 +0200 @@ -0,0 +1,139 @@ +/** + * Orthanc - A Lightweight, RESTful DICOM Store + * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics + * Department, University Hospital of Liege, Belgium + * Copyright (C) 2017-2018 Osimis S.A., Belgium + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Affero General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + **/ + + +#pragma once + +#include "BagOfTasks.h" + +#include + +#include +#include + +namespace Orthanc +{ + class BagOfTasksProcessor : public boost::noncopyable + { + private: + enum BagStatus + { + BagStatus_Running, + BagStatus_Canceled, + BagStatus_Failed + }; + + + struct Bag + { + size_t size_; + size_t done_; + BagStatus status_; + + Bag() : + size_(0), + done_(0), + status_(BagStatus_Failed) + { + } + + explicit Bag(size_t size) : + size_(size), + done_(0), + status_(BagStatus_Running) + { + } + }; + + class Task; + + + typedef std::map Bags; + typedef std::map ExitStatus; + + SharedMessageQueue queue_; + + boost::mutex mutex_; + uint64_t countBags_; + Bags bags_; + std::vector threads_; + ExitStatus exitStatus_; + bool continue_; + + boost::condition_variable bagFinished_; + + static void Worker(BagOfTasksProcessor* that); + + void Cancel(int64_t bag); + + bool Join(int64_t bag); + + float GetProgress(int64_t bag); + + void SignalProgress(Task& task, + Bag& bag); + + public: + class Handle : public boost::noncopyable + { + friend class BagOfTasksProcessor; + + private: + BagOfTasksProcessor& that_; + uint64_t bag_; + bool hasJoined_; + bool status_; + + Handle(BagOfTasksProcessor& that, + uint64_t bag, + bool empty) : + that_(that), + bag_(bag), + hasJoined_(empty) + { + } + + public: + ~Handle() + { + Join(); + } + + void Cancel() + { + that_.Cancel(bag_); + } + + bool Join(); + + float GetProgress() + { + return that_.GetProgress(bag_); + } + }; + + + explicit BagOfTasksProcessor(size_t countThreads); + + ~BagOfTasksProcessor(); + + Handle* Submit(BagOfTasks& tasks); + }; +} diff -r 442102e14933 -r fb8d4cd2f618 Framework/MultiThreading/Semaphore.cpp --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Framework/MultiThreading/Semaphore.cpp Tue Jul 17 10:06:39 2018 +0200 @@ -0,0 +1,50 @@ +/** + * Orthanc - A Lightweight, RESTful DICOM Store + * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics + * Department, University Hospital of Liege, Belgium + * Copyright (C) 2017-2018 Osimis S.A., Belgium + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Affero General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + **/ + + +#include "Semaphore.h" + + +namespace OrthancWSI +{ + Semaphore::Semaphore(unsigned int count) : count_(count) + { + } + + void Semaphore::Release() + { + boost::mutex::scoped_lock lock(mutex_); + + count_++; + condition_.notify_one(); + } + + void Semaphore::Acquire() + { + boost::mutex::scoped_lock lock(mutex_); + + while (count_ == 0) + { + condition_.wait(lock); + } + + count_++; + } +} diff -r 442102e14933 -r fb8d4cd2f618 Framework/MultiThreading/Semaphore.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/Framework/MultiThreading/Semaphore.h Tue Jul 17 10:06:39 2018 +0200 @@ -0,0 +1,61 @@ +/** + * Orthanc - A Lightweight, RESTful DICOM Store + * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics + * Department, University Hospital of Liege, Belgium + * Copyright (C) 2017-2018 Osimis S.A., Belgium + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Affero General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Affero General Public License for more details. + * + * You should have received a copy of the GNU Affero General Public License + * along with this program. If not, see . + **/ + + +#pragma once + +#include +#include + +namespace OrthancWSI +{ + class Semaphore : public boost::noncopyable + { + private: + unsigned int count_; + boost::mutex mutex_; + boost::condition_variable condition_; + + public: + explicit Semaphore(unsigned int count); + + void Release(); + + void Acquire(); + + class Locker : public boost::noncopyable + { + private: + Semaphore& that_; + + public: + explicit Locker(Semaphore& that) : + that_(that) + { + that_.Acquire(); + } + + ~Locker() + { + that_.Release(); + } + }; + }; +} diff -r 442102e14933 -r fb8d4cd2f618 Framework/Semaphore.cpp --- a/Framework/Semaphore.cpp Tue Jul 17 09:55:24 2018 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,50 +0,0 @@ -/** - * Orthanc - A Lightweight, RESTful DICOM Store - * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics - * Department, University Hospital of Liege, Belgium - * Copyright (C) 2017-2018 Osimis S.A., Belgium - * - * This program is free software: you can redistribute it and/or - * modify it under the terms of the GNU Affero General Public License - * as published by the Free Software Foundation, either version 3 of - * the License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Affero General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - **/ - - -#include "Semaphore.h" - - -namespace OrthancWSI -{ - Semaphore::Semaphore(unsigned int count) : count_(count) - { - } - - void Semaphore::Release() - { - boost::mutex::scoped_lock lock(mutex_); - - count_++; - condition_.notify_one(); - } - - void Semaphore::Acquire() - { - boost::mutex::scoped_lock lock(mutex_); - - while (count_ == 0) - { - condition_.wait(lock); - } - - count_++; - } -} diff -r 442102e14933 -r fb8d4cd2f618 Framework/Semaphore.h --- a/Framework/Semaphore.h Tue Jul 17 09:55:24 2018 +0200 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,61 +0,0 @@ -/** - * Orthanc - A Lightweight, RESTful DICOM Store - * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics - * Department, University Hospital of Liege, Belgium - * Copyright (C) 2017-2018 Osimis S.A., Belgium - * - * This program is free software: you can redistribute it and/or - * modify it under the terms of the GNU Affero General Public License - * as published by the Free Software Foundation, either version 3 of - * the License, or (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, but - * WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU - * Affero General Public License for more details. - * - * You should have received a copy of the GNU Affero General Public License - * along with this program. If not, see . - **/ - - -#pragma once - -#include -#include - -namespace OrthancWSI -{ - class Semaphore : public boost::noncopyable - { - private: - unsigned int count_; - boost::mutex mutex_; - boost::condition_variable condition_; - - public: - explicit Semaphore(unsigned int count); - - void Release(); - - void Acquire(); - - class Locker : public boost::noncopyable - { - private: - Semaphore& that_; - - public: - explicit Locker(Semaphore& that) : - that_(that) - { - that_.Acquire(); - } - - ~Locker() - { - that_.Release(); - } - }; - }; -} diff -r 442102e14933 -r fb8d4cd2f618 ViewerPlugin/CMakeLists.txt --- a/ViewerPlugin/CMakeLists.txt Tue Jul 17 09:55:24 2018 +0200 +++ b/ViewerPlugin/CMakeLists.txt Tue Jul 17 10:06:39 2018 +0200 @@ -154,7 +154,7 @@ ${ORTHANC_WSI_DIR}/Framework/Inputs/PyramidWithRawTiles.cpp ${ORTHANC_WSI_DIR}/Framework/Jpeg2000Reader.cpp ${ORTHANC_WSI_DIR}/Framework/Jpeg2000Writer.cpp - ${ORTHANC_WSI_DIR}/Framework/Semaphore.cpp + ${ORTHANC_WSI_DIR}/Framework/MultiThreading/Semaphore.cpp ${ORTHANC_ROOT}/Plugins/Samples/Common/DicomDatasetReader.cpp ${ORTHANC_ROOT}/Plugins/Samples/Common/DicomPath.cpp diff -r 442102e14933 -r fb8d4cd2f618 ViewerPlugin/Plugin.cpp --- a/ViewerPlugin/Plugin.cpp Tue Jul 17 09:55:24 2018 +0200 +++ b/ViewerPlugin/Plugin.cpp Tue Jul 17 10:06:39 2018 +0200 @@ -23,8 +23,9 @@ #include "DicomPyramidCache.h" #include "../Framework/Jpeg2000Reader.h" -#include "../Framework/Semaphore.h" +#include "../Framework/MultiThreading/Semaphore.h" +#include #include #include #include @@ -309,6 +310,8 @@ return -1; } + Orthanc::Logging::Initialize(context); + // Limit the number of PNG transcoders to the number of available // hardware threads (e.g. number of CPUs or cores or // hyperthreading units)