Mercurial > hg > orthanc

annotate OrthancServer/Sources/ServerJobs/ThreadedSetOfInstancesJob.cpp @ 5136:e71b22a43c0b

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
author Alain Mazy <am@osimis.io>
date Tue, 17 Jan 2023 17:54:38 +0100
parents 252385892197
children 15109c3f0f7d
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
1 /**
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
2 * Orthanc - A Lightweight, RESTful DICOM Store
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
3 * Copyright (C) 2012-2016 Sebastien Jodogne, Medical Physics
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
4 * Department, University Hospital of Liege, Belgium
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
5 * Copyright (C) 2017-2022 Osimis S.A., Belgium
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
6 * Copyright (C) 2021-2022 Sebastien Jodogne, ICTEAM UCLouvain, Belgium
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
7 *
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
8 * This program is free software: you can redistribute it and/or
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
9 * modify it under the terms of the GNU Lesser General Public License
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
10 * as published by the Free Software Foundation, either version 3 of
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
11 * the License, or (at your option) any later version.
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
12 *
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
13 * This program is distributed in the hope that it will be useful, but
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
14 * WITHOUT ANY WARRANTY; without even the implied warranty of
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
15 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
16 * Lesser General Public License for more details.
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
17 *
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
18 * You should have received a copy of the GNU Lesser General Public
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
19 * License along with this program. If not, see
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
20 * <http://www.gnu.org/licenses/>.
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
21 **/
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
22
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
23
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
24 #include "ThreadedSetOfInstancesJob.h"
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
25
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
26 #include "../../../OrthancFramework/Sources/Logging.h"
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
27 #include "../../../OrthancFramework/Sources/OrthancException.h"
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
28 #include "../../../OrthancFramework/Sources/SerializationToolbox.h"
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
29 #include "../ServerContext.h"
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
30
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
31 #include <boost/lexical_cast.hpp>
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
32 #include <cassert>
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
33
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
34 namespace Orthanc
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
35 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
36 static const char* EXIT_WORKER_MESSAGE = "exit";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
37
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
38 ThreadedSetOfInstancesJob::ThreadedSetOfInstancesJob(ServerContext& context,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
39 bool hasPostProcessing,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
40 bool keepSource,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
41 size_t workersCount) :
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
42 hasPostProcessing_(hasPostProcessing),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
43 started_(false),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
44 stopRequested_(false),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
45 permissive_(false),
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
46 currentStep_(ThreadedJobStep_ProcessingInstances),
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
47 workersCount_(workersCount),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
48 context_(context),
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
49 keepSource_(keepSource),
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
50 errorCode_(ErrorCode_Success)
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
51 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
52 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
53
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
54
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
55 ThreadedSetOfInstancesJob::~ThreadedSetOfInstancesJob()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
56 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
57 // no need to lock mutex here since we access variables used only by the "master" thread
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
58
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
59 StopWorkers();
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
60 WaitWorkersComplete();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
61 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
62
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
63
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
64 void ThreadedSetOfInstancesJob::InitWorkers(size_t workersCount)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
65 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
66 // no need to lock mutex here since we access variables used only by the "master" thread
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
67
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
68 for (size_t i = 0; i < workersCount; i++)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
69 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
70 instancesWorkers_.push_back(boost::shared_ptr<boost::thread>(new boost::thread(InstanceWorkerThread, this)));
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
71 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
72 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
73
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
74
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
75 void ThreadedSetOfInstancesJob::WaitWorkersComplete()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
76 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
77 // no need to lock mutex here since we access variables used only by the "master" thread
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
78
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
79 // send a dummy "exit" message to all workers such that they stop waiting for messages on the queue
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
80 for (size_t i = 0; i < instancesWorkers_.size(); i++)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
81 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
82 instancesToProcessQueue_.Enqueue(new SingleValueObject<std::string>(EXIT_WORKER_MESSAGE));
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
83 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
84
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
85 for (size_t i = 0; i < instancesWorkers_.size(); i++)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
86 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
87 if (instancesWorkers_[i]->joinable())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
88 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
89 instancesWorkers_[i]->join();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
90 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
91 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
92
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
93 instancesWorkers_.clear();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
94 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
95
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
96
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
97 void ThreadedSetOfInstancesJob::StopWorkers()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
98 {
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
99 boost::recursive_mutex::scoped_lock lock(mutex_);
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
100
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
101 stopRequested_ = true;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
102 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
103
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
104
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
105 void ThreadedSetOfInstancesJob::Stop(JobStopReason reason)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
106 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
107 // no need to lock mutex here since we access variables used or set only by the "master" thread
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
108
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
109 if (reason == JobStopReason_Canceled ||
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
110 reason == JobStopReason_Failure ||
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
111 reason == JobStopReason_Retry)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
112 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
113 // deallocate resources
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
114 StopWorkers();
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
115 WaitWorkersComplete();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
116 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
117 else if (reason == JobStopReason_Paused)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
118 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
119 // keep resources allocated.
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
120 // note that, right now, since all instances are queued from the start, this kind of jobs is not paused while in ProcessingInstances state
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
121 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
122 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
123
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
124
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
125 JobStepResult ThreadedSetOfInstancesJob::Step(const std::string& jobId)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
126 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
127 // no need to lock mutex here since we access variables used or set only by the "master" thread
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
128
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
129 if (!started_)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
130 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
131 throw OrthancException(ErrorCode_InternalError);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
132 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
133
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
134 if (GetInstancesCount() == 0)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
135 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
136 // No instances to handle: We're done
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
137 return JobStepResult::Success();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
138 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
139
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
140 try
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
141 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
142 if (currentStep_ == ThreadedJobStep_ProcessingInstances)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
143 {
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
144 // create the workers and enqueue all instances
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
145 for (std::set<std::string>::const_iterator it = instancesToProcess_.begin(); it != instancesToProcess_.end(); ++it)
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
146 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
147 instancesToProcessQueue_.Enqueue(new SingleValueObject<std::string>(*it));
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
148 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
149
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
150 InitWorkers(workersCount_);
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
151 // wait until all instances are processed by the workers
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
152 WaitWorkersComplete();
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
153
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
154 // check job has really completed !!! it might have been interrupted because of an error
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
155 if ((processedInstances_.size() != instancesToProcess_.size())
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
156 || (!IsPermissive() && failedInstances_.size() > 0))
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
157 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
158 return JobStepResult::Failure(GetErrorCode(), NULL);
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
159 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
160
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
161 currentStep_ = ThreadedJobStep_PostProcessingInstances;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
162 return JobStepResult::Continue();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
163 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
164 else if (currentStep_ == ThreadedJobStep_PostProcessingInstances)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
165 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
166 if (HasPostProcessingStep())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
167 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
168 PostProcessInstances();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
169 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
170
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
171 currentStep_ = ThreadedJobStep_Cleanup;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
172 return JobStepResult::Continue();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
173 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
174 else if (currentStep_ == ThreadedJobStep_Cleanup)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
175 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
176 // clean after the post processing step
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
177 if (HasCleanupStep())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
178 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
179 for (std::set<std::string>::const_iterator it = instancesToProcess_.begin(); it != instancesToProcess_.end(); ++it)
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
180 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
181 Json::Value tmp;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
182 context_.DeleteResource(tmp, *it, ResourceType_Instance);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
183 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
184 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
185
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
186 currentStep_ = ThreadedJobStep_Done;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
187 return JobStepResult::Success();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
188 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
189 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
190 catch (OrthancException& e)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
191 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
192 if (permissive_)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
193 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
194 LOG(WARNING) << "Ignoring an error in a permissive job: " << e.What();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
195 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
196 else
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
197 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
198 return JobStepResult::Failure(e);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
199 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
200 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
201
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
202 return JobStepResult::Continue();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
203 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
204
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
205
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
206 bool ThreadedSetOfInstancesJob::HasPostProcessingStep() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
207 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
208 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
209
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
210 return hasPostProcessing_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
211 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
212
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
213 void ThreadedSetOfInstancesJob::PostProcessInstances()
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
214 {
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
215 if (HasPostProcessingStep())
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
216 {
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
217 throw OrthancException(ErrorCode_InternalError, "Job with post-processing should override PostProcessInstances");
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
218 }
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
219 }
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
220
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
221
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
222 bool ThreadedSetOfInstancesJob::HasCleanupStep() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
223 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
224 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
225
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
226 return !keepSource_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
227 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
228
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
229
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
230 void ThreadedSetOfInstancesJob::InstanceWorkerThread(ThreadedSetOfInstancesJob* that)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
231 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
232 while (true)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
233 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
234 std::unique_ptr<SingleValueObject<std::string> > instanceId(dynamic_cast<SingleValueObject<std::string>*>(that->instancesToProcessQueue_.Dequeue(0)));
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
235 if (that->stopRequested_ // no lock(mutex) to access this variable, this is safe since it's just reading a boolean
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
236 || instanceId->GetValue() == EXIT_WORKER_MESSAGE)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
237 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
238 return;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
239 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
240
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
241 bool processed = false;
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
242
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
243 try
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
244 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
245 processed = that->HandleInstance(instanceId->GetValue());
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
246 }
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
247 catch (const Orthanc::OrthancException& e)
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
248 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
249 if (that->IsPermissive())
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
250 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
251 LOG(WARNING) << "Ignoring an error in a permissive job: " << e.What();
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
252 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
253 else
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
254 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
255 LOG(ERROR) << "Error in a non-permissive job: " << e.What();
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
256 that->SetErrorCode(e.GetErrorCode());
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
257 that->StopWorkers();
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
258 return;
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
259 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
260 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
261 catch (...)
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
262 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
263 LOG(ERROR) << "Native exception while executing a job";
5135
Alain Mazy <am@osimis.io>
parents: 5134
diff changeset
264 that->SetErrorCode(ErrorCode_InternalError);
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
265 that->StopWorkers();
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
266 return;
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
267 }
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
268
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
269 {
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
270 boost::recursive_mutex::scoped_lock lock(that->mutex_);
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
271
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
272 that->processedInstances_.insert(instanceId->GetValue());
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
273
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
274 if (!processed)
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
275 {
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
276 that->failedInstances_.insert(instanceId->GetValue());
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
277 }
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
278 }
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
279
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
280 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
281 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
282
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
283
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
284 bool ThreadedSetOfInstancesJob::GetOutput(std::string &output,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
285 MimeType &mime,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
286 std::string& filename,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
287 const std::string &key)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
288 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
289 return false;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
290 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
291
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
292
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
293 size_t ThreadedSetOfInstancesJob::GetInstancesCount() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
294 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
295 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
296
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
297 return instancesToProcess_.size();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
298 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
299
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
300
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
301 void ThreadedSetOfInstancesJob::GetFailedInstances(std::set<std::string>& target) const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
302 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
303 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
304
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
305 target = failedInstances_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
306 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
307
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
308
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
309 void ThreadedSetOfInstancesJob::GetInstances(std::set<std::string>& target) const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
310 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
311 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
312
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
313 target = instancesToProcess_;
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
314 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
315
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
316
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
317 bool ThreadedSetOfInstancesJob::IsFailedInstance(const std::string &instance) const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
318 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
319 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
320
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
321 return failedInstances_.find(instance) != failedInstances_.end();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
322 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
323
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
324
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
325 void ThreadedSetOfInstancesJob::Start()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
326 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
327 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
328
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
329 started_ = true;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
330 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
331
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
332
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
333 // Reset is called when resubmitting a failed job
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
334 void ThreadedSetOfInstancesJob::Reset()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
335 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
336 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
337
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
338 if (started_)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
339 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
340 // TODO: cleanup the instances that have been generated during the previous run
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
341 currentStep_ = ThreadedJobStep_ProcessingInstances;
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
342 stopRequested_ = false;
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
343 processedInstances_.clear();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
344 failedInstances_.clear();
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
345 instancesToProcessQueue_.Clear();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
346 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
347 else
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
348 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
349 throw OrthancException(ErrorCode_BadSequenceOfCalls);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
350 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
351 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
352
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
353
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
354 static const char* KEY_FAILED_INSTANCES = "FailedInstances";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
355 static const char* KEY_PARENT_RESOURCES = "ParentResources";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
356 static const char* KEY_DESCRIPTION = "Description";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
357 static const char* KEY_PERMISSIVE = "Permissive";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
358 static const char* KEY_CURRENT_STEP = "CurrentStep";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
359 static const char* KEY_TYPE = "Type";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
360 static const char* KEY_INSTANCES = "Instances";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
361 static const char* KEY_INSTANCES_COUNT = "InstancesCount";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
362 static const char* KEY_FAILED_INSTANCES_COUNT = "FailedInstancesCount";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
363 static const char* KEY_KEEP_SOURCE = "KeepSource";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
364 static const char* KEY_WORKERS_COUNT = "WorkersCount";
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
365
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
366
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
367 void ThreadedSetOfInstancesJob::GetPublicContent(Json::Value& target)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
368 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
369 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
370
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
371 target[KEY_DESCRIPTION] = GetDescription();
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
372 target[KEY_INSTANCES_COUNT] = static_cast<uint32_t>(GetInstancesCount());
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
373 target[KEY_FAILED_INSTANCES_COUNT] = static_cast<uint32_t>(failedInstances_.size());
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
374
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
375 if (!parentResources_.empty())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
376 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
377 SerializationToolbox::WriteSetOfStrings(target, parentResources_, KEY_PARENT_RESOURCES);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
378 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
379 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
380
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
381
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
382 bool ThreadedSetOfInstancesJob::Serialize(Json::Value& target)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
383 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
384 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
385
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
386 target = Json::objectValue;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
387
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
388 std::string type;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
389 GetJobType(type);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
390 target[KEY_TYPE] = type;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
391
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
392 target[KEY_PERMISSIVE] = permissive_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
393 target[KEY_CURRENT_STEP] = static_cast<unsigned int>(currentStep_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
394 target[KEY_DESCRIPTION] = description_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
395 target[KEY_KEEP_SOURCE] = keepSource_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
396 target[KEY_WORKERS_COUNT] = static_cast<unsigned int>(workersCount_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
397
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
398 SerializationToolbox::WriteSetOfStrings(target, instancesToProcess_, KEY_INSTANCES);
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
399 SerializationToolbox::WriteSetOfStrings(target, failedInstances_, KEY_FAILED_INSTANCES);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
400 SerializationToolbox::WriteSetOfStrings(target, parentResources_, KEY_PARENT_RESOURCES);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
401
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
402 return true;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
403 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
404
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
405
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
406 ThreadedSetOfInstancesJob::ThreadedSetOfInstancesJob(ServerContext& context,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
407 const Json::Value& source,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
408 bool hasPostProcessing,
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
409 bool defaultKeepSource) :
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
410 hasPostProcessing_(hasPostProcessing),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
411 started_(false),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
412 stopRequested_(false),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
413 permissive_(false),
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
414 currentStep_(ThreadedJobStep_ProcessingInstances),
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
415 workersCount_(1),
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
416 context_(context),
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
417 keepSource_(defaultKeepSource),
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
418 errorCode_(ErrorCode_Success)
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
419 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
420 SerializationToolbox::ReadSetOfStrings(failedInstances_, source, KEY_FAILED_INSTANCES);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
421
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
422 if (source.isMember(KEY_PARENT_RESOURCES))
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
423 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
424 // Backward compatibility with Orthanc <= 1.5.6
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
425 SerializationToolbox::ReadSetOfStrings(parentResources_, source, KEY_PARENT_RESOURCES);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
426 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
427
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
428 if (source.isMember(KEY_KEEP_SOURCE))
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
429 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
430 keepSource_ = SerializationToolbox::ReadBoolean(source, KEY_KEEP_SOURCE);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
431 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
432
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
433 if (source.isMember(KEY_WORKERS_COUNT))
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
434 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
435 workersCount_ = SerializationToolbox::ReadUnsignedInteger(source, KEY_WORKERS_COUNT);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
436 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
437
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
438 if (source.isMember(KEY_INSTANCES))
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
439 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
440 SerializationToolbox::ReadSetOfStrings(instancesToProcess_, source, KEY_INSTANCES);
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
441 }
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
442
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
443 if (source.isMember(KEY_CURRENT_STEP))
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
444 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
445 currentStep_ = static_cast<ThreadedJobStep>(SerializationToolbox::ReadUnsignedInteger(source, KEY_CURRENT_STEP));
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
446 }
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
447 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
448
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
449
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
450 void ThreadedSetOfInstancesJob::SetKeepSource(bool keep)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
451 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
452 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
453
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
454 if (IsStarted())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
455 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
456 throw OrthancException(ErrorCode_BadSequenceOfCalls);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
457 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
458
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
459 keepSource_ = keep;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
460 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
461
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
462
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
463 float ThreadedSetOfInstancesJob::GetProgress()
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
464 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
465 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
466
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
467 if (GetInstancesCount() == 0)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
468 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
469 return 1;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
470 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
471 else
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
472 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
473 size_t totalProgress = GetInstancesCount();
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
474 size_t currentProgress = processedInstances_.size();
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
475
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
476 if (HasPostProcessingStep())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
477 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
478 ++totalProgress;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
479 if (currentStep_ > ThreadedJobStep_PostProcessingInstances)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
480 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
481 ++currentProgress;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
482 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
483 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
484
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
485 if (HasCleanupStep())
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
486 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
487 ++totalProgress;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
488 if (currentStep_ > ThreadedJobStep_Cleanup)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
489 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
490 ++currentProgress;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
491 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
492 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
493
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
494 return (static_cast<float>(currentProgress) /
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
495 static_cast<float>(totalProgress));
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
496 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
497 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
498
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
499
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
500 ThreadedSetOfInstancesJob::ThreadedJobStep ThreadedSetOfInstancesJob::GetCurrentStep() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
501 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
502 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
503
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
504 return currentStep_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
505 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
506
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
507
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
508 void ThreadedSetOfInstancesJob::SetDescription(const std::string &description)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
509 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
510 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
511
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
512 description_ = description;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
513 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
514
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
515
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
516 const std::string& ThreadedSetOfInstancesJob::GetDescription() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
517 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
518 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
519
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
520 return description_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
521 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
522
5134
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
523 void ThreadedSetOfInstancesJob::SetErrorCode(ErrorCode errorCode)
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
524 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
525 boost::recursive_mutex::scoped_lock lock(mutex_);
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
526
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
527 errorCode_ = errorCode;
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
528 }
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
529
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
530 ErrorCode ThreadedSetOfInstancesJob::GetErrorCode() const
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
531 {
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
532 boost::recursive_mutex::scoped_lock lock(mutex_);
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
533
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
534 return errorCode_;
6aa41d86b948 fix ModificationJob state machine
Alain Mazy <am@osimis.io>
parents: 5130
diff changeset
535 }
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
536
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
537 bool ThreadedSetOfInstancesJob::IsPermissive() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
538 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
539 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
540
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
541 return permissive_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
542 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
543
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
544
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
545 void ThreadedSetOfInstancesJob::SetPermissive(bool permissive)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
546 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
547 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
548
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
549 if (started_)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
550 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
551 throw OrthancException(ErrorCode_BadSequenceOfCalls);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
552 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
553 else
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
554 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
555 permissive_ = permissive;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
556 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
557 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
558
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
559
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
560 bool ThreadedSetOfInstancesJob::IsStarted() const
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
561 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
562 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
563
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
564 return started_;
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
565 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
566
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
567
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
568 void ThreadedSetOfInstancesJob::AddInstances(const std::list<std::string>& instances)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
569 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
570 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
571
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
572 for (std::list<std::string>::const_iterator
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
573 it = instances.begin(); it != instances.end(); ++it)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
574 {
5136
e71b22a43c0b Threaded modifications continued: call ReconstructInstance at the end of the modification to update the DB model
Alain Mazy <am@osimis.io>
parents: 5135
diff changeset
575 instancesToProcess_.insert(*it);
5130
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
576 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
577 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
578
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
579
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
580 void ThreadedSetOfInstancesJob::AddParentResource(const std::string &resource)
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
581 {
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
582 boost::recursive_mutex::scoped_lock lock(mutex_);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
583
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
584 parentResources_.insert(resource);
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
585 }
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
586
f2dcdbe05884 ResourceModification jobs can now use multiple threads
Alain Mazy <am@osimis.io>
parents:
diff changeset
587 }