Mercurial > hg > orthanc-stone

comparison OrthancStone/Resources/Documentation/stone-object-model-reference.md @ 1586:b5417e377636

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
reorganization
author Sebastien Jodogne <s.jodogne@gmail.com>
date Thu, 22 Oct 2020 16:17:10 +0200
parents OrthancStone/Docs/stone-object-model-reference.md@244ad1e4e76a
children
comparison
equal deleted inserted replaced
1585:94edbfa64c97 1586:b5417e377636
1 ## Scene2D and viewport-related object reference
2
3 ### `Scene2D`
4
5 Represents a collection of layers that display 2D data.
6
7 These layers must implement `ISceneLayer`
8
9 The layers must be created externally and set to a specific Z-order index
10 with the `SetLayer` method.
11
12 The `Scene2D` object merely acts as a layer container. It has no rendering
13 or layer creation facility on its own.
14
15 The `Scene2D` contains an `AffineTransform2D` structure that defines how
16 the various layer item coordinates are transformed before being displayed
17 on the viewport (aka canvas)
18
19 It is up to each layer type-specific renderer to choose how this transformation
20 is used. See the various kinds of layer below for more details.
21
22 Examining the `Scene2D` contents can be done either by implementing the
23 `Scene2D::IVisitor` interface and calling `Apply(IVisitor& visitor)` or by
24 iterating between `GetMinDepth()` and `GetMaxDepth()` and calling the
25 `ISceneLayer& GetLayer(int depth)` getter.
26
27 ### `ISceneLayer`
28
29 Interface that must be implemented by `Scene2D` layers. This is a closed list
30 that, as of 2020-03, contains:
31
32 ```
33 Type_InfoPanel,
34 Type_ColorTexture,
35 Type_Polyline,
36 Type_Text,
37 Type_FloatTexture,
38 Type_LookupTableTexture
39 ```
40
41 Please note that this interface mandates the implementation of a `GetRevision`
42 method returning an `uint64_t`.
43
44 The idea is that when a model gets converted to a set of `ISceneLayer`
45 instances, changes in the model that result in changes to the layers must
46 increase the revision number of these layers.
47
48 That allows the rendering process to safely assume that a given layers whose
49 revision does not change hasn't been modified (this helps with caching).
50
51 Every mutable method in `ISceneLayer` instances that possibly change the visual
52 representation of an `ISceneLayer` must increase this revision number.
53
54 ### Implementation: `FloatTextureSceneLayer`
55
56 Layer that renders an `Orthanc::ImageAccessor` object that must be convertible
57 to `Float32` image.
58
59 The constructor only uses the image accessor to perform a copy. It can safely
60 be deleted afterwards.
61
62 The input values are mapped to the output values by taking into account various
63 properties that can be modified with:
64
65 - `SetWindowing`: uses windowing presets like "bone" or "lung"
66 - `SetCustomWindowing`: with manual window center and width
67 - `SetInverted`: toggles black <-> white inversion after windowing
68 - `SetApplyLog`: uses a non-linear response curve described in
69 https://theailearner.com/2019/01/01/log-transformation/ that expands contrast
70 in dark areas while compressing contrast in bright ones. This is **not**
71 implemented in the OpenGL renderer!
72
73 The corresponding renderers are `OpenGLFloatTextureRenderer` and
74 `CairoFloatTextureRenderer`. The scene transformation is applied during
75 rendering.
76
77 ### Implementation: `ColorTextureSceneLayer`
78
79 Layer that renders an `Orthanc::ImageAccessor` object an RGBA image (alpha must
80 be premultiplied).
81
82 The constructor only uses the image accessor to perform a copy. It can safely
83 be deleted afterwards.
84
85 The corresponding renderers are `OpenGLColorTextureRenderer` and
86 `CairoColorTextureRenderer`. The scene transformation is applied during
87 rendering.
88
89 ### Implementation: `LookupTableTextureSceneLayer`
90
91 Layer that renders an `Orthanc::ImageAccessor` object that must be convertible
92 to `Float32` image.
93
94 The constructor only uses the image accessor to perform a copy. It can safely
95 be deleted afterwards.
96
97 The final on-screen color of each pixel is determined by passing the input
98 `Float32` value through a 256-entry look-up table (LUT) that can be passed as
99 an array of either 256 x 3 bytes (for opaque RGB colors) or 256 x 4 bytes (for
100 RGBA pixels). The LUT is not specified at construction time, but with
101 calls to `SetLookupTable` or `SetLookupTableGrayscale` (that fills the LUT
102 with a gradient from black to white, fully opaque)
103
104 The range of input values that is mapped to the entirety of the LUT is, by
105 default, the full image range, but can be customized with `SetRange`.
106
107 The corresponding renderers are `OpenGLLookupTableTextureRenderer` and
108 `CairoLookupTableTextureRenderer`. The scene transformation is applied during
109 rendering.
110
111 ### Implementation: `PolylineSceneLayer`
112
113 Layer that renders vector-based polygonal lines.
114
115 Polylines can be added with the `AddChain` method, that accepts a `Chain`, that
116 is a typedef to `std::vector<ScenePoint2D>`, a flag to specify whether the
117 chain must be automatically close (last point of the vector connected to the
118 first one) and the chain color (a `Color` structure).
119
120 Please note that the line thickness is, contrary to the color, specified
121 per-chain but rather per-layer.
122
123 If you need multiple line thicknesses, multiple `PolylineSceneLayer` must be
124 created.
125
126 The corresponding renderers are `OpenGLAdvancedPolylineRenderer` and
127 `CairoPolylineRenderer`. The scene transformation is applied during
128 rendering.
129
130 ### Implementation: `TextSceneLayer`
131
132 This layers renders a paragraph of text.
133
134 The inputs to the layer can be changed after creation and are:
135 - The text iself, supplied as an UTF-8 encoded string in `SetText`
136 - The font used for rendering, set by `SetFontIndex`.
137 - The text anchoring, through `SetAnchor`: the text can be anchored to
138 various positions, such as top lef, center, bottom center,... These
139 various anchors are part of the `BitmapAnchor` enumeration.
140 - The text position, relative to its anchor, through `SetPosition`.
141
142 The font is supplied as an index. This is an index in the set of fonts
143 that has been registered in the viewport compositor. The following code
144 shows how to set such a font:
145
146 ```
147 std::unique_ptr<OrthancStone::IViewport::ILock> lock(viewport_.Lock());
148 lock->GetCompositor().SetFont(0,
149 Orthanc::EmbeddedResources::UBUNTU_FONT,
150 32, Orthanc::Encoding_Latin1);
151 // where 32 is the font size in pixels
152 ```
153
154 This call uses the embedded `UBUNTU_FONT` resource that has been defined in
155 the `CMakeLists.txt` file with:
156
157 ```
158 EmbedResources(
159 UBUNTU_FONT ${CMAKE_BINARY_DIR}/ubuntu-font-family-0.83/Ubuntu-R.ttf
160 )
161 ```
162
163 Please note that you must supply a font: there is no default font provided by
164 the OpenGL or Cairo compositors.
165
166 The corresponding renderers are `OpenGLTextRenderer` and
167 `CairoTextRenderer`. The scene transformation is not applied during rendering,
168 because the text anchoring, position and scaling are computed relative to the
169 viewport/canvas.
170
171 ### Implementation: `InfoPanelSceneLayer`
172
173 This layer is designed to display an image, supplied through an
174 `Orthanc::ImageAccessor` reference (only used at construction time).
175
176 The image is not transformed according to the normal layer transformation but
177 is rather positioned relative to the canvas, with the same mechanism as the
178 `TextSceneLayer` described above.
179
180 The image position is specified with the sole means of the `SetAnchor` method.
181
182 The corresponding renderers are `OpenGLInfoPanelRenderer` and
183 `CairoInfoPanelRenderer`.
184
185 ### `IViewport`
186
187 https://bitbucket.org/sjodogne/orthanc-stone/src/broker/Framework/Viewport/IViewport.h
188
189 (**not** the one in `Deprecated`)
190 - Implemented by classes that:
191 - manage the on-screen display of a `Scene2D` trough a compositor.
192 - Own the `ICompositor` object that performs the rendering.
193 - Own the `Scene2D` (TODO: currently through `ViewportController` --> `Scene2D`)
194 - Provide a `Lock` method that returns a RAII, that must be kept alive when
195 modifying the underlying objects (controller, compositor, scene), but not
196 longer.
197
198 #### Implementation: `SdlOpenGLViewport`
199 - Implementation of a viewport rendered on a SDL window, that uses OpenGL for
200 rendering.
201 - Instantiating this object creates an SDL window. Automatic scaling for hiDPI
202 displays can be toggled on or off.
203
204 #### Implementation: `WebGLViewport`
205 - Implementation of a viewport rendered on a DOM canvas, that uses OpenGL for
206 rendering.
207 - Contrary to the SDL OpenGL viewport, the canvas must already be existing
208 when the ctor is called.
209
210 ### `ICompositor`
211 The interface providing a rendering service for `Scene2D` objects.
212
213 **Subclasses:** `CairoCompositor`, `OpenGLCompositor`
214
215 You do not need to create compositor instances. They are created for you when
216 instantiating a viewport.
217
218 ### `ViewportController`
219 This concrete class is instantiated by its `IViewport` owner.
220
221 **TODO:** its functionality is not well defined and should be moved into the
222 viewport base class. Support for measuring tools should be moved to a special
223 interactor.
224
225 - contains:
226 - array of `MeasureTool`
227 - ref to `IViewport`
228 - `activeTracker_`
229 - owns a `Scene2D`
230 - weak ref to `UndoStack`
231 - cached `canvasToSceneFactor_`
232
233 - contains logic to:
234 - pass commands to undostack (trivial)
235 - logic to locate `MeasureTool` in the HitTest
236 - OTOH, the meat of the measuring tool logic (highlighting etc..) is
237 done in app-specific code (`VolumeSlicerWidget`)
238 - accept new Scene transform and notify listeners
239 - **the code that uses the interactor** (`HandleMousePress`) is only
240 called by the new `WebAssemblyViewport` !!! **TODO** clean this mess
241
242 ### `IViewportInteractor`
243 - must provide logic to respond to `CreateTracker`
244
245 ### `DefaultViewportInteractor`
246 - provides Pan+Rotate+Zoom trackers
247
248 ### `WebGLViewportsRegistry`
249
250 This class is a singleton (accessible through `GetWebGLViewportsRegistry()`
251 that deals with context losses in the WebGL contexts.
252
253 You use it by creating a WebGLViewport in the following fashion:
254
255 ```
256 boost::shared_ptr<OrthancStone::WebGLViewport> viewport(
257 OrthancStone::GetWebGLViewportsRegistry().Add(canvasId));
258 ```
259
260 ## Source data related
261
262 ### `IVolumeSlicer`
263
264 A very simple interface with a single method:
265 `IVolumeSlicer::IExtractedSlice* ExtractSlice(const CoordinateSystem3D& cuttingPlane)`
266
267 ### `IVolumeSlicer::IExtractedSlice`
268
269 On a slice has been extracted from a volume by an `IVolumeSlicer`, it can
270 report its *revision number*.
271
272 If another call to `ExtractSlice` with the same cutting plane is made, but
273 the returned slice revision is different, it means that the volume has
274 changed and the scene layer must be refreshed.
275
276 Please see `VolumeSceneLayerSource::Update` to check how this logic is
277 implemented.
278
279
280 ### `OrthancSeriesVolumeProgressiveLoader`
281
282 This class implements `IVolumeSlicer` (and `IObservable`) and can be used to
283 load a volume stored in a Dicom series on an Orthanc server.
284
285 Over the course of the series loading, various notifications are sent:
286
287 The first one is `OrthancStone::DicomVolumeImage::GeometryReadyMessage` that
288 is sent when the volume extent and geometric properties are known.
289
290 Then, as slices get loaded and the volume is filled,
291 `OrthancStone::DicomVolumeImage::ContentUpdatedMessage` are sent.
292
293 Once all the highest-quality slices have been loaded, the
294 `OrthancSeriesVolumeProgressiveLoader::VolumeImageReadyInHighQuality`
295 notification is sent.
296
297 Please note that calling `ExtractSlice` *before* the geometry is loaded will
298 yield an instance of `InvalidSlice` that cannot be used to create a layer.
299
300 On the other hand,
301
302 ### `VolumeSceneLayerSource`
303
304 This class makes the bridge between a volume (supplied by an `IVolumeSlicer`
305 interface) and a `Scene2D`.
306
307 Please note that the bulk of the work is done the objects implementing
308 `IVolumeSlicer` and this object merely connects things together.
309
310 For instance, deciding whether an image (texture) or vector (polyline) layer
311 is done by the `IVolumeSlicer` implementation.
312
313 - contains:
314 - reference to Scene2D
315 - `layerIndex_` (fixed at ctor) that is the index, in the Scene2D layer
316 stack, of the layer that will be created/updated
317 - `IVolumeSlicer`
318
319 - contains logic to:
320 - extract a slice from the slicer and set/refresh the Scene2D layer at
321 the supplied `layerIndex_`
322 - refresh this based on the slice revision or configuration revision
323 - accept a configuration that will be applied to the layer
324 - the `Update()` method will
325
326 ## Updates and the configurators
327
328 `ISceneLayer` does not expose mutable methods.
329
330 The way to change a layer once it has been created is through configurator
331 objets.
332
333 If you plan to set (even only once) or modify some layer properties after
334 layer creation, you need to create a matching configurator objet.
335
336 For instance, in the `VolumeSceneLayerSource`, the `SetConfigurator` method
337 will store a `ILayerStyleConfigurator* configurator_`.
338
339 In the `OrthancView` ctor, you can see how it is used:
340
341 ```
342 std::unique_ptr<GrayscaleStyleConfigurator> style(
343 new GrayscaleStyleConfigurator);
344
345 style->SetLinearInterpolation(true);
346
347 ...<some more code>...
348
349 std::unique_ptr<LookupTableStyleConfigurator> config(
350 new LookupTableStyleConfigurator);
351
352 config->SetLookupTable(Orthanc::EmbeddedResources::COLORMAP_HOT);
353
354 ```
355
356 The configurator type are created according to the type of layer.ΒΈ
357
358 Later, in `VolumeSceneLayerSource::Update(const CoordinateSystem3D& plane)`,
359 if the cutting plane has **not** changed and if the layer revision has **not**
360 changed, we test `configurator_->GetRevision() != lastConfiguratorRevision_`
361 and, if different, we call `configurator_->ApplyStyle(scene_.GetLayer(layerDepth_));`
362
363 This allows to change layer properties that do not depend on the layer model
364 contents.
365
366 On the other hand, if the layer revision has changed, when compared to the
367 last time it has been rendered (stored in `lastRevision_`), then we need to
368 ask the slice to create a brand new layer.
369
370 Another way to see it is that layer rendering depend on model data and view
371 data. The model data is not mutable in the layer and, if the model changes, the
372 layer must be recreated.
373
374 If only the view properties change (the configurator), we call ApplyStyle
375 (that **will** mutate some of the layer internals)
376
377 Please note that the renderer does **not** know about the configurator : the
378 renderer uses properies in the layer and does not care whether those have
379 been set once at construction time or at every frame (configuration time).
380
381
382 ## Cookbook
383
384 ### Simple application
385
386 #### Building
387
388 In order to create a Stone application, you need to:
389
390 - CMake-based application:
391 ```
392 include(${STONE_SOURCES_DIR}/Resources/CMake/OrthancStoneConfiguration.cmake)
393 ```
394 with this library target that you have to define:
395 ```
396 add_library(OrthancStone STATIC ${ORTHANC_STONE_SOURCES})
397 ```
398 then link with this library:
399 ```
400 target_link_libraries(MyStoneApplication OrthancStone)
401 ```
402
403 Building is supported with emscripten, Visual C++ (>= 9.0), gcc...
404
405 emscripten recommended version >= 1.38.41
406
407 These are very rough guidelines. See the `Samples` folder for actual examples.
408
409 #### Structure
410
411 The code requires a loader (object that )
412
413 Initialize:
414
415 ```
416 Orthanc::Logging::Initialize();
417 Orthanc::Logging::EnableInfoLevel(true);
418 ```
419 Call, in WASM:
420 ```
421 DISPATCH_JAVASCRIPT_EVENT("StoneInitialized");
422 ```
423
424 # Notes
425
426 - It is NOT possible to abandon the existing loaders : they contain too much loader-specific getters
427
428
429