annotate TODO @ 522:c49136b34891 large-queries tip

use a prepared statement for InsertOrUpdateMetadata
author Alain Mazy <am@orthanc.team>
date Fri, 05 Jul 2024 09:15:54 +0200
parents a164d8aebe0e
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
1
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
2 ==============================
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
3 == Orthanc database plugins ==
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
4 ==============================
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
5
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
6 -----------------------
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
7 Common - Database index
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
8 -----------------------
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
9
258
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
10 * Performance of joins in LookupResources: Create cached statement for
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
11 LookupResources, that are grouped to search up to, say, 10 tags,
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
12 instead of recompiling for each request
371
Alain Mazy <am@osimis.io>
parents: 365
diff changeset
13 * Do not log "DatabaseCannotSerialize" errors in the plugin but only
Alain Mazy <am@osimis.io>
parents: 365
diff changeset
14 in Orthanc after all retries have been made.
424
95c413106a66 Introduced a procedure to delete resources (to be tested)
Alain Mazy <am@osimis.io>
parents: 371
diff changeset
15 * Try to avoid the use of temporary tables:
95c413106a66 Introduced a procedure to delete resources (to be tested)
Alain Mazy <am@osimis.io>
parents: 371
diff changeset
16 https://discourse.orthanc-server.org/t/image-insert-are-too-slow-databse-performance-too-poor/3820
95c413106a66 Introduced a procedure to delete resources (to be tested)
Alain Mazy <am@osimis.io>
parents: 371
diff changeset
17
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
18
502
bf4b9c7cf338 todo & notes
Alain Mazy <am@orthanc.team>
parents: 482
diff changeset
19 * Implement "large queries" for:
bf4b9c7cf338 todo & notes
Alain Mazy <am@orthanc.team>
parents: 482
diff changeset
20 - updating all metadata of a resource at once
bf4b9c7cf338 todo & notes
Alain Mazy <am@orthanc.team>
parents: 482
diff changeset
21 - update all maindicomtags of 4 resource levels at once
bf4b9c7cf338 todo & notes
Alain Mazy <am@orthanc.team>
parents: 482
diff changeset
22
bf4b9c7cf338 todo & notes
Alain Mazy <am@orthanc.team>
parents: 482
diff changeset
23
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
24 ---------------------
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
25 Common - Storage area
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
26 ---------------------
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
27
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
28 * Implement the "StoreDicom" option from the default filesystem
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
29 storage area to run MySQL/PostgreSQL storage in index-only mode:
426
d700c8f9fc24 migration to UCLouvain servers
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 371
diff changeset
30 https://orthanc.uclouvain.be/book/contributing.html
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
31
259
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 258
diff changeset
32 * Performance: Add a pool of connections to PostgreSQL/MySQL, as
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 258
diff changeset
33 already done in the database index plugins
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 258
diff changeset
34
360
Alain Mazy <am@osimis.io>
parents: 266
diff changeset
35 ----------
Alain Mazy <am@osimis.io>
parents: 266
diff changeset
36 PostgreSQL
Alain Mazy <am@osimis.io>
parents: 266
diff changeset
37 ----------
Alain Mazy <am@osimis.io>
parents: 266
diff changeset
38
477
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
39 * Check if we can force the schema that is used. By default, Orthanc
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
40 is using the 'public' schema but, after a wrong command, we have seen
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
41 a DB where there was a 'AttachedFiles' table in the public schema and another one in a
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
42 'MyPacs' schema and Orthanc was actually using the 'MyPacs.AttachedFiles' table !!!
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
43 Orthanc was then seeing only the most recent attached files !!!
Alain Mazy <am@osimis.io>
parents: 473
diff changeset
44
473
15e609c439d1 Fixed the MaximumStorageSize & MaximumPatientCount.
Alain Mazy <am@osimis.io>
parents: 445
diff changeset
45 * Have a separate "thread" to UpdateStatistics to avoid large computations ?
15e609c439d1 Fixed the MaximumStorageSize & MaximumPatientCount.
Alain Mazy <am@osimis.io>
parents: 445
diff changeset
46
360
Alain Mazy <am@osimis.io>
parents: 266
diff changeset
47
365
Alain Mazy <am@osimis.io>
parents: 364
diff changeset
48 * Seems Orthanc might deadlock when there are plenty of conflicting transactions:
Alain Mazy <am@osimis.io>
parents: 364
diff changeset
49 https://groups.google.com/g/orthanc-users/c/xQelEcKqL9U/m/HsvxwlkvAQAJ
Alain Mazy <am@osimis.io>
parents: 364
diff changeset
50 https://groups.google.com/g/orthanc-users/c/1bkClfZ0KBA/m/s4AlwVh3CQAJ
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
51
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
52 -----
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
53 MySQL
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
54 -----
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
55
266
cc7af42d4f23 Store revisions for metadata and attachments in PostgreSQL
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 259
diff changeset
56 * Store revisions for metadata and attachments in MySQL (this is
cc7af42d4f23 Store revisions for metadata and attachments in PostgreSQL
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 259
diff changeset
57 already implemented in PostgreSQL)
cc7af42d4f23 Store revisions for metadata and attachments in PostgreSQL
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 259
diff changeset
58
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
59 * MySQL performance => implement GlobalProperty_GetTotalSizeIsFast:
Sebastien Jodogne <s.jodogne@gmail.com>
parents:
diff changeset
60 https://groups.google.com/d/msg/orthanc-users/kSR4a110zDo/D7e4ITR8BwAJ
258
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
61
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
62 * Add index to speed up wildcard search, as already done in PostgreSQL:
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
63
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
64 - https://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html +
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
65 ALTER TABLE table ADD FULLTEXT index_name(column1);
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
66
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 257
diff changeset
67 - https://dev.mysql.com/doc/refman/8.0/en/index-btree-hash.html
482
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
68
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
69
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
70 ----
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
71 ODBC
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
72 ----
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
73
Alain Mazy <am@osimis.io>
parents: 479
diff changeset
74 * https://orthanc.uclouvain.be/bugs/show_bug.cgi?id=224