annotate NOTES @ 431:7c1fe5d6c12c pg-transactions

PG: IncrementGlobalProperty
author Alain Mazy <am@osimis.io>
date Thu, 07 Dec 2023 12:13:43 +0100
parents
children 8b7c1c423367
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
431
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
1 Resources:
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
2 *********
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
3 - PG transaction modes explained: https://www.postgresql.org/files/developer/concurrency.pdf
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
4 - Isoolation level explained (PG + MySQL): https://amirsoleimani.medium.com/understanding-database-isolation-level-via-examples-mysql-and-postgres-a86b5502d404
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
5
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
6
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
7 Sample SQL code that you can execute in DBeaver to test new functions/procedures:
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
8
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
9 CreateInstance
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
10 ************************************************************************
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
11
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
12 CREATE OR replace FUNCTION CreateInstance(
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
13 IN patient TEXT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
14 IN study TEXT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
15 IN series TEXT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
16 IN instance TEXT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
17 OUT isNewPatient BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
18 OUT isNewStudy BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
19 OUT isNewSeries BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
20 OUT isNewInstance BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
21 OUT patientKey BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
22 OUT studyKey BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
23 OUT seriesKey BIGINT,
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
24 OUT instanceKey BIGINT)
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
25 AS $$
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
26 begin
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
27 isNewPatient := 1;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
28 isNewStudy := 1;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
29 isNewSeries := 1;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
30 isNewInstance := 1;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
31
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
32 BEGIN
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
33 INSERT INTO "resources" VALUES (DEFAULT, 0, patient, NULL);
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
34 exception
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
35 when unique_violation then
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
36 isNewPatient := 0;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
37 end;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
38 select internalid into patientKey from "resources" where publicId=patient and resourcetype = 0;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
39
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
40 BEGIN
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
41 INSERT INTO "resources" VALUES (DEFAULT, 1, study, patientKey);
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
42 exception
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
43 when unique_violation then
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
44 isNewStudy := 0;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
45 end;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
46 select internalid into studyKey from "resources" where publicId=study and resourcetype = 1;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
47
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
48 BEGIN
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
49 INSERT INTO "resources" VALUES (DEFAULT, 2, series, studyKey);
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
50 exception
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
51 when unique_violation then
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
52 isNewSeries := 0;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
53 end;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
54 select internalid into seriesKey from "resources" where publicId=series and resourcetype = 2;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
55
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
56 BEGIN
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
57 INSERT INTO "resources" VALUES (DEFAULT, 3, instance, seriesKey);
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
58 exception
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
59 when unique_violation then
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
60 isNewInstance := 0;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
61 end;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
62 select internalid into instanceKey from "resources" where publicId=instance and resourcetype = 3;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
63
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
64 END;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
65 $$ LANGUAGE plpgsql;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
66
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
67 DO $$
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
68 DECLARE
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
69 result record;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
70 begin
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
71 delete from "resources";
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
72
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
73 SELECT * INTO result from CreateInstance('patient1', 'study1', 'series1', 'instance1');
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
74
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
75 RAISE NOTICE 'Value patientInternalId: %, is_new: %', result.patientKey, result.isNewPatient;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
76 RAISE NOTICE 'Value studyInternalId: %, is_new: %', result.studyKey, result.isNewStudy;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
77 RAISE NOTICE 'Value seriesInternalId: %, is_new: %', result.seriesKey, result.isNewSeries;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
78 RAISE NOTICE 'Value instanceInternalId: %, is_new: %', result.instanceKey, result.isNewInstance;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
79 RAISE NOTICE '--------------';
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
80
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
81 SELECT * INTO result from CreateInstance('patient1', 'study1', 'series1', 'instance2');
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
82
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
83 RAISE NOTICE 'Value patientInternalId: %, is_new: %', result.patientKey, result.isNewPatient;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
84 RAISE NOTICE 'Value studyInternalId: %, is_new: %', result.studyKey, result.isNewStudy;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
85 RAISE NOTICE 'Value seriesInternalId: %, is_new: %', result.seriesKey, result.isNewSeries;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
86 RAISE NOTICE 'Value instanceInternalId: %, is_new: %', result.instanceKey, result.isNewInstance;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
87 RAISE NOTICE '--------------';
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
88
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
89 SELECT * INTO result from CreateInstance('patient1', 'study1', 'series1', 'instance2');
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
90
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
91 RAISE NOTICE 'Value patientInternalId: %, is_new: %', result.patientKey, result.isNewPatient;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
92 RAISE NOTICE 'Value studyInternalId: %, is_new: %', result.studyKey, result.isNewStudy;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
93 RAISE NOTICE 'Value seriesInternalId: %, is_new: %', result.seriesKey, result.isNewSeries;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
94 RAISE NOTICE 'Value instanceInternalId: %, is_new: %', result.instanceKey, result.isNewInstance;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
95 RAISE NOTICE '--------------';
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
96
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
97 END $$;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
98
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
99 -- \set patient_key 'patient_key'
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
100 -- SELECT CreateInstance('patient', 'study', 'series', 'instance', patient_key) ;
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
101
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
102 -- drop function CreateInstance
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
103 -- select * from "resources";
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
104 -- delete from "resources";
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
105 -- INSERT INTO "resources" VALUES (DEFAULT, 0, 'patient', NULL)
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
106
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
107
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
108
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
109 ************************************************************************
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
110
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
111 In debug, no verbose logs
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
112 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 51.703 s
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
113 Orthanc mainline + PG maineline (serializable mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 51.913 s
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
114 Orthanc mainline + PG maineline (read-committed mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 24.754 s
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
115
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
116 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_uploads_same_study with 40 workers and 1x repeat: 55.932 s
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
117 Orthanc mainline + PG maineline (serializable mode) : fails: No new instance while overwriting; this should not happen
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
118 Orthanc mainline + PG maineline (read-committed mode) :
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
119
7c1fe5d6c12c PG: IncrementGlobalProperty
Alain Mazy <am@osimis.io>
parents:
diff changeset
120 TODO: