Mercurial > hg > orthanc-databases
comparison NOTES @ 443:2a48f8fcec6e pg-transactions
cleanup
author | Alain Mazy <am@osimis.io> |
---|---|
date | Wed, 10 Jan 2024 09:04:05 +0100 |
parents | d979f25e60cf |
children | 2ca939d02d39 |
comparison
equal
deleted
inserted
replaced
439:2bc65c678b1b | 443:2a48f8fcec6e |
---|---|
165 | 165 |
166 | 166 |
167 | 167 |
168 ************************************************************************ | 168 ************************************************************************ |
169 | 169 |
170 In debug, no verbose logs | 170 In debug, no verbose logs, 10 connections |
171 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 43.957 s | 171 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 43.957 s |
172 Orthanc mainline + PG mainline (read-committed mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 15.744 s | 172 Orthanc mainline + PG mainline (read-committed mode) : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 15.744 s |
173 test_concurrent_anonymize_same_study deletion took: 18.8 s | 173 test_concurrent_anonymize_same_study deletion took: 18.8 s |
174 | 174 |
175 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 21.214 s | 175 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 21.214 s |
176 Orthanc mainline + PG mainline (read-committed mode) : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 6.57 s (with temporary tables) | 176 Orthanc mainline + PG mainline (read-committed mode) : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 6.57 s (with temporary tables) |
177 | 177 |
178 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 23.016 s | 178 Orthanc 1.12.1 + PG 5.1 (serializable mode) : test_upload_delete_same_study_from_multiple_threads with 5 workers and 3x repeat (10x): 23.016 s |
179 Orthanc mainline + PG mainline (read-committed mode) : test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 7.129 s | 179 Orthanc mainline + PG mainline (read-committed mode) : test_upload_delete_same_study_from_multiple_threads with 5 workers and 3x repeat (10x): 7.129 s |
180 | 180 |
181 With Docker with 10 connections SQL: | 181 With Docker with 10 connections SQL: |
182 osimis/orthanc:23.11.0: TIMING test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 25.047 s | 182 osimis/orthanc:23.11.0: TIMING test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 25.047 s |
183 osimis/orthanc:current: TIMING test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 8.649 s | 183 osimis/orthanc:current: TIMING test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 8.649 s |
184 | 184 |
189 osimis/orthanc:current: TIMING test_concurrent_uploads_same_study with 20 workers and 1x repeat: 3.465 s | 189 osimis/orthanc:current: TIMING test_concurrent_uploads_same_study with 20 workers and 1x repeat: 3.465 s |
190 | 190 |
191 osimis/orthanc:23.11.0: TIMING test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 11.966 s | 191 osimis/orthanc:23.11.0: TIMING test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 11.966 s |
192 osimis/orthanc:current: TIMING test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 5.092 s | 192 osimis/orthanc:current: TIMING test_upload_delete_same_study_from_multiple_threads with 5 workers and 30x repeat: 5.092 s |
193 | 193 |
194 Testing the connecions (note: Orthanc and PG server running on the same server) | |
195 10 connections : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 15.744 s | |
196 1 connection : test_concurrent_anonymize_same_study with 4 workers and 10x repeat: 21.341 s | |
197 10 connections : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 6.57 s | |
198 1 connection : test_concurrent_uploads_same_study with 20 workers and 1x repeat: 10.223 s | |
199 10 connections : test_upload_delete_same_study_from_multiple_threads with 5 workers and 3x repeat (10x): 7.129 s | |
200 1 connection : test_upload_delete_same_study_from_multiple_threads with 5 workers and 3x repeat (10x): 11.172 s | |
201 | |
194 | 202 |
195 TODO: | 203 TODO: |
204 - have a separate "thread" to UpdateStatistics ? | |
205 - check https://discourse.orthanc-server.org/t/image-insert-are-too-slow-databse-performance-too-poor-when-using-mysql-mariadb/3820 | |
206 - implement a downgrade script ? And test it in PotgresUpgrades integ tests | |
207 - In Docker images, re-enable MySQL & ODBC plugins + tests | |
208 | |
209 - this error sometimes occur... | |
210 ====================================================================== | |
211 FAIL: test_upload_delete_same_study_from_multiple_threads (Concurrency.test_concurrency.TestConcurrency) | |
212 ---------------------------------------------------------------------- | |
213 Traceback (most recent call last): | |
214 File "/home/runner/work/orthanc-builder/orthanc-builder/docker/integration-tests/orthanc-tests-repo-normal/NewTests/Concurrency/test_concurrency.py", line 254, in test_upload_delete_same_study_from_multiple_threads | |
215 self.assertEqual(0, len(self.o.studies.get_all_ids())) | |
216 AssertionError: 0 != 1 | |
217 | |
218 2023-12-21 17:10:04.737 UTC [69] ERROR: deadlock detected | |
219 2023-12-21 17:10:04.737 UTC [69] DETAIL: Process 69 waits for ShareLock on transaction 20657; blocked by process 70. | |
220 Process 70 waits for ShareLock on transaction 18193; blocked by process 69. | |
221 Process 69: SELECT * FROM DeleteResource($1) | |
222 Process 70: SELECT * FROM CreateInstance($1, $2, $3, $4) | |
223 2023-12-21 17:10:04.737 UTC [69] HINT: See server log for query details. | |
224 2023-12-21 17:10:04.737 UTC [69] CONTEXT: while deleting tuple (134,66) in relation "resources" | |
225 SQL statement "DELETE FROM ONLY "public"."resources" WHERE $1 OPERATOR(pg_catalog.=) "parentid"" | |
226 SQL statement "DELETE FROM Resources WHERE internalId = old.parentId | |
227 AND NOT EXISTS (SELECT 1 FROM Resources WHERE parentId = old.parentId)" | |
228 PL/pgSQL function resourcedeletedfunc() line 7 at SQL statement | |
229 SQL statement "DELETE FROM Resources WHERE internalId=id RETURNING *" | |
230 PL/pgSQL function deleteresource(bigint) line 26 at SQL statement | |
231 2023-12-21 17:10:04.737 UTC [69] STATEMENT: SELECT * FROM DeleteResource($1) | |
232 | |
233 DONE: | |
196 - reenable PatientRecyclingOrder | 234 - reenable PatientRecyclingOrder |
197 - have a separate "thread" to UpdateStatistics ? | |
198 - test events generation StableSeries .... | |
199 - check https://discourse.orthanc-server.org/t/image-insert-are-too-slow-databse-performance-too-poor-when-using-mysql-mariadb/3820 | |
200 - validate upgrade DB from previous Orthanc and from scratch | |
201 - check minimal version of PG (9.5 - 9.6 ? for create index if not exists) | |
202 - implement a downgrade script ? | |
203 - test with older version of PG | |
204 - In Docker images, re-enable MySQL & ODBC plugins + tests | |
205 | |
206 DONE: | |
207 - force the create/update DB transaction to be serializable (not needed: this is handled by POSTGRESQL_LOCK_DATABASE_SETUP) | 235 - force the create/update DB transaction to be serializable (not needed: this is handled by POSTGRESQL_LOCK_DATABASE_SETUP) |
208 - PatientAddedFunc contains an IF (check if other IF/THEN/ELSE pattern remains) | 236 - PatientAddedFunc contains an IF (check if other IF/THEN/ELSE pattern remains) |
237 - validate upgrade DB from previous Orthanc and from scratch | |
238 - check minimal version of PG (9.5 - 9.6 ? for create index if not exists): seems to work with 9.5 cfr PotgresUpgrades integ tests | |
239 - test events generation StableSeries .... (count the NewSeries, NewInstances event and make sure they match the numb) |