annotate PerfsDb/README.md @ 476:77f868063bb3

cleanup
author Alain Mazy <am@osimis.io>
date Tue, 03 May 2022 17:10:29 +0200
parents 705fa4447154
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
1 Performance Db tests
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
2 ====================
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
3
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
4 Introduction
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
5 ------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
6
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
7 This project performs benchmark tests of Orthanc with its various DB servers.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
8
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
9 It can be used to:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
10
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
11 - compare DB servers performance
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
12 - compare performance between small and big DB
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
13 - test the effectiveness of some code refactoring
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
14
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
15 In a first step, the project creates a set of DB servers and populates them.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
16 Then, it will perform a set of basic operations on Orthanc with each of these servers
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
17 and measure the time required for each operation.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
18
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
19 Timings are measured at the API level. Since we want to measure mainly the DB performance,
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
20 we'll mainly use very small DICOM files (without pixels data).
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
21
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
22 Prerequisites
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
23 -------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
24
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
25 - install standard tools
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
26
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
27 ```bash
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
28 sudo apt-get install -y mercurial wget curl
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
29 ```
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
30
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
31 - install python3, pip3 and pipenv
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
32
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
33 ```bash
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
34 sudo apt-get install -y python3 python3-pip python3-venv
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
35 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
36
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
37 - [install Docker-CE](https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository)
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
38 - have access to docker without typing `sudo`. This is done by typing: `sudo groupadd docker` and `sudo usermod -aG docker $USER`
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
39 - have Orthanc and its DB plugins natively installed or compiled on your host system
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
40
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
41 Once all prerequisites are installed, you should always execute all commands from a python virtual-env. To initialize the virtual env the first time:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
42
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
43 ```bash
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
44 python3 -m venv .env
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
45 source .env/bin/activate
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
46 pip install -r requirements.txt
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
47 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
48
198
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 197
diff changeset
49 To enter the virtual-env the next times (on benchmark VM):
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
50
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
51 ```bash
198
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 197
diff changeset
52 cd /data-hdd-3g/orthanc-tests/PerfsDb
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
53 source .env/bin/activate
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
54 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
55
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
56 Initializing a DB before tests
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
57 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
58
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
59 ```bash
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
60 on amazy PC:
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
61 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/mysql-release/ --init --mysql-tiny
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
62
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
63 on benchmark VM:
198
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 197
diff changeset
64 python Run.py --orthanc-path=/data-hdd-3g/orthanc-binaries/ --plugins-path=/data-hdd-3g/orthanc-binaries/ --init --pg9bis-small
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
65 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
66
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
67 Clearing a DB
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
68 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
69
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
70 ```bash
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
71 on amazy PC:
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
72 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/mysql-release/ --clear --mysql-tiny
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
73
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
74 on benchmark VM:
198
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 197
diff changeset
75 python Run.py --orthanc-path=/data-hdd-3g/orthanc-binaries/ --plugins-path=/data-hdd-3g/orthanc-binaries/ --clear --pg9bis-small
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
76 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
77
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
78 Runing tests on multiple DBs
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
79 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
80
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
81 ```bash
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
82 on amazy PC:
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
83 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/orthanc-build-release/ --run --pg9-tiny --pg10-tiny --pg11-tiny --mysql-tiny --sqlite-tiny --mssql-tiny
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
84
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
85 on benchmark VM:
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
86 python Run.py --orthanc-path=/data-hdd-3g/orthanc-binaries/ --plugins-path=/data-hdd-3g/orthanc-binaries/ --run --pg9-tiny --pg10-tiny --pg11-tiny --mysql-tiny --sqlite-tiny --mssql-tiny
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
87
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
88 ```
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
89
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
90 Command line options
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
91 --------------------
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
92
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
93 ```bash
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
94 --clear to clear the DB
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
95 --init to initialize the DB
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
96 --run to run the tests
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
97
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
98 --config to include the config in the test run/init/clear
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
99
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
100 --orthanc-path path to the folder where the Orthanc execuble lies
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
101 --plugins-path path to the folder where the Orthanc plugins lie
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
102
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
103 --test-filter=*ByPatient* to filter the tests to execute
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
104 --repeat=20 to repeate each test N times (default is 20)
196
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
105 ```