annotate PerfsDb/README.md @ 186:8feebc2bf9fc

test options expand and limit to /queries/.../answers
author Sebastien Jodogne <s.jodogne@gmail.com>
date Thu, 29 Nov 2018 20:36:08 +0100
parents ff939d07989f
children 29cb1ee97035
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
1 Performance Db tests
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
2 ====================
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
3
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
4 Introduction
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
5 ------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
6
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
7 This project performs benchmark tests of Orthanc with its various DB servers.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
8
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
9 It can be used to:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
10
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
11 - compare DB servers performance
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
12 - compare performance between small and big DB
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
13 - test the effectiveness of some code refactoring
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
14
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
15 In a first step, the project creates a set of DB servers and populates them.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
16 Then, it will perform a set of basic operations on Orthanc with each of these servers
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
17 and measure the time required for each operation.
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
18
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
19 Timings are measured at the API level. Since we want to measure mainly the DB performance,
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
20 we'll mainly use very small DICOM files (without pixels data).
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
21
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
22 Prerequisites
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
23 -------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
24
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
25 - install standard tools
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
26
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
27 ```bash
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
28 sudo apt-get install -y mercurial wget curl
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
29 ```
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
30
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
31 - install python3, pip3 and pipenv
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
32
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
33 ```bash
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
34 sudo apt-get install -y python3 python3-pip python3-venv
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
35 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
36
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
37 - [install Docker-CE](https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository)
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
38 - have access to docker without typing `sudo`. This is done by typing: `sudo groupadd docker` and `sudo usermod -aG docker $USER`
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
39 - have Orthanc and its DB plugins natively installed or compiled on your host system
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
40
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
41 Once all prerequisites are installed, you should always execute all commands from a python virtual-env. To initialize the virtual env the first time:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
42
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
43 ```bash
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
44 python3 -m venv .env
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
45 source .env/bin/activate
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
46 pip install -r requirements.txt
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
47 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
48
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
49 To enter the virtual-env the next times:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
50
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
51 ```bash
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
52 source .env/bin/activate
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
53 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
54
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
55 Initializing a DB before tests
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
56 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
57
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
58 ```bash
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
59 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/mysql-release/ --init --mysql-tiny
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
60 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
61
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
62 Clearing a DB
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
63 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
64
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
65 ```bash
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
66 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/mysql-release/ --clear --mysql-tiny
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
67 ```
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
68
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
69 Runing tests on multiple DBs
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
70 -----------------
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
71
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
72 ```bash
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
73 on amazy PC:
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
74 python Run.py --orthanc-path=/home/amazy/builds/orthanc-build-release/ --plugins-path=/home/amazy/builds/orthanc-build-release/ --run --pg9-tiny --pg10-tiny --pg11-tiny --mysql-tiny --sqlite-tiny --mssql-tiny
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
75
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
76 on benchmark VM:
159
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
77 python Run.py --orthanc-path=/data/orthanc-binaries/ --plugins-path=/data/orthanc-binaries/ --run --pg9-tiny --pg10-tiny --pg11-tiny --mysql-tiny --sqlite-tiny --mssql-tiny
616da104a996 fix for ubuntu 18.04
am@osimis.io
parents: 158
diff changeset
78
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
79 ```
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
80
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
81 Command line options
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
82 --------------------
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
83
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
84 ```bash
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
85 --clear to clear the DB
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
86 --init to initialize the DB
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
87 --run to run the tests
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
88
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
89 --config to include the config in the test run/init/clear
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
90
164
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
91 --orthanc-path path to the folder where the Orthanc execuble lies
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
92 --plugins-path path to the folder where the Orthanc plugins lie
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
93
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
94 --test-filter=*ByPatient* to filter the tests to execute
ff939d07989f saving results
am@osimis.io
parents: 160
diff changeset
95 --repeat=20 to repeate each test N times (default is 20)
160
6995d5d12d88 filtering tests
am@osimis.io
parents: 159
diff changeset
96 ```