Mercurial > hg > orthanc-tests

annotate PerfsDb/Test.py @ 231:4087505ddfe3

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
catching error codes of movescu
author Sebastien Jodogne <s.jodogne@gmail.com>
date Tue, 05 Mar 2019 07:02:26 +0100
parents f42c10dab862
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
200
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
1 import os
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
2 import time
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
3 import statistics
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
4 from orthancRestApi import OrthancClient
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
5
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
6 from TestResult import TestResult
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
7
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
8 class Test:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
9
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
10 def __init__(self, name: str):
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
11 self.name = name
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
12 self._orthanc = None
164
ff939d07989f saving results
am@osimis.io
parents: 161
diff changeset
13 self.repeatCount = 20
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
14
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
15 def setOrthancClient(self, orthanc: OrthancClient):
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
16 self._orthanc = orthanc
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
17
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
18 def setRepeatCount(self, repeatCount: int):
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
19 self.repeatCount = repeatCount
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
20
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
21 def beforeAll(self):
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
22 """
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
23 Code to execute before the execution of all repetitions of a test; i.e: upload a file.
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
24 This code is not included in timings
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
25 """
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
26 pass
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
27
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
28 def beforeEach(self):
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
29 """
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
30 Code to execute before the execution of each repetition of a test.
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
31 This code is not included in timings
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
32 """
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
33 pass
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
34
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
35 def test(self):
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
36 """
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
37 Code whose execution time will be measured
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
38 """
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
39 pass
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
40
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
41 def afterEach(self):
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
42 """
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
43 Code to execute after the execution of each repetition of a test.
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
44 This code is not included in timings
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
45 """
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
46 pass
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
47
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
48 def afterAll(self):
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
49 """
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
50 Code to execute after the execution of all repetitions of a test.
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
51 This code is not included in timings
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
52 """
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
53 pass
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
54
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
55 def run(self) -> TestResult:
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
56 result = TestResult(self.name)
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
57
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
58 self.beforeAll()
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
59
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
60 for i in range(0, self.repeatCount):
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
61 self.beforeEach()
200
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
62
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
63 try:
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
64 with open('/proc/sys/vm/drop_caches', 'w') as f:
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
65 f.write('3')
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
66 except:
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
67 print('Please run this script as root to be able to flush the disk cache, and have reproducible runtimes')
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
68
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
69 startTime = time.time()
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
70 self.test()
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
71 endTime = time.time()
161
27b3b0df5f90 2 upload tests
am@osimis.io
parents: 158
diff changeset
72
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
73 self.afterEach()
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
74
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
75 result.add((endTime - startTime) * 1000)
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
76
158
df1f9946571c perfs db continued: tests working with tiny DBs on all setup but the sqliteplugin
am@osimis.io
parents: 156
diff changeset
77 self.afterAll()
156
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
78 result.compute()
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
79 return result
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
80
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
81 def __str__(self):
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
82 return "{name:<40}: {avg:>8.2f} ms {min:>8.2f} ms {max:>8.2f} ms".format(
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
83 name=self.name,
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
84 avg = self.averageTimeInMs,
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
85 min=self.minTimeInMs,
f1a75985caa8 first Db test framework - work in progress
am@osimis.io
parents:
diff changeset
86 max=self.maxTimeInMs
200
f42c10dab862 drop cache
Sebastien Jodogne <s.jodogne@gmail.com>
parents: 164
diff changeset
87 )