Bug 32

Summary: HttpServer does not support multiple HTTP requests in the same TCP stream
Product: Orthanc Reporter: Sébastien Jodogne <s.jodogne>
Component: Orthanc CoreAssignee: Sébastien Jodogne <s.jodogne>
Status: RESOLVED FIXED    
Severity: normal    
Priority: ---    
Version: unspecified   
Hardware: All   
OS: All   
Attachments: 2729995299-Capture.PNG

Description Sébastien Jodogne 2020-06-29 15:11:57 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2017-02-08.11:29:21]

When accessing the Orthanc API through urllib3 (python), Orthanc sometimes closes the connection.  This happens when multiple requests happen in the same TCP stream: 

![Capture.PNG](2729995299-Capture.PNG)

On sever side:

- Setting KeepAlive: true in the config file does not help

On client side:

- Setting 'connection: close' in the HTTP request does not help

Workaround on client-side:

- Explicitly closing the connection after each HTTP request


We should probably replace Mongoose by CivetWeb but we could not make sure this is fixed in CivetWeb
Comment 1 Sébastien Jodogne 2020-06-29 15:11:59 CEST
Created attachment 1 [details]
2729995299-Capture.PNG
Comment 2 Sébastien Jodogne 2020-06-29 15:21:12 CEST
[BitBucket user: Thibault Nélis]
[BitBucket date: 2018-02-19.14:00:34]

RE CivetWeb switch; at least one distro reconsidering upgrading Mongoose:

> When CESANTA bought the software rights from its original author, back in 2013 they had introduced a twisted LICENCE as well some issues to retrieve the source code. I've avoided updating it initially due to that, as it's better to have software working despite old, than bring in fancy new stuff with legal disputes.
Seems that they're clearly stating on the github, now, that they're dual licenced and the redistribution can be done under GPL-V2 umbrella.

https://bugzilla.redhat.com/show_bug.cgi?id=1354468#c5

Depending on how drop-in (or not) CivetWeb is, staying with Mongoose now looks to be a reasonable option too.
Comment 3 Sébastien Jodogne 2020-06-29 15:21:12 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2018-02-19.15:04:54]

Since Orthanc 1.3.1, the `-DENABLE_CIVETWEB=ON` option to CMake can be used to replace Mongoose by CivetWeb. It would be worth giving a try to see whether this issue is solved by CivetWeb.
Comment 4 Sébastien Jodogne 2020-06-29 15:21:25 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-03-07.14:02:57]

Here's some python3 code to reproduce (note: don't configure Orthanc in verbose mode since outputting to stdout might slow down the whole process and hide the problem)

```
import requests
http = requests.Session()
while (True):
    if http.get("http://127.0.0.1:8042/studies").status_code != 200:
        print("error")
```
After 10-20 seconds, you should actually get a 'ConnectionError' exception thrown (and not catched)
Comment 5 Sébastien Jodogne 2020-06-29 15:23:48 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2019-01-15.10:36:37]

just made a few tests:

- Windows 10 with Orthanc 1.5.0 running as a service -> the python script crashes after 2-3 seconds
- Windows 10 with Orthanc mainline running in debug in Visual Studio -> the python script crashes after 2-3 seconds
- Ubuntu 18.10, Orthanc 1.3.2 LSB -> NO crash after 5 minutes
- Ubuntu 18.10, Orthanc mainline built locally (not LSB) -> the python script crashes after 2-3 minutes
- Ubunut 18.10, Orthanc running in Docker -> the python script crashes after 2-3 seconds
Comment 6 Sébastien Jodogne 2020-06-29 15:23:49 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2019-01-15.11:08:50]

Note that, in python, we get the same error as the one documented here: https://github.com/requests/requests/issues/4664

However, we've observed the issue with Java client too ... (needs sample code)
Comment 7 Sébastien Jodogne 2020-06-29 15:23:49 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2019-01-15.11:30:15]

FYI, tried to reproduce it on Windows with a .Net Core HTTP client.  No error after a few minutes while it crashed after 2-3 secs with a python client.  So it seems that the issue can indeed be in the client

For the record, here's the test code:

```
using System;
using System.Net.Http;
using System.Threading.Tasks;

namespace OrthancIssue32
{
  class Program
  {
    private static readonly HttpClient client = new HttpClient();

    static void Main(string[] args)
    {
      Console.WriteLine(DateTime.Now.ToString());
      Console.WriteLine("Start sending requests!");

      try
      {
        while (true)
        {
          GetStudiesFromOrthanc().Wait();
        }
      }
      catch(Exception ex)
      {
        Console.WriteLine(DateTime.Now.ToString());
        Console.WriteLine(ex.Message);
        Console.WriteLine(ex.StackTrace);
      }
    }

    private static async Task GetStudiesFromOrthanc()
    {
      var stringTask = client.GetStringAsync("http://localhost:8044/studies");
      var msg = await stringTask;
    }
  }
}

```
Comment 8 Sébastien Jodogne 2020-06-29 15:23:50 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2019-01-15.12:07:31]

Another attempt to reproduce the issue in C++, but without success: https://hg.orthanc-server.com/orthanc/changeset/df4f977c2f882228d0fdb186c3ddf17682248f4b
Comment 9 Sébastien Jodogne 2020-06-29 15:23:52 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2019-01-15.17:49:21]

Just managed to systematically reproduce the issue using Java, at least on Ubuntu 16.04. Code source lies in folder `Resources/Testing/Issue32/Java/`:  https://hg.orthanc-server.com/orthanc/changeset/58ea4ef84c924533eeb1f8dd27fa11140eb9f692
Comment 10 Sébastien Jodogne 2020-06-29 15:23:54 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2019-01-16.08:13:20]

The issue disappears after setting the option `KeepAlive` to `true` in the configuration file of Orthanc (that is `false` by default since its introduction in Orthanc 0.8.3). It it thus related to HTTP keep-alive (aka. [HTTP persistent connections](https://en.wikipedia.org/wiki/HTTP_persistent_connection)). I now commit a change to make this option `true` by default.
Comment 11 Sébastien Jodogne 2020-06-29 15:23:55 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2019-01-16.08:52:12]

fix issue #32 (HTTP keep-alive), make CivetWeb the default HTTP server

→ https://hg.orthanc-server.com/orthanc/changeset/b1787ba94eef