Bug 21 - DICOM files missing after uploading with Firefox in Orthanc Explorer
Summary: DICOM files missing after uploading with Firefox in Orthanc Explorer
Status: RESOLVED FIXED
Alias: None
Product: Orthanc
Classification: Unclassified
Component: Orthanc Explorer (show other bugs)
Version: unspecified
Hardware: All All
: --- normal
Assignee: Sébastien Jodogne
URL:
Depends on:
Blocks:
 
Reported: 2020-06-29 15:11 CEST by Sébastien Jodogne
Modified: 2021-01-19 12:05 CET (History)
0 users

See Also:


Attachments
3848876971-orthanc_21.PNG (259.46 KB, image/png)
2020-06-29 15:23 CEST, Sébastien Jodogne
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Sébastien Jodogne 2020-06-29 15:11:38 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2016-07-15.09:47:19]

**What steps will reproduce the problem?**

Open Mozilla Firefox. Upload the files from a study with several dozens of DICOM instances, with the "Upload" tab of Orthanc Explorer. I used the TOUTATIX sample from OsiriX (that contains 2028 files for about 200MB).

**What is the expected output? What do you see instead?**

The transfer is tagged as successful, but the statistics of the Orthanc core report only 155 instances:


```
#!text

$ curl http://localhost:8042/patients/b2160db0-765812e8-d65568b3-3748b5c0-0b86bea2/statistics
{
   "CountInstances" : 155,
   "CountSeries" : 1,
   "CountStudies" : 1,
   "DiskSize" : "17527506",
   "DiskSizeMB" : 16,
   "UncompressedSize" : "17527506",
   "UncompressedSizeMB" : 16
}

```


**What version of the product are you using? On what operating system?**

* Orthanc 1.1.0 under Linux
* Mozilla Firefox 47.0

This problem does **not** seem to affect Chromium and Google Chrome.

**Temporary solution**

* Use Chromium or Google Chrome, or
* Use the ImportDicomFiles.py sample script.
Comment 1 Sébastien Jodogne 2020-06-29 15:19:23 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2016-12-03.11:20:51]

Just added a warning message about this issue. This message will be visible as of Orthanc 1.2.0, until the issue is solved.
https://hg.orthanc-server.com/orthanc/changeset/79d259b86aa9978eca34d7d1c688217deff6c27f
Comment 2 Sébastien Jodogne 2020-06-29 15:23:03 CEST
[BitBucket user: Benjamin Golinvaux]
[BitBucket date: 2018-12-21.07:33:03]

I am seeing this with Chrome (Version 71.0.3578.98 (Official Build) (64-bit)) with Orthanc 1.3.0 running on Synology DS918+

Is that only supposed to occur with Chrom(e|ium) ?

A few directions to debug the issue maybe? :) 

![orthanc_21.PNG](3848876971-orthanc_21.PNG)
Comment 3 Sébastien Jodogne 2020-06-29 15:23:06 CEST
Created attachment 101 [details]
3848876971-orthanc_21.PNG
Comment 4 Sébastien Jodogne 2020-06-29 15:23:07 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-12-21.08:38:45]

When you say "seeing this"; do you mean you see the popup or you observed that some files are missing after the upload.
Actually, the popup shows whatever your browser.  

So right now, next step would be to implement a browser detection in the explorer js code to show the popup only to firefox users (That's probably as fast as writing this phrase).
Second step would be to re-write the file upload mechanisms in the explorer js code too.  I think that last time Seb tried that he entered a JS dependency hell and gave up.
Comment 5 Sébastien Jodogne 2020-06-29 15:23:08 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2018-12-21.08:43:43]

This issue normally only arises only on Firefox, but this assumption has not been thoroughly tested, so the popup warning is displayed on any browser (including Chrome/Chromium even if it should not be affected).

Solving this issue would require a refactoring of Orthanc Explorer (update of the "jquery-file-upload" library, that would in turn need an update of jQuery Mobile, which would eventually lead to very deep modifications because backward compatibility is not ensured in the framework). As an alternative solution, we could separate the upload page from the main jQuery Mobile application, enabling us to use a different framework.

Check also the remarks about "Orthanc Explorer 2": http://book.orthanc-server.com/faq/improving-interface.html
Comment 6 Sébastien Jodogne 2020-06-29 15:23:09 CEST
[BitBucket user: Benjamin Golinvaux]
[BitBucket date: 2018-12-21.20:17:51]

For some reason, I thought there **was** a browser detection mechanism and that it was bogus. Probably because I underestimated  the consequences of implementing a browser detection mechanism, as explained by Seb.

So, there is no other issue than the dialog box showing up under Chrome. Sorry for the trouble!

And thanks for the technical explanation. Having a look at the code before actually creating issues would be a sensible move on my end :tongue:
Comment 7 Sébastien Jodogne 2020-06-29 15:23:10 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-12-26.16:56:15]

Just created a branch and updated jquery & co: https://hg.orthanc-server.com/orthanc/changeset/5713952f60c0b6e57d31f8bbc5d5a33d07204fb1

My problem right now is that on my Ubuntu machine, drag and drop from Nautilus to Firefox hardly ever work (files are not listed).  This is, I think, totally independent from Orthanc because I observe it with other sites too -> installing a new VM for that.

On windows, with Firefox, I could not reproduce the problem with 1.5.1 so I can not tell if this upgrade helped.  -> to be continued.
Comment 8 Sébastien Jodogne 2020-06-29 15:23:11 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2018-12-26.17:31:43]

Wow... impressive changeset... For reference, jQuery Upload could be upgraded from v5.32.0 to [v9.28.0](https://github.com/blueimp/jQuery-File-Upload/releases), which could fix your observed issue.

Furthermore, the [jQuery core](https://en.wikipedia.org/wiki/JQuery#Release_history) could be upgraded from v1.9.1 to either v1.12.4 or v2.2.4. Note that jQuery 3.x is known to be [incompatible](https://github.com/jquery/jquery-mobile/issues/8546#issuecomment-271163317) with jQuery Mobile <= 1.4.5.
Comment 9 Sébastien Jodogne 2020-06-29 15:23:12 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-12-27.09:19:16]

Actually, I've upgraded jQuery core to 1.9.1 because that was the one known to be compatible with jQuery Mobile 1.4.5.
For the file upload, I've upgraded to the released a bit after jQuery Core 1.9.1.

Indeed, we can try upgrading the file upload to a yet newer release but first, I need to be able to reproduce the issue.
Comment 10 Sébastien Jodogne 2020-06-29 15:23:13 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-12-27.15:23:56]

Just created an independent upload page in plain HTML/JS: https://hg.orthanc-server.com/orthanc/changeset/e4f146b7d2d9903354f5c1feda6059e9c60ba168

If using the "browse" button, all files are always uploaded even when selecting 150 of them.
If using drag and drop, with a few files, it usually works but, as soon as you try to drop 150files, it takes firefox around 10 seconds to trigger the "dragover" and "drop" events and not all files that are dropped are listed in the events.  

This has been tested on Ubuntu 18.10 with Firefox 64.0.  I can not reproduce it on Windows.  

To me, the problem lies in the interaction between the Ubuntu file manager and Firefox.
On Ubuntu, I've tried uploading the same files to WeTransfer and Firefox is equally unresponsive when I drop 150 files.
So it looks like the only sure way to upload files via Firefox on Ubuntu is using the "browse" button (unfortunately, it's also incredibly slow and sometimes need 10 seconds to open the file browser dialog).

So it looks like upgrading jQuery ... was not necessary at all since the plain HTML/JS fails the same way.  I would recommend that:
- we forbid drag-drop for Firefox
- we add a "browse" button for all browsers
- instead of showing the issue #21 popup, on Firefox, we display a jQuery warning to tell users that drag-drop is not available

to be discussed ...
Comment 11 Sébastien Jodogne 2020-06-29 15:23:15 CEST
[BitBucket user: Sébastien Jodogne]
[BitBucket date: 2018-12-28.08:03:15]

Maybe linked to this issue? https://askubuntu.com/a/981426

On Ubuntu 14.04 with Firefox 60.0.1, using your new "upload.html" page, uploading 1000 DICOM files with the "Browse..." button is fast and perfectly usable.

In either case, I am not in favor of blocking drag-and-drop depending on the browser, as this issue is visibly independent of Orthanc, and could be fixed in future releases of Ubuntu and Firefox. Furthermore, maybe this issue is very specific to Ubuntu/Nautilus: KDE-based distributions might be unaffected. Finally, Firefox on Windows is apparently not affected, so disabling drag-and-drop on any Firefox is definitely not desirable.

If this popup warning is too intrusive, I would simply recommend to add a textual warning at the top of the upload form, possibly only displayed if the browser is Firefox.
Comment 12 Sébastien Jodogne 2020-06-29 15:23:17 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2018-12-28.08:49:51]

concerning https://askubuntu.com/a/981426, I had seen it and tried the workaround without any luck.

I agree with you: we shouldn't lock drag-and-drop so the solution is:

- on Firefox, add a textual warning
- add a Browse button for everyone
Comment 13 Sébastien Jodogne 2020-06-29 15:23:29 CEST
[BitBucket user: Alain Mazy]
[BitBucket date: 2019-01-10.09:53:21]

this has been done in this commit: https://hg.orthanc-server.com/orthanc/changeset/9b18c8d4d45981d69c9eff8e8e4dbfb44762efe8

Since the bug is not in Orthanc anymore but in Nautilus/Firefox interaction, I'll close the issue.
Comment 14 Sébastien Jodogne 2021-01-19 12:05:39 CET
This issue might have been solved for good by the following changeset, that will be part of Orthanc 1.9.0:
https://hg.orthanc-server.com/orthanc/rev/f20a7655fb1c