[BitBucket user: Alain Mazy] [BitBucket date: 2019-01-31.11:15:36] Disclaimer: I have not reproduced it myself but I saw it at one of our user. - This happened with the Debian packages and Orthanc 1.4.2. - Orthanc is running as a service as the `orthanc` user - Disks had been filled 100%. The user had removed the entire Orthanc Storage and tried to restart Orthanc. - Orthanc fails to start without any meaningful error message: ``` ... last message before this was the one about Loading Dicom dictionaries ... E0130 15:52:41.839830 main.cpp:1331] Uncaught exception, stopping now: [Unable to create a subdirectory or a file in the file storage] (code 2001) W0130 15:52:41.840499 main.cpp:1364] Orthanc has stopped ``` After a long investigation and finaly enabling `strace`, we discovered that Orthanc was trying to create a file in /tmp but, since the /tmp partition was full, it failed. Note that the /tmp partition was around 350MB and was full with previous temporary Orthanc files (generated from previous /archive jobs). I have no idea what Orthanc was trying to create in /tmp at that time of the startup. I think we should: - display a verbose log showing what Orthanc was doing at that time with, a possible recommandation to increase /tmp size - not count on /tmp being deleted when the computer restarts (this one never restarts). So, remaining temporary files of interrupted/failed archive jobs shall be removed somehow (maybe that is what Orthanc was trying to do at startup). - eventually be able to define a large "temporary storage" on a HDD in the configuration file. Now that there are 4GB studies out there, using RAM as the temporary storage might be too short .... Note that with Orthanc running as `root`, the problem did not happen (maybe root had more privileges on /tmp and the ability to overwrite existing data or ...)
[BitBucket user: Alain Mazy] [BitBucket date: 2019-01-31.11:16:50] @sjodogne unless you've a clear idea of the problem and know how to solve it, I can try to make a setup to reproduce it. Let's talk about it tomorrow.
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2019-01-31.11:37:52] If Orthanc didn't stop cleanly, it can't remove its temporary files (I have just validated that the integration tests don't let any temporary file if Orthanc is stopped in a clean way). On the other hand, it would be a very bad idea to do an automatic cleanup of files in `/tmp` at the Orthanc startup, as this might remove files that are still in use (e.g. if two instances of Orthanc are running on the computer). I think the only possibility is indeed to add a `TemporaryDirectory` configuration option that would defaults to `boost::filesystem::temp_directory_path()` (e.g. `/tmp/` on UNIX-like systems). System administrators would have the responsibility of removing the content of this folder as the first step of their script that launches the Orthanc service. The relevant file in Orthanc is: https://hg.orthanc-server.com/orthanc/file/default/Core/TemporaryFile.cpp Note that *"maybe that is what Orthanc was trying to do at startup"* is a wrong assumption: Orthanc only removes the temporary files it has created during its own execution. Regarding what Orthanc was trying to created in `/tmp/` at the startup, I have absolutely no idea. It might indeed be interested to check this, by combining this new `TemporaryDirectory` option with the `tmpfs` setup described in issue #103.
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2019-01-31.13:00:06] Just discovered the culprit. During the startup of Orthanc, if built with "-DSTANDALONE_BUILD=ON", Orthanc creates a temporary file that contains the DICOM dictionary. This is because DCMTK 3.6.2 does not provide a primitive to read the dictionary from a memory buffer. Check out function "LoadEmbeddedDictionary()" in "FromDcmtkBridge.cpp": https://hg.orthanc-server.com/orthanc/file/Orthanc-1.5.3/Core/DicomParsing/FromDcmtkBridge.cpp#L113 That being said, this file is small and immediately removed, so it makes perfect sense to store it into system-wide `/tmp/` folder. The folder `TemporaryDirectory` should only be used for large temporary files.
[BitBucket user: Sébastien Jodogne] [BitBucket date: 2019-01-31.14:33:33] Fix issue #130 (Orthanc failed to start when /tmp partition was full) → https://hg.orthanc-server.com/orthanc/changeset/6fd38327e777