Bug 217 - Folder indexer crashing when in the folder there's a 7GB file
Summary: Folder indexer crashing when in the folder there's a 7GB file
Status: CONFIRMED
Alias: None
Product: Orthanc
Classification: Unclassified
Component: Plugin - Other (show other bugs)
Version: unspecified
Hardware: Macintosh Mac OS
: --- normal
Assignee: telari7455
URL:
Depends on:
Blocks:
 
Reported: 2023-08-29 13:03 CEST by telari7455
Modified: 2023-12-20 12:17 CET (History)
1 user (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description telari7455 2023-08-29 13:03:29 CEST 
I have a folder with a single 7GB file that causes the indexer plugin to crash. The 7GB file is not a DICOM file so presumably it should not crash. Note that the size is about the size of my RAM allocated.
Comment 1 telari7455 2023-08-29 13:07:52 CEST 
Latest indexer cloned with "hg clone <indexer mercurial link>". Orthanc 12.2
Comment 2 telari7455 2023-08-29 13:27:29 CEST 
I experimentally determined that the exact way of crashing is that if the file size is greater than RAM size PLUS the swap size then on Ubuntu the exact error message is just "Killed". Which means that somewhere there is "malloc(entire_file_size)" which then crashes
Comment 3 telari7455 2023-08-29 13:30:32 CEST 
The culprit

https://orthanc.uclouvain.be/hg/orthanc-indexer/file/4bde2c2dc03c/Sources/Plugin.cpp#l102

I think it'd be nice to have it in the TODO to use mmap or something else to do this check without loading the whole file
Comment 4 telari7455 2023-08-29 19:54:51 CEST 
I self-assign this
Comment 5 Sébastien Jodogne 2023-12-20 12:17:48 CET 
For reference, this issue is related to the following discussion on Discourse:
https://discourse.orthanc-server.org/t/patch-for-orthanc-indexer-plugin-crashing-on-big-non-dicom-files/3849/7