OOM when load_file for special folder about pugixml HOT 5 CLOSED

For checking it and getting file size you can use something like this:

#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>

int get_file_size(const char* filename, size_t& out_result) {
   struct stat st;
   if(stat(filename, &st) != 0) {
      // return error
   }
   if (!S_ISREG(st.st_mode)) {
      // return error
   }
   out_result = st.st_size;
   // return ok
}

Or use Filesystem library (since C++17) for getting size: std::filesystem::path with std::filesystem::file_size

In case of using it with dirs it throws exception:

terminate called after throwing an instance of 'std::filesystem::__cxx11::filesystem_error'
  what():  filesystem error: cannot get file size: Is a directory [/]

from pugixml.

The bug was found with Futag

from pugixml.

Thanks! This should not have production impact, as the code correctly handles the OOM, but is certainly inconvenient for fuzzing. ftell here returns the maximum signed integer of the relevant type instead of returning a negative number for some reason.

std::filesystem can't be used in pugixml as it doesn't use STL except for STL-specific API helpers that can be disabled, and carries compatibility burden on systems like macOS. It is probably better to solve this by explicitly checking the size; stat could work but has some other issues and an extra syscall per file open is not optimal for some workloads. That said, maybe stat is actually fine; I'll look into this further.

from pugixml.

Ah, looks like I hit this before :) 7664bbf

from pugixml.

Looks like fseek/ftell use fstatat under the hood, so stat is actually more efficient (only matters for small files, but right now it takes us 7 syscalls to read a file, and with stat it only takes 5; one of these is done by fread and can be removed by switching to open/read or by disabling buffering via setvbuf).

from pugixml.