In supporting asynchronous writes, I added an optimization where a file that is newly created by a user always trusts its cache.

Consider the scenario where a user creates a file and writes bytes [1000, 1024]. Bytes in [0, 999] form a hole. Normally, reading any bytes in this hole would fall back to the log since they aren't cached. However, with the optimization, the read will be satisfied with zeros for a newly created file, since the cache is guaranteed to be up to date.

However, when we remove this optimization, then it is possible for a read in the range [0, 999] to reach the GDP concurrently with the request for write bytes [1000, 1024]. This isn't incorrect since the latest log entry technically isn't needed to satisfy the request, but I've observed the GDP providing a "ghost record" of length 0 in place of the one that is about to be appended. This is a bug that could inconvenience us later and that needs to be tracked down.

File creation behavior is weird. It sometimes succeeds but usually this requires running stuff more than once. Directory creation fails completely.

When adding new entries to a directory, sometimes existing entries get overwritten. At other times, entries remain after unlinking.

To reproduce do something like this

touch a
echo "hello" >> a
rm a
touch a
cat a

Currently we read the log backwards using negative record numbers. If someone appends to the log during a read this could cause an inconsistent result.

Logs should automatically be created. They can have random GUIDs since the users never need to interact with their GUIDs.

Everything should work even if you don't run GDPFS with the -f and -s flags. This means we need to audit our code base and add locks to prevent any potential race conditions.