Comments (4)
thank you for explanation. it makes sense. is it documented in some places?
It's not documented very well currently. Eventually it should be. But with changes in the works it's been hard to prioritize soon-to-be-deprecated behavior when that time could be spent getting to the better behavior sooner. Some hard decisions time-management-wise here.
does directory work is a similar way?
Ah this is a good question.
It would be quite nice if directories worked this way, since it would solve the modify-while-readdir issue you mentioned. Unfortunately because of technical limitations littlefs can't really snapshot directories. Files in littlefs are copy-on-write, but directories are not***.
The modify-while-readdir issue is a really interesting one, and I've read it has caused quite a bit of headache for other filesystems.
If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified.
It doesn't really seem to comment on if modifying a directory is allowed to effect readdir in other ways. My understanding is the expectation is that readdir should at minimum return all files in the directory at the time of
opendir
. littlefs should be doing this, though it may not have the best test coverage.At the very least we do have tests over some common modify-while-readdir patterns: test_dirs_recursive_remove
***:
Ok, so technically you could snapshot a directory by marking its mdirs as "frozen", and forcing any modifications to the mdir to copy first. This may be interesting in the future for full filesystem-level snapshots, which is a feature in other copy-on-write filesystems, but not currently possible in littlefs. But this would be too heavy-handed/expensive for readdir operations.
when i was working on some filesystems decades ago, the access pattern of cvs caused a lot of trouble
wrt readdir. i don't remember details though.
hopefully there are less applications assuming UFS-like behaviors these days.
(snapshoting a directory might make the situation worse for those apps.)
from littlefs.
Ah. Yes, this is well defined, though the behavior may be unintuitive coming from POSIX filesystems.
The way this works right now is that every open file handle gets a "snapshot" of the file.
So if you open a file twice, say with lfs_file_t A and B, and then wrote to B, A would still contain the original contents.
If you open a file twice and write to both A and B, the last handle to be synced or closed will be what ends up on disk.
The reason for this behavior is because it derives naturally from files in littlefs being copy-on-write. But users have noted it's unintuitive and initially confusing.
There are some changes in the works to make littlefs behave a bit more like POSIX here (by broadcasting file changes to other open file handles on sync/close), but these changes will need to wait for a major release with API changes, so it will take some time before they land.
from littlefs.
thank you for explanation. it makes sense. is it documented in some places?
does directory work is a similar way?
i vaguely remember that some traditional applications, which modify a directory (create/remove entries) while readdir'ing it, have some assumptions on what readdir contents can contain/can not contain.
from littlefs.
thank you for explanation. it makes sense. is it documented in some places?
It's not documented very well currently. Eventually it should be. But with changes in the works it's been hard to prioritize soon-to-be-deprecated behavior when that time could be spent getting to the better behavior sooner. Some hard decisions time-management-wise here.
does directory work is a similar way?
Ah this is a good question.
It would be quite nice if directories worked this way, since it would solve the modify-while-readdir issue you mentioned. Unfortunately because of technical limitations littlefs can't really snapshot directories. Files in littlefs are copy-on-write, but directories are not***.
The modify-while-readdir issue is a really interesting one, and I've read it has caused quite a bit of headache for other filesystems.
If a file is removed from or added to the directory after the most recent call to opendir() or rewinddir(), whether a subsequent call to readdir() returns an entry for that file is unspecified.
It doesn't really seem to comment on if modifying a directory is allowed to effect readdir in other ways. My understanding is the expectation is that readdir should at minimum return all files in the directory at the time of opendir
. littlefs should be doing this, though it may not have the best test coverage.
At the very least we do have tests over some common modify-while-readdir patterns: test_dirs_recursive_remove
***:
Ok, so technically you could snapshot a directory by marking its mdirs as "frozen", and forcing any modifications to the mdir to copy first. This may be interesting in the future for full filesystem-level snapshots, which is a feature in other copy-on-write filesystems, but not currently possible in littlefs. But this would be too heavy-handed/expensive for readdir operations.
from littlefs.
Related Issues (20)
- Block count defined twice HOT 1
- What is the "bd" folder? HOT 1
- why t1_buf is sample of t2_buf at sometime?
- Expanding superblock deletes File HOT 7
- '-fcallgraph-info=su' breaking builds on some platforms HOT 9
- All the data is lost after update from disk 2.0 to disk 2.1 HOT 3
- v2.7 : long delay in open/close for spi nor HOT 8
- Data Flash values after erase is "undefined" - Would LittleFS work? HOT 1
- Do we really need directory structures in meta data layer for simple structure variable read/write? HOT 1
- running into RAM limitations for freertos tasks HOT 1
- LittleFS Display Directory Content HOT 3
- File creation with full path in the name HOT 2
- Query: failing to find files after a soak test HOT 8
- Usage of lfs_dir_seek HOT 2
- File system corruption when creating directories HOT 2
- used and free memory HOT 2
- issue when append data to a file HOT 2
- Wrong printf specifiers
- examples about use BDs provided (in particular lfs_filebd)
- Where do we find the version in the code
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from littlefs.