sboudrias / mem-fs Goto Github PK

View Code? Open in Web Editor NEW

123.0 10.0 14.0 243 KB

Simple in-memory vinyl file store.

License: MIT License

TypeScript 100.00%

mem-fs's Introduction

mem-fs

Simple in-memory vinyl file store.

Usage

Loading a file

You access a file using store#get() method. If the file is in memory, it will be used. Otherwise, we'll load the file from the file-system.

import { create } from 'mem-fs';

const store = create();
store.get('/test/file.txt');

When trying to load a file we cannot read from disk, an empty Vinyl file will be returned. The contents of this file will be set to null.

Trying to get a directory or any invalid files will also return an empty Vinyl file pointer.

Adding/updating a file

You update file references by using store#add() method. This method take a vinyl file object as parameter.

import File from 'vinyl';
import { create } from 'mem-fs';

const coffeeFile = new File({
  cwd: '/',
  base: '/test/',
  path: '/test/file.coffee',
  contents: new Buffer('test = 123'),
});

const store = create();
store.add(coffeeFile);

Iterating over the file system

Using store#each(cb(file, index)), you can iterate over every file stored in the file system.

Get all files

Using store#all(), you can get every file stored in the file system.

Check existence in the file system

Using store#existsInMemory(), you can check if the file already exists in the file system without loading it from disk.

Stream every file stored in the file system

Using store#stream(), you can create a stream with every file stored in the file system.

Pass stored files through a pipeline

store#pipeline() generates a new map with yielded files in transforms. If no transform is passed, files references are updated.

mem-fs's People

Contributors

Stargazers

Watchers

Forkers

nvdnkpr vinyljs olsonpm liangklfang rodrigodmd jakub300 sallar com-xubo mshima askvortsov1 jimmywarting bonniss

mem-fs's Issues

intent of 'change' event?

In lines 39-40 on index.js

  Store.prototype.add = function (file) {
    store[file.path] = file;
    this.emit('change');
    return this;
  };

I'm assuming this.emit('change') is meant to indicate the internal store object has been modified.

If this is true, then Store.prototype.get should also emit a change event due to the load function modifying the internal store.

I created a few branches:

If it's not a bug, please explain the intent so I understand the code better.

Thanks much

TypeScript Declaration is broken

The current type-declaration require esModuleInterop top be enabled (which is disabled by default - the decision whether or not to use esmoduleInterop should be left to the user consuming this module).

#stream() to provide an object stream out of vinyl files

New utility method to create an object stream out of every loaded vinyl files.

Incorrect use of Vinyl#base

I believe that file.base is being set incorrectly. In the source:

file = new File({
    cwd: process.cwd(),
    base: path.basename(filepath),
    ...

According to the documentation, file.base is intended to be a directory and file.basename is the file's basename. Setting file.base to the basename of the file results in an incorrect value being returned from the file.relative getter.

I ran into this issue while attempting to create a utility that would override mem-fs-editor's .commit() method to generate a zip file using gulp-zip.

Consider adding typings

I've added TypeScript typings for mem-fs in DefinitelyTyped/DefinitelyTyped#24434

Would you consider providing typings for this module? Alternatively if you don't want to do that, could the information about the typings be provided in the project README.

I'm happy to contribute PRs with these changes, but didn't want to make a PR for something of no interest to the project.

Aaaaa

Alternative solution

I find vinyl files to be abnormal files. it's not so web-ish.
NodeJS now also ships with Blob built in (and Files too, ...in a roundabout way)
this blobs have .stream(), .arrayBuffer(), .text(), .slice(), name & type something that can easily be shared with consistently with deno, browsers and Deno.land without needing any custom file wrappers.

if you are using fetch-blob then you can get files/blob's that are just references point to some location on the disc (just like vinyl files) You can also construct own in memory Files.
this blobs are immutable, re-readable and also slice:able (and when you slice a blob then you are just creating a new ref point with a new start/end offset

And for the storage you could just use Map, WeakMap, Set, or WeakSet. and maybe just use the path as the key.

so all in all:

import { File, fileFromSync, Blob } from 'fetch-blob/from.js'

var store = new Map();

var coffeeFile = new File(['test = 123'], 'test/file.coffee', { type, lastModified });
var pkg = fileFromSync('./package.json')

// will not allocate much memory, will just be 2 references point to other  files to read from
var both = new File([coffeeFile, pkg], '') 

store.set(coffeeFile.name, coffeeFile)
store.set(pkg.name, pkg)

// Map also has a set/delete/has/forEach method...

// to stream every file in the store: use something like:
new Blob(store.values()).stream()

You could also extend Map to override the get method and fallback to using the fs if the file isn't stored

Add a Method to Check If a File Is Loaded

Currently there is no way to check if a file is loaded into the memory store. The get method only works to check for files that don't exists on disk or in mem-fs.

I propose adding a method called exists or loaded that returns a boolean indicating if the file is currently present in mem-fs.

Consider using `stream` instead of `through`.

As in, just the stream module, since mem-fs targets node>=4 and stream is standardized for that. duplexer3 did something similar, I think.

Document states

Need to consider documenting state of a file and making it more obvious.

I probably want to namespace state: file['mem-fs:state']. State should tell if the file is an empty pointer (no file existed) or if it is a file pointer (file was found on disk).

Currently there's this vague "state" property that is assigned. This need to be normalized.

Stricter #get() operation

Right now #get() will return file pointer if we try to get a directory. We should be stricter about that and throw if what we try to get is not a file.

Same thing about permissions issue. We shouldn't have a catch all clause - ideally we'd only catch read errors.