gliviu / dir-compare Goto Github PK
View Code? Open in Web Editor NEWNode JS directory compare
Home Page: https://www.npmjs.com/package/dir-compare
License: Other
Node JS directory compare
Home Page: https://www.npmjs.com/package/dir-compare
License: Other
I suggest to Travis CI to automatically run your tests on OS X
and Linux
and AppVeyor to run your tests x86 and x64 Windows machines automatically.
mkdir -p tmp/test1.good
mkdir -p tmp/test2.good
mkdir -p tmp/test3.bad
touch tmp/test1.good/test.txt
touch tmp/test2.good/test.txt
touch tmp/test3.bad/test.txt
sudo chown root:root tmp/test3.bad/ -R
sudo chmod og= tmp/test3.bad/
cd tmp/
npm install dir-compare-cli
./node_modules/.bin/dircompare test1.good test2.good
Entries are identical
total: 1, equal: 1, distinct: 0, only left: 0, only right: 0
./node_modules/.bin/dircompare test1.good test3.bad
Error occurred: Error: EACCES: permission denied, scandir '/home/joseph/tmp/test3.bad'
at Object.readdirSync (fs.js:981:3)
at getEntries (/home/joseph/tmp/node_modules/dir-compare-cli/node_modules/dir-compare/src/compareSync.js:18:26)
at compare (/home/joseph/tmp/node_modules/dir-compare-cli/node_modules/dir-compare/src/compareSync.js:33:20)
at Object.compareSync (/home/joseph/tmp/node_modules/dir-compare-cli/node_modules/dir-compare/src/index.js:27:5)
at /home/joseph/tmp/node_modules/dir-compare-cli/src/cli/dircompare.js:113:44
at new Promise ()
at run (/home/joseph/tmp/node_modules/dir-compare-cli/src/cli/dircompare.js:112:38)
at Object. (/home/joseph/tmp/node_modules/dir-compare-cli/src/cli/dircompare.js:147:1)
at Module._compile (internal/modules/cjs/loader.js:1075:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1096:10)
Copied from moshfeu/vscode-compare-folders#71 (comment)
If a symlink points to a file that doesn't exist, it should not break the diff.
It looks like symlinks are not handled well in general.
#16 inspired me to make a test of the following case:
$ tree -C /tmp/22 /tmp/11
/tmp/22
├── 1
├── 2
└── 3 -> 2
/tmp/11
├── 1
├── 2
└── 3 -> 1
6 directories, 0 files
dir-compare reported that everything is fine:
$ dircompare.js -c -lrd /tmp/11/ /tmp/22
Entries are identical
total: 0, equal: 0, distinct: 0, only left: 0, only right: 0
It's usually not a big deal, however, on the way of a potential implementation of #16 it might be worth to add a check if a synlink points to the same place (both relative and absolute). I don't know how it is implemented in the current version, but I think that dir-compare shouldn't follow the link - just check if a destination is the same.
buffer-equal uses node 0.11+ buffer.equals when available, buffertools require compilation (problematic on windows)
git diff
is able to detect when a file has been renamed, based on a "similarity index":
diff --git a/a/a.txt b/b/b.txt
similarity index 85%
rename from a/a.txt
rename to b/b.txt
index b8ee8a7..8054674 100644
--- a/a/a.txt
+++ b/b/b.txt
@@ -1,3 +1,3 @@
asdfdsfasdf
gsdfgsdfgdsfgsdfg
-asdf
\ No newline at end of file
+asdf1
\ No newline at end of file
currently dir-compare
treats all renames as a file "missing" on each side
Hi, I notice there are some breaking behavior changes when upgrading from 1.4 to 1.5:
compare()
function will throw instead of returning a rejected promise when the input path does not exist.Are these intentional?
Especially the second one, my guess is that it is a regression.
Thanks for this very useful library thou. 🌷
My use case is looking at whether we need to do git operations by comparing a pair of directories as current and to-be states. Recently ran into a problem where someone had created an empty directory in the to-be version, which caused dir-compare to (correctly) return that the directories were not the same, but then no git operations were needed (because git will ignore empty directories). Bottom line:
is there an option I'm missing that would allow me to tell dir-compare to ignore empty directories, and if not, is this something that would be worth me doing a PR for (vs what I'm going to do right now which is check the result sets after the fact)
i see you are capturing the date in the diff, why don't you have an option to compare those?
Hi,
thanks for the work on the library 🙇
I'm integrating it in a personal app using typescript and noticed that that the Difference
interface doesn't currently cover the actual result of the difference.
For example:
{
"path2": "/<base-path>/LUTs/ImpulZ Ultimate/ImpulZ Ultimate for Vision Color/29 ImpulZ - Sony FS700 Cine 4",
"relativePath": "/LUTs/ImpulZ Ultimate/ImpulZ Ultimate for Vision Color/29 ImpulZ - Sony FS700 Cine 4",
"name2": "Kodak Vision3 250D 5207 (NEG)_VS.cube",
"state": "right",
"type1": "missing",
"type2": "file",
"level": 4,
"size2": 137798,
"date2": "2014-05-30T16:54:26.000Z"
}
Doesn't contain name1
, size1
, date1
, equally for "state": "right"
, there isn't a properties called name2
, size2
, date2
.
I propose changing the typescript interface of all possible solutions to a union of a specific type of response based on state.
interface Difference {..}
to type Difference = DifferenceEqual | DifferenceLeft | DifferenceRight | DifferenceDistinct
As a user of the library, I had a problem to understand that type of result combinations would I get depending on the state.
I still haven't caught a case when would I get a "state": "distinct"
as left
& right
cover the "distinct" cases.
If using the only state
property is a valid way of splitting up the differences, I'm ready to help with a PR.
Going through the library code understood that state
:
left
- Entry is only present on the left sideright
- Entry is only present on the right sideequal
- Entry exists on both sides and depending on given options entries are considered "equal"distinct
- Entry exists on both sides and depending on given option entries aren't considered "equal"offending line: fileDescriptorQueue:43
fs.close(fd, callback); // <--------- callback is undefined here
Doesn't seem to affect functionality. I only noticed due to node v7's warning
(node:48849) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
To reproduce
git clone https://github.com/olsonpm/dir-compare-deprecation.git
cd dir-compare-deprecation
npm i
# must have node 7 installed
nvm use node
node index
# outputs
# done
# (node:48849) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
# (node:48849) DeprecationWarning: Calling an asynchronous function without callback is deprecated.
Since it doesn't affect functionality, I wasn't sure what the correct fix would be. If you want, I can look into a fix as well.
Thanks for the module - it helped me out a lot :)
As the title says, skipSubdirs
still compares directories themselves. Not sure what the intended functionality is, but I expected it to be ignored all together in the compare.
This extends gliviu/dir-compare-cli#1
.\compare
│ d_file2.txt
│ m_file1.txt
│ m_file3.txt
│
├───directory1
│ d_file2.txt
│ m_file1.txt
│ m_file3.txt
│
└───directory2
d_file2.txt
m_file1.txt
m_file3.txt
const diff = require("dir-compare")
const options = {
skipSubdirs: true,
}
const input = {
path1: "./compare/directory1",
path2: "./compare/",
}
diff
.compare(input.path1, input.path2, options)
.then((results) => {
results.diffSet.forEach((d) => {
const differenceString = `${d.relativePath}${d.name1} (${d.type1}): ${d.state} - ${d.name2} (${d.type2})`
console.log(differenceString)
})
})
.catch((error) => {
throw new Error(error)
})
> node .\test.js
\undefined (missing): right - directory1 (directory)
\undefined (missing): right - directory2 (directory)
\d_file2.txt (file): equal - d_file2.txt (file)
\m_file1.txt (file): equal - m_file1.txt (file)
\m_file3.txt (file): equal - m_file3.txt (file)
skipSubdirs: false
> node .\test.js
\undefined (missing): right - directory1 (directory)
\directory1undefined (missing): right - d_file2.txt (file)
\directory1undefined (missing): right - m_file1.txt (file)
\directory1undefined (missing): right - m_file3.txt (file)
\undefined (missing): right - directory2 (directory)
\directory2undefined (missing): right - d_file2.txt (file)
\directory2undefined (missing): right - m_file1.txt (file)
\directory2undefined (missing): right - m_file3.txt (file)
\d_file2.txt (file): equal - d_file2.txt (file)
\m_file1.txt (file): equal - m_file1.txt (file)
\m_file3.txt (file): equal - m_file3.txt (file)
edit- typo
I'm using this great library in my vscode extension.
There is a feature request to ignore files extension so the it will compare only by files name.
What do you think?
Hello
I have created some TypeScript type definitions so that I can use this library in a project. Would you be interested in a pull request for them?
Thanks for the great library 👍
I want to compare my mounted backup with the real directory to verify it's data integrity (that the backed up data are the same as the local). I started with the classic diff
, but it doesn't support **
in the exclusion patterns. dir-compare
on the other hand uses minimatch
which claims to do that. Unfortunately I wasn't able to make it works.
Simple case. I excluded the internal backup subdirectory (made by the problem itself) from external backup.
$ dircompare.js -c /home/foo/somepath/ /tmp/mounted-backup/
Entries are different
total: 326, equal: 26, distinct: 0, only left: 300, only right: 0
With -a
I see that:
[/users/xxx/backup/20191019_151830] tasks.xml->missing
I can add -x xxx
and it works, but that directory name could be reused in some other content, so I want to use globstar patterns which are also used by my backup tools (Borg backup). And this doesn't work:
$ dircompare.js -c -x "**/users/xxx/backup/**" /home/foo/somepath/ /tmp/mounted-backup/
Entries are different
total: 326, equal: 26, distinct: 0, only left: 300, only right: 0
How can I configure dircompare.js
with **
?
Consider the following structure
The library never compare index.t and index.cc. The thing is that I want to ignore the extensions only for pair of extensions. In this example, I want the library to treat index.t and index.cc as the same file by the following code:
if (
// options.ignoreExtension is ["t", "cc"]
options.ignoreExtension.some((exts: [string, string]) => {
return exts.includes(extnameOnly(name1)) && exts.includes(extnameOnly(name2));
})
) {
name1 = path.basename(name1, path.extname(name1));
name2 = path.basename(name2, path.extname(name2));
}
Test repo: https://github.com/moshfeu/dir-compare-test
We want an option ignoreContentCase.
When we develop the vba extensions, vba kindly does case conversions automatically.
So, we compare files, dir-compare detects the difference.
We think adding the code below is ok, but we can not test.
at function compareLine in compareLines.ts
if (options.ignoreContentCase) {
line1 = Line1.toLocaleLowerCase()
line2 = line2.toLocaleLowerCase()
}
Or, someone tell us, how to extend dir-compare to ignore Content Case.
thank you.
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates are currently rate-limited. Click on a checkbox below to force their creation now.
@typescript-eslint/eslint-plugin
, @typescript-eslint/parser
)These updates have all been created already. Click a checkbox below to force a retry/rebase of any.
@typescript-eslint/eslint-plugin
, @typescript-eslint/parser
)These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
package.json
minimatch ^3.0.4
p-limit ^3.1.0
@types/node 16.11.12
@typescript-eslint/eslint-plugin 5.6.0
@typescript-eslint/parser 5.6.0
colors 1.4.0
copyfiles 2.4.1
eslint 7.32.0
memory-streams 0.1.0
semver 7.3.5
tar-fs 2.1.1
temp 0.9.4
ts-node 10.4.0
tsc-watch 4.5.0
typedoc v0.22.10
typescript 4.5.4
In my test code I am just testing if the non-whitespace is equal. I don't care if line endings are different.
Can we get an option to specify that line endings are ignored in the buffer comparison?
I'm basically trying to make a directory compare tool that list all the files that are different between two directories. I can share my code if it helps.
Error: UNKNOWN: unknown error, lstat 'M:\User\xx\Node Projects\next-learn-demo\3-create-dynamic-pages\node_modules\core-js\fn\object\define.js'
at Object.lstatSync (fs.js:943:3)
at getStatIgnoreBrokenLink (C:\Node\fileDiff\node_modules\dir-compare\src\entry\entryBuilder.js:50:17)
at Object.buildEntry (C:\Node\fileDiff\node_modules\dir-compare\src\entry\entryBuilder.js:32:15)
at Object.buildDirEntries (C:\Node\fileDiff\node_modules\dir-compare\src\entry\entryBuilder.js:19:21)
at C:\Node\fileDiff\node_modules\dir-compare\src\compareAsync.js:21:37
at async Promise.all (index 1)
at async Promise.all (index 9)
at async Promise.all (index 6)
at async Promise.all (index 92)
at async Promise.all (index 3) {
errno: -2146893818,
syscall: 'lstat',
code: 'UNKNOWN',
path: 'M:\\User\\xx\\Node Projects\\next-learn-demo\\3-create-dynamic-pages\\node_modules\\core-js\\fn\\object\\define.js'
}
If the arguments to compareSync
are two files, then the result of comparison is a little odd: it seems they are treated as if they were each contained in a directory, so the files are only considered the same if their names are the same.
But compareSync
applied to two directories compares their contents; it should do the same for files.
I came across this issue because I have some code that normally compares directories, but in some cases it may only compare single files, and I had to special case this and not use dir-compare
for the single files.
Thanks for dir-compare
! A very handy piece of code.
Just an idea, not a necessity:
An option could be added to compare the checksums of the two directories, for example using the dirsum package.
When using the "includeFilter" option, it's impossible to filter folders.
Checking the code (
dir-compare/src/Entry/EntryBuilder.ts
Line 110 in 484b59b
Could it be possible to replace the line
if ((entry.stat.isFile() && options.includeFilter) && (!match(path, options.includeFilter))) {
by
if (options.includeFilter && !match(path, options.includeFilter)) {
It then will match what is done for the excludeFilter.
What do you think about it ?
Thanks in advance.
Line based comparator ignoreLineEnding doesn't work correctly, when the files to be compared are bigger, than the BUF_SIZE (100000). In that case two files, which are only differs in line endings, such as one has CRLF and the other has LF, are deemed to be different by the algorithm.
Great library. Any chance to get version that supports node 18?
I am getting an error while installing:
npm install dir-compare
npm ERR! code EBADENGINE
npm ERR! engine Unsupported engine
npm ERR! notsup Required: {"npm":">=8.0.0 <9.0.0","node":">=16.0.0 <17.0.0"}
npm ERR! notsup Actual: {"npm":"8.19.2","node":"v18.12.1"}
You say the actual version is v2.1.0 but the real version in package json is 2.0.1
Originally asked in moshfeu/vscode-compare-folders#65
I'm in desperate need of the ability to do some diffs on some project directories that should be very similar but have a lot of whitespace differences scattered around, making it impossible to easily figure out what's meaningfully different between them. Any chance you might be able to add something equivalent to running diff -b or diff -w?
For the record, I have the ignoreWhiteSpaces option set which does ignore some whitespace differences like those at the start or end of a line, but that's it. It still shows differences in the number of newlines or whitespace differences in the middle of a line, like between a bracket and a comment.
@gliviu is it a reasonable feature?
Thanks
I want to know which file is different
I implemented the gitignoreFilter as specified here https://github.com/gliviu/dir-compare?tab=readme-ov-file#implement-gitignore-filter.
However, I started getting errors like:
Error: Path /private/var/folder/a.txt is not in cwd /var/folder
(I shortened the paths for the example)
I discovered that /var/folder
is actually a symlink to /private/var/folder
. dir-compare turns the symlink into an absolute path, but the GlobbyFilterFunction
in gitignoreFilter.ts
uses the original symlinked path, not the absolute path, hence the mismatch.
I fixed this for myself by modifying line 26 of gitignoreFilter.ts
to use entry.path
instead of entry.absolutePath
.
https://github.com/gliviu/dir-compare/blob/master/test/extended/gitignoreSupport/gitignoreFilter.ts#L26
You could probably also resolve the absolute paths before calling isGitIgnoredSync
.
Here are my test results:
> node tests/runTests.js
test1: !!!!FAILED!!!!
test2: !!!!FAILED!!!!
test3: !!!!FAILED!!!!
test4: !!!!FAILED!!!!
test5: !!!!FAILED!!!!
test6: !!!!FAILED!!!!
test7: !!!!FAILED!!!!
test8: !!!!FAILED!!!!
test100: !!!!FAILED!!!!
test101: !!!!FAILED!!!!
test102: !!!!FAILED!!!!
test103: !!!!FAILED!!!!
test104: !!!!FAILED!!!!
test105: !!!!FAILED!!!!
test106: !!!!FAILED!!!!
test200: !!!!FAILED!!!!
test201: !!!!FAILED!!!!
test300: Passed
test301: Passed
test302: Passed
test303: Passed
test304: Passed
test400: !!!!FAILED!!!!
test500: !!!!FAILED!!!!
test501: !!!!FAILED!!!!
test600: !!!!FAILED!!!!
test601: !!!!FAILED!!!!
Tests: 27, failed: 22, succeeded: 5
compareSync:119 and compareAsync:147
I'm having a problem with the compareDate option on Windows.
If I copy a file and preserve the timestamp by copying the modification date with utimesSync
, it only copies the seconds, not the milliseconds.
The compareDate
functionality is using getTime()
which returns milliseconds.
E.g. I have two files with two timestamps 1480843824000
and 1480843824030
in which I copied the mtime of the second to the first one. The seconds are the same, but the milliseconds are not, as they are not copied. So compareDate will always return false.
Maybe you could add an option to compare seconds and use getTime() / 1000
to compare the date.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.