mnagel / clustergit Goto Github PK

Clustergit rightnow only reports unpushed commits in the current branch of a repo. Could it add support to find all unpushed commits in other branches?

My use case: I often work on multiple branches at the same time, for example bug fixes and new features, and it's easy to leave some commits in a branch forgot to push to upstream. I can use command git --no-pager log --branches --not --remotes --no-walk --decorate --oneline to list branches containing unpushed commits in a repo. It's good to make it works on multiple repos by clustergit.

Since git, Github, and the community at large have updated best practices to use the label main in place of master as the default branch name, clustergit should be updated to follow this pattern.

Reference:
https://github.com/github/renaming

Would love the ability to skip symlinks. I hope to find time to investigate this myself, but time will tell.
Great tool!

Howdy 👋

For whatever reason, one of my git projects refuses to fetch - it doesn't error, it just hangs forever. (Whatever, it works when I specifically cd in and fetch myself, not a huge deal)

I thus exclude that folder with -e folder-name. Epic. But is there some kind of "if nothing happens for X seconds, move on" functionality? I ask because, well, it's late and I could very well have missed it.

Cheers!

#45 introduced a new feature enabled by --all-branches.

I don't see it in the documentation yet. I had to experiment with it to figure out how it works.

Perhaps it would be useful to add simple usage examples or sample output to the README. At least least the --all-branches option should appear there.

$ git -C clustergit --no-pager log --branches --not --remotes --no-walk --decorate --oneline
b09d21c (HEAD -> master) Add pic
3210f8a (otherbranch) Add file
$ clustergit -d .
Scanning sub directories of ['.']
./clustergit                            : No Changes, Unpushed commits
Done
$ clustergit -d . --all-branches
Scanning sub directories of ['.']
./clustergit                            : No Changes, Unpushed commits, Unpushed commits on branches: [otherbranch]
Done

@mnagel as you can see here. I have a repository with unpushed commits and unpulled commits..

clustergit shows all projects as clean though...

How is this possible?

I use clustergit in a script that I run at the end of every session to ensure all my local repos have their changes committed and pushed before I log off.

The script filters the output of clustergit like this to operate on repos that need attention.

unclean="$(
  clustergit --branch '' --recursive --exclude=/\.terraform/ |
  grep ':' |
  grep -v Clean |
  awk '{print $1}'
)"

for repo in $unclean; do
  cd $unclean;
  ...
done

(The exclude option is to ignore temporary data created by the terraform init command that happen to be git repos.)

This used to work great until I started getting a weird error.

cd: Waiting: No such file or directory 

It turns out that an extra string is output for slower repos so that the first word of the line is not always the repo name. When the repo processing is complete the line no longer appears in the terminal but it remains in stdout. The '\r' means that in the terminal the line gets overwritten but in the stream it's prepended to the line.

Scanning sub directories of .
Waiting for ./aws-notes .    (1/4)^M./aws-notes                             : ESC[93mUntracked filesESC[0m
./aws-solutions-architect-professional  : ESC[92mCleanESC[0m
./life-notes                            : ESC[92mCleanESC[0m
./foo                                   : ESC[91mChangesESC[0m
Done

I tried using --quiet and it does suppress the first "Scanning" line but not the "Waiting" line.

Would it be possible to make --quiet to suppress the "Waiting" message as well?

My current workaround is to add tr '\r' '\n' to the filter command.

ImportError when Clustergit is run from cmd.

Traceback (most recent call last):
  File "C:\CS\clustergit", line 20, in <module>
    from ctypes import windll, c_ulong
  File "C:\python\lib\ctypes\__init__.py", line 8, in <module>
    from _ctypes import Union, Structure, Array
ImportError: DLL load failed while importing _ctypes: The specified module was not found.

Tried reinstalling mambaforge, error still there. My system:

• Python 3.10 (recent mambaforge)
• Windows 11

The --recursive option doesn't seem to work across a symlinked directory structure. More specifically, I have a directory structure that I've created a symlink to within a directory I'm running clusergit --recursive in. For example:

.
├── dir1
│   └── repo1
└── dir2
    ├── repo2
    └── linked -> foo/dir1/

Git v2.9.1 changed the response from a clean git status to working tree clean instead of working directory clean; this change breaks clustergit for 2.9.1 users.

as seen here:

Awesome. Suggestions: recursive push to origin master, recursive pull from origin master, give examples that show -v turned on and off. Also, is there a way to add or commit in each directory? – David Y. Stephenson Jul 12 at 14:35

Hi,

I have a failing test because the path of the generated tmp folder is longer than you expect it to be. I have less whitespace characters between the path and : Changes

I would suggest to make the String comparison less strict. What about an expression to check if the output is partially as expected? Ignoring two many whitespace characters?

Cheers,
Ben

======================================================================
FAIL: tests.clustergit_test.TestClustergit.test_check_fresh_repo
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/Users/ben/dev/projects/oss/clustergit/tests/tests/clustergit_test.py", line 55, in test_check_fresh_repo
    same("stdout should be alright", expected_out, actual_out)
  File "/Users/ben/dev/projects/oss/clustergit/tests/tests/tools.py", line 4, in same
    eq_(actual, expected, '%s. We expected [%s] but got [%s]' % (message, expected, actual))
AssertionError: stdout should be alright. We expected [Starting git status...
Scanning sub directories of /var/folders/qz/pkd6t5v52d9b2w01nc2f587h0000gp/T/tmpuqsbOv
/var/folders/qz/pkd6t5v52d9b2w01nc2f587h0000gp/T/tmpuqsbOv/mygit                    : Changes
Done
] but got [Starting git status...
Scanning sub directories of /var/folders/qz/pkd6t5v52d9b2w01nc2f587h0000gp/T/tmpuqsbOv
/var/folders/qz/pkd6t5v52d9b2w01nc2f587h0000gp/T/tmpuqsbOv/mygit: Changes
Done
]

I made a batch file clustergit.cmd so I could add the program to my PATH on Windows:

@echo off
python "%~dp0\clustergit" %*

It wasn't getting any output from the commands, maybe because of the && usage.
I was able to remedy this by pasting in some code from an SO answer:

from subprocess import check_output, CalledProcessError, STDOUT

def get_status_output(cmd):
    try:
        output = check_output(cmd, shell=True, universal_newlines=True, stderr=STDOUT)
        status = 0
    except CalledProcessError as ex:
        output = ex.output
        status = ex.returncode
    if output[-1:] == '\n':
        output = output[:-1]
    return status, output

This code can go away:

try:
    import commands
except Exception:
    # python3
    import subprocess as commands
    commands.get_output = commands.check_output

Also I added a bit of error handling for the lack of output:

            if len(out) is 0:
                raise ChildProcessError('No output from child process')

I'm not sure whether that should be a ChildProcessError or a ValueError or just an Exception.

As the title says. When I do a clustergit -v at the root of a number of repos, I get given a red "Changes" notice when, actually, no commits have yet been made to that repo.
Great tool. Saves me so much effort. Thanks very much.

In my case I see no reason to demand that repos must be checked out to master branch.

Github now defaults to "main" for new repos. https://github.com/github/renaming

Azure Devops wikis use "wikiMaster" for the default branch. https://docs.microsoft.com/en-us/azure/devops/project/wiki/wiki-update-offline?view=azure-devops

I need to work with all three of these defaults at a time and I want clustergit to handle it.

$ clustergit
Scanning sub directories of .
./Infra                          : Clean
./Infra.wiki                     : On branch wikiMaster, No Changes
./multi-region-org-config-rules-terraform: Clean
./terraform-aws-remote-state-s3-backend : Clean
./terraform-aws-secure-baseline         : Clean
./terraform-provider-aws                : On branch main, No Changes
Done

These should all say "clean".

The exclude option already accepts a regex.

If the branch option supported regex too, I would expect to be able to solve it like this:

clustergit --branch "master|main|wikiMaster"

The branch option default "master" continues to be sensible because of the all the existing Git repos that use it as the default branch.

It would be useful if clustergit descended into sub-subdirectories or supported a --depth argument similar to find. I organize my repos in a variety of ways, usually ~/code ~/personal ~/external but any of those might be further organized like ~/external/netflix or ~/external/scala...

https://github.com/mnagel/clustergit#license seems to be narrowed to cover only clustergit, README.* files.

Hi @mnagel ! 👋

I've been using this CLI for years now (since at least 2021, when I first contributed to clustergit: af79ab4) - loved it back then, and I still love it and use it daily!

Context

For a while now, I've been encountering an 'undesired' behavior - specifically, I have a bunch of repos (order of magnitude in the hundreds) that I use this tool against, commonly fetching/pulling the latest changes, and most-commonly against GitHub-backed repositories. When I run clustergit, I've been encountering rate limiting from GitHub in the middle of fetching repos. So far, I've been bumping down --worker, from the default of 4, to 2, now down to 1 - I'm still hitting GitHub rate limits (booo! 😞 ).

"Feature" request

What I'm looking for here is a 'feature' (in quotes, because I personally feel like this slightly flies in the face of this CLI's ability to do operations in many workers) to delay between iterations of the analyze() method.

Testing

I've actually made this change to my local copy of clustergit (see diff below), and plan to test this over the next few weeks.

• If you're open to this 'feature', I'll happily open a PR.
• If you're open to this 'feature', and are ok to "not wait a few weeks" for me to test locally (it's a simple change), I'll happily open a PR sooner.
• If you're not open to this 'feature', I totally understand, and will likely just keep this delta for myself. :)

Let me know your thoughts!

@mnagel this would be nice to auto include in my dotfiles via brew or pip ;) What do you think?

very nice tool btw ⭐️

I scan entire filesystem - but there are certain known directories that have uncommitted changes and it is acceptable (for example clone of https://github.com/StevenBlack/hosts, with update scheduled in cron, location that keeps partial/modified copy of real repository what is not a cause for alarm etc).

Currently I have no idea how this option should be named --except with short version -e collides a bit with --execute.

git status is not intended to be machine parsable, and is liable to change without notice. For true future compatibility, git status --porcelain should be used, which gives easily parsable output. I'm happy to work on this in the coming weeks, as well as work to make the script windows compatible (it runs but gives lots of errors for me; I'll open a separate issue for that though).