Feature Request / Question: Can git-filter-repo merge commits into a repo rather than replace? about git-filter-repo HOT 5 CLOSED

Why do you need to write directly to the target monorepo? Are you attempting to stitch commits together so that it looks like commits from different repositories were always interleaved or something? Are you trying to combine many branches? What about when both the monorepo and the separate repo have a branch with the same name, such as 'master' -- which do you keep or how did you want them to somehow be combined?

Most cases I've seen of using filter-repo to assist with creating a monorepo out of a bunch of separate repositories have usually been of the form: (a) use filter-repo on a separate repo to rewrite things in prep for merging into the monorepo, (b) cd to the monorepo and use git pull with the --allow-unrelated-histories flag to pull in and merge history from the newly rewritten repo into the monorepo, (c) repeat steps a & b for all the other repos. This style, of course, assumes that you just need to merge in one branch of the separate repo into one branch of the monorepo and that before that merge commit the histories of the two repositories remain separate.

Does that help, or can you explain more about what you're trying to achieve and why?

from git-filter-repo.

That's more or less what I found. It's a little error prone, but I can automate it. I was wondering if it made sense to have an option for filter-repo to interleave, as you say, commits instead of replacing them. This only makes sense if the source and target repos are different. If they are the same, it would obviously make a mess.

from git-filter-repo.

Interleaving commits is not a simple operation, and I'm pretty sure that no matter what is implemented is guaranteed to be wrong for some repos. If there are multiple branches and tags in both repos, which ones are interleaved? If multiple, which combinations of branches and tags?
What if none of the branch names match, or one side has a bunch of branches that the other doesn't? When history is non-linear, how do you decide which side(s) of history match up with the other repositories' history? And if both repositories have highly non-linear history, is there any sane strategy for interleaving those commits at all?

Unless someone can demonstrate some kind of super clever idea here that just eludes me right now, there's no way I'll implement any kind of interleaving strategy in core filter-repo. While it could certainly be possible to implement something useful for some folks, it'd just be asking for trouble from people with slightly different history setups. However, implementing some very basic interleaving strategy as a demo in the contrib directory may make sense (e.g. interleaving just first-parent history using commit dates as the guide), though even the simplest strategy I could think of before got really complex fast with multiple branches and tags, which caused me to punt the last time I looked at it. Even with cases where you have really nice hints, such as trying to turn a submodule into a real subdirectory and making it one unified repository (where you have nice hints because submodule commits are part of the parent commit history giving you guidelines for how to stitch things back together) got really complicated because of e.g. the possibility to just totally switch branches in the submodule or revert commits making the subdirectory move backward in time while the outside repository moves forward.

Even checking for when assumptions were valid about when simpler stitching rules could safely be employed seemed a bit involved (so that people don't abuse the demo and mis-use it on cases that are more complex).

I do have some notes somewhere when I started on a script to try to do submodule extraction and subsumption, but I'm not sure if they'd be useful to anyone else separate from my comments above.

That's probably not what you want to hear, but short of someone coming along with a clever idea and more of a desire to tackle this problem than me, I'm unlikely to work on this anytime soon and even if I eventually do it'd probably only be a special case contrib example.

from git-filter-repo.

I'll go ahead and close this out as I don't see a path forward with interleaving commits automatically and we've addressed the alternative route of filtering out pieces you need and then merging into a bigger repo. Let me know if I've missed anything, though.

from git-filter-repo.

turn a submodule into a real subdirectory and making it one unified repository
I'm trying to do that exact scenario. Is there any way to do this at the moment? or at least guidelines for possible directions?

I have a repo with a single submodule, that I'll like to integrate / stich into the main one.

Basically I'd like to go from this (Ax are commits on the main repo, Sx on the submodule, and M the "merge" commits / .gitmodules commits):

M
|\
| S3
|/
A2
|
M
|\
| S2
| |
| S1
|
A1

to this:

S3
|
A2
|
S2
|
S1
|
A1

got really complicated because of e.g. the possibility to just totally switch branches in the submodule or revert commits making the subdirectory move backward in time while the outside repository moves forward.

I don't think I have any of that. Would ignoring these scenarios make it easier? Would it be possible to at least detect them and log that they're a problem?

Is git-filter-repo even the right tool for this kind of job? Or would it make more sense to use some other, lower level tool (lib2git?)
I also found https://metacpan.org/pod/git-stitch-repo but it doesn't look well maintained (?).

from git-filter-repo.