Very big repository about eaopt HOT 22 CLOSED

Hey! Extracting the examples/ folder is a very good idea! @AP-Nothize you have my approval :)

from eaopt.

It's much faster to go get gago now. :)~

PS. thank gago(and you) so that I can write a GA solver in golang quickly!

from eaopt.

Also, when I have to vendor the library, I don't want to have to manually delete the examples so that my tests won't fail when I run in CI

from eaopt.

How about rewriting the history and extract the examples as git sub-modules?

If @MaxHalford is okay with this, I can try to do this. :)

This looks like promising

from eaopt.

While shrinking the repo, I found that there are some png (eg. logo.png) file in the history but no longer exist now. Could those files be removed as well?

from eaopt.

Yep go for it, I moved some of the pictures to Google Drive and created a link to them.

from eaopt.

Yo, I've uploaded the repo to https://github.com/AP-Nothize/gago so that you can preview what will it look like!

Summary of changes:

1. Extracted the subdirectory "examples" as another repo (using git subtree and filter-branch)
2. Removed all *.png history from the main repo (using BFG Repo-Cleaner)
3. Added the new examples repo back as a submodule on the subdirectory "examples". (For github to show the charts correctly) << I will revert this

Ow......just tested..... "go get" will retrieve the submodules automatically which is what I don't want...... so we need to see how to avoid that or don't add examples as a submodule whilst being able to show the charts correctly.

from eaopt.

It seems that there is no way to prevent go get from downloading the submodule. Would you mind referencing the charts (in README.md) from another repo (ie. gago-examples)?

from eaopt.

Yeah actually I was precisely going to say that linking from another repo is fine :); thanks for looking into all this!

from eaopt.

Ok done reverted submodule!

from eaopt.

Nice!

Thanks a lot, I've still got some stuff I'm going to implement to make it even easier, I'm thinking of implementing a bunch of operators to handle tree structures.

So what should be the next steps? Do you want to send a pull request with the removal of the examples and the switch of the links in the README file?

from eaopt.

Due to history rewrite, the original gago and the new gago repo are considered different by git, thus pull request is not possible. Moreover, there is problem when the golang users try to go get -u gago again because it will result in rebase failure.

On one hand, publishing the rewritten history repo to a new name won't have those problems but you must use another repo name which I'm not sure is good or bad....

On the other hand, you could overwrite the current gago repo by the new one but old users may be surprised when they try to update and the better way for them is to remove their local version and "go get" / clone from github again.

The switch of the links depends on your creation of the gago-examples repo (which doesn't exist yet) for verification purpose.

So you will need to:

1. Fork/create the gago-examples (from https://github.com/AP-Nothize/gago-examples) under your account
2. Either you or me update README.md to reflect the new location of the charts
3. Publish the new gago repo
   3.1 Backup the original gago repo (rename to something else?) and push the new repo to github by force; OR
   3.2 Keep the original gago repo and frozen it and push the new gago repo to another name to maintain backward compatibility with existing users.
4. Cherry-pick any new changes not in the new gago repo yet and frozen the old gago repo (no more new commits)

from eaopt.

I'm a bit busy at the moment (actually I'm going on holiday) so we can discuss this in, say, 10 days :). Again, thanks for all the hard work figuring this out.

from eaopt.

Okay I did all the procedures that had to be done. There is now an examples repository. @AP-Nothize are you happy with how it looks?

from eaopt.

@MaxHalford , the repository looks a little bit smaller than the original but there are a few issues:

1. There should have no commits showing that the examples folder is being deleted.
2. I have used bfg-repo-cleaner for removing the png file only but no other, it seems that you have used it for doing something more as there are some "Former-commit-id".
3. Your gago repo size is 4.6M including the .git folder, my gago repo size is 618K including the .git folder.
4. Your gago-examples repo has not preserved the history.

In general, the gago repository after splitting the examples should look exactly as before except the commit ids are all changed and some commits are removed if they are only related to the big files and examples.

The gago-example repository extracted should look exactly as before except the commit ids are all changed and some commits are removed if they are only related to the big files not desired anymore (eg. the png files that was added and later removed).

Since you decide to overwrite the github gago repository, it will be better if the original gago repostiory is saved as another name on github as a permanent backup. (although I am not aware of any problem after rewritting the history, there are always risks associated with such operation that some desired histories maybe accidentally removed)

If you trust my work, you can just take my gago and gago-examples. I kept googling and experimenting while doing my first surgery so no repeatable command is recorded. :(

from eaopt.

Yes you're right I didn't do this very cleanly... I actually do mind creating a new repository, one of the reasons being losing all the stars and hits it has.

Can you not pull a branch from a previous commit on master and do your magic there?

from eaopt.

Assume [email protected]:MaxHalford/gago.git is the original repository (though it isn't now), the commands to do this:

# Since your github repo is already rewritten, u should do this from a local backup of the original gago repository
# git clone [email protected]:MaxHalford/gago.git
# cd gago
cd /your/original/gago/repository
# create a backup repository on github called gago_backup
git remote add backup [email protected]:MaxHalford/gago_backup.git
git push backup -u --all
git push backup --tag
# so now you have a complete backup of the repo, then u could be brave to do whatever to the original one at github
cd /to/somewhere
git clone [email protected]:AP-Nothize/gago.git
cd gago
git remote add max [email protected]:MaxHalford/gago.git
# push to your github gago repo by force, I'm not sure if --all -f will work or not......in case the tags and branches are incorrect, push them too by force.
git push max master -f
# now your github gago repo should look identical to my github gago repo, and the stars and forks are still there.

from eaopt.

Ok I think we're good! As for the examples repository I don't think it matters that the related commits are not there.

Thank you for the commands, I've learnt a lot.

from eaopt.

It looks good except that the tag 0.0.1 needs to be updated as well because that is pointing to the old history which consumes of several MiB.

I compared the commit time and commit message and conclude that the right commit id for 0.0.1 is 75686fa.

So in your local working directory, you can do this:

# Re-tag 0.0.1 to commit id 75686f
git tag 0.0.1 75686f -f
# Overwrite the tag 0.0.1 on github by force
git push origin 0.0.1 -f

Then your repository will become much smaller by a fresh clone, 417K for the .git directory.

As for the gago-examples repository, you can do something similar as my previous post to overwrite your repository from mine.

History is something that you will never know when do you need it but when you need it, you'll be glad that it is there. Like the tag 0.0.1 I've just fixed mine as well by tracing from the history.

Moreover, due to the same reason, I strongly recommend creating a gago-backup that preserve the original big big repository that is before the surgery so that we can gain the benefit of a smaller repository while still keeping the full history. (ps. my network speed is 10x faster at home than at work)

from eaopt.

Okay I did the same for gago-examples, you're right you never when the history could be useful. I also updated the 0.0.1 tag.

Thanks again!

from eaopt.

Perfect! Now the gago can be go got much faster!

Off-topic a bit, have you used gopkg.in? I just aware of it recently and then I think its quite good as a workaround for dependency management for golang. go getting some old repository may easily result in a broken state if some of the dependency are updated to break the API being depended on.

I think it's a brilliant idea by using v1, v2 for pointing to a stable API and at the same time allow bug fix to be updated via v1.0.1, v2.2.3 and so on.

from eaopt.

Actually I think I'm going to wait for dep to be officially released and then I'll start using it.

Anyway I'm closing this issue; thanks again for your work, we did well.

from eaopt.