Giter VIP home page Giter VIP logo

vcstool's Introduction

What is vcstool?

Vcstool is a version control system (VCS) tool, designed to make working with multiple repositories easier.

Note:

This tool should not be confused with vcstools (with a trailing s) which provides a Python API for interacting with different version control systems. The biggest differences between the two are:

  • vcstool doesn't use any state beside the repository working copies available in the filesystem.
  • The file format of vcstool export uses the relative paths of the repositories as keys in YAML which avoids collisions by design.
  • vcstool has significantly fewer lines of code than vcstools including the command line tools built on top.

Python 2.7 / <= 3.4 support

The latest version supporting Python 2.7 and Python <= 3.4 is 0.2.x from the 0.2.x branch.

How does it work?

Vcstool operates on any folder from where it recursively searches for supported repositories. On these repositories vcstool invokes the native VCS client with the requested command (i.e. diff).

Which VCS types are supported?

Vcstool supports Git, Mercurial, Subversion, Bazaar.

How to use vcstool?

The script vcs can be used similarly to the VCS clients git, hg etc. The help command provides a list of available commands with an additional description:

vcs help

By default vcstool searches for repositories under the current folder. Optionally one path (or multiple paths) can be passed to search for repositories at different locations:

vcs status /path/to/several/repos /path/to/other/repos /path/to/single/repo

Exporting and importing sets of repositories

Vcstool can export and import all the information required to reproduce the versions of a set of repositories. Vcstool uses a simple YAML format to encode this information. This format includes a root key repositories under which each local repository is described by a dictionary keyed by its relative path. Each of these dictionaries contains keys type, url, and version. If the version key is omitted the default branch is being used.

This results in something similar to the following for a set of two repositories (vcstool cloned via Git and rosinstall checked out via Subversion):

repositories:
  vcstool:
    type: git
    url: [email protected]:dirk-thomas/vcstool.git
    version: master
  old_tools/rosinstall:
    type: svn
    url: https://github.com/vcstools/rosinstall/trunk
    version: 748

Export set of repositories

The vcs export command outputs the path, vcs type, URL and version information for all repositories in YAML format. The output is usually piped to a file:

vcs export > my.repos

If the repository is currently on the tip of a branch the branch is followed. This implies that a later import might fetch a newer revision if the branch has evolved in the meantime. Furthermore if the local branch has evolved from the remote repository an import might not result in the exact same state.

To make sure to store the exact revision in the exported data use the command line argument --exact. Since a specific revision is not tied to neither a branch nor a remote (for Git and Mercurial) the tool will check if the current hash exists in any of the remotes. If it exists in multiple the remotes origin and upstream are considered before any other in alphabetical order.

Import set of repositories

The vcs import command clones all repositories which are passed in via stdin in YAML format. Usually the data of a previously exported file is piped in:

vcs import < my.repos

The import command also supports input in the rosinstall file format. Beside passing a file path the command also supports passing a URL.

Only for this command vcstool supports the pseudo clients tar and zip which fetch a tarball / zipfile from a URL and unpack its content. For those two types the version key is optional. If specified only entries from the archive which are in the subfolder specified by the version value are being extracted.

Validate repositories file

The vcs validate command takes a YAML file which is passed in via stdin and validates its contents and format. The data of a previously-exported file or hand-generated file are piped in:

vcs validate < my.repos

The validate command also supports input in the rosinstall file format.

Advanced features

Show log since last tag

The vcs log command supports the argument --limit-untagged which will output the log for all commits since the last tag.

Parallelization and stdin

By default vcs parallelizes the work across multiple repositories based on the number of CPU cores. In the case that the invoked commands require input from stdin that parallelization is a problem. In order to be able to provide input to each command separately these commands must run sequentially. When needing to e.g. interactively provide credentials all commands should be executed sequentially by passing:

--workers 1

In the case repositories are using SSH git@ URLs but the host is not known yet vcs import automatically falls back to a single worker.

Run arbitrary commands

The vcs custom command enables to pass arbitrary user-specified arguments to the vcs invocation. The set of repositories to operate on can optionally be restricted by the type:

vcs custom --git --args log --oneline -n 10

If the command should work on multiple repositories make sure to pass only generic arguments which work for all of these repository types.

How to install vcstool?

On Debian-based platforms the recommended method is to install the package python3-vcstool. On Ubuntu this is done using apt-get:

If you are using ROS you can get the package directly from the ROS repository:

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt install curl # if you haven't already installed curl
curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get install python3-vcstool

If you are not using ROS or if you want the latest release as soon as possible you can get the package from packagecloud.io :

curl -s https://packagecloud.io/install/repositories/dirk-thomas/vcstool/script.deb.sh | sudo bash
sudo apt-get update
sudo apt-get install python3-vcstool

On other systems, use the PyPI package:

sudo pip install vcstool

Setup auto-completion

For the shells bash, tcsh and zsh vcstool can provide auto-completion of the various VCS commands. In order to enable that feature the shell specific completion file must be sourced.

For bash append the following line to the ~/.bashrc file:

source /usr/share/vcstool-completion/vcs.bash

For tcsh append the following line to the ~/.cshrc file:

source /usr/share/vcstool-completion/vcs.tcsh

For zsh append the following line to the ~/.zshrc file:

source /usr/share/vcstool-completion/vcs.zsh

For fish append the following line to the ~/.config/fishconfig.fish file:

source /usr/share/vcstool-completion/vcs.fish

How to contribute?

How to report problems?

Before reporting a problem please make sure to use the latest version. Issues can be filled on GitHub after making sure that this problem has not yet been reported.

Please make sure to include as much information, i.e. version numbers from vcstool, operating system, Python and a reproducible example of the commands which expose the problem.

How to try the latest changes?

Sourcing the setup.sh file prepends the src folder to the PYTHONPATH and the scripts folder to the PATH. Then vcstool can be used with the commands vcs-COMMAND (note the hyphen between vcs and command instead of a space).

Alternatively the -e/--editable flag of pip can be used:

# from the top level of this repo
pip3 install --user -e .

vcstool's People

Contributors

130s avatar bergercookie avatar chapulina avatar christophebedard avatar codebot avatar cottsay avatar dirk-thomas avatar eduidl avatar esteve avatar firemark avatar jbohren avatar jenisys avatar mikaelarguedas avatar mxgrey avatar nuclearsandwich avatar rotu avatar sloretz avatar stonier avatar wjwwood avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vcstool's Issues

please advertise the tool

I know you sent an email to ros users but please add a tiny page on the ROS wiki. Thx, it's awesome.

vcs export --exact reports tag names instead of commit hash

Version: vcs 0.2.4

How to Reproduce:
Run vcs export --exact on a set of repos.

Expected behavior:
The version field of each repo will report the exact commit hash like this:

  src/ros2/rclpy:
    type: git
    url: https://github.com/ros2/rclpy.git
    version: 76d26a125da7f9df19ba35beb60cb16838019a10

Actual Behavior:
Frequently the hash is replaced by a tag name.

  src/ros2/rcl_interfaces:
    type: git
    url: https://github.com/ros2/rcl_interfaces.git
    version: 0.8.0

I think when the --exact parameter is specified, it is preferable to provide the commit hash.

python3-setuptools is a dependency

When using on a fresh debian (or armbian in my case) install I get an error:

Traceback (most recent call last):
  File "/usr/bin/vcs", line 6, in <module>
    from pkg_resources import load_entry_point
ImportError: No module named 'pkg_resources'

This is resolved by apt-get install python3-setuptools. Perhaps this should be added to the dependencies of the python3-vcstools package?

Problem installin vcs on OSX

Hi,

I am trying to install vcs on OSX but I get the following error:

creating /System/Library/Frameworks/Python.framework/Versions/2.7/share
    error: could not create '/System/Library/Frameworks/Python.framework/Versions/2.7/share': Operation not permitted
    
    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-build-gxcIWT/vcstool/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-KDkxWu-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-gxcIWT/vcstool/

Some idea to fix the problem?

Juan Antonio

pypi package should depend on yaml

On a fresh osx install, I get the following after installing vcstool and trying to run it:

$ vcs st
Traceback (most recent call last):
  File "/usr/local/bin/vcs", line 8, in <module>
    load_entry_point('vcstool==0.1.4', 'console_scripts', 'vcs')()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 318, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 2221, in load_entry_point
    return ep.load()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py", line 1954, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/Library/Python/2.7/site-packages/vcstool/commands/__init__.py", line 5, in <module>
    from .import_ import ImportCommand
  File "/Library/Python/2.7/site-packages/vcstool/commands/import_.py", line 6, in <module>
    import yaml
ImportError: No module named yaml

Improve error message when executable is not available

=== /home/hiya/ros2_ws/src/ament/ament_cmake (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ament/ament_lint (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ament/ament_package (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ament/ament_tools (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ament/osrf_pycommon (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ament/uncrustify (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/examples (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/launch (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rcl (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rcl_interfaces (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rclc (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rclcpp (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rmw (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rmw_connext (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rmw_implementation (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rmw_opensplice (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rosidl (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)
=== /home/hiya/ros2_ws/src/ros2/rosidl_dds (git) ===
Invocation of command 'import' on client 'git' failed: TypeError: sequence item 0: expected str instance, NoneType found (/usr/lib/python3/dist-packages/vcstool/clients/vcs_base.py:51)

I am using Ubuntu 14.04 LTS.

recursive submodules only work with new repositories

When switching the remote URL of an already existing git repo by forcing the import from a yaml file, e.g.
vcs import --recursive --force --input repo.yaml
git submodules are not checked out.

I manually need to delete the repo with submodules and repeat the command to also have submodules to be checked out.

Offline tests

Would be possible to have a cut down version of the test suite which uses offline data.

I'm packaging this for openSUSE https://build.opensuse.org/package/show/home:jayvdb:ros/python-vcstool , and the rpm build environment is isolated from the internet.

I can add tarballs of other repos from the tests repo.list to the build environment, and manually import them.

Ideally the test runner would recognise that the repo is already imported, and not try to update it, and then run the tests.

`export` should strip the `src` path element

Generally, when working with vcs, my working directory contains a subdirectory called src and all the actions I take operate on the contents of src without me referring to it explicitly.

But when I vcs export to capture the state of my source tree, if I do it from above src, then the local paths that are produced start with src/. If I subsequently vcs import that result somewhere else in the normal way, I end up with the checkout going into src/src/.

To get what I consider to be the right answer, I have to run vcs export inside src. It seems to me that running vcs export at the top should give me the local paths without the leading src/ path element.

I'm happy to make a PR to implement the change, but I first want to confirm that I have the right perspective.

vcs import fails when credentials are needed

Running vcs import with a config file that contains two or more repos that need either:

  • http login credentials
  • private key password

does not work as the prompts clobber each other.

Running 0.1.15 from PyPI

Add option to `import` to retry clone of repositories until success or timeout

Some ci builds fail while using vcs to download dependencies because of temporary network issues (example).

07:36:31 === src/eProsima/Fast-RTPS (git) ===
07:36:31 Could not clone repository 'https://github.com/eProsima/Fast-RTPS.git': Cloning into '.'...
07:36:31 error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
07:36:31 fatal: The remote end hung up unexpectedly
07:36:31 fatal: early EOF
07:36:31 fatal: index-pack failed

I suspect this would had succeeded if vcs waited a few seconds and tried again. Adding an option to import which would retry cloning with some delay or increasing backoff may reduce the number of failing jobs on the build farm.

Submodule transition support

vcs status does not show any output for submodules inside a git repository.

I am not quite sure if this is intended behaviour, but vcs could still be a helpful tool for managing the submodules.

importing more than 20 repos causes fails

Hi, I am using latest version of the vcstool (0.1.20).
I have a YAML file consisting more than 35 repos. When I try importing repos to a single folder after 20~ repos it is throwing errors and stuck downloading next repos. I have tried splitting YAML file to 3 different files and running them in order to same folder. Every one of them has around 12 repos. First one goes without problem but second one either throws error or stuck downloading. I am able to clone all the repos manually meaning that there is no permission issue.

I get something like this :
.....................EE..EE....

ZIP file support

Please add a handler for zip files (similar to the tar file handler). Thanks.

nested folders don't work

I get the error below even with the --force option:

=== src/localization (hg) ===
Could not clone repository 'https://mercurial.asirobots.com:8443/scm/hg/localization': abort: destination '.' is not empty

It seems a little weird that it creates the "localization" folder prematurely. From my repos file:

  localization:
    type: hg
    url: https://mercurial.asirobots.com:8443/scm/hg/localization
    version: default
  localization/src/AutoGenFusor:
    type: tar
    url: https://builder.domain.local/guestAuth/repository/download/MatlabScripts_FusorCodeGen/1.5.5c5faf48e415.91/builderOutput.tgz
    version: 1.5.5c5faf48e415.91
  localization/src/AutoGenMatchMaker:
    type: tar
    url: https://builder.domain.local/guestAuth/repository/download/MatlabScripts_PFCodeGen/1.8.e8c4885e9249.103/builderOutput.tgz
    version: 1.8.e8c4885e9249.103

Can't import tarballs

In the README.rst is mentions that tar is supported as a pseudo client, to fetch a tarball at a URL and extract it.
When I try to use the tool for this, it always fails with the following error:
Invocation of command 'import' on client 'tar' failed: TypeError: initial_value must be str or None, not bytes (/usr/lib/python3/dist-packages/vcstool/clients/tar.py:72)

I went in and tinkered with the tar.py file and changed the StringIO fileobj on the tarfile.open to be a BytesIO. This had the affect of making the script happy, and printing out that my tarball was extracted, but there's no data anywhere. I can't find the contents of the tarball where it is expected to be.

[feature] Make vcs status --hide show when branch diverged from their remote

If it is of any interest I can try to take a shot at it. For the git client for example, we could apply something like:

-            cmd = [GitClient._executable, 'status', '-s']
+            cmd = [GitClient._executable, 'status', '-s', '-b']
             if command.quiet:
                 cmd += ['--untracked-files=no']
             result = self._run_command(cmd)
-            if result['returncode'] or not result['output']:
+            if result['returncode'] or ('\n' not in result['output'] and 'ahead' not in result['output'] and 'behind' not in result['output']):
+                result['output'] = ''

Not sure how it would work for locales though. Didn't look at other vcs systems to see if something similar can be done easily

Store multiple named remotes

As I understand vcs is mainly used to replicate the set of currently tracked git (and others) repos.
I often use multiple remotes when working with forks to simplify rebasing on master. E.g. a remote origin pointing to the upstream projects and a remote fork pointing to my fork that contains feature branches.

When having the feature branch checked out, vcs will pick up the fork repo in the url. If I replicate this setup via vcs import on another PC, this fork url will be set to origin again and it will not create the upstream remote.
As a consequence:

  • the original upstream repo needs to be manually added to the second workspace (e.g. as remote upstream) to rebase on master
  • the remote names differ between workspaces, e.g. the original upstream repo will be either origin or upstream and the forked feature branch will be either fork/feature or origin/feature

It would be useful, if vcs would store multiple remotes as key-value pairs, e.g.:

url:
  origin: <url to upstream>
  fork: <url to fork>
version:
  - origin/master
  - fork/feature

Raspberry Pi 3 restarts

I was trying to follow the tutorial from https://github.com/ros2/ros2/wiki/Linux-Development-Setup . I was trying to do it with my raspberry pi. While doing vcs-import src < ros2.repos. I used the with the --debug flag, but it tells me nothing about the crash.

DEBUG:vcstool.executor:ongoing ['src/eProsima/Fast-RTPS', 'src/ros2/urdfdom', 'src/ament/uncrustify', 'src/ros2/rmw_fastrtps', 'src/ros2/rosidl', 'src/ros2/realtime_support', 'src/eProsima/Fast-CDR', 'src/ament/ament_package', 'src/ament/ament_index', 'src/ros2/rosidl_typesupport', 'src/ros2/urdfdom_headers', 'src/ament/ament_python', 'src/ros2/poco_vendor', 'src/ros2/demos', 'src/ament/gtest_vendor', 'src/ros2/sros2', 'src/ros2/examples', 'src/ament/ament_lint', 'src/ros2/rclcpp', 'src/ros2/ros1_bridge']
Write failed: Broken pipe

In the previous line you see the las debug message and just after just a broken pipe because there is no more ssh connection with the RPI because it is restarting.

Any ideas?

Git LFS Support

I think some of ROS package may includes large binary.
So, I want to add LFS support for vcstool.

Easier ways to create repository collections

I find myself frequently creating unique collections of repositories. In both the ROS workspace framework and our own framework for Java software, one may select a set of repositories to build from source rather than linking to pre-compiled versions.

The fastest way to initialize a collection is to mkdir -p and git clone by hand, so I often skip the step to create the .repos file. Additionally, I find that I'm copying URLs from a browser window and pasting into git clone.

I was wondering if you had any recommended solutions to quickly creating a .repos file.

One idea I had was a web browser plugin in which right-clicking a VCS URI (such as a .git link) would provide a context menu entry to "Add to Workspace". It could provide a setting for default workspaces directory and automatically creating and modifying the .repos files. Maybe also a link to "Open Workspace in Terminal" once you're done.

I was just hoping to bounce off this idea and I didn't quite know where to put it.

tar extraction fails on Windows

The os-specific separator is not used when extracting tarballs in python. So, on windows, all files are using the '/' separator. So in tar.py, the file extraction/replacement in get_members fails to work properly.
By removing the os.sep from the prefix, the code works cross-platform.

def get_members(tar, prefix):
    for tar_info in tar.getmembers():
        if tar_info.name.startswith(prefix):
            tar_info.name = tar_info.name[len(prefix)+1:] # new +1 here to compensate
            yield tar_info
prefix = command.version #+ os.sep
tar.extractall(self.path, get_members(tar, prefix))

I've made this change locally on Windows and Linux and they both work. Maybe you can think of a more elegant solution.

support nested repos in all verbs

We use a git repo that configures a set of repositories within it. This works by gitignoring everything except a few key config files. However, vcstool does not automatically pick up these repositories.

VCS fails when connecting to unknown host via SSH

Steps to reproduce:

  1. Create a .repos file with ssh URLS
  2. Delete your ~/.ssh/known_hosts file
  3. Import the .repos file using vcs and watch the output look something like the following:

EEEThe authenticity of host 'gitlab.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)? The authenticity of host 'gitlab.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)? The authenticity of host 'gitlab.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)? The authenticity of host 'gitlab.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)? .The authenticity of host 'gitlab.com (35.231.145.151)' can't be established.
ECDSA key fingerprint is SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw.
Are you sure you want to continue connecting (yes/no)? ......

I tried typing "yes" a few times and more dots appeared, but then it hanged for quite a while and seemed to get stuck.

Possible Solutions:

  • Add an arg to auto-accept the connection (possibly dangerous).
  • Improve the user experience by presenting a (y/n) prompt that actually works.

Ubuntu 12.04 no longer supported?

I installed a new 12.04.5 Deskop system (supporting an aging robot configuration) and set up the ROS Precise apt repo.

$ sudo apt-get install python-vcstool
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 python-vcstool : Depends: python:any (>= 2.7.5-5~)
E: Unable to correct problems, you have held broken packages.

AFAIK, 12.04.5 isn't going to get Python 2.7.5. It's stuck at:

$ python --version
Python 2.7.3

Can the python-vcstool package in the ROS Precise apt repo be rolled back? Is version 2.7.5-5~ actually required, or is that an artifact of a build server decision? Or can you recommend a workaround?

VCS tool 0.1.35 failing for svn repos

I'm trying to checkout svn repos with vcs tool 0.1.35 and they're failing with the error:

Invocation of command 'import' on client 'svn' failed: TypeError: %d format: a number is required, not str (C:\Python36\lib\site-packages\vcstool\clients\svn.py:128)

The same repos file works fine with vcstool 0.1.29

Jenkins Integration

We use this tool on our project and are very happy with the simplicity that it brings in managing multiple repositories. We were having some issues integrating with Jenkins though.

Any suggestions on how this can be integrated with Continuous Integration tool (Jenkins).
We could clone all repositories using shell script. Need to figure out a way to trigger build on any commit on the list of repositories.

Thanks for your help on this.

vcstool doesn't take uri as import option

wstool supported the notion of using a uri to pull down a remote rosinstall file. It seems that vcstool should support taking a local file or pulling down a remote yaml file for the import command.

`--skip-existing` not working for entries with a version

I am unsure whether I am doing something wrong or --skip-existing doesn't work in a way I expect it to work.

Steps to reproduce:

mkdir -p ~/tmp_ws/src
cd ~/tmp_ws
wget https://gist.githubusercontent.com/Karsten1987/fcb84771fdcdb05287aaa8029569814c/raw/b0ea145c8758e7fb72682b007a3721a61559030d/rosbag2.repos
vcs import src < rosbag2.repos
cd src/rosbag2_bag_v2
git checkout -b custom_branch
cd ../../
vcs import --skip-existing src < rosbag2.repos

Expected behavior:
the git branch within the rosbag2_bag_v2 repository is still on custom_branch.

Actual behavior:

 ➭ vcs import src --skip-existing < rosbag2.repos
=== src/ros2/rosbag2_bag_v2 (git) ===

Switched to branch 'master'
Your branch is up to date with 'origin/master'.

Export tags?

I have a bunch of repos that are all checked out to a git tagged release. When I export my workspace, the versions are all recorded as git hashes. I didn't see a way to export the tag instead. I'd like the tagged release so I can more easily understand how packages are being updated. Is this possible?

Git flow init pauses with no output

When I use the commands below I would expect to be able to initialize each repository to git flow one by one using the w 1 option. However, nothing ever prints out and the process pauses forever.

~/dev $ vcs import < robots.repos
~/dev $ vcs custom -w 1 robots --args flow init

My .repos file is structured like:

repositories:
  robots/project-1:
    type: git
    url: https://path.git
    version: master
  robots/project-2:
    type: git
    url: https://path.git
    version: develop

Upon keyboard interruption:

~/dev $ vcs custom -w 1 robots --args flow init
Traceback (most recent call last):
  File "c:\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Python37\Scripts\vcs.exe\__main__.py", line 9, in <module>
  File "c:\python37\lib\site-packages\vcstool\commands\vcs.py", line 23, in main
    return entrypoint(args)
  File "c:\python37\lib\site-packages\vcstool\commands\custom.py", line 84, in main
    debug_jobs=args.debug)
  File "c:\python37\lib\site-packages\vcstool\executor.py", line 103, in execute_jobs
    (job, result) = result_queue.get()
  File "c:\python37\lib\queue.py", line 170, in get
    self.not_empty.wait()
  File "c:\python37\lib\threading.py", line 296, in wait
    waiter.acquire()
KeyboardInterrupt

Using vcs 0.1.36 and git version 2.20.1.windows.1.

Credential input broken when importing multiple repos

When importing multiple repos that require username/password, it will ask for multiple usernames on the same line and everything will get out of wack.

This seems to be fixed by passing -w 1 to the import command to make it only use 1 thread. Can 1 thread be changed to be default?

git.py should not require a version to clone repo

Git supports the notion of cloning a repository without specifying a branch. While vcstool allows the 'version' to be optional, git.py requires a version minatorily. This requires my vcstool import yaml file to specify a version/branch. It seems one should be able to clone the default branch from the source git repository. This way the owner of the remote repository can set the default branch.

MacOS vcstool-completion not found

I am trying to use vcstool to install ROS2 on a Mac, and it doesn't seem like the vcs is being found correctly. I installed 0.2.4 via pip3, but I am not seeing anything in /usr/share. Is there something I may be doing wrong?

Thanks, and let me know if there is any other info you may need.

vcs import hangs if environment variables are in the --input path

We use vcstool in our CI server to clone repos files with lines like,

vcs import --input $CI_PROJECT_DIR/repos_file.repos $MY_WORKSPACE/src

This has been working fine for us until yesterday. Now it hangs on any import lines where there are environment variables in the paths. I also see there might have been an update yesterday? This is all going on in a docker, but I can confirm that the docker is using vcs 0.1.26.

Edit: Can also confirm that on another computer trying the same thing with vcs 0.1.22 that it has no problem with environment variables in the args.

Edit+: Can also (unsurprisingly)show that it's not actually the environment variables causing the problem. With valid manually entered paths I can make it hang. I know this doesn't really help much, but the line i'm using is,

vcs import --input /builds/DD/repos_dir/repos_file.repos /builds/DD/deps_ws/src

Edit++: If I modify the path arg to be . then it works, so I guess it's something with the way it's handling the path arg

Edit+++: Sorry for all the edits, just trying to provide as much info as possible, but I think I have shown that if the path arg is any absolute path then it hangs. Tried to import to /tmp and it hung(was in a docker, so I was root).

vcs export fails to add nesting path prefix to nested repos

Starting with an existing nested repository structure in ~/dev:

repository-group
├── .git
├── nested-repo-one
│   └── .git
└── nested-repo-two
    └── .git

and running

~/dev $ vcs export --nested repository-group

the result is

repositories:
  repository-group:
    type: git
    url: https://<url>.git
    version: master
  nested-repo-one:
    type: git
    url: https://<url>.git
    version: develop
  nested-repo-two:
    type: git
    url: https://<url>.git
    version: develop

but should be

repositories:
  repository-group:
    type: git
    url: https://<url>.git
    version: master
  repository-group/nested-repo-one:
    type: git
    url: https://<url>.git
    version: develop
  repository-group/nested-repo-two:
    type: git
    url: https://<url>.git
    version: develop

I had forgotten about the export command for a while. @dirk-thomas thanks for reminding me. Since we use nested repos in the IHMC build framework, this breaks that convenience for me. I'm guessing it's an easy fix. 😄

Cannot export all my repos

Hi, I am trying to use vcstool to export the repository states to a file. All the repos share the same folder, so that I run vcs export > repos.cfg and the output is almost OK:

folderA/repo1: Could not determine ref: fatal: No upstream configured for branch 'branch1'
folderB/repo2:: Could not determine ref: fatal: No upstream configured for branch 'branch2'

The configuration for these repos and the others are exactly the same (non of them have an upstream remote, but a origin). Why does this error exactly mean ?

python-vcstool is not in the Ubuntu apt server

The python-vcstool debian package is located in the ROS apt server. Here, for example.

Adding instructions in the README to add the ROS apt server would reduce confusion when attempting to install on a vanilla Ubuntu installation, i.e. per http://wiki.ros.org/indigo/Installation/Ubuntu

sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
sudo apt-key adv --keyserver hkp://pool.sks-keyservers.net --recv-key 0xB01FA116
sudo apt-get update

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.