Giter VIP home page Giter VIP logo

libvcs's Introduction

libvcs · Python Package License Code Coverage

libvcs is a lite, typed, pythonic tool box for detection and parsing of URLs, commanding, and syncing with git, hg, and svn. Powers vcspull.

Overview

Supports Python 3.9 and above

Features for Git, Subversion, and Mercurial:

  • Detect and parse VCS URLs
  • Command VCS via python API
  • Sync repos locally
  • Test fixtures for temporary local repos and working copies

To get started, see the quickstart for more.

$ pip install --user libvcs

URL Parser

You can validate and parse Git, Mercurial, and Subversion URLs through libvcs.url:

Validate:

>>> from libvcs.url.git import GitURL

>>> GitURL.is_valid(url='https://github.com/vcs-python/libvcs.git')
True

Parse and adjust a Git URL:

>>> from libvcs.url.git import GitURL

>>> git_location = GitURL(url='[email protected]:vcs-python/libvcs.git')

>>> git_location
GitURL(url=git@github.com:vcs-python/libvcs.git,
        user=git,
        hostname=github.com,
        path=vcs-python/libvcs,
        suffix=.git,
        rule=core-git-scp)

Switch repo libvcs -> vcspull:

>>> from libvcs.url.git import GitURL

>>> git_location = GitURL(url='[email protected]:vcs-python/libvcs.git')

>>> git_location.path = 'vcs-python/vcspull'

>>> git_location.to_url()
'[email protected]:vcs-python/vcspull.git'

# Switch them to gitlab:
>>> git_location.hostname = 'gitlab.com'

# Export to a `git clone` compatible URL.
>>> git_location.to_url()
'[email protected]:vcs-python/vcspull.git'

See more in the parser document.

Commands

Simple subprocess wrappers around git(1), hg(1), svn(1). Here is Git w/ Git.clone:

import pathlib
from libvcs.cmd.git import Git

git = Git(path=pathlib.Path.cwd() / 'my_git_repo')
git.clone(url='https://github.com/vcs-python/libvcs.git')

Sync

Create a GitSync object of the project to inspect / checkout / update:

import pathlib
from libvcs.sync.git import GitSync

repo = GitSync(
   url="https://github.com/vcs-python/libvcs",
   path=pathlib.Path().cwd() / "my_repo",
   remotes={
       'gitlab': 'https://gitlab.com/vcs-python/libvcs'
   }
)

# Update / clone repo:
>>> repo.update_repo()

# Get revision:
>>> repo.get_revision()
u'5c227e6ab4aab44bf097da2e088b0ff947370ab8'

Pytest plugin

libvcs also provides a test rig for local repositories. It automatically can provide clean local repositories and working copies for git, svn, and mercurial. They are automatically cleaned up after each test.

It works by bootstrapping a temporary $HOME environment in a TmpPathFactory for automatic cleanup.

import pathlib

from libvcs.pytest_plugin import CreateProjectCallbackFixtureProtocol
from libvcs.sync.git import GitSync


def test_repo_git_remote_checkout(
    create_git_remote_repo: CreateProjectCallbackFixtureProtocol,
    tmp_path: pathlib.Path,
    projects_path: pathlib.Path,
) -> None:
    git_server = create_git_remote_repo()
    git_repo_checkout_dir = projects_path / "my_git_checkout"
    git_repo = GitSync(path=str(git_repo_checkout_dir), url=f"file://{git_server!s}")

    git_repo.obtain()
    git_repo.update_repo()

    assert git_repo.get_revision() == "initial"

    assert git_repo_checkout_dir.exists()
    assert pathlib.Path(git_repo_checkout_dir / ".git").exists()

Learn more on the docs at https://libvcs.git-pull.com/pytest-plugin.html

Donations

Your donations fund development of new features, testing and support. Your money will go directly to maintenance and development of the project. If you are an individual, feel free to give whatever feels right for the value you get out of the project.

See donation options at https://www.git-pull.com/support.html.

More information

Docs Build Status

libvcs's People

Contributors

dependabot-preview[bot] avatar jcfr avatar jensens avatar jfpedroza avatar pre-commit-ci[bot] avatar pyup-bot avatar tony avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

libvcs's Issues

Python 3

Using only Python 3 would make this much easier since there's access to Pathlib and type hints

`GitRepo`, `MercurialRepo`, etc.: Mirror commands

GitRepo

  • clone
    • --template
    • --bare
    • --mirror
    • --no-hard-links
    • --recursive-submodules
    • .. etc
  • pull

Differences from GitPython:

`run`: Refactor for controllability

subprocess executes immediately. it should be possible to pass this around and defer to run() or execute()

  • subprocess
  • to the extent possible, annotate and pass through to the standard library
  • the instantiated wrapper class should
    • allow introspection

    • mutability

    • terraria dreamworld:

      • potentially an option to freeze
        • tamper-proof object (is such a thing possible?)
  • ...

(For StackOverflow) Pasting MyST-Parser roles into an issue

re: https://stackoverflow.com/questions/72424748/myst-parser-auto-linking-linkifying-references-to-bug-tracker-issues

I wish I could just use link to #341 as #341 in MyST-Parser. To stay in scope I removed autodoc roles. Here is what it looks like when copy pasting (a lot of work to dredge out the issues):

libvcs 0.13.0 (unreleased)

Breaking changes

  • {issue}343: libvcs.cmd.core (including ~libvcs._internal.run.run) have been moved to
    libvcs._internal.run. It will be supported as an unstable, internal API.

  • {issue}361: ~libvcs._internal.run.run's params are now a pass-through to
    subprocess.Popen.

    • run(cmd, ...) is now run(args, ...) to match Popen's convention.

What's new

  • Commands: Experimental command wrappers added ({issue}346):

    • libvcs.cmd.git.Git

      • libvcs.cmd.git.Git.help
      • libvcs.cmd.git.Git.reset
      • libvcs.cmd.git.Git.checkout
      • libvcs.cmd.git.Git.status
      • libvcs.cmd.git.Git.config via {issue}360
  • Command: Now support -C (which accepts .git dirs, see git's manual) in addition to cwd
    (subprocess-passthrough), {issue}360

Bug fixes

  • Fix argument input for git commands, e.g. git config --get color.diff would not properly
    pass-through to subprocess. {issue}360

Internals

  • {issue}345 libvcs.utils -> libvcs._internal to make it more obvious the APIs are strictly
    closed.

  • StrOrPath -> StrPath

  • {issue}336: ~libvcs._internal.subprocess.SubprocessCommand: Encapsulated
    subprocess call in a dataclasses.dataclass for introspecting, modifying, mocking
    and controlling execution.

  • Dataclass helper: ~libvcs._internal.dataclasses.SkipDefaultFieldsReprMixin

    Skip default fields in object representations.

    Credit: Pietro Oldrati, 2022-05-08,
    StackOverflow Post

Documentation

  • Document libvcs.types

_internal/README.md

These are MIT licensed python parts. You're welcome to copy them and augment them in your project as you see fit.

However, they're not supported. Maintainers can break these APIs between patch releases without warning (no deprecation policy).

If you find something in here useful, feel free to file an issue that'd you find it helpful to stabilize the API or see it in a project of its own. Also as it's MIT licensed, you can make your own package based on it at any time.

Remote branches are not checked out (git)

Precondition: #293 is applied.

Expected: If one passes an pipurl with a ...@some-branch the branch should be picked.

Current behavior: It does not checkout the branch, but instead rebases the remote branch into master (or main). Which sometimes fails, dependent if there are conflicts or not. Anyway, the result is wrong.

Solution: Checkout the remote branch before rebase.

Abstract away pip url / url pattern support

Inspiration: https://wiki.archlinux.org/index.php/VCS_package_guidelines

Other ideas:

  • Allow registering regex's on startups (sort of similar to how mimetypes initializes as a singleton. And it also allows adding more types at runtime
  • have group regexes catch
  • Have vcs detection hint callbacks (and registry). For cases where a vcs is a certain time, a detection callback for having a git/ directory is a sign its a probably git repo. If it has common git-like files, it may be a bare git repo

This allows a directory to be given and a guess of what repo it is

but also its enough to just use the plain old repo object

Allow injecting `Executor` class to control how commands are made / ran

Default behavior is the same

Executors:

  • Default: run() w/ a pass-through to show real time stderr of progress

  • subprocess.run pass-through

  • SubprocessCommand generates functions in dataclasses you can invoke later

    Rather than immediately running a SubprocessCommand, you can introspect, pickle it, adjust it, etc.

    Git(executor=SubprocessCommand)

  • Bring your own

    If you're not satisfied and don't think a PR will do, create your own executor class.

    If you do, you may want to either 1.) override the class's #run() or 2.) pass a run_fn to override it.

Object/dict filtering

We need a way to return a list of objects. Should be:

  • Simplicity over performance
  • Not meant for speed. ~100 objects.
  • Iterable as a list through a for loop
  • Iterable as a dictionary (pk, obj) via .items()
  • Support direct lookups via a pk (e.g. git remote name)
  • Support any (fixed) type of list item
  • Type annotated

Nice to have:

  • Should be comparable to other queryset objects

    Including inner data, if possible

  • One file

  • Simple

breaking!: Overhaul

I wrote this in 2013 when I was new to python programming. I am looking back
surprised at myself:

"I wrote that?".

Over the next releases, you will see two things:

And maybe, libvcs.contrib for pip-style URLs (or it'll become a gist)

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

`run`: Move libvcs to new `SubprocessCommand`, document + test overriding

requires #337

  • run() should convert to Popen params for compatibility with SubprocessCommand

  • run() should support legacy_params (existing) to deprecate safely

class CommandPopenMixin:
    _run_fn = Subprocess.Popen


class CommandRunMixin:
    _run_fn = Subprocess.run

class Command:
    _run_fn = SubprocessCommand




class Command:
    _run_fn = SubprocessCommand

This turns git command into a subprocess command factory:

class Git(Command):
   ...


git_factory = Git(url='...')
status_cmd = git_factory.status()
status_cmd.run()
status_cmd.Popen()

Subprocess.Popen
Subprocess.run

Revision ignored if create_repo_from_pip_url is used

Expected: If one passes an pipurl with a ...@some-branch the branch should be picked.

Current behavior: It ignores the given branch and uses the default branch (like master or main).

Analysis:

  • rev is parsed correctly
  • rev is passed to the specialized classes constructor as kwarg, i.e. for Git.
  • from there rev is passed to the base class constructor as kwarg.
  • In the base class constructor all kwargs are ignored.

Solution:
If a kwargs rev is passed to the base constructor, stored it as an instance variable.

Report progress of long operations like `git clone` or `git update`

It is convenient for used of the libvcs to (optionally) be able to report progress.

It used to be reported with vcspull, which approach would you recommend to "restore" this behavior ?

I was initially thinking about having a report_progress_callback parameter initially set to None

Open question: Should libvcs depends on package like svn, GitPython or python-hglib

There are already few packages specialized in handling interaction with these repository:

I wonder if libvcs should make use of them ?

It is also nice to keep libvcs as a light wrapper on top of the executable.

In the FAQ, user could be guided to use these more specialized package once the source are checked out by libvcs ?

`GitPipURL`: `to_pip_url()`

Since pip URLs are incompatible with plain-old git URLs we need this

Incompatible with git(1): git+ssh, etc. URL, and revs, e.g. @branch-name

VCS Libs: Use shell commands, Driver objects

Git <- Shell object, wraps subprocess commands and a cwd
GitDriver <- High level tool for setting state, cloning data
Repo <- one class, implements a driver so multiple vcs can be managed via one command

just like libtmux, git/svn/etc should have params that imitate actual shell commands

e.g. for a GitRepo: repo.pull, repo.clone

Git can be used to be a subprocess wrapper around a config and a cwd.

GitDriver invokes Git's commands to in a Repo object.

GitDriver can be used to accept path_hint / something else to see if a working directory is a git directory or not.

GitDriver can also accept a scan function to pull directories that are git repos

Repo registers drivers on startups

This way Repo can iterate through drivers to see if a directory is a certain vcs

And Repo can be a generic object that can connect to a vcs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.