qiskit / qiskit-bot Goto Github PK

We should have an option to include a link to the hosted release notes in the generated changelog output. The changlog output is just a subset of the git history for the release and the full release notes contain all the details. To do this we likely need a config flag somewhere to give qiskit-bot a hint to the url for the release notes and then it can generate the version suffix for the url to link to it.

In order to better track PRs from the Qiskit community, it would be great if qiksit-bot could tag as community PR PRs that are not from members of the organization. The data seems to be available as author_association in the PR object https://api.github.com/repos/Qiskit/qiskit-terra/pulls?state=open

Right now qiskit-bot's release automation is triggered by by an authorized contributor pushing a tag to a tracked repo which triggers the changelog generation, github release page creation, etc. However, in practice this has proven to be somewhat error prone, while mistakes in manually tagging aren't common they happen occasionally (typically either tagging the wrong branch or the wrong commit). These mistakes are avoidable though if we just have qiskit-bot do the tagging.

A workflow I've been thinking about since the aer 0.4.2/0.5.0 release (which was caused by an admin on aer at the time tagging 0.4.2 on master instead of the stable branch) is to trigger the release automation from the release prep PR somehow and let qiskit-bot do the tagging (either directly or via github's release api). The two trigger conditions I'm debating between are either a PR merge with a release tag, or a user comment like @qiskit-bot release on a merged PR. I'm leaning towards the latter because it enables us to limit the users via configuration on who are allowed to trigger a release (more users are allowed to tag a PR than just those with write permissions) and it's more explicit in behavior.

Assuming we go with a comment trigger the expected behavior here would be adding a config field to list the authorized usernames on a repo, then on each PR comment check for the expected trigger comment text (probably just @qiskit-bot release) that will then trigger the tag creation from the sha1 of the merged commit for the pr (merge_commit_sha in the PR details). The rest of the release workflow should be the same.

As pointed out in: #9 (review) a better forward looking default value for the default_branch config options added in #9 is main. Once the qiskit repos currently using qiskit-bot (qiskit/qiskit, terra, aer, ignis, aqua, nature, optimization, finance, and machine-learning) all migrate to use main for their default branch we should change the default value.

For large releases, being able to tag a PR as explicitly not needing a changelog entry would be helpful in order to triage which PRs have and haven't yet had their changelog status reviewed. This could be from a label like 'Changelog: None' which exclude the labeled PR both from any of the changelog categories and from the ## No changelog entry list.

It would be great if we could have the qiskit bot do certain tasks related to activity from external community members. There are a few different parts to this outlined below, any further comments welcome :)

Part 1 - when HW/GFI label is added to an issue

1. create new webhook function for event_type=issues
2. when GFI or HW labels are added to an issue (i.e. data['action']=labeled and data[label]=gfi or hw) do the following:
   • add that issue to contributor monitoring project (github org level project beta)

I don't think PyGithub has support yet for editing github projects (beta), but they are editable via the github graphql api, if we can add a python graphql library or something

useful docs:

• webhook api ref
• github graphql api for updating projects

Part 2 - comments on HW/GFI issues from external contributors

1. create new webhook function for event_type=issue_comment
2. when a comment is left on an issue by a non-member (i.e. data['author_association'] != MEMBER) with GFI/HW (i.e. data['issue']['label']['name]=gfi or hw) do the following:
   • trigger notification for community-reviewers team

useful docs:

• webhook api ref

It would be great if the bot can comment when somebody modifies qiskit/visualization/ with the binder to test the visualization module: https://mybinder.org/v2/gh/<github_user>/<repo>/<branch>?urlpath=apps/test/ipynb/mpl_tester.ipynb and that can locally be run with jupyter notebook test/ipynb/mpl_tester.ipynb.

In private discussion with @mtreinish, we realize that people tend to ignore the qiskit-bot PR notification (for example: Qiskit/qiskit#8592 (comment)). A wild idea we have is to use some kind of AI service to generate the same message using different tones each time to make it more varied so that people will pay attention to it.

The simplest way to implement this is just to generate a bunch of these messages offline and store it in this repo and the bot randomly pick one version of it each time.

Since the release of qiskit 1.0.0 the changelog generation that runs as part of release process has been failing. Looking at the logs the git log command that's generated is not valid and getting confused by the concept of major versions (which to be fair for almost it's entire life weren't a thing). For example:

2024-05-16 20:02:49,238: 350 ERROR qiskit_bot.git [-] Failed to get git log
stdout:
b''
stderr:
b"fatal: ambiguous argument '1.1.0...0.0.0': unknown revision or path not in the working tree.\nUse '--' to separate paths from revisions, like this:\n'git <command> [<revision>...] -- [<file>...]'\n"
Traceback (most recent call last):
  File "/qiskit-bot/qiskit_bot/git.py", line 77, in get_git_log
    res = subprocess.run(['git', 'log', '--oneline', sha1],
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['git', 'log', '--oneline', '1.1.0...0.0.0']' returned non-zero exit status 128.
Process Process-3:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/qiskit-bot/qiskit_bot/release_process.py", line 259, in _finish_release__changelog_process
    create_github_release(repo, log_string, version_number,
  File "/qiskit-bot/qiskit_bot/release_process.py", line 213, in create_github_release
    changelog = _generate_changelog(repo, log_string, categories)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/qiskit-bot/qiskit_bot/release_process.py", line 142, in _generate_changelog
    git_log = git.get_git_log(repo, log_string).decode('utf8')
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'decode'

We should fix the release processing to correctly generate the git log from 1.1.0...1.0.0 as intended.

The only manual step that's still part of the release workflow (besides reviewing, approving, and merging PRs) is bumping the element version after a successful release. For example, see:

Qiskit/qiskit#3803

We should add support to the release automation path of qiskit-bot to do this automatically. This way it's less error prone and something we won't forget after pushing a release.

This automated comment is useless and noisy if the contributor has already made fifty PRs. (or five or maybe two).

Is there a way to check if the contributor has already made several PRs to the repo in question (eg. qiskit-terra) ?

Or even keep a static list of frequent contributors? I poked around in the code a bit. But I don't see where the github id that opened a PR is available.

Github doesn't natively give granular level of control for notifications on subdirectories. If an active contributor wants to get notified of open PR or issues that only effect a subset of the repo their only option right now is to subscribe to the whole project and get emails for every piece of activity. Using codeowners has been tried to solve this problem for those users who have write access to the repo, but it's not ideal because this couples conditional notifications with approval authority.

Adding a feature to qiskit-bot should not be hard to implement this, it already has all the capabilities to implement it. When qiskit-bot is configured to watch a repo it already knows how to read a local config file (for configurable tags) and it also gets events for every event. The missing piece is the step to take in PR and issue events and check them against the local_config_file for a configured element to see if a user should be notified. If there are users who need to be notified the bot can leave a mention for each configured user as a comment on the PR or issue. This will then generate a notification email for users with github's default notification settings.

While an original motivating feature for the development of the qiskit-bot release automation was to handle the metapackage sync between all the qiskit elements and automate as much of that process as we could. With the release of qiskit 0.44.1 the use of the metapackage has been official retired. Moving forward all qiskit releases are from a single repository and we don't need all the complex logic involved with bumping version numbers and requirements files anymore. While it's not a strong priority right now, we should drop all the metapackage functionality from qiskit-bot to simplify the code base.

The bot tagged one of my Terra PRs (Qiskit/qiskit#8627) with "Community PR", even though I appear as a member of the Qiskit org. When I curl'd the API hook for that PR, my association appears as "CONTRIBUTOR", not "MEMBER" (what the bot checks for).

I think we might want to swap to making a second API request to the pr["user"]["organizations_url"] API point to ask if the user is part of Qiskit directly, rather than trying to infer it from the association, since contributor seems to take precedence over member.

cc: @javabster (since I couldn't assign you for some reason).

Looking to future releases it would be good to support doing release candidate releases prior to publishing a final release. To support this qiskit-bot will need to be improved to recognize a prerelease tag (vs a real release flag). On a pre-release tag qiskit-bot should create the stable branch, and the changelog, but not update the metapackage. Then on subsequent release candidates from the stable branch we should only generate the changelog. Then on the full release we will treat it like a stable point release but for the changelog generation we will want the from the previous full release to the new one (not just the incremental from the last release candidate) and to update the metapackage. We also probably should add a config flag to say whether the project supports release candidates or not because it does influence stable branch creation.

• ask to add release notes if needed
• ask to add test if needed
• qiskit-bot to tag Documentation and Changelog: None if the PR is only contain documentation changes

Qiskit bot already watches the meta-package repo when it opens up a version bump pr like:

Qiskit/qiskit#802

so it can close it when the commit is finished. The next step of automation is that the bot should automatically create a release from the merged pr (which will trigger the travis job to upload the sdist). It also already has all the information for the release description with the updated versions since it uses that as the commit message. Right now this is a manual process and error prone and it should be automated.