Currently, i am working on this https://github.com/icf20/espanso-oficina/blob/main/packages/oficina/0.1.0/README.md

Based on the documentation, there is no way for make a package that works only on certain programs or URL? did i understand that correctly.

To do what i want to do, users need to manually move and rename files? is this correct?

Hi all!

I'm writing this issue to align all the team and contributors on the current state of the Espanso Hub. Hopefully, this will give everyone enough context to participate in the discussion.

The core principle that motivates many of the choices detailed below is: Security should be a top priority.

Current state

The Espanso Hub is currently stuck. The last package was merged 9 months ago, and we have >20 packages waiting to be reviewed and merged. This is my fault and I'm sorry about it, I became a bottleneck because I'm (currently) the only one with permission to approve and merge packages, and I've been unable to dedicate time to it in the past few months.

Motivation

The community raised some really valid points in espanso/espanso#1742, which I'll try to address here

Why can't package authors update their packages independently? That would speed up the process

Nothing would stop a package maintainer from publishing a good looking package, and then publishing an update that contains a match like:

  - trigger: "a"
    replace: "{{output}}"
    vars:
      - name: output
        type: shell
        params:
          cmd: "rm -Rf ~"

so that as soon as the user types a, all their files are gone. Script and shell extensions are particularly dangerous.

On one hand, I'm against walled gardens (like Apple's AppStore), but on the other, I'd love to maintain the user expectation that every package they download from the Hub is vetted and safe. There are approaches to mitigate the effort (like improving automatic checks), but removing the review process (even for known package creators) could lead to dangerous packages being released. Keeping in mind that users can still share their packages with others outside the Hub, I still believe that this model is better (although scaling the reviewers team is for sure necessary, because I'm definitely a bottleneck here)

Seems to me you just gave excellent reasons for excluding script and shell extensions from the hub, on principle.

Excluding shell and script extensions is an option, but would limit the usefulness of many packages currently available in the Hub. Examples:

• https://hub.espanso.org/get-ip
• https://hub.espanso.org/rand-tools

Proper human reviews are the only solution I can think of to allow those useful packages without posing a security risk for users

Maybe create a CI that validates the packages?

We already do: https://github.com/espanso/hub/tree/main/.github/scripts/validate
There's a list of rules currently being checked, and when run, they verify that the package satisfies a set of must-have conditions.
Unfortunately, there's a significant problem with the current approach, and again, it's motivated by security.

A bit of context:

Every time a package is merged on the main branch, a Github Action pushes it on the hub-frontend: https://github.com/espanso/hub-frontend so that it can be displayed in the Hub
This action is a sensitive operation, because it requires a token that grants write access to the Espanso Hub

If you want to run GitHub actions against PRs opened by external contributors, things get tricky. If permissions are not configured correctly, a malicious user could modify the GitHub actions configuration as part of the PR and extract secrets (for more information, see: https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/#improvements-for-public-repository-forks). At the time, I went with the easiest solution that also guaranteed good security: on each new PR, I would manually review that the PR did not change the GitHub actions configuration, and then approve the workflow manually. Clearly, this does not scale very well.

A solution to allow automated workflows while also guaranteeing good security exists, but it takes time to configure correctly, and unfortunately I didn't have capacity in the past few months. See: https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

Reality check: How does Maven Central do it? Or PyPi? Or Rubygems? Or sdkman? Or GitHub Packages?

That's a good question. Due to the nature of Espanso, most users are fairly technical. That said, downloading a package from PyPi and from Espanso might trigger different expectations: when you download a python package, you are downloading an executable, which is dangerous by nature. For many users, Espanso packages might seem safer (most of them are just string replacements after all), and with the Hub, I wouldn't want to break this expectation. If a package is on the Hub, it should be safe to run (dangerous packages can be hosted in external repositories, if necessary)

Thoughts?

cc @AucaCoyan @smeech

It's currently difficult to find the source of a package without manually searching for it yourself. Thus, it would be great if there was a link back to the source listed on every package's page. It would also help users with being able to trust that the package they are installing is safe.

https://hub.espanso.org/calc-macos, the page linked to from the topmost item in https://hub.espanso.org/search?t=math just shows a blank page rather than the package page like the rest I have viewed.

To check the current PRs, I use act, but checking pr 91 this happened

The issue I had is that python can't create a file because is already generated 🤔 . Maybe the script doesn't delete the file if it exists, or it doesn't overwrite the file in any case, or it's a permission issue.

This is a main blocker to check the dated PRs, so we need to fix this soon to continue

Hi @federico-terzi, Would you mind adding hacktoberfest topic to this repo (and possibly other), so it can participate in this Yer Hacktober Fest (https://hacktoberfest.com/)? I'm going to contrubute to this repo next month and I would like my contribution to be counted during this contest. I'ts free from your perspective and might bring more contributors to your repos.
Thanks.

Here is example of how to add topic

To trigger a new deployment of the frontend application, the publish-packages workflow should send a repository_dispatch event to the frontend repo webhook.

For example, via curl as an additional step after publishing the packages.

The JSON data to be posted has to follow the below schema:

{
   "event_type": "new-hub-release",
   "client_payload":{
      "PACKAGE_INDEX_URL": $newPackageIndexUrl
   }
}

A personal access token with repo permissions is required to perform the webhook call successfully.

Hello,

I have been making a package following the steps in the documentation (I forked and cloned this repo, duplicated the dummy folder, wrote stuff in the package.yaml file, etc. ) and I want to be able to test out my package as I am writing it. I tried simply dragging the folder to the directory with all my installed packages (/Library/Preferences/espanso/packages) but it is not showing up. Thanks!

This new package system strives to accomplish these goals:

• The package submission process should be easy for contributors
• The verification process should be easy for maintainers

New package format

In its basic form, a package is composed of 3 files:

• A _manifest.yml file containing the package metadata:

name: "dummy-package"
title: "Dummy package"
description: A dummy package for testing
version: 0.1.0
author: Federico Terzi

• A README.md file containing the package description (and documentation, if necessary)
• The package.yml file, which contains the actual snippets:

matches:
  - trigger: ":hello"
    replace: "world"

These three files MUST be present in each package, but the package might also contain other files, such as other yml files containing more snippets.

To recap, the package directory content should be:

_manifest.yml
README.md
package.yml

Checks

In order to maintain a good quality while minimizing the amount of time spent by maintainers to keep the hub running, it's critical that we implement a series of automated checks that verify PRs against a set of basic rules.

These should run on each submitted PR, as well as each repository push. The goal is to always keep the repository consistent and to be notified when something is off.

Each of the following sections describes a check that we should implement through a Github Actions job.

Naming convention

Package names should be only composed of alphanumeric characters and the hyphen - character. For example:

• best-emojis and emojis123 are valid names
• best_emojis and best@emojis are not.

NOTE: this should also be enforced at the path level, meaning that the name should follow this convention both in the _manifest.yml file and in the actual package structure, as described in the "Path coherence" section below.

Manifest format

Each _manifest.yml file MUST contain the following fields: name, title, description, version and author. If any of them is missing, the check should fail.

Required files

Each package directory MUST contain the _manifest.yml, README.md and package.yml files. The check should fail if any of these files is missing, but packages can include other additional files.

Path coherence

Packages should follow a standard format and location. For example, a package named best-emojis and having version 0.1.1 should be located in the packages/best-emojis/0.1.1/... directory

If a package has multiple versions, we should follow this structure:

packages/
  best-emojis/
    0.1.0/
      _manifest.yml
      ...
    0.1.1/
      _manifest.yml
      ...

This path should be coherent with the contents of the _manifest.yml file, in particular, the name and version attributes of the manifest should reflect the actual path structure.

YAML syntax

All YAML files contained by a package should be valid YAML. Therefore, we should always attempt to parse each .yml and .yaml file and fail if a syntax error is present.

It is very boring for a new update do a new PR everytime manually creates everything etc and the documentation doesn't help with something automatic.