Giter VIP home page Giter VIP logo

prefect-collection-template's Introduction

prefect-collection-template

Welcome!

prefect-collection-template is a cookiecutter template for quickly bootstrapping a Prefect Collection.

When you bootstrap a Prefect Collection with prefect-collection-template you'll automatically be set up with the tools necessary to build and maintain your collection:

  • mkdocs for automatic documentation generation
  • black, isort, and flake8 for automatic code formatting and linting
  • pytest for unit testing
  • interrogate for documentation coverage analysis
  • Coverage.py for code coverage analysis
  • pre-commit to automatically run code formatting and linting prior to git commit
  • versioneer for automatic package versioning
  • GitHub Actions workflows for continuous integration and deployment of your collection and its documentation

We're excited to see what you build!

Quickstart

Install cruft if you have not already:

pip install cruft

Generate a Prefect Collection project:

cruft create https://github.com/PrefectHQ/prefect-collection-template

Refer to the MAINTAINERS.md in the generated project for how to get started developing Prefect tasks and flow.

Contributing

To start contributing to prefect-collection-template, run:

pip install -r requirements-dev.txt

To test generation of a Prefect Collection with your changes, run:

cruft create .

To run tests, from the base directory of the repository, run:

pytest tests

prefect-collection-template's People

Contributors

ahuang11 avatar cicdw avatar dependabot[bot] avatar desertaxle avatar discdiver avatar giorgiobasile avatar jawnsy avatar tpdorsey avatar zanieb avatar zzstoatzz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

prefect-collection-template's Issues

Move current README to MAINTAINERS

The README currently contains information for users to get started with their new collection, but this results in a nasty diff when updating the README to contain information about the collection itself.

Moving the information currently in the README to MAINTAINERS would allow users to easily keep the documentation on how to develop and maintain their collection while also giving developers a better starting point for their own READMEs.

Improve documentation instructions in MAINTAINERS.md

I think it would be worthwhile to add some more detail to the Writing Documentation section. A quick blurb about how to create a new file for a module and what to put in the file to automatically pull in the documentation for that module along with how to add navigation for that new page should suffice.

Upgrade GitHub Actions in template

Dependabot has submitting PRs to update some of the GitHub actions in our collection repos. We can make the following updates to the GitHub Actions workflows in the template:

  • Bump actions/setup-python from 2 to 3
  • Bump actions/checkout from 2 to 3
  • Bump JamesIves/github-pages-deploy-action from 4.2.3 to 4.2.5

Improve handling of Prefect specific workflows

When community members generate a collection from our template, they get our 'Template Synchronization' and 'Add issues to integrations board' workflows. Those workflows will fail if the corresponding secrets are not configured. We should avoid cluttering the actions logs of repos that don't want or need these workflows. We should consider making those workflows opt-in during collection creation and/or skipping execution of those workflows if the needed secrets are not configured.

Why not pyproject.toml ?

I like the idea of having a template for creating the collections. As more and more python projects are moving to pyproject.toml, I'm wondering why not use that new project format for the template?

This would allow:

  • use more modern tools
  • have all the configurations in one file
  • better support for dependencies
  • Enable lock file support

Add CI/CD to automate listing of collections on conda-forge

Making prefect-managed collections available via the conda-forge channel could help improve the accessibility of prefect collections for conda users.

Since there is a submission / listing process independent of PyPI's, we'd probably want to design and implement some CI/CD that would take care of providing a valid package recipe to the conda forge reviewers and kicking off that review process.

More info on contributing packages to the conda-forge channel

The collection name and collection slug are not a valid Prefect collection name

Hello,
I developing on prefect project and I want give new name for collection and slug names. But I've got an error when I was get install command. How can I set new name?


import re
import sys


MODULE_REGEX = r"^[_a-zA-Z][_a-zA-Z0-9]+$"
PREFECT_COLLECTION_MODULE_REGEX = r"^prefect_[_a-zA-Z][_a-zA-Z0-9]+$"
PREFECT_COLLECTION_NAME_REGEX = r"^prefect-[-a-zA-Z][-a-zA-Z0-9]+$"

collection_slug = "my_collection"
collection_name = "my-collection"

if not re.match(MODULE_REGEX, collection_slug):
    print(
        f"ERROR: The collection slug ({collection_slug}) is not a valid Python module name. "
        "Please do not use - and use _ instead"
    )
    sys.exit(1)

if not re.match(PREFECT_COLLECTION_MODULE_REGEX, collection_slug):
    print(
        f"ERROR: The collection slug ({collection_slug}) is not a valid Prefect collection module name. "
        "Please ensure your collection slug is prefixed with 'prefect_'"
    )

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.