Giter VIP home page Giter VIP logo

fluvio-website's Introduction

Fluvio Website

Public/Nightly Websites

Cloudflare Pages watches the default branch, and updates the live site automatically after merge

If you create branches in this repo (i.e., not on a fork), Cloudflare will build and provide a url to the Hugo site for each PR. Their URLS are dynamic, based on the name of your branch + .fluvio-website-preview.pages.dev

For more info read: https://developers.cloudflare.com/pages/platform/preview-deployments/#preview-aliases

Run Website on Local Machine

The website is generated using Hugo Framework. To run the website on your local machine:

  1. Install Hugo

  2. Run Hugo

    cargo run -- hugo 
  3. Website is rendered at

    http://localhost:1313/
    
  4. Checkout WRITERS.md for formatting and Fluvio customizations.

Hugo watches for file changes and automatically updates website.

Connector reference docs

A preview of the fluvio-cms CLI can be used to update the reference docs for connectors in the Hub.

Connectors are organized inscripts/fluvio-cms/src/connectors/mod.rs, and define:

  • Protocol used by connector
  • Direction of the data flow with respect to a Fluvio cluster.
  • Location in Hugo repo where connector README is stored
  • Location in Hugo repo where the content template is stored

At the time of this writing, the latest connector docs are collected by parsing the output of fluvio hub connector list

Add new connector docs

2 areas need to be updated before fluvio-cms will support a new connector's docs

  1. For new protocols, add to the DataService enum.
  2. For new connectors, add to the OfficialConnector enum

Update existing connector docs

Downloading from the public git repo:

Ex. Download MQTT docs from github repo

$ cargo run -- connector --service mqtt --direction inbound

Using README from local git clone:

Ex. Copy SQL docs

$ cargo run -- connector --service sql --direction outbound --file /path/to/README

CLI docs

Update help text

The list of CLI commands is located at /data/cli-commands.yml

Running this command will iterate through the list of CLI commands and capture the output of the --help flag

$ cargo run -- cli

This creates files in /embed/cli/help with the filename pattern <cmd>[-<subcmd1>][-<subcmd2][-subcmd3etc].md. The help output for the command fluvio cluster list would be located at /embed/cli/help/fluvio-cluster-list.md

The templates for the CLI pages in https://www.fluvio.io/cli/ are not generated. Any commands that are added or removed may need to modify additional files.

fluvio-website's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fluvio-website's Issues

[Bug]: Update consumer CLI document with new keywords & add examples

Update CLI document with additional keywords, such as --array-map and --extra-params.

        --array-map <array-map>             Path to a SmartStream array_map wasm file

    -e, --extra-params <extra-params>...
            (Optional) Extra input parameters passed to the smartstream module. They should be
            passed using key=value format Eg. fluvio consume topic-name --filter filter.wasm -E
            foo=bar -E key=value -E one=1

Add examples.

Add options for topic compression

This should also be changed in the CLI help text, but we support

fluvio topic create testtopic --compression-type none --dry-run
fluvio topic create testtopic --compression-type gzip --dry-run
fluvio topic create testtopic --compression-type snappy --dry-run
fluvio topic create testtopic --compression-type lz4 --dry-run

Architecture Docs: Describe storage and election

  • Elections
    • Which components participate in elections?
    • When do elections occur, and how are they conducted?
    • How do Fluvio elections compare with elections in other distributed systems?
  • Storage
    • What is the layout of persisted objects?
    • How are checksums used?

Add autogenerate of This Week in Fluvio links to News template

It's really easy to forget to update the newsletter list when we create a new post. This means we can accidentally orphan a This Week in Fluvio post, (especially with #184 forcing readers to rely on this list being accurate).

We have enough info about the post through the section organization of posts, and the front-matter that we can generate a new link whenever we add a new post.

Connector development walkthrough

Current docs cover the high-level project setup, but we could use a more detailed walkthrough where we provide an example connector project skeleton and have some example commands and outputs to set expectations for how to use connectors in development.

Brainstorm new content hierarchy for website

As the project is growing and we have more documentation content we want to put on the website, we're realizing that the current hierarchy of doc pages is not the most natural fit to hold these contents in a clean and understandable way. I want to use this issue for us to collect some ideas of what we think the biggest areas of content are going to be in the coming year, and to brainstorm new possible page hierarchies for sorting that content.

Here are a few items I'm thinking of in terms of content:

  • CLI docs
  • Language clients
    • Including wasm client when we get there
  • Smart streams
  • Architecture
  • Glossary
    • A section where we briefly explain common terms like "Topic" and "Partition" that are not unique to us but rather domain-specific

Update Download page

Add next steps -> Getting started
Be upfront w/ Linux/MacOS support using tabs (like Consul), but no windows

Document cluster behavior when produced data exceeds storage limit

Fluvio Cloud specifies a 1GB storage limit, but does not describe the behavior of the system when it exceeds the limit. I think we should add documentation that describes the expected behavior when this occurs. For example should it:

  • Refuse the newly-produced records and send an error response to the client?
  • Roll the logs and drop the oldest records?

It seems to me like either of these options may be desirable, so we should make it configurable, though we should open a different issue for that. For this issue, we should identify the default behavior and figure out how and where to document it.

Add prev/next links to News template

Today I have to click back in the browser, and select another post.

It would make it a lot easier to read all of the This Week in Fluvio posts if there were links that connected them together. This is particularly useful if people visit an older This Week in Fluvio post, in order to simply communicate that there are newer posts to read.

I was thinking we might test out placing them just at the bottom of the post?

Add example of a replica assignment file for `fluvio topic create`

Noticed this option in fluvio topic create

$ fluvio topic create -h
[...]
    -f, --replica-assignment <file.json>    Replica assignment file

I've never used this option before, and I don't know what the format should look like. No one else will be patient enough to figure this out, so we should show this off with an example.

Add detail to Producer Behavior

We are going to launch the website with Architecture > Producer Behavior hidden.

Need to add detail to set expectations on Producer's behavior. There are some questions in the body that we should answer.

Add Fluvio upgrade guide

We need a doc page on how to upgrade a Fluvio cluster. Right now we do not have an actual "upgrade" command, so the process is simply to fluvio cluster delete and then fluvio cluster start. Once we have upgrade implemented, we can update the doc to reflect the new process.

Add SmartModule support w/ WASI workflow in docs

In This week in Fluvio #24 we walk through compiling Fluvio w/ WASI support.

This is a valuable workflow for a (very) technical user developing SmartModules, and it belongs somewhere in the docs so we don't lose it.

The current structure doesn't provide an obvious place for this to live, so this may need to wait for reorganization to occur first.

Update /docs/ page with SmartStream description

SmartStreams are now a core feature of Fluvio, but we don't mention them in the front docs page. We should add a section describing what they are.

We should also do some general upkeep such as describing new language clients that are now available rather than pending.

getting started tutorial for mac m1 has incorrect port mappings for kind configuration

In the getting started tutorial for the Mac M1 it suggest using Kind and exposes the K8's cluster through various node ports.

The kind config suggest mapping ports in the 31xxx range but the default ports used by the helm charts are 30xxx as you can see here:
https://www.fluvio.io/docs/get-started/mac_m1/#start-a-kubernetes-cluster

The helm chart is using different ports seen here:
https://github.com/infinyon/fluvio/blob/d71161d76c15bf8646621eab7b264d1106e6f4b2/k8-util/helm/fluvio-app/values.yaml#L27

This tripped me up and took a little bit of debugging to get the cluster up and running.

I'm guessing the docs just need to be aligned with the helm chart. I'm happy to send a PR to update the docs?

Write a tutorial for using a multiple-partition topic

Now that we have good record-partitioning semantics, it makes sense for us to put together a tutorial page that describes how to leverage partitioning. Note that this does not yet assume multiple SPU. I imagine the topics we will want to cover are:

  • How to create a topic with multiple partitions
  • How do the various Producer APIs distribute records between partitions
  • How to view the record count in each partition to see thing working
  • How to consume records from the partitions

Create shortcode for external links

As convention, we want external links to open in another tab/window.

Shortcode should look similar to this:

<a href="{{ url }}" target="_blank">

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.