Giter VIP home page Giter VIP logo

edgex-docs's Introduction

Documentation for developing with EdgeX-Docs

Build Status GitHub Pull Requests GitHub Contributors GitHub Committers GitHub Commit Activity

Local Development (docker) (recommended)

The most common use case for local development of edgex-docs will be to verify changes to the HTML. To facilitate docs verification you can run the following command:

make serve

Once running, you can view the rendered content locally and makes changes to your documentation and preview them in realtime with a browser at:
http://localhost:8008


If you only want to verify the documentation will build successfully you can run the following command:

make build

When done, you can clean up with:

make clean

Local Development (native)

In order to render and preview the site locally (without docker) you will need a few things to get started.

  1. You will need to install python and pip
  2. After python is installed, you'll need the following python dependencies:

pip install mkdocs pip install mkdocs-material==8.2.1

  1. Once you have all the pre-reqs installed. You can simply run mkdocs serve and view the rendered content locally and makes changes to your documentation and preview them in realtime with a browser at http://0.0.0.0:8001/edgex-docs.

Checking for broken links when developing docs

To check that all the links in the documentation set are valid:

  1. Install the htmlproofer plugin (native only):

    Note: if using the docker method, this is already installed in the image

    pip install mkdocs-htmlproofer-plugin
  2. Export the ENABLED_HTMLPROOFER environment variable.

    Note: This adds about 5 minutes each time a change is made, so it is recommended to do once all changes are ready.

    export ENABLED_HTMLPROOFER=true
  3. Run make build or make serve. Broken links will be listed at the end of the build process.

Warning: the check for invalid / broken links does take some time and will add significantly to the build and serve times.

"Publishing" your changes

Publishing is done by the jenkins pipeline. Once a PR is merged, the changes will be reflected on the documentation site, hosted under gh-pages branch and served by Github Pages.

The different versions of the documentation are maintained in separate branches. The main branch hosts the source files for the version that is under development as well as the following production site files:

The pipeline copies the files to separate directories inside gh-pages branch. For example, when the dev version is 2.2:

Source Production
main/docs/CNAME gh-pages/CNAME
main/docs/index.html gh-pages/index.html
main/docs/versions.json gh-pages/versions.json
main/docs_src/* gh-pages/2.2/*
jakarta/docs_src/* gh-pages/2.1/*
ireland/docs_src/* gh-pages/2.0/*

Other files such as for CI checks and guidelines are also copied from all branches.

Versioning the docs

When a new version of EdgeX is released, we version the docs as well. There are four steps to make this happen:

  1. Create a branch without production site files

    i) Create a branch from main for the released documentation The branch name should be the new EdgeX release name. For example, for 2.2, a kamakura branch is created.

    ii) Remove production site files from the branch, listed here. This is necessary to avoid overriding production files; see #680.

  2. Add the version to be added to the docs/versions.json file. This file will populate the drop down in the site deployed at https://docs.edgexfoundry.org

[
    {"version": "1.1", "title": "1.1-Fuji", "aliases": []},
    {"version": "1.2", "title": "1.2-Geneva", "aliases": []}
    {"version": "[new version number here]", "title": "[name that is visible in the drop down]", "aliases": []}
]
  1. The value placed in version property in the json above MUST match the name of the folder that contains the versioned content in the gh-pages branch. This is specified by updating the site_dir: property in the mkdocs.yml file:
site_name: EdgeX Foundry Documentation
docs_dir: ./docs_src
site_dir: ./docs/1.2 #UPDATE THE VERSION NUMBER HERE TO MATCH WHATS IN THE VERSION.JSON
site_description: 'Documentation for use of EdgeX Foundry'
site_author: 'Michael Johanson'
site_url: 'https://edgexfoundry.github.io/edgex-docs/'
repo_url: 'https://github.com/edgexfoundry/edgex-go'
repo_name: 'edgex/edgex-go'
copyright: 'Copyright © 2020 EdgeX Foundry'
...

Once this is done and merged, the build job will place content in the specified folder in the gh-pages branch.

  1. Update the docs/index.html to redirect from / to the most recent release directory. For example, if the latest release is 2.1:
<!DOCTYPE html>
<html>
<head>
<title>Redirecting</title>
<script>
    window.location.replace("2.1"); //UPDATE ME
</script>
</head>
<body>
</body>
</html>

edgex-docs's People

Contributors

ajcasagrande avatar akramtexas avatar alexcuse avatar bill-mahoney avatar bnevis-i avatar cherrycl avatar christiandarr-personal avatar cloudxxx8 avatar edgex-jenkins avatar ejlee3 avatar ernestojeda avatar farshidtz avatar felixting avatar iain-anderson avatar jim-wang-intel avatar jinlinguan avatar jpwhitemn avatar judehung avatar jumpingliu avatar lenny-goodell avatar lindseysimple avatar marcpfuller avatar mhall119 avatar monicaisher avatar presatish avatar rsdmike avatar tombrennan-eaton avatar tonyespy avatar vli11 avatar weichou1229 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

edgex-docs's Issues

Broken download links in GettingStartedSDKGo

The download links for random-generator-device.yaml and configuration.toml are broken.

They appear as:

random-generator-device.yaml <random-generator-device.yaml>{.interpreted-text role="download"}

Feel free to assign this to me.

Make JSON schema in docs readable

The current docs site has a rather unreadable formatting in Chrome for the schema, as it's not formatted at all and as such is a really long single line:

bad json output

It would be nice if at the bare minimum the JSON output was pretty printed so it scales down the page instead of across the page.

Maybe helpful for Guide 6.2.3.3. Add Device Service to docker-compose File

to enbale beginners quickly pass the guide documents, maybe some changes can be made.

  • the deviceprofile "GS1-10P5.profile.yaml" dont work well directly for ModbusPal and docker-compose-delhi-0.7.1.
    in order to make it work, I changed some of the content to following
    name: RPM
        description: "RPM speed of the motor."
        attributes: 
            { primaryTable: "HOLDING_REGISTERS", startingAddress: "8455" }
        properties: 
            value:
                { type: "INT16", readWrite: "RW", minimum: "0", maximum: "1500", defaultValue: "1"}
            units:
                { type: "String", readWrite: "R", defaultValue: "rpm"}
    name: OutputVoltage
        description: "Output Voltage driving the motor."
        attributes: 
            { primaryTable: "HOLDING_REGISTERS", startingAddress: "8454" }
        properties: 
            value:
                { type: "INT16", readWrite: "RW", defaultValue: "1"}
            units:
                { type: "String", readWrite: "R", defaultValue: "Volts"}
    name: MotorControl
        get:
          - { index: "1", operation: "get", object: "RPM", parameter: "RPM", property: "value" }
          - { index: "2", operation: "get", object: "OutputVoltage", parameter: "OutputVoltage", property: "value" }
        set:
          - { operation: "set", resource: "RPM", parameter: "RPM" }
    
  • The addressable given in the guide
    { “name”: “Motor”, “method”: “GET”, “protocol”: “HTTP”, “address”: “10.0.1.29”, “port”: 502 }
    
    needs to be modified to following(Pay attention to the format of symbols)
    { "name": "Motor", "method": "GET", "protocol": "HTTP", "address": "172.17.0.1", "path": "1", "port": 502 }
    
  • It will be much easier if you run ModbusPal.jar at the same localhost of docker. And the "Learn" mode is really useful.

Add deployment best practices in security chapter

Some best practices:

  1. Launch container as non-root user where possible
  2. Close un-used ports:
    a) source - in docker-compose only list ports that we use, cleaning up any cruft
    b) some ports will still be open for on-host inter-service requests. The micro services are on the same network, so they shall be able to access these without issue.
    c) In EdgeX secure-mode, add iptables rules to allow access only to the security proxy for all non-local accesses

Document Port Numbers Used by Services

Discussed on the 22-Jul-2019 Device Service WG call.

There isn't anywhere that the port numbers used by our services are documented. A tabular format would be handy in the case where new services are coming online and need to be assigned a port. This will allow us to see what the appropriate range of ports is (Core = 48000, Device Services = 49000, etc) for the reference implementation of EdgeX, and we can see at a glance whether a port number is already taken.

Document new query parameter support for Core Command and Device SDKs

Support was recently added to Core Command and the Device SDKs to handle query parameters for command REST requests. These parameters are copied "as is" from a Core Command REST request to the request made to the device service, which then adds them to the attributes map passed to the device service ProtocolDriver.

This issue covers adding documentation for Core Command and the Device Service SDKs to explain how these query parameters are handled.

This documentation should also mention that query parameters with the prefix "ds-".

For reference see the following PRs:

Update markdown version of "Sending and Consuming Binary Data From EdgeX Device Services" with steps for creating payloads for PUT commands.

  • During some recent work (to fix a CBOR Content Header-related issue), the need arose for creating payloads to accompany PUT commands.
  • This issue will address making updates to the markdown version of "Sending and Consuming Binary Data From EdgeX Device Services" with steps for creating payloads for PUT commands, to include the steps (including the golang code used to create the said payload).

Up-level Configuration Docs, Include Overrides (cmd line, env var)

It has been requested to up-level configuration documentation so that there's a top-level page that includes definitions for configuration settings that are common to all services. Documentation applicable to common overrides coming from environment variables and cmd line parameters could also be included. Service-specific keys and values can be defined in documentation applicable to the given service.

A proposed way to layout this content can be found here.

Edinburgh Updates

References to older versions of Docker Compose file (may not be complete):

  • security/Ch-APIGateway.rst
  • examples/Ch-ExamplesAddingModbusDevice.rst
  • examples/Ch-ExamplesAddingSNMPDevice.rst
  • quick-start/Ch-QuickStart.rst
  • security/Ch-SecretStore.rst

Edinburgh->Fuji updates

Locations in the current docs that reference either "Edinburgh" or "1.0" that may need to be updated to reflect the Fuji release:

  • ./application/Ch-ApplicationServices.rst:7:The intention of Application Services are to address scalability concerns of the existing EdgeX Export Client and Distribution Services (Export Services) as well as provide a flexible solution for exporting data outside of EdgeX without encumbering the EdgeX development community itself with trying to support all major cloud providers and export solutions. For the Edinburgh release cycle, the existing Client and Export Service remain supported and are still considered the primary way to export data out of EdgeX. However, it is encouraged for new development efforts adopting EdgeX that the App Functions SDK and resulting Application Services be leveraged moving forward with the intention that by the Fuji release, the SDK will be moved into release status and become the primary method of exporting data from EdgeX.
  • ./configuration/Ch-Configuration.rst:68: - 1.0
  • ./configuration/Ch-Configuration.rst:74: - 1.0
  • ./configuration/Ch-Configuration.rst:80:The versions shown correspond to major versions of the given services. These are not necessarily equated with long term support (LTS) releases. For all minor/patch versions associated with a major version, the respective service keys live under the major version in configuration (such as 1.0). Changes to the configuration structure that may be required during the associated minor version development cycles can only be additive. That is, key names will not be removed or changed once set in a major version, nor will sections of the configuration tree be moved from one place to another. In this way backward compatibility for the lifetime of the major version is maintained.
  • ./configuration/Ch-Configuration.rst:89:- /edgex/core/1.0/edgex-core-data/Clients
  • ./configuration/Ch-Configuration.rst:90:- /edgex/core/1.0/edgex-core-data/Databases
  • ./configuration/Ch-Configuration.rst:91:- /edgex/core /1.0/edgex-core-data/Logging
  • ./configuration/Ch-Configuration.rst:92:- /edgex/core/1.0/edgex-core-data/MessageQueue
  • ./configuration/Ch-Configuration.rst:93:- /edgex/core/1.0/edgex-core-data/Registry
  • ./configuration/Ch-Configuration.rst:94:- /edgex/core/1.0/edgex-core-data/Service
  • ./configuration/Ch-Configuration.rst:95:- /edgex/core/1.0/edgex-core-data/Writable
  • ./examples/Ch-ExamplesAddingMQTTDevice.rst:325:Download the docker-compose file from https://github.com/edgexfoundry/developer-scripts/blob/master/releases/**edinburgh**/compose-files/docker-compose-edinburgh-1.0.0.yml.
  • ./examples/Ch-ExamplesAddingMQTTDevice.rst:332: image: edgexfoundry/docker-device-mqtt-go:1.0.0
  • ./examples/Ch-ExamplesAddingMQTTDevice.rst:5:EdgeX - Edinburgh Release
  • ./examples/Ch-ExamplesVirtualDeviceService.rst:105: image: edgexfoundry/docker-device-virtual-go:1.0.0
  • ./general/conf.py:62:#version = u'1.0'
  • ./general/conf.py:64:#release = u'1.0'
  • ./getting-started/Ch-GettingStartedSDK-C.rst:25:The next step is to download and build the EdgeX Device SDK for C. You always want to use the release of the SDK that matches the release of EdgeX you are targeting. As of this writing the edinburgh release is the current stable release of EdgeX, so we will be using the edinburgh branch of the C SDK.
  • ./getting-started/Ch-GettingStartedSDK-C.rst:27:#. First, clone the edinburgh branch of device-sdk-c from Github::
  • ./getting-started/Ch-GettingStartedSDK-C.rst:29: git clone -b edinburgh https://github.com/edgexfoundry/device-sdk-c.git
  • ./getting-started/Ch-GettingStartedSDK-C.rst:59: export CSDK_DIR=../device-sdk-c/build/release/_CPack_Packages/Linux/TGZ/csdk-1.0.0
  • ./security/Ch-SecretStore.rst:156: edgex-core-consul | Version: 'v1.1.0'
  • ./security/Ch-SecretStore.rst:552: edgex-core-consul 127.0.0.1:8301 alive server 1.1.0 2 dc1

Add Swagger V1 API documents into API reference

Add ADR for V2 API Specification

This issue is a compliment to the following PR
edgexfoundry/edgex-go#2309

This issue is to track the need for addition of the ADR capturing the design principles and goals of the new V2 API discussed since the Geneva face-to-face in Phoenix, November 2019.

The name of pre-defined devices inside Virtual Device needs to be updated

The name of those pre-defined devices as listed in section 6.2.3 needs to be updated, as they don't match to what are defined in https://github.com/edgexfoundry/device-virtual-go/blob/master/cmd/res/configuration.toml

"Random-Boolean-Generator01" should be "Random-Boolean-Device"
"Random-Float-Generator01" should be "Random-Float-Device"
"Random-Integer-Generator01" should be "Random-Integer-Device"
"Random-UnsignedInteger-Generator01" should be "Random-UnsignedInteger-Device"

ADR for microservice communication security

🚀 Feature Request

Description

The EdgeX framework currently relies on network segmentation for security. An API gateway separates external traffic from internal traffic, authenticates callers, and forwards properly authenticated requests to internal services. Services behind the gateway share a common network, and communication between microservices is unrestricted.

The EdgeX framework should do a better job of protecting itself against local attacks such as unauthorized software processes running on the host or malicious containers. The secret management subsystem of EdgeX is the current sole exception to the unrestricted communication posture:

  • The secret store must enforce role-based access control policies.
  • The secret store must protect the confidentiality of secrets in transit.
  • The secret store must be available for proper system functioning.

Besides secret management, there are other sensitive services in EdgeX, such as the systems management agent, that execute with a high level of privilege and expose control-plane level functionality. These other microservices also require confidentiality and integrity protections on data, as well as role-based access controls that require the caller to authenticate itself. Moreover, not all microservice communications are many-to-many; for example, the only microservice that needs to access the PostgresSQL database is the API gateway.

Describe the solution you'd like

This feature request is a placeholder to begin discussion of:

  • Implementation of TLS security between microservices to provide confidentiality and integrity protection of messages.
  • Implementation of microservice-to-microservice authentication.
  • Implementation of a common authorization framework for microservice implementers.
  • Implementation of policy-based networking.

The solution needs to comprehend:

  • Security-enabled vs non-security enabled EdgeX stack
  • Authentication/authorization of requests proxied through the API gateway
  • Authentication/authorization of internal service-to-service requests

Describe the solution space (hand-coded option)

Confidentiality and Integrity

Server-side TLS is the most convenient option here. The security-secrets-setup tool initializes PKI for Vault and could be extended to other services. Alternatively, security-secrets-setup can be modified to create an intermediate CA that is loaded into the secret store's PKI engine, and services can directly go there to get TLS certificates.

Microservice authentication

Mutual-auth TLS is one option here. This option combines confidentiality, integrity, and authentication together, but it imposes a tax: encryption is mandatory, which could add an unacceptable level of overhead for high-bandwidth or latency-sensitive applications.

Another option is token-based authentication. There are various options for selection of a token.

  • Use the pre-existing secret store authentication token. The server could use this token to verify the client's identity with the secret store, but it could also steal all the client's secrets. Not a good solution
  • Use the secret store token accessor. The server could use this token to query metadata about the caller out of the secret store. The benefit--and drawback--of this solution is that it allows for unlimited delegation: a far-removed microservice can perform actions on behalf of the original caller.
  • Use a JWT token. The benefit of this approach is that use of an audience field can be used to prevent delegation of the token. The JWT could be signed by a JWT provider that everyone trusts, or a service could sign its own JWT. Self-signed JWT's require that the public key database be held in some kind of trusted registry (so that an attacker couldn't replace the key in the registry and then claim to be that microservice). Signed objects need to deal with TTL issues.

Microservice authorization

For the hand-coded option, this means that code must be added to each microservice to authorize each operation. This creates a high barrier for developers.

Network policy

Policy-based networking can be done to some extent with Docker networks. It is an effective even if authentication and authorization fail.

Describe the solution space (service-mesh option)

Many microservice architectures introduce a service mesh to deal with microservice encryption and authentication. Some popular service meshes include, but are not limited to:

  • Consul Connect
  • Linkderd
  • Istio
  • Envoy
  • Conduit

Service meshes often solve the problems of confidentiality and integrity, authentication, authorization, and network policy at the same time. A network intermediary could combine authorization and network policy by blocking unauthorized requests. However, fine-grained authorization might require special support from the service mesh, such as by adding context to a request before it is delivered to a microservice.

Services known to be incapable of microservice communication security

  • Redis. Redis supports only a single password and does not support TLS at all.

GettingStarted updates (Edinburgh)

Per Michael Hall the docs.edgexfoundry.org site is published from the edinburgh branch.

To promote a successful user and developer experience with EdgeX until Fuji release we should update/release this hosted documentation.

Note that the latest documentation in master branch has already been recently updated (with PR#12).

Indication
The current documentation hosted at https://docs.edgexfoundry.org/Ch-GettingStartedUsers.html (for example in section 4.1.3.2) points to a broken link for the docker-compose files, which are now contained in a 'releases' subfolder:

The collection of the EdgeX Foundry Docker compose files are found here: https://github.com/edgexfoundry/developer-scripts/tree/master/compose-files

Correct link for above ^:
https://github.com/edgexfoundry/developer-scripts/tree/master/releases/

Document minimum recommended integrations for new services

Document what API implementations and integrations a service should have to work properly with an EdgeX deployment. This should include:

  • Using the Registry service for configurations
  • Connecting to and using the Logging service
  • Implementing API endpoints for System Management Agent

Update Go Developers Getting Started guide

Currently the go developer getting started guide states that we use go 1.9 and that newer versions of go have not been tested. This is causing issues for new developers that come to the project that explicitly install go 1.9 not knowing that EdgeX now uses modules which isn't official until go 1.12. Following the current guide leads to an instant dead end on getting the project built. It should also be recommended that the repos be cloned outside of your gopath contrary to what the document states again because of modules.

https://github.com/edgexfoundry/edgex-docs/blob/master/getting-started/Ch-GettingStartedGoDevelopers.rst

0005-Service-Self-Config ADR Discussion Items

Tracking discussion/review items from Core WG 27-Feb-2020 on the referenced ADR.

Please refer to the Working Group meeting notes for more information.

  • It was agreed that environment variables should override all source configuration. That is, whether a configProvider has been specified or not (local file) if an environment variable override is present, it will always take precedence.
  • We discussed the use of the --override switch and how it could cause some issues if a service is restarted with this flag. In particular, if the service's config is present in Consul and was manually edited for some reason, that manual edit will get clobbered. Call attendees thought it would be good to add Use with caution verbiage.
  • --profile will continue to be supported and someone could continue to support the current Docker profile in this way. However the project team has agreed that we would like to eliminate the res/docker configuration.toml files and instead provide the subset of overridden values to support docker in the Geneva release docker-compose file.

@jpwhitemn I'll assign this issue to you for any follow-on edits to the ADR and for you to associate with the resulting PR

@tonyespy @lenny-intel @michaelestrin @AnthonyMBonafide for visibility

Update Metadata Docs for DeviceProfile Contract Changes

pkisetup tool incorrect link

The link to the PKI setup tool in this sentence

The CA name (EdgeXFoundryCA) was defined by the pkisetup tool during the Vault image build process. This tool is also responsible for all the TLS server configuration and creation tasks.

is incorrect.

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.