Giter VIP home page Giter VIP logo

iot-plugandplay-models-tools's Introduction

Azure/iot-plugandplay-models-tools

This repository contains tools to manage DTDL models submitted to the Azure IoT Plug and Play models repository Azure/iot-plugandplay-models.

Repo structure

Wiki Spec

You can find the specs and design documents in the Wiki.

Samples

The directory samples contains sample code for .NET and Node to resolve models from any models repository following Azure IoT conventions.

Clients

The directory clients contains client tools to accelerate integration and management of models repositories following Azure IoT conventions.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit the Microsoft Open Source Contributor License Agreement.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

iot-plugandplay-models-tools's People

Contributors

ashokkhurana avatar avagraw avatar c-ryan-k avatar cartertinney avatar digimaun avatar elsie4ever avatar microsoft-github-operations[bot] avatar microsoftopensource avatar montgomp avatar msterin avatar rido-min avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

iot-plugandplay-models-tools's Issues

[Spec] Architecture and Design

After x-team review, we need to clarify:

  • Federation. Resolving from multiple repos
  • Client resolver performance #req
  • Owners
  • GitHub and JEDI
  • Dependencies

Architecture and Design x-team sign-off

x-team sign-off on:

  • DTMI -> Path (case sensitive requirements)
  • Unique IDs
  • Dependencies
  • CodeOwners
  • Federation

"As an internal partner stakeholder, I have reviewed and signed off on the conventions to be
implemented for model resolution."

Tool to pre-validate new models locally

"As a customer developing a model to later be submitted to the DMR, I have local tools for
pre-validating my models to increase my confidence that my submission will be accepted."

  • Check File/Folder consistency with DTMI
  • Check Unique IDs
  • Check Dependencies are available
  • Check DTDL conformance

[resolver] Settings phase 1

This task is to support the following initial settings, which can be optionally passed to the ResolverClient

[ ] - Include dependencies
[ ] - Use pre-computed dependencies if available.

If no settings are provided, the ResolverClient will use sensible default settings.

Node Resolver Plan

Questions we need answered

  • Separate library rather than incorporating it into the existing service SDKs
  • Location of the code
  • Namespaces
  • Signing process
  • Package distribution

Checks during PR validation pipeline

"As a customer submitting a model to the DMR, I can see checks as part of the PR process telling
me that if my model is valid or not."

  • Check File/Folder consistency with DTMI
  • Check Unique IDs
  • Check Dependencies are available
  • Check DTDL conformance

Doc updates

  • Publishing Guidelines

  • Resolution sample

  • Update existing pnp docs.ms.com

    "As an external or internal customer, I have the documentation I need to use and interact
    with the DMR."

Python resolver

Questions we need answered

  • Separate library rather than incorporating it into the existing service SDKs
  • Location of the code
  • Namespaces
  • Signing process
  • Package distribution

Push script skips valid DTMIs

Migrating Public models from Mesh https://github.com/rido-min/azure-dtdl-repository/pull/7
Validation passed, but after merging the push scripts says:

Skipping dtmi/meshsystems/txs_blereader-1.json. It doesn't appear to be a DTDL document.
Skipping dtmi/meshsystems/txs_cellular-1.json. It doesn't appear to be a DTDL document.
Skipping dtmi/meshsystems/txs_cellular-2.json. It doesn't appear to be a DTDL document.
Skipping dtmi/meshsystems/txs_cellular-3.json. It doesn't appear to be a DTDL document.
Skipping dtmi/meshsystems/txs_location-1.json. It doesn't appear to be a DTDL document.

https://github.com/rido-min/azure-dtdl-repository/runs/1197467000?check_suite_focus=true#step:4:13

wondering if the push script should fail if there are json files under dtmi that cannot be uploaded. Without detailed inspection this issue would be difficult to detect.

Metrics and Monitoring

How many failed resolutions?
What is our SLA attainment over the past whatever time period
How many new models have been published/day/week/month?
Trends for all of this

Others?

Tool to support PR approval operations

"As an internal team member processing incoming PR submissions, I have the tools I need to test, validate and otherwise verify that the model is ready to be accepted into the DMR."

  • Check File/Folder consistency with DTMI
  • Check Unique IDs
  • Check Dependencies are available
  • Check DTDL conformance
  • Check reserved words
  • Check with internal tools PolyCheck

Contribution guidelines published

Include instructions to allow external users to submit new models.

  • Add instructions to README
  • Create external PR playbook
  • PR/Issue templates

"As a customer seeking to submit a new model to the DMR, I can easily find help, guidance, and possibly tools to help me prepare and submit my new model(s)."

Certification uses the DMR

Certification Tests (Portal and CLI) uses DMR.

Catalog uses the DMR

  • Commitment from Jacob to adopt this change

"As a customer using the device certification workflow, I can leverage the DMR to resolve the models my device reports."

Migration/Deprecation plan

The existing Model Repo:

Will be shutdown when we start operating the new DMR. We need a plan to cover.

  • Deprecation/Sunset dates
  • Communicate to internal teams
  • Migrate existing models in the public repo
  • Provide options to save models in private repos
  • Advertise the dates in the current UX/CLI/Explorer
  • Communicate to existing users
  • Identify breaking changes, and impact on existing customers

"As a customer who has previously used the model repository*, I understand what will happen to those models when the new repository goes live, and I know what steps I have to take (if any) to migrate to it."
* - Until now customers could ONLY use company repos, so this doesn't actually affect models in the public repo.

Communication to partners

Prepare comms to tell the partners about the new model repo

  • New Model Repo email ready for internal and external partners
  • Update Certification requirements
  • CSS readiness

"As an external partner using PnP, I have recieved clear communication from Microsoft so I understand exactly what is happening with DMR and how that will affect me."

Public models migrated

as described in #42

  • Export/Import script
  • Validate models in test

"As a customer using existing public models, I know that they are available in the new DMR."

CLI and Resolver

EXE to be invoked from the Command line

  • uses the parser
  • uses the resolver
  • Command Line Lib
  • CI script

SDK integration plan

Figure out how to integrate with other Azure IoT SDKs

  • Track 2 ??
  • decide namespaces
  • packaging system (GPR? NuGet?)

Maintaining SLA plan

Should we have 2 instances of storage set up? Assuming we are using geo redundant storage, can we cut over seamlessly?
Should we set up the CDN to fall through to the GitHub pages (assuming we publish there too) in case of storage outage?
Do we need to do anything to protect against a CDN outage? Probably no
How will we detect that we are down? Will it trigger an ICM alert?

Resolution Samples for NodeJS

"As a customer seeking to use the resolution tools, I have appropriate samples and guidance
to help me be successful."

  • Resolve single ID in .NET
  • README

CELA, SDL, PLR signoff

Compliance sign offs

  • SDL
  • CELA
  • PLR

"As an external or internal customer, I have confidence in the service because I know that it
passed through the required Microsoft pre-deployment checks."

E2E Test for MVP

Submit PRs with different models, and edge cases

  • Unique IDs
  • Dependencies
  • Case sensitive DTMIs
  • Use IoTEx to resolver
  • Use README samples
  • AICS test

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.