This is a request for help with spec for the dialog CLI.

See Specs folder for sample templates, but at the very least we need to know what should come there, the list of command groups, options, corresponding functionality and existing implementations reference.
The spec must be reviewed before any substantial coding begins (other than maybe preparatory work).

The Contribution Guide should provide sufficient guidance for Microsoft developer (first, and maybe eventually OSS community) to add a command group from scratch to manage a related Bot management functionality.

The guide should include:
• Pre-requisites, and dependencies
• Required prior knowledge
• Structure orientation
• Step by step contribution "get started" guide along with source code snippets or pointers as necessary
• Acceptance requirements:
○ Style, lint
○ Test coverage requirements
○ Best practices
○ Authorized review any special submissions requirements
• Usage of common utilities such as global configuration

After installation bf cli will prompt the user the telemetry preference the first time the tool is used. If the first command has piped data the telemetry prompt will use the this as input.

Please review the initial spec (draft placeholder) for CLI Foundation Spec.

• Provide further guidance to what should or should not be there.
• Trim as needed (this is to inspire, not mandate)
• Ensure sufficiency for spec review
• Consider to use a sample for future specs (for CLI plugins or otherwise).
• Examine corresponding new-cli-feature dev spec template
  Note: @cleemullins

Tool

Name: Ludown
Version: 1.3.3
OS: Windows 10

Describe the bug

Ludown cannot parse utterance "[[this]is] a new form (a | b)"

To Reproduce

1- Create lu file
2- Add Intention with utterance "[[this]is] a new form (a | b)"
3- Run command "ludown parse toluis ..."

Expected behavior

The file is parsed and the json file is generated

Screenshots

[ERROR]: Invalid LU File Ref: "[[this]is] a new form (a | b)".
 Reference needs a qualifier - either a #Intent-Name or #?
Stopping further processing.

[bug]

Tool

• Name: Ludown
• Version: 1.3.3
• OS: macOS 10.14.6

Describe the bug

Ludown cannot translate text when used as a library, most likely due to the use of this and export

To Reproduce

Steps to reproduce the behavior:

const ludown = require('ludown');

// valid values not need to reproduce
const key = 'abc';
const text = '# Intent';
const lang = 'xx';
ludown.translate.parseAndTranslate(text, key, lang).then(console.log, console.error)

Expected behavior

The text is translated

Screenshots

TypeError: this.addSegment is not a function
    at Object.parseAndTranslate (/<snip>/node_modules/ludown/lib/translate-helpers.js:76:26)

Additional context

The functions parseAndTranslate and addSegment are both methods of the translateHelpers object, however parseAndTranslate is being exported alone, which is probably why this.anything cannot be resolved. Exports.js should export the full object, not just a method, or else the functions should be converted to not use this

A workaround is to import the helper directly:

const translateHelpers = require('ludown/lib/translate-helpers')
translateHelpers.parseAndTranslate(text, key, 'xx').then(console.log, console.error)

[bug]

Review command line guidelines and ensure chatdown conforms to new guidelines.

Tracking.

Although most CLI will be within GitHub repos, I think we need some basic documentation, maybe full sample scenarios or pointers to basic tool usage, downloads?

Consider/discuss?

We need a spec and implementation for LuisGen

Automate the process of adding, removing, and updating skill configuration for a Bot Framework Bot.

Core Skill Registration Steps (Action and Utterance Invocation)

1. Validate Skill Manifest. Specifically to ensure Id, AppId, SkillEndpoint, etc. is valid. Given the current SDK approach requires developers to create this manually it leads to many errors.

2. Extract core manifest information and update Bot configuration. Id, AppId, Endpoint, Name and Description are required on the Bot side. AppId is used as part of the secure skill communication mechanism.

"botFrameworkSkills": {
"id": "<SKILL_ID>",
"appId": "<SKILL_APPID>",
"skillEndpoint": "<SKILL_ENDPOINT>",
"name": "<SKILL_NAME>",
"description": "<SKILL_DESCRIPTION>"
}

3. Populate skillHostEndpoint Uri. This setting is part of the Bots configuration file and provides the endpoint that a Skill calls back with response activities. Not setting this leads to obscure HTTP 500 errors and is highly confusing to developers. As developers move from dev envrionments through to production this needs to be set as part of CI/CD. Can also be looked up based on other information in configuration.

   "skillHostEndpoint": "<VA-SKILL_ENDPOINT>"

Additional Skill Registration Steps required for Utterance Invocation scenarios

1. Retrieve LU training data. Manifest will point to LU source(s), one per supported locale. These need to be resolved and contents downloaded. Local File, Remote File and also directly from the LUIS model for Microsoft-centric and enterprise scenarios whereby a separate static file is not required.

2. Filter out unneeded intents. Manifest optionally enables certain intents to be "exposed" (made public) so LU data may be filtered down to just the intents the manifest specifies. May not be needed. No intents specified will default to all intents.

3. Train Dispatcher. A new "label" is added to Dispatcher/Orchestrator and the training utterances from previous step are added. This label is mapped to the Skill Name enabling the Skill to be identified at runtime.

4. Multi-Locale. Above steps are repeated for each and every locale. Each locale will have a different dispatch model.

5. Strongly typed class. Generated an updated Dispatch "model" class ensuring the (now extended) Dispatch result can be deserialized by the assistant. This avoids any code changes when adding a Skill.

Additional Considerations for Utterance Invocation Scenarios

• Utterance Invocation in multi-locale scenarios requires a consistent way of modelling the cognitive models in your configuration (e.g. Each locale will have a Dispatcher, N LUIS Models and N QnAMaker knowledgebases). These need to be reliably retrieved at runtime for both the bot but also CLIs that are refreshing dispatcher. This is problematic for a "SDK" which doesn't want to impose too much structure/regiment on developers - easier for an assistant platform layered on the SDK.

• When registering a series of Skills the dispatch retraining cycle between each skill is less than ideal especially in CI/CD pipelines. We have implemented a "noRefresh" flag where you can skip dispatch refresh until say the last skill registration in a batch meaning you only refresh dispatch once.

The LUIS portal provides some useful training evaluation analytics about my LUIS model:

As a developer, I would like to have an API that returns this data so I can invoke it from my integration tests and perform assertions on the status of the training model, for example:

[Fact]
public async Task AnalyticThresholds()
{
    var analyzer = new LuisAnalyzer();
    var sut = await analyzer.AnalyzeAsync("flightbooking.lu");

    Assert.True(sut.IncorrectPredictions.Count == 0);
    Assert.True(sut.ImbalancedIntents.Count == 0);
    Assert.True(sut.UnclearPredictions.Count == 0);
}

Preferably, I would like the LuisAnalyzer to work locally on my machine, but it may also acceptable to pass an authoring key and have the analyzer create, train and tear down my model based on the lu file or a json file.

• Investigation (e.g. app-insights or others)
• PII
• Instrumentation

Tool

Name: Ludown
Version: 1.3.3
OS: Windows 10

Describe the bug

When traslating to zh-Hans, ',' in phraselist is translated to chinese '，', '、'.

To Reproduce

Steps to reproduce the behavior:

1. add phraselist to lu file
2. ludown translate --to_lang zh-Hans

Expected behavior

Still ','.

[bug]

• Linting, Parser, Pattern for cmd line (file struct for opts):
  • Version, Help, Learna or equivalent

Need Information for compliance item:
Compliance 151124: [CA-1] Clients Must Support the Ability to Securely Upgrade its User Base
(details in Email)

• Investigate: cp az cli

We need to have a consistent deprecation message for all old tools (along with deprecation roadmap).

Example:

|-------------------------------------------------------------|
|                         DEPRECATED                                           
| Ludown CLI has been deprecated.                                  
| All functionality has been migrated to the new BF CLI.  
| See also: https://aka.ms/newbfcli                                   
|-------------------------------------------------------------|

Specifically:

1. Need to show all configuration values. Use Show (not get)
2. Config:show displays all values whereas Config:show:telemetry shows only telemetry values

When trying to create a new plugin:

c:\botroot\cli>npm run build

> root@ build c:\\botroot\cli
> lerna bootstrap && lerna run build && lerna bootstrap

lerna notice cli v3.13.2
lerna info Bootstrapping 6 packages
lerna info Symlinking packages and binaries
lerna info lifecycle @microsoft/[email protected]~postinstall: @microsoft/[email protected]

> @microsoft/[email protected] postinstall c:\workspace\botroot\cli\packages\cli
> node scripts/postinstall.js

lerna success Bootstrapped 6 packages
lerna notice cli v3.13.2
lerna info Executing command in 1 package: "npm run build"
lerna ERR! npm run build exited 1 in '@microsoft/bf-cli-command'
lerna ERR! npm run build stdout:

> @microsoft/[email protected] build c:\workspace\botroot\cli\packages\command
> rm -rf lib && tsc -b


lerna ERR! npm run build stderr:
'rm' is not recognized as an internal or external command,
operable program or batch file.

Create a way to view my current configuration settings, and update corresponding privacy section in the readme.

bf config:show

Npmjs.com: Fix Install command, and ToC (double usage/commands)

As soon as the new tools is live, GA add the following message to the old tool.

#127

Noted during a quick sanity test. Prompt showed up every time on my machine (default is off, enable doesn't take effect).

Per this issue: microsoft/botbuilder-tools#1227

We should add the ability to pass in an ARM token via file to get around the problem with Windows taking in command lines with a length of less than 8192 characters.

After testing the command in VS Code, it's not a problem in the code base. The dummy token wasn't rejected because of its length, just because it was an invalid token.

This requirement on length is actually due to Windows: https://support.microsoft.com/en-us/help/830473/command-prompt-cmd-exe-command-line-string-limitation

After searching for workarounds, it seems that using a relatively unladen service principal might be a valid workaround. Alternatively there could be an enhancement to the CLI tool in the future that makes it possible to pass in a file that contains the ARM token instead of passing it in directly via the cmd.

@emiliomunoz FYI

Comment from the relevant issue.

We need to ensure the metrics we collect are accurate & complete.

1. Need a dashboard (as simple as set of queries)
2. Need to run UT as part of regular releases/checkins

The current luis CLI provides some primitives to create, publish, train, etc. LUIS models but these are too granular and require writing some additional powershell to wrap them into common operations.

As a developer, I would like to simply say:

luis push -luFile FlightBooking.lu -luisAppId MyAppId-luisAuthoringKey myAuthoringKey -luisSubscriptionKey mySubscriptionKey -luisAppVersion myVersion

And have my model created, trained, published and ready to use.

• luisSubscriptionKey: can be optional but it should get assigned to my app if provided
• luisAppVersion should be optional and If it is not provided, it should just overwrite the latest or create a 0.1 one.

To get a a model, I would like to just say:

luis pull-luFile FlightBooking.lu -luisAppId MyAppId-luisAuthoringKey myAuthoringKey -luisAppVersion myVersion

And get a lu file with the app from LUIS:

• luisAppVersion should be optional and If it is not provided, it should just get the currently active and published model.

Additional info

• A sample LuisPull written in powershell can be found here
• A sample LuisPush written in powershell can be found here
• These two scripts rely on a helper powershell file that uses the LUIS primitives and can be found here

Per email/OneNote communications fix QnAMaker command group

Issue

We have multiple implementations of transcript stores

• Azure Blob Storage
• CosmosDb
• File system
• etc.
  But we don't have a way to easily access transcripts from my developer box.

Proposed change

We should create a transcripts cli which allows you to consume and manipulate transcript stores.

Example usage

c:\ transcripts list conversations --channel=teams --azureStorageKey=xxxxx --botId MyBot
<show list of conversations Ids>
c:\ transcripts get conversation -channel=skype --convereation=conversationId --folder=folderPath 
<outputs the .transcript file>
c:\ transcripts delete conversation -channel=skype --conversation=conversationId 
transcript activities have been deleted for skype/conversationId

Component Impact

No component updates, it's a new component.

Customer Impact

None

Tracking Status

Docs

• PR
• Merged

Tools

• PR
• Merged

[dcr]

• Version
• Help
• Verbose-output
• Quiet
• File-capture (ala Out-File) (cp az cli)

As soon as the new tools is live, GA add the following message to the old tool.

#127

As soon as the new tools is live, GA add the following message to the old tool.

#127

We need a PM spec for MSLG.

See Mini Spec template and Contribution guidelines in specs folder

• Inspection
• Management
• Retention

We need more guidance on how to create a plugin, how to answer these questions in prompt (for internal contributions)

? npm package name 
? description 
? author 
? version 
? license 
? Who is the GitHub owner of repository (https://github.com/OWNER/repo) 
? What is the GitHub name of repository (https://github.com/owner/REPO) 
? Select a package manager 
? TypeScript 
? Use tslint (linter for TypeScript) 
? Use mocha (testing framework) 
? Add CI service config (Press <space> to select, <a> to toggle all, <i> to invert selection)

Tool

Name: Ludown
Version: 1.3.3
OS: os version

Describe the bug

Hospitality.txt
See attached. Failed to translate this by
ludown translate -k XXX --in Hospitality.lu -t zh

The error is :

Other options like parse ToLuis works well.

Expected:
Ludown translate support composite entity like:

• stay {NumNights=three more {HotelNights=days}}
  Which is generated by ludown itself.

Tool

Name: Ludown

Describe the problem

In lu file, list entities is defined as

$Name:Value=
- Synonyms

However, only Synonyms are translated since Value must not be translated (or it will be confusing when using).
So in the other language, it does not detect Value.

Describe the solution you'd like

Add Value automatically to Synonyms when translating.

Describe alternatives you've considered

Output a warning to inform user to add Value to Synonyms.

Additional context

[enhancement]

See Task 36858
In particular follow this video

Migrate QnAMaker tool from here:
https://github.com/Microsoft/botbuilder-tools/tree/master/packages/QnAMaker

• CI
• Nightly (npm)
• Functional

Pull from nightly|prod build

[+PM]

Details tracked in VSO issues due to compliance needs.

When displaying help, all commands must specify whether they're mandatory or optional and any default values.

Example

bf SomeCommand
Perform some command....
USAGE
$ bf SomeCommand [SUB-COMMANDS] [OPTIONS]
SUB-COMMANDS
All sub-commands are separated by ':'
$ SubCmd1 Performs sub command
$ SubCmd2 Performs sub command 2
OPTIONS
-x, --xyz Modifies SomeCommand to do xyz [mandatory]
-y, --yzx Modifies SomeCommand to do yzx [optional]
-z, --zxy=number Modifies SomeCommand to do zxy [default=123]

Refer to Vishwac's spec.
https://github.com/microsoft/botframework-cli/blob/master/specs/LUQnAAndDispatch.md

Need to discuss:

• LUIS command (merge in LUIS API).
• Terminology (convert, generate...)

Oclif generates a test file per command automatically and provides a library to execute the command in a simple way.
https://oclif.io/docs/testing