dotnet / templating Goto Github PK

The matches are not anchored and so can match partial paths. Possibly other issues with overlap between what are meant to be disjoint match sets.

Update defaultinstalls to pull packages correctly again

There are many places in the code where invalid config is quietly ignored. It would be quite helpful to users to get diagnostic info about their config.

SimpleConfigModel.ProduceOperationSetup(SpecialOperationConfigParams, bool, ICustomFileGlobModel) hard codes the condition evaluator to C++.
ConditionalConfig.ConditionalSetup(ConditionalType, string, bool, bool, string) hard codes the evaluator type for MSBuild (for cases where the selected conditional type is MSBuild)

This should be fixable by:

• Adding string EvaluatorName { get; } to SpecialOperationConfigParams (and reading it from the underlying configuration or defaults)
• Passing defaultModel.EvaluatorName ?? "C++" to ConditionalConfig.ConditionalSetup(ConditionalType, string, bool, bool, string) instead of the hard coded C++

Any --debug:* switches should not be shown in the output of --help, but all other switches should be

Add --debug:help to show only debug switches, but do not include that in --help

Add descriptions (and any other metadata) to each of the libraries so that the packages don't just say "Package Description" on the feed.

GlobalRunSpec stores the special RunSpec's in a dictionary, so the order defined in IRunnableProjectConfig.SpecialOperationConfig is lost. Make appropriate modifications to ensure the processing order is preserved.

• This is redundant with the "generated -> evaluated" symbol type, use the syntax of computed, and get rid of "generated -> evaluated" type symbols
• The evaluator will need to be able to be specified, right now it always assumes C++ style
• value should behave like a switch, like postActions does (keep evaluating conditions until one is true or no condition is specified) to determine the value to assign

When I run dotnet --help the first few lines are shown below.

c:\data\mycode\templating>dotnet --help
.NET Command Line Tools (1.0.0-preview5-004275)
Usage: dotnet [host-options] [command] [arguments] [common-options]

When I run dotnet new3 --help the first few lines are

c:\data\mycode\templating>dotnet new3 --help


Usage:  [arguments] [options]

To make this more consistent we should:

• Remove empty lines that are printing before "Usage"
• Update usage statement to be "Usage: dotnet new [arguments] [options]"

When I run dotnet new3 mvc --help I see the following.

c:\temp\dotnetnew3>dotnet new3 mvc --help
MVC ASP.NET Web Application (C#)
Author: Microsoft
Options:
  -au|--auth           The type of authentication to use
                       None, Individual
                       Default: None

  -uld|--use-local-db  Whether or not to setup a local DB
                       bool - Optional
                       Default: false

From the dotnet new3 --help the following are relevant during project creation.

  -n|--name         The name for the output being created. If no name is specified, the name of the current directory is used.
  -o|--output       Location to place the generated output.

It would be nice if we can pass these through so the user will see the full set of options they can pass during invocation.

This happens for all parameters, even when help is specified

Input:

dotnet new3 empty -rv 1.1 --help -ffv net555

Output:

Empty ASP.NET Web Application
Author: Microsoft
Input param named [NetTargetFrameworkVersion] must be of data type = [choice] but was provided the invalid value = [net555]
Additional information:
Choice is invalid. Valid choices are: [net45,net451,net452,net46,net461,net462]
   at Microsoft.TemplateEngine.Orchestrator.RunnableProjects.RunnableProjectGenerator.DataTypeSpecifiedConvertLiteral(ITemplateParameter param, String literal)
   at Microsoft.TemplateEngine.Orchestrator.RunnableProjects.RunnableProjectGenerator.InternalConvertParameterValueToType(ITemplateParameter parameter, String untypedValue)
   at Microsoft.TemplateEngine.Orchestrator.RunnableProjects.RunnableProjectGenerator.ConvertParameterValueToType(ITemplateParameter parameter, String untypedValue)
   at Microsoft.TemplateEngine.Edge.Template.TemplateCreator.ResolveUserParameters(ITemplate template, IParameterSet templateParams, IReadOnlyDictionary`2 inputParameters)
   at dotnet_new3.New3Command.TemplateHelp(ITemplateInfo templateInfo, ExtendedCommandParser app, IReadOnlyDictionary`2 userParameters) in C:\Projects\templating\src\dotnet-new3\New3Command.cs:line 486
   at dotnet_new3.New3Command.DisplayHelp(String templateNames, ExtendedCommandParser app, IReadOnlyDictionary`2 userParameters) in C:\Projects\templating\src\dotnet-new3\New3Command.cs:line 462
   at dotnet_new3.New3Command.MaintenanceAndInfo(ExtendedCommandParser app, String templateName, Boolean& shouldExit) in C:\Projects\templating\src\dotnet-new3\New3Command.cs:line 313
   at dotnet_new3.New3Command.<>c__DisplayClass4_0.<<Run>b__0>d.MoveNext() in C:\Projects\templating\src\dotnet-new3\New3Command.cs:line 120

Sources can have conditioned modifiers, but sources themselves should support conditions as well

We should show two examples of how to use the command when a list is displayed (unless the -l|--list switch was specified)

Remove the -d|--directory switch
Add a new switch -o|--output that accepts a path (relative or absolute)

• This switch should be visible in help
• The directory should be created if it does not currently exist
• The directory should be the target for the instantiated template

The general purpose expression evaluator should be used for this as well - bitwise operators & value comparisons will need to be introduced

Give a message about no (or multiple) templates matching the specified search term, give possible corrections:

List templates that match the corrected forms for:

• typos
• things that are filtered out by other search criteria

Also list if there are multiple matches

Make sure to add the "do you mean" type message & link it back to the user's input so it makes sense

Several items here:

• Port an MSBUILD style condition evaluator https://github.com/mono/monodevelop/blob/master/main/src/core/MonoDevelop.Core/MonoDevelop.Projects.MSBuild/MSBuildEvaluationContext.cs and/or this https://github.com/Microsoft/msbuild/blob/master/src/XMakeBuildEngine/Evaluation/ConditionEvaluator.cs for syntax support.
• Brace balancing should not be authoritative for expression ends (the value of the conditioning attribute can be pre-read and processed, searching for the end-quote, collecting the expression along the way)
• The root variable collection should be passed with a format string of $({0})
• Files matching the pattern **/*.*proj should use this new conditional evaluator instead of the existing Conditional provider
• **/*.*proj files should have a default attribute for evaluation called Condition
  • This may produce several variations that consume different common surrounding whitespace configurations
  • The basic form of this would be something like Condition="
• If the condition evaluates to false...
  • Reposition the output stream back to the start tag (by scanning backwards for <), or earlier depending on the leading whitespace behavior
  • Set the "open element" count to 1
  • Scan forward in the input buffer
    • If the end element start token </ is encountered, decrement the open element count and seek to the element end token >
    • If the start element start token < is encountered, increment the open element count
    • If the self closing token /> is encountered, decrement the open element count
    • If the open element count is 0, process the trailing whitespace behavior
• In order to avoid having to use flags settings to disable and enable conditional behavior for things like the default property groups, references to unresolved symbols should not be treated as false-y (as they are in the C++ style condition evaluator), instead, the expression (and originally matched conditional token) should be emitted to the output stream if the expression contains any unresolved symbols

Ex.

<root>
  <element if="'$(variable)' == 'value'"/>
  <element if="'$(variable)' != 'value'">
    <child>
        <grandchild />
    </child>
  </element>
</root>

If variable == "value", the output would be

<root>
  <element />
</root>

If variable != "value", the output would be

<root>
  <element>
    <child>
        <grandchild />
    </child>
  </element>
</root>

Fixed by #79

Work for this is in progress in #28.

SimpleConfig and GlobalRunSpec are hard to follow and deal directly with the JSON for determining what to do. Fixing this involves:

• Creating a clear progression in which parameters/variables are populated
• The configuration of operations and macros are standardized (no special calls to configure a specific operation or macro - they should be delegated to the relevant read logic supplied by the operation or macro provider)
• Enhancements such as #26 and #24 should be more intuitive to add
• Adding custom configuration for both the standard and the special (file glob specific) operations, including: Custom operations, custom variable formatting, custom flags.

In a future release, we'll add it back in cases where one or more templates have aliases

There will need to be some kind of a notion of what the result of a creation is so that consuming systems can add the artifact to the thing it was generated into (if required)

Two approaches come to mind:

• A new tag that indicates to the engine some value (that the host understands) use host defined rules in deciding whether or not to include the template in the list output (it'd still be specifiable if asked for by name)
• A set of context rules for suggestion like "suggest if there's a file by name {x} in search direction {y} from the output directory"

This is a big item but will enable lookarounds in token matching. This will address ligershark/microsoft-templateengine#63

For things like running npm install or what have you, templates should be able to suggest running actions if they're available and/or providing instructions on how to do this manually.

Actions should:

• Have some sort of an identifiable "key" that a caller would be able to use to determine whether the command is supported.
• Have some sort of an "if unsupported" behavior (returning a message to display is the only behavior that has been asked for so far)
• Have some sort of an indication of how to specify what arguments/parameters the action should be run with

From dotnet new3 mvc --help I see

Examples:
    dotnet new3 mvc --auth None --use-local-db false
    dotnet new3 mstest
    dotnet new3 --help

Is the --use-local-db false a relevant flag in the mvc template? I tried setting to true and then to false and the same content was produced.

Moved from https://github.com/dotnet/cli/3617 on behalf of @kichalla

Steps to reproduce

• dotnet new -t Web in a folder1
• dotnet new -t Web in a folder2
• compare the generated project.json content and you should notice the same value for userSecretsId value

Expected behavior

userSecretsId value should be different as they are different apps and can have different config values.

Actual behavior

Not sure if this is hard coded, but I always see "userSecretsId": "aspnet-WebApplication-0799fe3e-6eaf-4c5f-b40e-7c6bfd5dfa9a", when creating apps.

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.0-preview2-003102)

Product Information:
Version: 1.0.0-preview2-003102
Commit SHA-1 hash: 0589496b98

Runtime Environment:
OS Name: Windows
OS Version: 10.0.10586
OS Platform: Windows
RID: win10-x64

@ajaybhargavb

Because of this, listing templates doesn't show locale specific.

Might need to use a full ILocalizationModel when working with template caches, instead of ILocalizationLocator.

When I run dotnet new3 --help for arguments I see the following.

Usage:  [arguments] [options]

Arguments:
  template  The template to instantiate.

I think it would be more consistent with dotnet --help if we output the following.

Usage:  [template] [options]

Arguments:
  [template]  Name of the template used for creation
  [options]   Options

Options:
  -l|--list         List templates containing the specified name.
  -lang|--language  Specifies the language of the template to create
  -n|--name         The name for the output being created. If no name is specified, the name of the current directory is used.
  -o|--output       Location to place the generated output.
  -h|--help         Displays help for this command.

Needs investigation

The templates that are in here (for testing) should be updated to the current template content

Currently, templates must be explicitly installed in order to compute the display information/runtime dependencies. It'd be great to have a way of having this information pre-computed and shipped with the binaries.

Something like:

Setting a preference for this particular host (host identifier/version specific)

dotnet new3 --set:preference language=F#

Setting a preference for all hosts that identify the same way as this one (host identifier specific)

dotnet new3 --host:set:preference language=F#

Setting a preference for all hosts (not specific to the current host identifier or version)

dotnet new3 --global:set:preference language=F#

When I do dotnet new3 --help I see the following listed in options.

Options:
  -l|--list         List templates containing the specified name.
  -lang|--language  Specifies the language of the template to create
  -n|--name         The name for the output being created. If no name is specified, the name of the current directory is used.
  -o|--output       Location to place the generated output.
  -h|--help         Displays help for this command.

The -lang description is the only one missing a period. I think it's better to remove the periods here though since dotnet --help doesn't use periods either.

When an invalid choice is specified, show a message to that effect (and invoke the parameter error handler on the host) but don't stop execution in the parameter processor so that if the help switch has been specified, help is still shown.

If the template would be created & has a bad argument specified, creation should be prevented. This could be accomplished by validating parameters just before help or creation instead of during the parse.

Also, if the user explicitly specifies the default value for the parameter, it should be shown as the current value in help

We should accomodate versioning of environments that use semantic versioning to disambiguate their releases rather than a x.x.x.x style version. To do that, we'll need to change this property to a string.

Ex. dotnet new3 asdfg

Would return an error like this

some text asdf some text

It should return text like this

some text "asdf" some text

• Enforce types
• Allow sets of options
• Support "presence = true" for boolean type parameters (in the case that they're false by default)

The empty web template (current short name of emptyweb) should just be web

High level summary of the current thinking:

• The host interface should always give a way of getting the current locale
• .netnew.json will have locale specific flavors (en_us.netnew.json, es_es.netnew.json, etc.)
  • *.netnew.json should never be emitted as content
  • All display properties should be taken from (locale).netnew.json if available, if a matching locale or property is missing, it should be taken from .netnew.json
• Each (locale).netnew.json should contain a localizations property whose value is an object
  • The property names in that object should be the full file paths in the template to the content containing localizations
  • The values of those properties should be objects
    • The properties of those objects should be the unlocalized value
    • The values of those properties should be the localized value that should be used in place of the unlocalized value
• The template display info cache should be differentiated on language (a cache for en_us, fr_fr, etc. instead of the one cache we currently have)
  • The appropriate localized display properties should be used when computing the locale specific cache
• At template instantiation, the normal matching rules should apply with the following changes
  • If no (locale).netnew.json applies to the current instantation, nothing changes
  • If one does, for each file to be processed:
    • The unlocalized -> localized value pairs that apply to the file (from (locale).netnew.json) should become a set of Replacement operations
    • If the file would be processed anyway the replacements should be added to the set of things that would be done to the file (clone and update the operation trie, add the operations to the set, etc.)
    • If the file wouldn't be processed (matches copyOnly), the file now needs to be processed, but only using the replacement operations without any other operations that would apply to the file based on any other pattern matches

The comment is in this block of code in ConditionalConfig.CStyleLineCommentsConditionalSetup

// temporary - one time test
IOperationProvider uncomment = new Replacement("//", string.Empty, uncommentOperationId);
IOperationProvider reduceComment = new Replacement("////", "//", reduceCommentsOperationId);

Evaluate whether this should be here & either remove the comment or remove the code.

This comment was introduced with commit 03b6f1c on 10/13, but the code it applies to was introduced in commit 1c0d078 on 9/22

Currently the config system has to understand everything, delegating the settings read to the operation factory itself seems like a better approach

Spec is here https://docs.nuget.org/ndocs/create-packages/package-types#dotnetclitool

We'll need to send a PR to call out/standardize our package types

I have started to look into porting the ASP.NET MVC Boilerplate project template from a VSIX to dotnet new. I have several questions:

1. What is the migration path? Do I totally switch to dotnet new or is it possible to somehow produce a dotnet new NuGet package, as well as a VSIX? Should I support VS 15 and/or 2017?
2. Is it too soon to do a port?
3. I followed the instructions in the ReadMe.md and could not get dotnet new3 -i C:\GitHub\templating\template_feed to actually install the templates in that directory. Is dotnet new3 broken at the moment?
4. I've had a look at the new templates in this repo. What's the best way to learn the new pre-processor syntax, .netnew.json syntax, computed symbol syntax and learn how to create a package template? Are there any docs or more examples I can look at?
5. ASP.NET MVC Boilerplate has it's own pre-processor syntax and feature selection dialogue. I've written functional tests to run dotnet restore, dotnet build, npm install, gulp default and start the MVC site using the TestHost API to ensure that every feature combination works. How can I select features and run the templating engine programmatically in my functional tests?
6. Can I execute arbitrary code to create a project from my template? Any examples of this?
7. I love how templates are just NuGet packages. How are multi-project templates or files external to the project (global.json as an example) handled.
8. ASP.NET MVC Boilerplate has a WPF UI. Are there plans for a VS UI ontop of dotnet new? If not, I'm thinking it would be fairly simple to write an electron app to do feature selection based on .netnew.json, would this be an interesting open source project to the tooling team at Microsoft? It would be nice to provide icons for each feature in .netnew.json.

Please excuse the long list of questions, there is just a lot to templating :)

Show the alias command in help

Add the alias column to the template list iff at least one template has an alias

In some conditional processing situations, an extra newline is emitted.
See ConditionalTests.VerifyExcludeNestedConditionInNonTakenBranch() for additional notes.
See ConditionalTests.VerifyExcludeNestedCondition() for an example where an extra newline is emitted. The expected result should not have the trailing newline.

See ligershark/microsoft-templateengine#56 for what the additional defaults should be

Get the build to produce appropriately signed artifacts
Get the outputs of builds saved somewhere