Build in support for the TypeScript static default props of arrow function.

While working with @Evomatic on an issue about branch names that include a slash, we noticed several points to improve.

There is something wrong about the error handling, at some point I got a message ERROR: followed by nothing, impossible to figure out what was wrong

At some point I got a weird error in the terminal, it looks like a HTML error page displayed character by character, this kind of error should be handled in a better way.

Pushing takes a lot of time sometimes while nothing is on the screen: a spinner could help (related #38 )

And last thing, when everything is fine, the final success message in blue includes an invalid character (I guess it was an emoji) that gets replaced by either a question mark or a square, depending on the terminal.

Build in support for the TypeScript Indexed signature property.

Build in support for the TypeScript enum property type.

One of our customers has a DS that is relying on the use of React's Forwarding Refs

There are two ways of declaring forwards refs that they are using:
React.forwardRef
React.FC<Props>

Issue

When using Merge in Windows using the WSL, experimental mode has a problem.

In the URL that it generates and opens, it will insert the ^ character after your port number. This results in the page not being able to load.

Workaround

The workaround is to manually delete the ^ character from the URL in the browser and that enables the page to load. N.B. this does not affect the normal uxpin-merge push command.

Idea for fix

If we can't prevent this character from being inserted (perhaps it comes from a bug in WSL), then we can just manually strip out ^ characters from final generated string.

Named arrow functions defined in TypeScript cannot be serialized by the CLI

The following simple example component will not work:

import * as React from 'react';

interface HelloProps {
    name: string;
}

const Hello = (props: HelloProps) => {
    return <div>Hello {props.name}</div>;
};

export default Hello;

When a component fails to render in experiment mode, it looks like this on the canvas:

(A red-shaded rectangle with an X). No error information is displayed alongside the broken component.

Enabling the JS console

The best way I've found to debug is to enable the web console and watch for JS errors. They do appear here. For example, see this log output when loading experiment mode w/a broken component. The critical portion:

Uncaught ReferenceError: blah is not defined
    Button Button.js:7

There is also this message, which is helpful:

The above error occurred in the <Button> component:
    in Button (created by i)
    in i (created by u)
    in u
    in UXPinWrapper

However, it is a 222-line file w/some duplication of the error message.

Storybook Error Handling

Storybook renders the error right in the UI. However, it is hard to track the error to a component JS as just the bundle is referenced:

Proposal

It'd be great if it was possible to display rendering errors for the component somewhere within our UI (versus the error console). For example, the properties panel when clicking on a component.

In #194, I mentioned the same error occurs when running uxpin-merge push from a detached HEAD git repo state. This state is common when running via a CI system as they checkout a specific commit, not a ref to a branch.

Because of this, uxpin-merge push may be unlikely to work currently when triggered via CI.

I'm creating a dedicated issue for this as it is a more immediate blocker to running Merge within CI than branch support and there's less tradeoffs to think through versus branch push support.

This list of desired TypeScript features came from the old roadmap:

• Serialize enums property
• Serialize date property
• Support React.forwardRef
• Support component with non inlined export statement

There are problems with Restoring Cache:

Error computing cache key: template: cacheKey:1:37: executing "cacheKey" at <checksum "package.js...>: error calling checksum: open /home/circleci/project/packages/uxpin-code-cli/package.json: no such file or directory

Saving Cache:

Error computing cache key: template: cacheKey:1:37: executing "cacheKey" at <checksum "package.js...>: error calling checksum: open /home/circleci/project/packages/uxpin-code-cli/package.json: no such file or directory

And building

#!/bin/bash -eo pipefail
make build
make: *** No rule to make target 'build'.  Stop.
Exited with code 2

See: https://circleci.com/gh/UXPin/uxpin-code-tools/288

Build in support for the TypeScript date property type.

Goal

Provide a better Developer eXperience and solve issues related to out-dated dependencies.

• Remove tslint dependency that is deprecated since 2019 🙀 (see palantir/tslint#4534)
• Use typescript-eslint instead and setup rules similar to the existing one
• Use Prettier to format the code to be able to format automatically the code in our favorite code editor, without worrying about : and spacing.

About tslint

The main problem I found when checking the code is that our setup forces us to declare the types of any variable in any single line of the code base, even if it's not needed because of TypeScript capacity to infer types.

It makes the code verbose and redundant.

It seems to come from this rule: https://github.com/UXPin/tslint-config-uxpin/blob/master/tslint.json

We should try to set up an environment that is up-to-date and as standard as possible in order to write "idiomatic TypeScript".

Action plan

• Remove tslint
• Setup typescript-eslint
• Setup prettier
• Update Make commands
• Clean the existing code using eslint --fix parameter
• Add commit hooks to trigger linting when files are committed: it should not be possible to commit files if style rules (from Prettier) and language rules (from both TypeScript and ESLint) are not met. The hook should show an error message instead of correcting the files automatically, to make the developer fix his/her environment.

At this stage, we should only fix files using automatic corrections provided by ESLint and Prettier, to avoid any risk of introducing bugs.

The main downside is that a lot of files would have to be automatically updated, which is annoying when you care about the Git history.

Build in support to allow the user to specify their TypeScript configuration.

Hi

We use typescript and I was testing a component that works perfectly in Storybook exposing the types but in Uxpin doesn't return any times.

The types used was

export type FooType = Omit<BarType, 'type'> & {
 type: boolean
}

Weirdly if I wrote it as

export interface FooType extends Omit<BarType, 'type'> {
 type: boolean
}

it works

Components need a presets directory and .jsx file with a specific name and in a specific format in order for Merge to be able to render the component.

Whilst this is understandable, it prevents a "just-work" approach.

Possible solution:

Could the CLI auto-generate the the directories and preset files and attempt to produce something that would define the component in a default state? Then all the user would have to do is make tweaks to the already working solution.

When you are integrating components with Merge, you have to manually create a presets directory and a 0-default.jsx preset file for every component.

Could we add a generate-presets mode to the CLI (we currently have modes such as push and dump) which would read the uxpin.config.js file and then create these directories and files for you.

As a first pass, just having the files created with some basic, stubbed information in them would be great, but if in future we can automatically fill them in by parsing the props then that would be even better.

To make a coded component more designer-friendly within the UXPin editor, sometimes a Higher-Order (HO) component is created that offers different properties than the base component. For example, a button base component may expect an icon and its properties as JSON while the UXPin version may separate out those icon settings into distinct settings (icon name, size, etc).

When viewing the preview spec mode, a developer will see the JSX for the HO component and not the underlying component. This means they cannot copy and paste the JSX code into their app and expect it to work.

It would be nice if the underlying component JSX could be presented in the preview spec mode.

When running make build in the root project directory with Node v12.18.1, I see the following error:

src/utils/browser/openUserBrowserOnSpecificUrl.ts:1:22 - error TS7016: Could not find a declaration file for module 'opn'. '/Users/dlite/projects/uxpin/uxpin-merge-tools/packages/uxpin-merge-cli/node_modules/opn/index.js' implicitly has an 'any' type.
  Try `npm install @types/opn` if it exists or add a new declaration (.d.ts) file containing `declare module 'opn';`

1 import opn = require('opn');

@matthewchigira - have you run into this?

Hi we are proxying you component library over the top of Material-UI and I wanted to see how a component looked in your preview.

I noticed there is a significant speed difference between Typescript and Javascript on loading your system locally.

Your material-ui-merge repo vs our version with only 5 components currently

So unless I've done something really silly my dump file time for 5 components is longer than the dump time for the whole material-ui-merge

For reference on the same library, storybook has a time to load of 20 seconds

uxpin.config.js

module.exports = {
  components: {
    categories: [
      {
        name: 'Inputs',
        include: [
          'src/inputs/buttons/button.tsx',
          'src/inputs/text-field/text-field.tsx',
          'src/inputs/select/simple-select.tsx',
        ],
      },
      {
        name: 'Layouts',
        include: ['src/layout/container/container.tsx'],
      },
      {
        name: 'Navigations',
        include: ['src/navigation/nav-list/nav-list.tsx'],
      },
    ],
    webpackConfig: 'webpack.config.js',
  },
  name: 'MyCRM React Design UI',
}

webpack.config.js

const path = require('path')

module.exports = {
  output: {
    path: path.resolve(__dirname, 'build'),
    filename: 'bundle.js',
    publicPath: '/',
  },
  resolve: {
    modules: [__dirname, 'node_modules'],
    extensions: ['.tsx', '.ts', '*', '.js', '.jsx'],
  },
  devtool: 'source-map',
  module: {
    rules: [
      {
        test: /\.svg$/,
        exclude: /node_modules/,
        loader: 'svg-react-loader',
      },
      {
        loader: 'babel-loader',
        test: /\.tsx?$/,
        exclude: /node_modules/,
      },
      {
        enforce: 'pre',
        test: /\.tsx?$/,
        loader: 'source-map-loader',
      },
    ],
  },
}

Could we create init mode in the CLI (we currently have modes such as push or dump) which generates a default uxpin.config.js and our "default" uxpin.webpack.config.js file? Perhaps we could also create the UXPinWrapper component file too.

This would save new users the hassle of manually having to create these files, and would give people a better starting point.

For the config file it would be good to stub out a "General" category with a single, commented-out component e.g. // 'src/components/Button/Button.js' showing best-practises. Also we could add the paths to webpack and wrapper files in this config too.

Currently the CLI only supports pushing from the master branch. It checks for changes made to the master branch of the project and pushes these changes.

This is an issue for onboarding because when people are experimenting with getting their components into Merge, they don't want to have to commit to master each time.

The current workaround at the moment is to create a new library in the UI and push there (this will push uncommitted changes) or to use the Experimental Mode locally (but this has issues in that it doesn't always work for all people).

e.g. We don't detect this case because

With our isExported function, we only check the inline export statement.

And here, we use isExported to check.

Sample component.

import * as React from 'react';

export interface Props {
  appearance: 'secondary' | 'primary' | 'link';
}

class Component extends React.Component<Props> {

  public static defaultProps: Partial<Props> = {
    appearance: 'secondary',
  };

  public render(): JSX.Element {
    const { appearance } = this.props;
    return (
      <div>
        <button className={appearance}>
           Hello
        </button>
      </div>
    );
  }
}

export default Component;

Currently, you'll get Error: No component found!

A lot of dependencies used by @uxpin/merge-cli are either deprecated, out-of-date or considered as vulnerable.

It leads to a lot of warning messages when installing the tool on a computer.

The purpose of this issue is to take a snapshot of the situation and to track the progress of the cleanup actions.

First audit • 2023-01-20

yarn audit

287 vulnerabilities found - Packages audited: 2554
Severity: 9 Low | 27 Moderate | 189 High | 62 Critical

Only production deps:

98 vulnerabilities found - Packages audited: 952
Severity: 11 Moderate | 62 High | 25 Critical

Using Snyk.io

Running snyk test command against 2.11.0:

Tested 741 dependencies for known issues, found 27 issues, 294 vulnerable paths.

Issues to fix by upgrading:

  Upgrade [email protected] to [email protected] to fix
  ✗ Regular Expression Denial of Service (ReDoS) [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-GLOBPARENT-1016905] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 1 other path(s)

  Upgrade [email protected] to [email protected] to fix
  ✗ Regular Expression Denial of Service (ReDoS) [High Severity][https://security.snyk.io/vuln/SNYK-JS-SSRI-1246392] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected]


Patchable issues:

  Patch available for [email protected]
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/npm:extend:20180424] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 1 other path(s)

  Patch available for [email protected]
  ✗ Prototype Pollution [Medium Severity][https://security.snyk.io/vuln/npm:hoek:20180212] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] and 3 other path(s)

  Patch available for [email protected]
  ✗ Uninitialized Memory Exposure [Medium Severity][https://security.snyk.io/vuln/npm:stringstream:20180511] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]


Issues with no direct upgrade or patch:
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-AJV-584908] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] and 3 other path(s)
  This issue was fixed in versions: 6.12.3
  ✗ Regular Expression Denial of Service (ReDoS) [High Severity][https://security.snyk.io/vuln/SNYK-JS-ANSIREGEX-1583908] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] and 2 other path(s)
  This issue was fixed in versions: 3.0.1, 4.1.1, 5.0.1, 6.0.1
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-ASYNC-2441827] in [email protected]
    introduced by [email protected] > [email protected]
  This issue was fixed in versions: 2.6.4, 3.2.2
  ✗ Arbitrary File Write via Archive Extraction (Zip Slip) [Critical Severity][https://security.snyk.io/vuln/SNYK-JS-DECOMPRESSZIP-73598] in [email protected]
    introduced by [email protected] > [email protected]
  This issue was fixed in versions: 0.2.2, 0.3.2
  ✗ Exposure of Resource to Wrong Sphere [Low Severity][https://security.snyk.io/vuln/SNYK-JS-FSEVENTS-5487987] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 1.2.11
  ✗ Regular Expression Denial of Service (ReDoS) [High Severity][https://security.snyk.io/vuln/SNYK-JS-HAWK-2808852] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 9.0.1
  ✗ Prototype Pollution [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-JSON5-3182856] in [email protected]
    introduced by @babel/[email protected] > [email protected] and 1 other path(s)
  This issue was fixed in versions: 1.0.2, 2.2.2
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-JSONSCHEMA-1920922] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 0.4.0
  ✗ Validation Bypass [Low Severity][https://security.snyk.io/vuln/SNYK-JS-KINDOF-537849] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 156 other path(s)
  This issue was fixed in versions: 6.0.3
  ✗ Regular Expression Denial of Service (ReDoS) [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-MINIMATCH-3050818] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 2 other path(s)
  This issue was fixed in versions: 3.0.5
  ✗ Prototype Pollution [Low Severity][https://security.snyk.io/vuln/SNYK-JS-MINIMIST-2429795] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] and 6 other path(s)
  This issue was fixed in versions: 0.2.4, 1.2.6
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-MIXINDEEP-450212] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected] and 15 other path(s)
  This issue was fixed in versions: 2.0.1, 1.3.2
  ✗ Prototype Poisoning [High Severity][https://security.snyk.io/vuln/SNYK-JS-QS-3153490] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 6.2.4, 6.3.3, 6.4.1, 6.5.3, 6.6.1, 6.7.3, 6.8.3, 6.9.7, 6.10.3
  ✗ Server-side Request Forgery (SSRF) [Medium Severity][https://security.snyk.io/vuln/SNYK-JS-REQUEST-3361831] in [email protected]
    introduced by [email protected] > [email protected]
  No upgrade or patch available
  ✗ Cross-site Scripting (XSS) [High Severity][https://security.snyk.io/vuln/SNYK-JS-SERIALIZEJAVASCRIPT-536840] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 2.1.1
  ✗ Arbitrary Code Injection [High Severity][https://security.snyk.io/vuln/SNYK-JS-SERIALIZEJAVASCRIPT-570062] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 3.1.0
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-SETVALUE-1540541] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] and 31 other path(s)
  This issue was fixed in versions: 4.0.1, 2.0.1
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-SETVALUE-450213] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] and 31 other path(s)
  This issue was fixed in versions: 2.0.1, 3.0.1
  ✗ Regular Expression Denial of Service (ReDoS) [High Severity][https://security.snyk.io/vuln/SNYK-JS-TRIM-1017038] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 0.0.3
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-UNSETVALUE-2400660] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected] > [email protected] > [email protected] and 15 other path(s)
  This issue was fixed in versions: 2.0.1
  ✗ Prototype Pollution [High Severity][https://security.snyk.io/vuln/SNYK-JS-Y18N-1021887] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 3.2.2, 4.0.1, 5.0.5
  ✗ Insecure Randomness [Medium Severity][https://security.snyk.io/vuln/npm:cryptiles:20180710] in [email protected]
    introduced by [email protected] > [email protected] > [email protected] > [email protected]
  This issue was fixed in versions: 3.1.3, 4.1.2

While make build appears to build the cli package, it does generate a number of errors.

See https://gist.github.com/itsderek23/95df96b8a7535e2b2540886e969b5feb for my make build output.

If a prop doesn't have a type or is set to the 'any' type, then we ignore it. This was a design decision to ensure good user experience, for example a prop with a Boolean is matched up with a checkbox etc.

However this is an onboarding stumbling block because at best it leaves customers guessing where their prop is and at worst can lead to display issues if key props relating to CSS are ignored.

Here is where we validate and ignore:
https://github.com/UXPin/uxpin-merge-tools/blob/master/packages/uxpin-merge-cli/src/steps/serialization/validation/props/isCustomTypeAllowedForType.ts

Here is the docs for how props match up to UI components:
https://www.uxpin.com/docs/merge/configuring-the-properties-panel/#control-types

Possible solution:
When no type is present, assume it's type and provide a textbox in the UI, but have a tooltip next to it explaining that you should define a type. Also show a warning in the CLI and in the UI too.

Motivation

Often, building a design system library takes a long time. We need to notify a user that the process is still in progress.

Acceptance criteria

We should print the following output in the console.
Output while performing discovery step:

[1/4] 🔍 Discovery

Output while performing serialization step:

[1/4] 🔍 Discovery
[2/4] ⏳ Serialization
Serializing: Button 
█████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 25%

Output while performing build step:

[1/4] 🔍 Discovery
[2/4] ⏳ Serialization
[3/4] 🏗 Building
⠏ Webpack: <here most recent log from webpack – if possible>

Output while performing upload step:

[1/4] 🔍 Discovery
[2/4] ⏳ Serialization
[3/4] 🏗 Building
[4/4] 🚀 Upload
█████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 25%

After all the steps finished successfully:

[1/4] 🔍 Discovery
[2/4] ⏳ Serialization
[3/4] 🏗 Building
[4/4] 🚀 Upload
✨ All done!

It's a very common pattern to include a slash / inside a branch name (feature/xxx, fix/yyy) but DXC customer has reported an issue when trying to delete a branch.

Note how the original slash/ is transformed into :2F.

export function encodeBranchName(branchName: string): string {
  return encodeURIComponent(branchName).split('%').join(':');
}

Also it seems there is something wrong when pushing a branch, as the branch name is not correctly displayed.

Reason: usually we are encoding the branch name from Merge CLI side and decoding it from uxpin-api side.

For the delete command, we don't decode the branch name from uxpin-api side.

We need to encode the branch name when its part of the API end-point we call, but we don't need to do that when the branch name is part of the JSON payload.

So I think we should fix both delete and push commands to remove the un-necessary encoding of the branch name.

Motivation

In Some Design Systems (e.g. nordnet-ui-kit) code examples include other component imports and therefore interfere with embedding components library. That causes throwing an error:

⛔ Error: unknown: Duplicate declaration

Source of bug

In given code example embedded library (imported in 1st line) is conflicted with Icon import (2nd line):

const { Alert, Animate, Avatar, Badge, Button, Dropdown, Flag, Icon, Input, LabeledValue, Li, Logo, Pane, RadioGroup, SegmentedControl, Spinner, Table, Tbody, Td, Tfoot, Th, Thead, Tooltip, Tr, Ul } = library;
const { Icon } = require('../../');
const color = require('../../styles/color').default;
<div style={{display: 'flex'}}>

Source: https://github.com/nordnet/nordnet-ui-kit/blame/d914edd5f97cadb284f7f47b783106c86fe430d9/src/components/avatar/avatar.md#L1

Acceptance criteria

Not supported code examples are rendered successfully in debug server mode and Design System Documentation (future).

The official name of the project is now Merge, so we need to:

• rename the repository to uxpin-merge-tools,
• rename the only package to uxpin-merge-cli,
• and the CLI command should be registered after npm i -g as uxpin-merge.

The colors package went rouge and leads to an infinite loop:
Marak/colors.js#285

Please lock down the colors package to 1.4.0.
Thanks!

Build in support for TypeScript array of enums property.

Hey Guys great work on the product, I'm a little concerned though with the typescript exports.

Firstly they have to be default exports which we don't want to use as it makes it much much more difficult for IntelliSense to keep up with them.

The second issue is the Arrow support, you have just added Arrow support which is great. But you are taking your component name from the file name rather the const name. For us, we use Kebab case so you are opinionating a file system that works for you but maybe not other people. As for example if I have a component in a folder simple-select.tsx it is being imported as import simple-select from ...

The struggles we are facing is that Storybook and our style guides say we should be exporting named arrow components and our file system should be Kabab case. And so far the only way to do this is to hack,

if we had a file name action-button.tsx

export type SomeProps = {}
/**
 * @uxpincomponent
 */
export function ActionButton(props: SomeProps){
 return <button>{props.label}</button>
}

export default ActionButton

then in the index file

import { SomeProps, ActionButton } from './action-button'

export {
   SomeProps,
   ActionButton
}

with these two errors its not a great experience