Giter VIP home page Giter VIP logo

ts-minify's Introduction

TS-Minify (Experimental) Build Status

About


TS-Minify is tool to aid in the reduction of code size for programs written in the TypeScript language. It is currently highly experimental.

This tool is developed on TypeScript and NodeJS, and transpiled to ES5; it uses the CommonJS module system.

There is currently no CLI or build-tool integration for TS-Minify.

Table of Contents


Motivation


Angular 2 (which is written in TypeScript) currently sits at around 135kb after being Uglified and compressed through GZIP. In comparison, Angular 1.4 is about 50kb, minified and compressed.

A smaller bundle size means that less data needs to be transferred and loaded by the browser. This contributes to a better user experience.

The impetus for this tool was to reduce the code size of the Angular 2 bundle, but TS-Minify is meant to be a generic tool that can be used on programs written in TypeScript.

What it Does


To achieve code size reduction, TS-Minify uses the idea of property renaming: take a TypeScript source file and rename properties to shorter property names, then re-emit the file as valid TypeScript.

TS-Minify role in minification pipeline This diagram demonstrates TS-Minify's role in the intended minification pipeline.

TS-Minify only targets property renaming. Minification tactics like whitespace removal, dead code removal, variable name mangling, etc. are taken care of by tools such as UglifyJS. TS-Minify specifically targets property renaming since it is something that is difficult to achieve safely without type information. As such, TS-Minify requires a program to be correctly and throughly typed, otherwise, unwanted renaming may occur.

How Does it Work? The TL;DR


The TypeScript Compiler API is utilized in order to access the Abstract Syntax Tree of a TypeScript source file. Identifiers that might be considered properties (such as identifiers in property declarations, property access expressions, method names in method declarations, etc.) are renamed to short names such as a, b, etc. after determining their renaming eligibility. A “renaming” global map is created with mappings from the original property name to the new generated property name so that re-namings are kept consistent between properties with the same names.

{ maybeHandleCall: g }
{ reportMissingType: h }
{ getHandler: i }
{ handlePropertyAccess: j }
{ emitExtraImports: k }
{ emit: l }
{ visitTypeName: m }

An example of some mappings from the original property name to the shorter, generated property name.

The renaming eligibility of a property takes several factors into consideration:

  • Does the property belong to an object declared in an external file (.d.ts file)?
  • Does the property belong to a standard built-in object?
  • Does the property name belong to the DOM?

External vs. Internal Types


TS-Minify is able to safely rename properties by understanding which types are external, and which are not, to one's program.

An external type is something that you are bringing into your program. It is a type which you have not defined. This includes objects and functions from the JavaScript standard library, and JavaScript libraries like underscore.js or jQuery. If you are using external libraries, it is imperative that you include their corresponding .d.ts files.

Typings for the standard library are included in the default library typings, lib.d.ts, which the TypeScript compiler uses during transpilation (it will typecheck your program at compile time) and does not need to be explicitly included.

Internal types are ones that you, the programmer, have defined yourselves. Declaring a class Foo in your program tells the minifier that properties on Foo objects are within the renaming scope. Internal types also include things like object literal types.

Note: If you want all internal sources to contain the same renamings, you should pass the relevant files together through a single pass of the minifier.

An example of what types are considered external and internal:

var x: { name: string, message: string }; // The object literal is an internal type

var x: Error; // Error is an external type

// Structurally, the two are compatible, which brings us to the next section...

Structural Typing


TypeScript uses a structural type system. This means that objects and variables can be casted from one type to another as long as they are structurally compatible. Sometimes this means that external objects might be casted to internal objects, or vise versa.

Here is an example of an external type being casted to an internal type:

function x(foo: { name: string, message: string }) {
	foo.name;
	return foo; 
} 

x(new Error());

Function x expects an internal object literal type. The call site x(new Error()) passes an Error object as the parameter.

Error is an object in the JavaScript standard library. Hence it is external to our small program above. Structurally, an Error object and the object literal { name: string, message: string } are compatible. However, this means that we do not want to rename the properties on the object literal because we will likely want to access the properties of the Error object as we know them: name and message. If we renamed the properties of the object literal, it means that if an Error object is passed into a function call for x(), foo.$some_renaming will throw an error, because Error does not have property $some_renaming.

Here is an example of an internal type being coerced into an external type:

function ff(e: Error) { 
	return e.name; 
} 

ff({ name: null, message: null });

The parameter e on function ff is an external type to our program. We do not want to rename its properties. At the function call site, we are passing in an object literal type with properties name and message, which is structurally compatible with Error. We do not want to rename properties in the object literal because we need to coerce it into an external typing, which is expected to have properties name and message.

All of this is to say, the tool relies on type information to figure out if two types can be coerced into each other and whether their properties can be renamed based on the internal/external distinction. Type your programs! :)

Usage


First clone the repo and run npm install.

$ git clone [email protected]:angular/ts-minify.git
$ cd ts-minify
$ npm install

Create a new TypeScript file:

import {Minifier, options} from './src/main';
var minifier = new Minifier();
minifier.renameProgram(['path/to/file.ts', 'path/to/another/file.ts', 'some/other/file.d.ts'], 'path/to/destination');

Transpile the above script into JavaScript using the TypeScript compiler and run it with Node:

tsc --module commonjs script.ts
node script.js

A new instance of the minifier takes an optional constructor argument:

MinifierOptions {
  failFast?: boolean;
  basePath?: string;
}

failFast: Setting failFast to true will throw an error when the minifier hits one.

basePath: Specifies the base path that the minifier uses to figure out to where to write the minfied TypeScript file. The basePath maybe be relative or absolute. All paths are resolved from the directory in which the minifier is executed. If there is no base path, the minified TypeScript file is outputted to the specified destination using a flattened file structure.

minifier.renameProgram() takes a list of file names and an optional destination path:

renameProgram(fileNames: string[], destination?: string)

renameProgram accepts TypeScript files and .d.ts files. If you are not explicitly including typings using the reference path syntax in your TypeScript file, please pass in the .d.ts file so that the minifier can use the type information.

The minifier will uniformly rename the properties (which are available for renaming) across all the files that were passed in.

TSConfig Users


If you are using a tsconfig.json file to reference your type definitions, pass in the .d.ts files for libraries used in your program to the minifier so that it can reference the type information for property renaming. The minfier does not have access to the tsconfig.json file. Otherwise, make sure to have /// <reference path = 'foo.d.ts' /> at the top of your programs if you are using any external libraries.

Scope of Minification


Currently, TS-Minify will do property renaming throughout all the files that are passed into it. The minifier excludes .d.ts files from renaming because those are the typing definitions of external libraries. If your TypeScript program exports objects across those files, the minifier will rename them uniformly in their declarations and their usage sites.

Caveats/Warnings


In order to get the most out of the minifier, it is recommended that the user types their program as specifically and thoroughly as possible. In general, you should avoid using the any type, and implicit anys in your code as this might result in unwanted naming.

We recommend that you compile your TypeScript program with the noImplicitAny compiler flag enabled before trying the minifier.

Please avoid explicit any casting:

// example of explicit any casting
var x = 7; 
<any>x;

Sample Measurements


TS-Minify was run on a well-typed TypeScript program with the following stats and results:

  • Approximately 2100 lines of code (split across 10 files)
  • Unminified and Uglified: 72kb
  • Minified and Uglified: 56kb (codesize reduction of about 20%)
  • About 6% - 8% codesize reduction after minification, Uglification, and GZIP compression.

Contributing


Clone the repository from GitHub:

$ git clone [email protected]:angular/ts-minify.git
$ cd ts-minify
$ npm install

Run the unit tests with gulp unit.test

Run the end-to-end tests with gulp e2e.test.

This project uses clang-format. Run gulp test.check-format to make sure code you write conforms to this standard.

  • If you need some guidance on understanding TypeScript, look at the TypeScript GitHub Wiki.
  • If you need a quick introduction to the TS Compiler API, take a look at the page on using the TS Compiler API.
  • Take a look at the typescript.d.ts for type signatures, and to understand what is available to you from the TS toolchain.

If you need some help debugging, there is a DEBUG flag that can be enabled in src/main.ts:

const DEBUG = true; // switch from false to true to enable the console.logs

There are some helpful print statements which print out:

  • the SyntaxKind of the nodes being traversed
  • the minified string output
  • a dictionary of external/internal type casts

Remember to switch the DEBUG flag off afterwards.

Contributors


Daria Jung (Google intern)

License: Apache 2.0


Copyright 2015 Google, Inc. http://angular.io

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

Future


Property renaming is hard and there are a lot of issues! If you're interesting in contributing to this project, please check some of them out.

Issues are labelled easy, medium, and hard. Some have a Needs Exploration label which means you might have to poke around a bit before reaching conclusions about how to tackle the problem!

ts-minify's People

Contributors

dariajung avatar rkirov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ts-minify's Issues

compile time casting to an external type

Example: <ts.TextChange>{ span: 'something', newText: 'foo!' };

The Object Literal's properties cannot be renamed because it is being cast to an external type.
This should be added to the renaming heuristic.

Check the type that an object is being cast to and add it to the cast from: to dictionary.

in renaming cases, if node has > 1 identifier child, all get renamed

For instance, with PropertyAssignments, we just want to rename name.

 interface PropertyAssignment extends ObjectLiteralElement {
        _propertyAssignmentBrand: any;
        name: DeclarationName;
        questionToken?: Node;
        initializer: Expression;
    }

original:

function getFileAndName(): { fileName: string, qname: string } {
    var fileName = 'hello';
    var blah = 'world';
    return { fileName, qname: blah };
}

renaming:

function getFileAndName(): { $: string, _: string } {
    var fileName = 'hello';
    var blah = 'world';
    return { fileName, _: a };
}

blah gets renamed to a even though it is not a property name because it is an identifier node.

in a PropertyAssignment:
initializer: Expression; but Expression might be an Identifier

Possible fix: Check if (<any>node).name exists, if so, compare to child. If node.name === child, we can rename?

Explict any casting results in emitting 'undefined'

Something like:

var x = <any>Some_Class; will result in an emit that looks like var x = undefined;

This is likely due to the fact that there is no specific type information associated with an any type.

This issue is related to #53.

Possible ways to approach this:

  • Throw an error to the user telling them to type their program more specifically.
  • Show the user a warning about using explicit any casting, and emit the object being cast as is, or unsafely rename it.

Bug: generateNextPropertyName

Because the alphabet is $, _, 0 - 9, a - z, A - Z, some property names that are generated begin with a number, which is invalid.

Fix: Add a check to see if the generated property name's first character begins with a number.

Failure to find symbol information

require('source-map-support').install(); throws an error.
args._ throws an error (ts2dart main.ts line 331)

args wasn't typed to minimist.ParsedArgs

args.destination throws error. destination is a property that comes from a named option; we have no idea of knowing how many named options there will be. Had to update the typings for ParsedArgs in minimist.d.ts to:

export interface ParsedArgs {
                       _: string[];
                       [key: string]: any;
        }
try {
} catch(e) {
 e.name; // breakage, need to cast e to Error, otherwise, e is of type any: (<Error>e).name
}

most seem to be typing related problems. have better error message + handling

shorthand assignment desugaring

A shorthand assignment is when an object literal is created without explicitly writing the property name using the name: expression syntax.

An example:

var title = "hello world";
var author = "Foo Bar";
var obj = { title, author }; 

is the same as writing

var obj = {title: title, author: author};

We would like to desugar this ourselves when we are in the visit step. If we see a node with SyntaxKind.ShorthandPropertyAssignment, rewrite the identifier like so, and check if the name of the property can be renamed.

property_name_identifier: expression_being_assigned

Difficulty: Between easy and medium

Properties on "module" objects get renamed

import base = require('./base');


base.ts

export class Foo {
   . . .
}

base is some object that represents the base module. base.Foo is seen as a property access expression but we don't want to rename the Foo class name.

Possible routes:

  • Forbid import * from 'foo' syntax
  • Detect import * from 'foo' and understand foo is a "module" bundle object in subsequent uses of foo
  • Rename all identifiers (like class names, etc.)
  • See if TypeScript has some sort of type/symbol information for modules

Library level renaming

If you intend for your TypeScript code to be used as an external library (in the case that you have a public API), exports cannot be renamed. Private properties should still be renameable, however.

At the moment, there is no support for this.

Pushing to self-typed Array field causes all assignments to be lost

Based on SHA ffbbb47

Pushing an object of a given type to an array field causes all properties of that type to no longer be minified.

Input:

export class Foo {
  private _foolist:Array<Foo>;
  add(f:Foo) {
    this._foolist.push(f);
  }
}

Output:

export class Foo {
  private _foolist:Array<Foo>;
  add(f:Foo) {
    this._foolist.push(f);
  }
}

If I change the array to be an array of string instead, here is the nicely-renamed output

export class Foo {
  private $:Array<string>;
  _(f:string) {
    this.$.push(f);
  }
}

Typescript preprocessor

thanks for this, I have been looking for it! Great job!


I am considering to write a TS pre/post processor which should take care of at least following actions:

  • Conditional compilation (known from other languages)
  • Minified name to unminified name mapping (i.e. for usage in string references to properties).
  • Eventually other actions such as
    • joining all the same namespaces / modules which are using the single class per file pattern to just single file in order to prevent resulting code to be polluted by many namespace/module definitions (as it is now).
    • maybe handy, but I am not sure how much at this time
      • Minified to unminified library file mappings (something like C header file generation)
      • Unminified to minified library linking

The preprocessor should be implemented like that:

the preprocessor.ts must be included in the TS project. It will "virtually" pollute the main object with following statements:

$DEFINE = function $DEFINE(statement: string) {...};
$IFDEF = function $IFDEF(statement: string) {...};
$IFNDEF = function $IFNDEF(statement: string) {...};
$ELSE = function $ELSE(statement: string) {...};
$ENDIF = function $ENDIF() {...};

If we are talking mainly about minification, something like this would be great:
$LIBRARY = function $LIBRARY(libname: string)
It should take all classes from the same module and put them to the single library file or take classes from various namespaces / modules and make a single library source code file from it. The main reason would be to try to avoid a lot of repetitive namespace/module definitions in the compiled code.

I am still looking for the way how to reference minified property by a string so I came with this concept:
$MINIFIEDNAME = function $MINIFIEDNAME(name: string, object: any);
As previous statements are I would say straightforward this one I guess requires deeper explanation.
The function returns the original unminified variable/object/property name defined in string but after minification the preprocessor will take the resultant object parameter and will replace whole definition in the code by the minified name. The syntax looks really horrible so I am still trying to find a better way, how to define this in order to look nicer.

Let me give you an example:

// original .ts class definition:
class Test {
   protected _computedValue: number;
   protected _someName: number;
   public set someName(value: number) {
      this._someName = value;
      this[$MINIFIEDNAME('_computedValue', this._computedValue)] = value + 3;
      console.log('Property ' + $MINIFIEDNAME('someName', this.someName) + ' was changed');
   }
}

// minified .ts:
class a {
   protected a: number;
   protected b: number;
   public set c(a: number) {
      this.b = a;
      console.log('Property ' + $MINIFIEDNAME('someName', this.b) + ' was changed');
      this[$MINIFIEDNAME('_computedValue', this.a)] = a + 3;      
      console.log('Property ' + $MINIFIEDNAME('someName', this.a) + ' was changed');
   }
}

// postprocessed .ts (before compilation):
class a {
   protected a: number;
   protecte b: number;
   public set c(a: number) {
      this.b = a;
      console.log('Property ' + 'b' + ' was changed');
      this['a'] = a + 3;
      console.log('Property ' + 'a' + ' was changed');
   }
}

The Pre-Post processor will be included in the build process like this:

Unminified program -> Preprocessor -> Clean code without DEFS/IFDEFS/LIBRARIES -> Minifier -> Minimized code -> MinPreprocessor -> String names replaced -> Compiler -> Compiled JS and .map to generated lib files -> Postprocessor-> Compiled JS with maps to original source code

It will preprocess all TS files in the project (eventually defined folder walked through recursively) before and after minification step in order to:

Before minification:

  • Remove the preprocessor.ts from the target compilation
  • Remove all the code defined (and including) $IF ans so on
  • Join all related files to specified library/libraries

Calls minifier to do his job

After minification:

  • Replaces all occurences of $MINFIELDNAME ... by the minified name of the variable / object / property string

Calls compiler to do his job

Post processing:

  • remaps all compiler generated map files to match the original source code (because of joined libraries) in order to be possible to debug it.

Main requirements:

  • don't modify the typescript compiler
  • the preprocessor directives must be a valid TS code as the Dev IDE must be able to recognize without any additional changes to the Dev IDE itself (it would be great to have syntax highlighting for preprocessor directives but it is not that important)
  • if the preprocessor is not included in the build process the code must be still valid and must work in the same way as the processed code (even while it will be slower)
  • support for "changed source" compilation - compile just those files which were changed

Any comments and recommendations are welcome. Especially ideas, how to exactly implement it in order to be compatible with existing build systems.

object literal destructuring

From Rado:

As discussed object literal destructuring cannot be handled well, because it contains a property 
declaration or access (things we rename) and a local variable expression (something we leave to
 uglifyJS).

var {a} = foo;

is sugar for

var a = foo.a;
which should be renamed to

var a = foo.$
but there is no way to do that while keeping the sugar.

Similarly for property declaration {a} is sugar for {a: a} and we rename that to {$: a}. 
For now I suggest we detect these and throw an error.

Unable to minify

Hello,

I'm looking for minify tool which can obfuscate whole my project.
I need to rename all variables, methods, properties and classes names (including "export" ones).

I'm trying to use ts-minify but currently have these problem:

  1. "Symbol information could not be extracted"
    I have this error near:
export function x(let el: Element) {
el.className = "xx"; // here
}

do I need to load ES5 library for ts-minify?

  1. If I set fastFail to false, error (1) goes out (!) but I see another error:
    "Error: ts-minify does not support accepting both internal and external types at a use site
    Symbol name: Dialog"

I have class Dialog:

export class Dialog {
..
}

and use it in this way (in other files):

import {Dialog} from "dir/dialog"

What am I doing wrong? TSC sees no problems in such code.

  1. Exports are not being renamed...
    I pass all linked to each other files as one array to renameProgram() method, so I expect ts-minify knows anything about classes and can rename them in all the files.
    I want to rename anything except one API class I specify (and its public methods). All other classes are internal, tough they have "export" keyword.
    How to pass API class name to ts-minify and let ts-minify to rename everything except it?

I will be very appreciated for any help :)

Property Name Generation bug

Running the minifier through ts2dart produces this interesting renaming dictionary...

screen shot 2015-08-13 at 2 00 18 pm

lastGeneratedPropName isn't being updated in one of the if clauses

Constructor parameter property declaration

Parameter properties need to be desguared, or handled otherwise:

class Animal {
    constructor(private name: string) { 
        console.log(name);
    }
}

is sugar for

class Animal {
    private name: string;
    constructor(name: string) { 
        this.name = name;
    }
}

name is a property that, if renamed in the parameter property declaration, also needs to be renamed in the constructor body if used.

declarations inside constructor not renamed

Parameter properties
The keywords 'public' and 'private' also give you a shorthand for creating and initializing members of 
your class, by creating parameter properties. 
The properties let you can create and initialize a member in one step.
 Here's a further revision of the previous example.
 Notice how we drop 'theName' altogether and just use the shortened 'private name: string' parameter on the constructor to create and initialize the 'name' member.

Issues minifying ts2dart

  • TypeScript objects not being typed to classes/interfaces defined in typescript.d.ts (in want of more generic/flexible types)
    • ie: n: ts.ModifiersArray vs n: {flags: number} in base.ts line 61
  • Typing problems - if maybeVisitTypeArguments takes a type not externally declared, typeArguments get renamed, but adding a more strict typing like ts.ExpressionWithTypeArguments brings up an error when passed something like an object of type ts.CallExpression
    • error TS2345: Argument of type 'CallExpression' is not assignable to parameter of type 'ExpressionWithTypeArguments'.

Breakage, in cases like these:

return {fileName, qname};

function getFileAndName(): { fileName: string, qname: string } // rename properties here? {
    var fileName = 'hello';
    var qname = 'world';
    return { fileName, qname }; // ShortPropertyAssignment
}

A TypeLiteral node is comprised of

  • First Punctuation
  • SyntaxList
  • CloseBraceToken

SyntaxListis where the interesting bit could happen:

  • PropertySignature
    • Identifier
    • ColonToken
    • StringKeyword

Another scenario:

function getFileAndName(): { fileName: string, qname: string } {
    var fileName = 'hello';
    var blah = 'world';
    return { fileName, qname: blah };
}

The current algorithm contextEmit will rename qname and blah because they are both identifiers, when we only want to rename qname.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.