novaleaf / xlib Goto Github PK

Your isomorphic toolbox

License: MIT License

TypeScript 96.37% Batchfile 0.05% JavaScript 2.66% HTML 0.20% CSS 0.57% Shell 0.14%

typescript cross-platform corelib isomorphic-javascript isomorphic xlib monorepo

xlib's Introduction

REWRITE IN PROGRESS

Xlib is being rewritten from first principles. The current master branch of this repo is dedicated to this rewrite. The current npm xlib package points to [email protected] which is a stable, high quality core/util library for node. See the v17 branch for it's source.

The v17 branch is:

highly stable
full featured
used in production
available as the default npm xlib package

The problems with `v17`

too opinionated: had debugging and logging interactions built into the runtime. v18 includes a logger and source-map-support but doesn't do these automatically.
obsolete features
- required Bluebird promises: bluebird was great before promises became a native part of javascript. v18 has xlib.promise which supports any promise implementation you use.
- brittle axios: axios "works" but fails for advanced features (proxy, network failures, authentication). v18 uses gaxios instead and is less strongly tied to it.
- no use of async/await: not in itself a reason to rewrite, but supporting async/await by default simplifies the api
node only: v18 targets browsers and node equally

The remainder of this readme is dedicated to the v18 rewrite.

XLIB v18+

Your isomorphic toolbox

Goals

Monorepo: the core xlib, tooling, and build verification projects all live in this repo
Docs: full online documentation
Promises: async / await by default
Isometric: full support for node and browser
Performance: take advantage of worker threads where it makes sense
Full Featured: aim to provide 80% of utility needs
Professional: no hacks, fully documented, deployment environment aware
Dev Experience: will include dev-env scafolding scripts and an easy build system
Lightweight: A library, not framework. No unexpected logging to console, tree-shake support

Non-Goals

Legacy browser support: Ignoring IE, but Edge-Classic support will be attempted
Legacy Node support: Development is done on Node 14.x. Node 12.x should work fine.
Dev Tooling Agnostism: Development is done on VSCode via ubuntu. You should be able to build/dev on windows, but it's not tested.
- Win10 Tip: Use WSL for development!

R&D Status

Current v18.x is of develop build quality (meaning: do not use this right now).

Critical R&D complete. Now porting most useful xlib features to the new codebase.

The following signifiers will be attached to the v18 rewrite as work progresses

develop: not suitable for any use.
alpha: usable for PoC projects only.
beta: can be used for production, but features may be missing, and breaking changes should be expected between xlib versions

COMPLETE

isomorphic: xlib feature parity for browser and node projects
- Full test suite runs (and passes) under Node and Evergreen Firefox + Chrome
typings: ensure dependent projects get sub-package typings automatically
- basically, tsconfig.json has to be setup properly. for example, esModuleInterop:true
debugging: ensure dependent projects can debug into xlib's *.ts source files
- dependent project's launch.json needs to properly setup the outFiles setting. see build-examples/xlib-node-basic
- browser debugging properly sourcemaps to original .ts files
monorepo: develop xlib ecosystem as seperate projects within the same repository
- use npm @microsoft/rush
upgradeability: ensure dependent modules can be properly upgraded in a reliable way
- use rush check and rush dep-check and rush dep-upgrade for this workflow
documentation: proper code documentation, auto-gen from source
- functional doc system using api-extractor via the @xlib/xlib-docs website project.
- if we need to change in the future, consider using https://github.com/tgreyuk/typedoc-plugin-markdown
testing: isometric testing supported: mocha used in browsers, jest used in node.
- jest doesn't work in browsers. didn't switch to full mocha because heft nicely runs jest tests when it builds typescript, and I don't want to spend the time figuring out to do similar with mocha.
- however there is a problem, in that jest's expect library isn't available cross-platform and it doesn't seem easy to register another assertion lib to globally override jest's in node.
- so currently working on building the xlib.diagnostics.logging module to handle this kind of work.

PENDING

publishing: ensure xlib ecosystem can be published to npm
basic react e2e: proof of concept using react to setup a real app (including ssl and dev vs production env)
doc site: a documenation site with full text search and versioning
modern promise libary
- probably just use builtin promises
enterprise grade logging system (without the enterprise plumbing requirement)
- ??? some plugable listener lib probably
isomorphic worker threads: currently using npm threads which works, but is brittle

xlib's People

Contributors

Stargazers

Watchers

Forkers

mcroker

xlib's Issues

add numHelper.aboutEqual()

function aboutEqual( input: number, checkAgainst: number, maxDeviationPercent: number, extraSpreadBuffer:number=0 ): boolean {
	return ( (input * ( 1 + maxDeviationPercent ))+extraSpreadBuffer > checkAgainst ) && (( input * ( 1 - maxDeviationPercent ))-extraSpreadBuffer < checkAgainst );
}

envLevel default to NODE_ENV if not explicitly set

if envLevel is not set, firstly default to whatever NODE_ENV is set to (production or not)

stringhelper.summarize() includes embeded quotes around output string

problem due to JSON.stringify() call on string.

if a string is passed in, avoid the stringify() call.

if a non-string is passed in, use jsonx.stringify()

feature: custom log targets (fluentd)

allow logger to direct output to https://github.com/fluent/fluent-logger-node

allow any fluentd target as a recipient.

this will allow projects to customize log recipients

Also as a bonus, projects running on google-cloud will have automatic log export to stackdriver logging. see: https://cloud.google.com/error-reporting/docs/setup/compute-engine#using_logging

code quality: ensure tslint can catch accidental function references (vs calls)

in some private code I found some accidental calls like such:

const loadAvg = ( nlib.os.loadavg[ 0 ] / nlib.os.cpus.length );

this is a bug because cpus is actually a function that needs to be called as such: cpus().length

same with loadavg. it's a function.

however it is valid JS code so compiler didn't catch it. find a tslint rule that can catch these mistakes and set it to error

add reflection.getFunctionName()

something like:

/** get the name of the current function, and all functions under it in the call stack */
function getFunctionName(
/** where to start on the call stack.   default is 0 (the immediate caller) */
stackDepth=0
): { name:  parameters:[] target:string} []

add threading support (webworkers) and make async versions of expensive operations

node v10 adds webworker support. all modern browsers now have it too.

create async versions of jsonx and make it the default usage.

security: update JSONX.parse() to reject proto keys

as shown here, json.parse() has exploit possibility: https://github.com/hapijs/bourne

either use bourne or port it's proto inspection regex into our jsonx util

this security issue isn't exploitable by default. it requires mallicious user input to be parsed and then assigned via object.assign() for the payload to become activated. This isn't a super common workflow, but This proposed fix would make the risk moot.

dependencies break node 4.x

some 3rd party library dependency (probably google cloud tools) breaks versions of node older than 6.x

we should track this down and try to revert the dependency or remove it.

see Novaleaf/phantomjscloud-node#6 for original report of this issue.

debloat: minimalize features

some features have been rarely used by novaleaf in the last few years.

these features increase load times, and potentially collide with typescript type definitions.

remove these and list as removed in the readme.

if we get load on demand working sometime in the future, these can be considered for readding

switch from ```axios``` to ```request```

axios has a lot of systemic bugs that you don't discover until using it a lot. for example: https proxy support, http(s) agent bugs (like lack of connection pooling), and unhandled promise exceptions.

rework the network sub-module to switch to the request library, however unsure if that works in a browser. maybe not.

improve time.Stopwatch functionality

add Stopwatch.start() static method and .pause() and .reset() instance members

usability: if new Exception message already contains innerError message, don't append it again

expected usage of innerError is:

const oldErr;
const err = new xlib.diagnostics.Exception("my msg",{innerError:oldErr});

the above will paste the innerError.message as part of the new err.message.

however, if the user manually pastes in the innerError.message as the following, we should not double-paste it.

const oldErr;
const err = new xlib.diagnostics.Exception("my msg.  innerErr="+oldErr.message,{innerError:oldErr});

consider doing this, however if node v12 comes out before hand, consider just doing #9 instead.

diagnostics.errorToJson() doesn't preserve error properties

only message,name, and stack are kept. need to preserve other properties too.

AsyncReaderWriterLock.writeEnd() requires a parameter even if TValue is void

make the write parameter optional