Giter VIP home page Giter VIP logo

askql's Introduction

AskQL is the next step after GraphQL and Serverless.

With AskQL developers can attach scripts to queries that are executed serverside. The AskQL parser accepts the GraphQL query format so there's no learning curve. Because the scripts are executed serverside and the results can be cached it's great for Web Vitals and app performance. Think of it as a programmable GraphQL.

Read a great articly on AskQL as a GraphQL alternative

Deploy your dynamic JAMStack, Mobile, CMS apps with no backend development required.

By doing so frontend developers needing additional API endpoints are no longer bound by the backend development release cycles. They can send the middleware/endpoint code along with the query. No deployments, no custom resolvers, lambdas required.

It's safe

AskQL uses the isolated Virtual Machine to execute the scripts and the resources concept that let you fully controll what integrations, collections and other data sources are accessible to the scripts. Moreover features like Access management and Secrets management are on their way.

Business loves it

We're working on a set of built in resources integrating AskQL with MACH data sources like eCommerce platforms (SFCC, Commercetools), databases (MongoDB, MySQL) etc. By having it all in - frontend devs can directly access the data sources, processing the data server-side, with no additional API endpoints, middlewares required. It' shortening the integration time a lot,

By the way, it's a Turning-complete query and programming language :-)

Getting started

AskQL comes with whole variety of default resources (resource is equivalent of GraphQL resolver). You should definitely read the Introduction to AskQL by @YonatanKra and AskQL Quickstart

Why and what for?

Benefits for development process:

  • Next milestone after GraphQL and REST API
  • New safe query language extedning the GraphQL syntax
  • Send code to servers without the need to deploy
  • Send executable code instead of JSONs
  • Give the frontend developers more freedom, simplifying the dev process to single layer

Benefits for programmers:

  • Asynchronous by default, no more await keyword - cleaner code
  • Processes only immutable data - fewer errors
  • Compiled to a clean functional code - clear logic

Benefits for business:

  • We're working on a set of built in resources integrating AskQL with MACH data sources like eCommerce platforms (SFCC, Commercetools), databases (MongoDB, MySQL) etc. It' shortening the integration time a lot,
  • Leaner, straightforward app development process and low maintenance cost - you built just the frontend app, no backend app is required.

Prerequisites

node >=12.14

Roadmap

You can use AskQL right away - both as the CLI scripting for extending your Node's app or as a GraphQL endpoint alternative. Howeve'r were working an some cool features making it even easier for business use-cases and You're invited to contribute :)

Quick Start

Installation

In your Node project run:

npm install askql

Usage

You can use AskQL interpreter for variety of use cases:

  • Ultimate endpoint accpeting the extended GraphQL queries

Sample server. Checkout full demo from @YonatanKra repo.

import askql from "askql";
import express from 'express';
import bodyParser from 'body-parser';

const { askExpressMiddleware } = middlewareFactory;
const { resources } = askql; // resources available to AskQL Scripts
const values = { }; // values available to AskQL scripts

export const askMiddleware = askExpressMiddleware(
    { resources, values },
    { callNext: true, passError: true }
);

const port = 8080;
const app = express();

app.use(express.static('public'));

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: true}));
app.post('/ask', [askMiddleware]);

app.listen(port, () => {
    console.log(`AskQL listening at http://localhost:${port}`);
});
  • CLI appps acting a query language

Sample index.js file:

const askql = require("askql");

(async () => {
  const result = await askql.runUntyped(
    { resources: askql.askvm.resources },
    askql.parse("ask { 'hello world!' }")
  );

  console.log(JSON.stringify(result, null, 2));
})();

πŸ‘‰ More examples

Examples

AskQL comes with whole variety of default resources (resource is equivalent of GraphQL resolver). You should definitely read the Introduction to AskQL by @YonatanKra and AskQL Quickstart

Query the Star Wars characters with AskQL and the fetch builtin resource:

ask {
	fetch('https://swapi.dev/api/people'):at('results'):map(fun(swCharacter) {
    {
      Name: swCharacter.name,
      Gender: swCharacter.gender,
      'Hair Color': swCharacter.hair_color
    }
  })
}

Development & Contributing

Please find all important information here:

Contributing guidelines

Installation from source

  1. Clone the repository

    git clone [email protected]:xFAANG/askql.git

  2. For Linux it is advised to install autoreconf as it is used by one of the Node packages used by AskScript Playground.

    For Ubuntu/Debian run: sudo apt-get install autoconf

  3. Install dependencies: npm i

  4. Build the project: npm run build

  5. Link the project to askql command: npm link

  6. Now you should be able to launch the interpreter (we use REPL for that): askql

Code examples

You can find all the examples in __tests__ folders (e.g. πŸ‘‰ AskScript tests) or in the Usage section below.

Documentation

Find AskQL documentation here.

The Documentation is divided into 4 parts:

Try It Yourself

Do not hesitate to try it out yourself! You can also find fellow AskQL devs in our Discord community.

Tools

CLI (AskScript interpreter)

Similar to python or node, AskScript CLI allows the user to type AskScript programs and get immediate result.

In order to run CLI:

  1. Build the code:

    npm run build
    
  2. Run:

    node dist/cli.js
    

Every input is treated as an AskScript program. For convenience, CLI expects just the body of your program, without ask{ }.

The editor has 2 modes - a default single-line mode and a multiline mode.

In order to enter the multiline mode, please type .editor.

At the end of your multiline input please press Ctrl+D.

    $ node dist/cli.js
    πŸ¦„ .editor
    // Entering editor mode (^D to finish, ^C to cancel)
    const a = 'Hello'
    a:concat(' world')

    (Ctrl+D pressed)

Result:

    string ask(let('a','Hello'),call(get('concat'),get('a'),' world'))
    'Hello world'

As the output CLI always prints AskCode (which would be sent to an AskVM machine if the code was executed over the network) and the result of the AskScript program.

Usage

  1. Write a hello world and test it out with the CLI interpreter! If you'd like to use the GraphQL like endpoint read this article.

In AskQL we only use single quotes:

πŸ¦„ 'Hello world'
string ask('Hello world')
'Hello world'

In the response you get a compiled version of the program that is sent asynchronously to the AskVM.

  1. There are two number types
πŸ¦„ 4
int ask(4)
4
πŸ¦„ 4.2
float ask(4.2)
4.2
  1. Let's say we've got more sophisticated example using the REST api and the fetch resource to get the current India's COVID19 stats:
ask {
  fetch('https://api.covid19india.org/data.json')['cases_time_series']
  :map(fun(dataSet) {
                         return {
                            data: dataSet['date'],
                            dailyconfirmed: dataSet['dailyconfirmed'],
                            dailydeceased: dataSet['dailydeceased'],
                            dailyrecovered: dataSet['dailyrecovered']
                          }
                        })
}
  1. Exit the console!

ctrl + d

  1. You finished the AskScript tutorial, congratulations! πŸŽ‰

Playground

Here is the link to our AskQL playground!

Developer info - how to run Playground frontend

  1. Copy .env.example to .env and set PLAYGROUND_PORT and PLAYGROUND_ASK_SERVER_URL appropriately. You can also set the optional GTM variable to your Google Tag Manager code.

    or

    You can also specify the variables in command line when running the Playground.

  2. Compile Playground:

    npm run playground:build
    

    or

    npm run build
    
  3. Run it:

    npm run playground:start
    

    You can specify port and server URL in command line:

    PLAYGROUND_PORT=8080 PLAYGROUND_ASK_SERVER_URL="http://localhost:1234" npm run playground:start
    

Additional notes

Some files in the Playground come or are inspired by https://github.com/microsoft/TypeScript-Node-Starter (MIT) and https://github.com/Coffeekraken/code-playground (MIT).

Developer info - how to run Playground backend

  1. Run:

    npm run build
    
  2. Run:

    node dist/playground-backend/express/demoAskScriptServer.js
    

    If you want to specify custom port, run:

    PORT=1234 node dist/playground-backend/express/demoAskScriptServer.js
    

    instead.

FAQ

What's the difference between ask { <askscript> } and eval( <javascript> )?

JavaScript's eval( <javascript> ) is terrible at ensuring security. One can execute there any code on any resources available in Javascript. Moreover there is no control over time of execution or stack size limit.

On contrary, Ask's ask { <askscript> } runs by default on a secure, sandboxed AskVM, which has a separate execution context. We have built in control mechanisms that only allow using external resources you configured. Ask programs are also run with the limits on execution time and stack size restrictions you define.

Troubleshooting

If you didn't find answers to your questions here, write on our Discord community. We will both help you with the first steps and discuss more advanced topics.

License

The code in this project is licensed under MIT license.

Core Developers

askql's People

Contributors

199201shubhamsahu avatar akeni avatar bardeutsch avatar bhargav-khalasi avatar cpelican avatar czerwinskilukasz1 avatar dastin-sandura avatar dependabot[bot] avatar fifciu avatar jamesd35 avatar kgajowy avatar km4 avatar markkulube avatar mhagmajer avatar piotrczubak avatar pkarw avatar richardm99 avatar sebastianwesolowski avatar tuhaj avatar undomalum avatar xfaang-ci avatar yonatankra avatar ysavoshevich avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

askql's Issues

< operator not defined

The following code fails because of an unknown < identifier:

πŸ¦„ .editor
// Entering editor mode (^D to finish, ^C to cancel)
let i = 0
for(i = 0; i < 5; i = i + 1) {
    
  }
  
Uncaught Error: Unknown identifier '<'!
    at Object.<anonymous> (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:47:15)
    at Generator.next (<anonymous>)
    at /Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:4:12)
    at Object.run (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:22:12)
πŸ¦„ 

Simplify AskScriptAst by inlining props

Currently AskScriptAst nodes are needlessly verbose with props.

For example, instead of having this come out from parseToAst in askscript library:

{
  "name": "if",
  "props": {
    "condition": {
      "name": "call",
      "props": {
        "name": "checkThis",
        "args": []
      }
    }
  },
  "children": [
    {
      "name": "return",
      "props": {
        "value": "your string"
      }
    }
  ]
}

We could decide that name (renamed to "type") and children are reserved prop names, and arrive at a simpler structure such as:

{
  "type": "if",
  "condition": {
    "type": "call",
    "name": "checkThis",
    "args": []
  },
  "children": [
    {
      "type": "return",
      "value": "your string"
    }
  ]
}

Syntax sugar for function type and rest arguments

Example 1
(int, int) -> int

const sum2: (int, int) -> int = fun (a:int, b:int) { a + b }
const sum2: fun(int, int, int) = fun (a:int, b:int) { a + b }

Example 2
(...array(int)) -> int

const sumN: (...array(int)) -> int = fun (...args:array(int)) { sum(args)) }
const sumN: fun(array(int), true) = fun { sum(resources:at('args')) }

where fun is the function type constructor introduced by #42.

Restore tests for 'remote' block

Currently all tests for remote are marked as not implemented.
image

What is needed is running a server (or a mock server) during the tests which would respond on the code from the remote block.

Build - pin dependencies

Currently, dependencies are not pinned to a particular version. It may cause serious issues, as some breaking changes may happen even in patch updates. To ensure that every build remains the same, even if dependencies have not changed, we should pin them by removing ^ from package.json (and of course updating .lock).
Otherwise, it may happen that triggering the very same build on different machines (depending on local node_modules) may lead to unpredicted issues.

Secondly, after pinning, I would recommend enabling Dependabot for constant updates to ensure stability and security. I used a few of similar services (like Renovate) and this one seems to be the most solid one.

Add comma as a way to list statements

All these cause syntax errors:

Example 1

ask { 1, 2 }
// returns 2

Example 2

ask {
  1, 2
}
// returns 2

Example 3

for(let i = 0; i < arr:size; i = i + 1, sum = sum + arr:at(i)) { }

npm run build fails on a fresh Debian 10

Steps to reproduce:
Follow instructions from "Development & Contributing" at https://github.com/xFAANG/askql on a clean Debian 10 server.
0. Install Node 12 by following this guide: https://www.digitalocean.com/community/tutorials/how-to-install-node-js-on-debian-10

  1. Clone the repository: git clone [email protected]:xFAANG/askql.git
  2. Install dependencies npm i
  3. Build the project npm run build

Expected result: npm run build succeeds.

Actual:
Got 38 errors.

image

(...)

image

CLI doesn't preserve environment between statements

πŸ¦„ let n = 5
int ask(let('n',5))
5
πŸ¦„ n
Uncaught Error: Unknown identifier 'n'!
    at Object.<anonymous> (/Users/marcin/Repositories/askql/dist/askvm/lib/run.js:47:15)
    at Generator.next (<anonymous>)
    at /Users/marcin/Repositories/askql/dist/askvm/lib/run.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/marcin/Repositories/askql/dist/askvm/lib/run.js:4:12)
    at Object.run (/Users/marcin/Repositories/askql/dist/askvm/lib/run.js:22:12)

Possible invalid regex

'**/__tests__/**/*.[jt]s?(x)',
    '**/?(*.)+(spec|test).[jt]s?(x)',

I think there's an issue with the suffix.
I believe it looks for files that end with jx or tx with a possible s in the middle.

call doesn't work in certain cases

Probably call in AskScript has different semantics than in AskCode due to dereferencing of identifiers. This causes different issues:

Example 1

ask {
  call(fun { 5 })
}

returns the AskCode but 5 is expected

Example 2

ask {
  let f = fun { 5 }
  f
}

returns 5 but AskCode for f is expected

Example 3

ask {
  f
}

returns null but the resource for f is expected (currently shorthand for fragment which is used in complex clauses such as if or while)

'remote' should accept arguments

While it's possible to run a procedure (a parameterless function) on a remote server, e.g. a factorial of a hardcoded number, it's not possible to pass any parameters, e.g. factorial of n.

`npm test` does not run `.test.ask` test files

How to reproduce:

  1. Open a *.test.ask file, e.g. src/askscript/tests/00-documentation-examples/documentation01-complete_example.test.ask
  2. Write an untrue expect().toBe() statement, e.g.: expect(2 + 2):toBe(5)
  3. Run npm test.

Expected: npm test fails with an error.
Actual: npm test succeeds with no error.
Note that also running npm test src/askscript/__tests__/00-documentation-examples/documentation01-complete_example.test.ask succeeds with a 'todo' mark:

PASS test src/askscript/tests/00-documentation-examples/documentation01-complete_example.test.ask
βœ“ starts
βœ“ compiles
✎ todo computes

Contribution Guide - missing flow / policies of contribution

It would be nice to add some policy on how one can contribute to the following cases:

  • Existing issue. Should I post a comment under issue and wait for some approval? How one can ensure that when wants to work on some issue, someone isn't already on it?
  • Is this allowed to do some minor changes in batch?
  • To what degree should the contribution be agreed on (before implementation) in case of
    • breaking changes
    • scout-boy rules
  • Missing PR Template would help a lot!

Thanks in advance for answering those questions, it would greatly increase understanding how to correctly contribute ⚑

Implement custom parser for AskScript

We already have a source code parser in askcode. I think once we have solidified the syntax for AskScript, we can translate it onto the custom parser letting go of pegjs dependency which doesn't seem to be maintained anymore. This could also mean huge savings in the amount of code in the askscript package. The complied parser alone is now 5268 locs and over 138KB of text.

Generate .askcode and .askjsx in GitHub actions

Maybe GitHub action could also take care of generating .askcode and .askjson files? Currently these files are generated on npm test. This way maybe they wouldn't have to be included in reviewed code. More reasearch is needed here on how this would work exactly.

AskVM not timing out on infinite loops

According to the FAQ, the AskVM should timeout on infinite loops.

However running the following in the AskVM generates a hang (infinite loop) that does not seem to resolve.

const f = fun():any {
  call(f)
}
call(f)

To replicate:

  • npm install askql
  • ./askql
  • .editor
  • Enter the above code
  • Ctrl+D

Can't define a variable in for

The following code results in an error:

πŸ¦„ .editor
// Entering editor mode (^D to finish, ^C to cancel)
for(let i = 0; i < 5; i = i + 1) {
    
  }
  
Uncaught Error: Unknown identifier 'i'!
    at Object.<anonymous> (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:47:15)
    at Generator.next (<anonymous>)
    at /Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:4:12)
    at Object.run (/Users/milimetr/Desktop/x/askql/dist/askvm/lib/run.js:22:12)

Don't build the test files

Currently, the build process builds the test files and they appear in the dist folder. This should be avoided.

Only *.test.ts files use jasmine2 runner in Jest

When running tests I see a strange issue.
Here's my test:

it('should run a simple query', function () {
    expect(false).toEqual(true);
  });

But when I run the tests I get that everything is passing:
image
I thought it is my test but - I've added the same expect to an existing test src/askjsx/__tests__/render.test.tsx:

import * as askjsx from '..';
import { toAskCode } from '../../askcode';
askjsx;

test('jsx', () => {
  const sum = <call name="sum" args={[4, 5]} />;
  expect(false).toEqual(true);
  expect(sum).toStrictEqual(
    toAskCode({
      name: 'call',
      params: [toAskCode({ name: 'get', params: ['sum'] }), 4, 5],
    })
  );
});

And now all 3 changed suites pass:
image

Create testing documentation

Some untold processes for testing like:

  1. running tsc after changing the runners
  2. Running npm test <filePath>
  3. Document the way the custom transformers and runners work (big files with no tests?)
    etc.

AskScript: TypeError: undefined is not iterable (cannot read property Symbol(Symbol.iterator))

Watching https://youtu.be/Su1FBJ5r7CU?t=221 shows running the current snippet
should be possible:

find(philosophers, fun(is(at(scorePerPhilosopher, get('$0')), max(scorePerPhilosopher))))

However, here is what happens:

πŸ¦„ find(philosophers, fun(is(at(scorePerPhilosopher, get('$0')), max(scorePerPhilosopher))))
Thrown:
TypeError: undefined is not iterable (cannot read property Symbol(Symbol.iterator))
    at Resource.<anonymous> (/Users/piotrzientara/xfaang/askql/dist/askvm/resources/core/get.js:19:29)
    at Generator.next (<anonymous>)
    at /Users/piotrzientara/xfaang/askql/dist/askvm/resources/core/get.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/piotrzientara/xfaang/askql/dist/askvm/resources/core/get.js:4:12)
    at Resource.compute (/Users/piotrzientara/xfaang/askql/dist/askvm/resources/core/get.js:18:16)
    at Object.<anonymous> (/Users/piotrzientara/xfaang/askql/dist/askvm/lib/run.js:37:56)
    at Generator.next (<anonymous>)

int[] syntax sugar for array(int) constructor

The purpose of this feature request is to support the same syntax sugar that TypeScript has for Array type definition which is T[]. When user writes T[] in type position in AskScript, the parser should interpreter it as array(T) where T like int

Dear new contributor,

If you have any questions on how to start, please write a comment with a tag: @czerwinskilukasz1 or @mhagmajer, or join our Discord Community and let us know there. We will be very happy to help you start! :)

Cheers,
Łukasz (@czerwinskilukasz1)
AskQL Core Developer

Build & Release - Conventional Commits & Changelog

This suggestion may be potentially breaking for the current system but I believe it is worth the initial effort.

Issues/Automation to address:

  • force conventional commits (git-cz)
  • generate changelog using commits messages
  • easier release

A helper for newcomers to this standard - would be useful for new contributors to reduce the initial effort to get the submission right.

In addition to commitizen package, cz-conventional-changelog, and standard-versions packages, it's quite easy to make an automated way of generating changelog (based on commit messages) and publishing new versions.

The flow could look like:

  1. All commits to develop come with conventional commit messages, strictly related to given modules.
  2. When release time has come, use standard-version to bump package, build changelog, create the tag and publish to npm

Example of the output here

It may seem to be a little complicated but trust me - it's great automation made in an easy way. It reduces the need to manually manage What's new.

PS

All commits in PR are squashed on merge. Please use Conventional Commits when naming your PR.

There is no point in using conventional commits everywhere, as long as the PR is squashed (only the first one will be used). Please consider allowing to not squash if a PR touches multiple modules/areas and the changes/commits are correctly separated.

Contribution Guide - avoid a need to configure github CLI

Turn off autocrlf in your Git.
This is advised because some of the test files have Windows line endings on purpose and we would like to keep them there.
git config core.autocrlf false

This step from contribution guide can be avoided as long as we add

{
  "endOfLine": "lf"
}

to .prettierc or config file.

Source

// Willing to take this on.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.