Giter VIP home page Giter VIP logo

ziggy's Introduction

Ziggy

A data serialization language for expressing clear API messages, config files, etc.

Status

Alpha, using Ziggy now means participating in its development.

At a glance

.id = @uuid("..."),
.time = 1710085168,
.payload = Command {
  .do = @action("clear_chat"),
  .sender = "kristoff-it",
  .roles = ["admin", "mod"],
  .extra = {
    "agent": "Mozilla/5.0",
    "os": "Linux/x64", 
  },
}

Value Types

Ziggy values can be of the following types:

  • Bytes "🧑‍🚀", "\x1B[?1000h gang", \\multiline
  • Numbers 123_000, 1.23, 0xff_ff_ff, 0o7_5_5, 0b01_01_01
  • Null null
  • Bool true, false
  • Custom Literals @date("2020-12-01"), @v("1.0.0"), @foo("bar")
  • Array [1, 2, 3]
  • Struct { .fixed = "schema" }, Named { .for = "unions of structs" }
  • Map { "custom": "keys" }

Documentation

See the official website: https://ziggy-lang.io

ziggy's People

Contributors

kristoff-it avatar travisstaloch avatar der-teufel-programming avatar ethanholz avatar mfashby avatar

Stargazers

const void* avatar M. Schiller avatar Sean Bohan avatar Nick Mazuk avatar Mohamed Akram avatar Yozachar avatar Niklas Stegmann avatar Calle Englund avatar Joachim Giron avatar Selman Kayrancioglu avatar Corey Raymond DeMarse avatar haruki7049 avatar  avatar Samuel Fiedler avatar alaska avatar Rafael Ristovski avatar Soroush Mirzaei avatar  avatar em jordan avatar Ian Johnson avatar Jan Ph. H. avatar Michael Chiu avatar  avatar Igor avatar Lee Cannon avatar LingFeng avatar Ludwig Pouey avatar Arnau Camprubí avatar Denny George avatar  avatar Brian Wo avatar Ruan G. Molinari avatar O. Tadashi avatar Jared Leonard avatar Artem avatar Amit D. avatar Jsiwa avatar mobus avatar Rhiza avatar Jens van de Wiel avatar William avatar Patrick Park avatar Jakob Udsholt avatar Gendon Tann avatar  avatar Julien Bisconti avatar  avatar Can Rau avatar Wesley Koerber avatar Phanuprat Suwannachan avatar Blazy avatar  avatar Fridge avatar fwcd avatar Oscar Aguinagalde avatar Hoang Phan avatar Leonard Aukea avatar Derick Rodriguez avatar orzogc avatar Kamil Shakirov avatar Eder Sosa avatar Slava avatar Daniel Hill avatar Rylee Alanza Lyman avatar George Thayamkery avatar David Granström avatar George Kontridze avatar Piotr Wera avatar  avatar Rene Schallner avatar Malcolm Still avatar sooriya avatar John Olheiser avatar Natalie Perret avatar  avatar Matías Cartes avatar yuekcc avatar Niranjan Anandkumar avatar Zhang Kaizhao avatar  avatar Shail Patel avatar Alex Rios avatar  avatar Ashutosh Sajan avatar James Hulse avatar LuizZzZzzZ avatar  avatar Sam Boysel avatar  avatar Lucas avatar Andrey Kirpach avatar  avatar  avatar Alex Kwiatkowski avatar  avatar Peter Li avatar Kai Norman Clasen avatar Dean avatar Andrew Meier avatar Benny Peake avatar

Watchers

 avatar  avatar chaosratel avatar

ziggy's Issues

real support for numbers

Right now number parsing is PoC quality.

Goal is to support everything that Zig supports, ideally by leveraging the Zig stdlib parser.

schema: complete validation

In the construction of a Schema struct, we currently check for:

  • Duplicate tag literal definitions
  • Usage of unknown tag literals
  • Duplicate struct definitions
  • Duplicate struct field
  • Usage of unknown struct names
  • Duplicate mentions of the same Struct inside of a struct union expression

What's missing:

  • Recognizing types of infinite size (ie self-references outside of a map or array container)
  • Recognizing definitions impossible to reach from the root expression

Note that the presence of any automatically colors everything.

How to get started

Here we perform the aforementioned analysis:

ziggy/src/schema/Schema.zig

Lines 168 to 176 in 4905290

log.debug("beginning analysis", .{});
try schema.analyzeRule(gpa, schema.root, nodes, code, diagnostic);
log.debug("root_rule analized", .{});
for (schema.structs.keys(), schema.structs.values()) |s_name, s| {
for (s.fields.keys(), s.fields.values()) |f_name, f| {
log.debug("analyzeRule '{s}.{s}'", .{ s_name, f_name });
try schema.analyzeRule(gpa, f.rule, nodes, code, diagnostic);
}
}

Below those lines it would make sense to call a new function that performs the remaining checks.
It might make sense to add a field or two to the existing types (eg StructRule, Rule).

implement the serializer

Look at src/Parser.zig for inspiration.

Interface prototype to start from:

pub fn stringify(value: anytype,  opts: StringifyOptions, writer: anytype) !void {}

pub const StringifyOptions = struct {
    whitespace: enum { minified, some_space_options, tab } = .minified,
    emit_null_fields: bool,
};

This is a good starting point, dealing with emitting tagged strings and other complications can be tackled in a follow up issue.

Follow up feature: support for a nonportable_numbers_as_strings: bool option.

real support for strings

Right now string parsing is PoC quality.

Goal is to support everything that Zig supports, ideally by leveraging the Zig stdlib parser.

Related design question: we want to be able to report an error at a given line and column. What's the correct way or reporting column number in the presence of things like emojis?

Add an easy way to toggle off serialization of a field with default values.

Sometimes there might be fields that have default values that for one reason or another might not need to be serialized / deserialized, as initing them with their default value is always the correct choice.

Right now this is needed to implement this feature in a type:

    pub const ziggy_options = struct {
        pub fn stringify(
            value: Assets,
            opts: ziggy.serializer.StringifyOptions,
            indent_level: usize,
            depth: usize,
            writer: anytype,
        ) !void {
            _ = value;
            _ = opts;
            _ = indent_level;
            _ = depth;

            try writer.writeAll("{}");
        }

        pub fn parse(
            p: *ziggy.Parser,
            first_tok: ziggy.Tokenizer.Token,
        ) !Assets {
            try p.must(first_tok, .lb);
            _ = try p.nextMust(.rb);
            return .{};
        }
    };
  1. Since the outer type has a field of this type, you must find a "placeholder" value to put there.
  2. You have to parse it correctly, otherwise you will leave in the stream tokens that need do be consumed.

Ideally this should be a setting that one can specify in ziggy_options

convert: improve JSON converter

Right now the JSON converter is using std.json to parse a tree of json.Value but this way we lose line and column information when producing diagnostics.

This means that right now we print error: missing field 'foo', but we can't tell at which line and column of the original json file this field was missing (ie where the surrounding object closes).

To make this happen, we need to switch to a different approach that involves not losing information about the original token stream.

One potentially interesting solution is to start from a JSON token stream and produce a Ziggy AST.

See src/cli/convert.zig for more information.

Also see #17 for the overall goal of completing ziggy convert.

Complete `ziggy convert`

Related issues

Additionally support for PATH mode still needs to be implemented, alongside all the other minor flags.

See ziggy convert --help for a full list of features that must be supported.

CBOR for the binary format?

The README currently says "Binary Format \ Note: this part is still vaporware \ An optimized binary representation that omits field names and avoids escaping strings." I suggest that designing a binary format from scratch may be some work and, while there are good reasons for creating a new text format like Ziggy, creating a new binary format may be too much. Why not using CBOR, standardized in RFC 8949?

  1. It already exists, with a lot of implementations,
  2. It is optimized for storage space, and simplicity of parsers,
  3. Its data model is very close from the one of JSON so it seems it would be fine for Ziggy.

convert: conversion strategies for struct unions

At the moment, given the following files, conversion will succeed:

build.json

{
        "name": "zine",
        "version": "0.0.1"
        "dependencies": {
                "foo": {
                        "Remote": {
                                "url": "https",
                                "hash": "aaa"
                        }
                },
                "bar": {
                        "Local": {
                                "path": "/usr"
                        }
                }
        }
}

build.ziggy-schema

root = Project

struct Project {
    name: bytes,
    version: bytes,
    dependencies: map[Remote | Local],
}

struct Remote {
   url: bytes,
   hash: bytes,   
}

struct Local {
  path: bytes,
} 

Result:

{
    .name = "zine",
    .version = "0.0.1",
    .dependencies = {
        "foo": Remote {
            .url = "https",
            .hash = "aaa",
        },
        "bar": Local {
            .path = "/usr",
        },
    },
}

The goal is to also make the following JSON files succeed at importing:

Shape matching:

{
	"name": "zine",
	"version": "0.0.1"
	"dependencies": {
		"foo": {
			"url": "https",
			"hash": "aaa"
		},
		"bar": {
			"path": "/usr"
		}
	}
}

type field:

{
	"name": "zine",
	"version": "0.0.1"
	"dependencies": {
		"foo": {
			"type": "Remote",
			"url": "https",
			"hash": "aaa"
		},
		"bar": {
			"type": "Local",
			"path": "/usr"
		}
	}
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.