Giter VIP home page Giter VIP logo

safeyaml's Introduction

SafeYAML

SafeYAML is an aggressively small subset of YAML. It's everything you need for human-readable-and-writable configuration files, and nothing more.

You don't need to integrate a new parser library: keep using your language's best-maintained YAML parser, and drop the safeyaml linter into your CI pipeline, pre-commit hook and/or text editor. It's a standalone script, so you don't have any new dependencies to worry about.

What's allowed?

It's best described as JSON plus the following:

  • You can use indentation for structure (braces are optional)
  • Keys can be unquoted (foo: 1, rather than "foo": 1), or quoted with '' instead
  • Single-line comments with #
  • Trailing commas allowed within [] or {}

More details are in the specification safeyaml.md

Here's an example:

title: "SafeYAML Example"

database:
  server: "192.168.1.1"

  ports:
    - 8000
    - 8001
    - 8002

  enabled: true

servers:
  # JSON-style objects
  alpha: {
    "ip": "10.0.0.1",
    "names": [
      "alpha",
      "alpha.server",
    ],
  }
  beta: {
    "ip": "10.0.0.2",
    "names": ["beta"],
  }

As for what's disallowed: a lot. String values must always be quoted. Boolean values must be written as true or false (yes, Y, y, on etc are not allowed). Indented blocks must start on their own line.

No anchors, no references. No multi-line strings. No multi-document streams. No custom tagged values. No Octal, or Hexadecimal. No sexagesimal numbers.

Why?

The prevalence of YAML as a configuration format is testament to the unfriendliness of JSON, but the YAML language is terrifyingly huge and full of pitfalls for people just trying to write configuration (the number of ways to write a Boolean value is a prime example).

There have been plenty of attempts to define other configuration languages (TOML, Hugo, HJSON, etc). Subjectively, most of them are less friendly than YAML (.ini-style formats quickly become cumbersome for structures with two or more levels of nesting). Objectively, all of them face the uphill struggle of needing a parser to be written and maintained in every popular programming language.

A language which is a subset of YAML, however, needs no new parser - just a linter to ensure that files conform. The safeyaml linter is an independent executable, so whatever language and tooling you're currently using, you can continue to use it - it's just one more step in your code quality process.

How do I use it?

The safeyaml executable will validate your YAML code, or fail with an error if it can't. Here's an example of a passing validation:

$ cat input.yaml
title: "My YAML file"

$ safeyaml input.yaml
title: "My YAML file"

Here's an example of an error:

$ cat input.yaml
command: yes

$ safeyaml input.yaml
input.yaml:1:11:Can't use 'yes' as a value. Please either surround it in quotes
if it's a string, or replace it with `true` if it's a boolean.

With the --fix option, safeyaml can automatically repair some problems within YAML files.

Here's an example file that has some problems:

$ cat input.yaml
name: sonic the hedgehog      # Unquoted string
settings:{a:1,b:2}            # Missing ' ' after ':'
list:
- "item"                      # Unindented list item

$ safeyaml --fix input.yaml
name: "sonic the hedgehog"
settings: {a: 1,b: 2}
list:
  - "item"

To rewrite the fixed YAML back to the input files, pass the --in-place flag:

$ safeyaml --fix --in-place input.yaml

You can turn individual "fix" rules off and on:

--fix-unquoted will put quotes around unquoted strings inside an indented map. This does not affect map keys (which must still be in identifier format, i.e a1.b2.c2).

--fix-nospace ensures that at least one space follows the : after every key.

--fix-nodent ensures list items inside maps are further indented than their parent key.

There are also some more forceful options which aren't included in --fix:

--force-string-keys turns every key into a string. This will replace any key that has a boolean or null ('true' etc) with the string version (i.e "true").

--force-commas ensures every non-empty list or map has a trailing comma.

Other Arguments

--json output JSON instead of YAML.

--quiet don't output YAML on success.

How do I generate it?

Don't. Generating YAML is almost always a bad idea. Generate JSON if you need to serialize data.

safeyaml's People

Contributors

tef avatar aanand avatar

Stargazers

 avatar Michael Floering avatar Kushal Shingote avatar Mark Steward avatar Jonty Wareing avatar  avatar Bueller avatar Darek Kedra avatar Ben Firshman avatar Masaki Kozuki avatar Andreas Jansson avatar Amio Jin avatar Jon avatar Joshua Pollack avatar Eamonn O'Brien-Strain avatar Ali S. Raza avatar Alex Willmer avatar  avatar Dave Evans avatar Scott Ivey avatar Matthieu Jacquot avatar Ahmed Ghoneim avatar bryfry avatar patrixia avatar Maurizio De Santis avatar Chase Tingley avatar cecil avatar Stephen Jung avatar  avatar Ken Salomon avatar Ranvir Singh avatar Thomas Widhalm avatar Charles FD avatar Guillaume avatar Dan Levy avatar Scott Smith avatar  avatar André Arko avatar Martin Schütte avatar Dan Mannock avatar Paulo Roberto Ribeiro avatar John Keyes avatar Carl Furrow avatar Henrique Cabral avatar 爱可可-爱生活 avatar  avatar Wim Jeantine-Glenn avatar Markus Zapke-Gründemann avatar Krzysztof Góralski avatar Ben Nugent avatar Sullivan S. avatar Karanja Denis avatar Fabian Neumann avatar Steffen Gransow avatar Zhao Xiaohong avatar Sérgio Silveira avatar Yuanwei Wu avatar Nikolay Kolev avatar Marcin Zajączkowski avatar Randall Degges avatar Mike Helmick avatar Marco Federighi avatar Nikolaus Schlemm avatar Étienne BERSAC avatar Thomas Wickham avatar Liqiang Lau avatar Pete Cornish avatar Sebastian Blask avatar Anderson Araujo avatar  avatar Paweł Adamczak avatar Ales Kuchar avatar George Kussumoto avatar Alex Lebedev avatar Ryan D avatar Robbert Korving avatar Benoit Brayer avatar m.l avatar Jonathan Egol avatar  avatar 0ri0n ☠ avatar  avatar Leonardo Rossi avatar Matthew French avatar Juuso Mikkonen avatar Etienne Marais avatar Martin Simon avatar Keith Chambers avatar curtis avatar Sergey Arkhipov avatar Christian avatar Phil Andrew avatar Bojan Delić avatar Christian Betz avatar Isagani Mendoza avatar Martin Suchanek avatar Ilia Averianov avatar Edwin Cheruiyot avatar Joelly.W avatar Tom Christie avatar

Watchers

 avatar Carlos Valiente avatar Fabian Neumann avatar Mark avatar Thomas Widhalm avatar James Cloos avatar Keith Chambers avatar Daniel M. Capella avatar Alex avatar  avatar Jeremy Suntheimer avatar

safeyaml's Issues

Proposal: --fix-unquoted doing line at a time repair

Unquoted handling:

For Inside {}'s, or []'s only identifier like keys or values. There isn't a good way to handle unquoted values inside of these, because of examples like 'a: 1, b: 2'

For maps, we have to pick and choose between nesting and error handling.

  • Scan the line for ':' followed by a space or a newline.

  • Error out if line/word starts with YAML controls ``@%!"'

  • if the line ends in a :, and does not contain "'s or spaces, then it is a bare key, and the next line must be indented further in, and be the start of an indented list, or new indented map)

  • If the line has one ': ', then split it into key, value. Key must not have spaces.

  • If value starts with a {, [, ',", or -+0123456789, parse as normal

  • Or take entire line, trimming whitespace at end and front, as value

For indented lists we can do the same, handling a value in the same way as a map,

  • search for a {, [, ',", or -+0123456789 and parse as normal.
  • Or take entire line, trimming whitespace at end and front, as value

Note: Because of negative numbers, nesting lists complicate nesting

However, a - name: foo can work.

Colons in keys and string values

Almost every docker-compose.yml in the world has multiple lines like this:

image: postgres:9.3

A lot of Ruby projects contain YAML where strings begin with colons, which is abominable but a fact of life:

:foo: bar

I'm working on a GitLab CI file at the moment which has colons in the middle of keys:

foo:bar: "value"

All of these should be fixed as follows:

image: "postgres:9.3"
":foo": "bar"
"foo:bar": "value"

Flake8 issues

You possibly don't care about this, but for the record here is the output from pipenv check --style .:

safeyaml.py:10:1: W293 blank line contains whitespace
safeyaml.py:12:1: W293 blank line contains whitespace
safeyaml.py:14:1: E402 module level import not at top of file
safeyaml.py:14:1: F811 redefinition of unused 'sys' from line 5
safeyaml.py:21:80: E501 line too long (113 > 79 characters)
safeyaml.py:23:80: E501 line too long (113 > 79 characters)
safeyaml.py:26:80: E501 line too long (86 > 79 characters)
safeyaml.py:28:80: E501 line too long (102 > 79 characters)
safeyaml.py:46:32: E261 at least two spaces before inline comment
safeyaml.py:47:1: E302 expected 2 blank lines, found 0
safeyaml.py:52:22: E703 statement ends with a semicolon
safeyaml.py:58:1: E302 expected 2 blank lines, found 1
safeyaml.py:59:80: E501 line too long (125 > 79 characters)
safeyaml.py:66:1: E302 expected 2 blank lines, found 1
safeyaml.py:69:5: E301 expected 1 blank line, found 0
safeyaml.py:89:1: E302 expected 2 blank lines, found 1
safeyaml.py:92:1: E302 expected 2 blank lines, found 1
safeyaml.py:99:1: E302 expected 2 blank lines, found 1
safeyaml.py:102:1: E302 expected 2 blank lines, found 1
safeyaml.py:105:1: E302 expected 2 blank lines, found 1
safeyaml.py:108:1: E302 expected 2 blank lines, found 1
safeyaml.py:111:1: E302 expected 2 blank lines, found 1
safeyaml.py:114:1: E302 expected 2 blank lines, found 1
safeyaml.py:117:1: E302 expected 2 blank lines, found 1
safeyaml.py:120:1: E302 expected 2 blank lines, found 1
safeyaml.py:123:1: E302 expected 2 blank lines, found 1
safeyaml.py:126:1: E302 expected 2 blank lines, found 1
safeyaml.py:156:1: E302 expected 2 blank lines, found 1
safeyaml.py:163:19: E225 missing whitespace around operator
safeyaml.py:165:19: E225 missing whitespace around operator
safeyaml.py:171:23: E225 missing whitespace around operator
safeyaml.py:175:26: W291 trailing whitespace
safeyaml.py:180:1: E302 expected 2 blank lines, found 1
safeyaml.py:198:80: E501 line too long (105 > 79 characters)
safeyaml.py:204:80: E501 line too long (190 > 79 characters)
safeyaml.py:207:80: E501 line too long (117 > 79 characters)
safeyaml.py:208:1: W293 blank line contains whitespace
safeyaml.py:215:80: E501 line too long (80 > 79 characters)
safeyaml.py:224:80: E501 line too long (112 > 79 characters)
safeyaml.py:226:80: E501 line too long (126 > 79 characters)
safeyaml.py:228:80: E501 line too long (83 > 79 characters)
safeyaml.py:237:15: E225 missing whitespace around operator
safeyaml.py:238:38: E231 missing whitespace after ','
safeyaml.py:239:80: E501 line too long (145 > 79 characters)
safeyaml.py:243:80: E501 line too long (107 > 79 characters)
safeyaml.py:249:80: E501 line too long (83 > 79 characters)
safeyaml.py:259:1: W293 blank line contains whitespace
safeyaml.py:262:1: E302 expected 2 blank lines, found 1
safeyaml.py:272:35: E231 missing whitespace after ','
safeyaml.py:272:80: E501 line too long (109 > 79 characters)
safeyaml.py:276:80: E501 line too long (114 > 79 characters)
safeyaml.py:278:80: E501 line too long (219 > 79 characters)
safeyaml.py:281:15: E225 missing whitespace around operator
safeyaml.py:282:38: E231 missing whitespace after ','
safeyaml.py:286:80: E501 line too long (135 > 79 characters)
safeyaml.py:286:118: E231 missing whitespace after ','
safeyaml.py:289:1: W293 blank line contains whitespace
safeyaml.py:295:80: E501 line too long (132 > 79 characters)
safeyaml.py:302:30: E701 multiple statements on one line (colon)
safeyaml.py:303:80: E501 line too long (98 > 79 characters)
safeyaml.py:314:1: W293 blank line contains whitespace
safeyaml.py:324:80: E501 line too long (190 > 79 characters)
safeyaml.py:327:80: E501 line too long (117 > 79 characters)
safeyaml.py:328:1: W293 blank line contains whitespace
safeyaml.py:330:5: E303 too many blank lines (2)
safeyaml.py:354:1: W293 blank line contains whitespace
safeyaml.py:358:35: E231 missing whitespace after ','
safeyaml.py:358:80: E501 line too long (81 > 79 characters)
safeyaml.py:364:9: E266 too many leading '#' for block comment
safeyaml.py:370:26: E201 whitespace after '('
safeyaml.py:370:80: E501 line too long (117 > 79 characters)
safeyaml.py:372:50: E231 missing whitespace after ','
safeyaml.py:376:80: E501 line too long (134 > 79 characters)
safeyaml.py:376:117: E231 missing whitespace after ','
safeyaml.py:395:80: E501 line too long (103 > 79 characters)
safeyaml.py:395:91: E231 missing whitespace after ','
safeyaml.py:398:26: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
safeyaml.py:403:1: E302 expected 2 blank lines, found 1
safeyaml.py:413:38: E231 missing whitespace after ','
safeyaml.py:413:80: E501 line too long (112 > 79 characters)
safeyaml.py:418:43: E231 missing whitespace after ','
safeyaml.py:418:80: E501 line too long (151 > 79 characters)
safeyaml.py:429:1: E302 expected 2 blank lines, found 1
safeyaml.py:452:29: E201 whitespace after '('
safeyaml.py:452:80: E501 line too long (112 > 79 characters)
safeyaml.py:454:26: E712 comparison to False should be 'if cond is False:' or 'if not cond:'
safeyaml.py:461:1: E302 expected 2 blank lines, found 1
safeyaml.py:522:1: E302 expected 2 blank lines, found 1
safeyaml.py:558:80: E501 line too long (86 > 79 characters)
safeyaml.py:564:1: E302 expected 2 blank lines, found 1
safeyaml.py:579:39: E231 missing whitespace after ','
safeyaml.py:579:80: E501 line too long (182 > 79 characters)
safeyaml.py:581:80: E501 line too long (128 > 79 characters)
safeyaml.py:590:80: E501 line too long (171 > 79 characters)
safeyaml.py:595:80: E501 line too long (113 > 79 characters)
safeyaml.py:599:80: E501 line too long (123 > 79 characters)
safeyaml.py:601:80: E501 line too long (111 > 79 characters)
safeyaml.py:602:80: E501 line too long (104 > 79 characters)
safeyaml.py:603:80: E501 line too long (153 > 79 characters)
safeyaml.py:604:80: E501 line too long (131 > 79 characters)
safeyaml.py:605:80: E501 line too long (126 > 79 characters)
safeyaml.py:606:80: E501 line too long (112 > 79 characters)
safeyaml.py:607:80: E501 line too long (102 > 79 characters)
safeyaml.py:608:80: E501 line too long (103 > 79 characters)
safeyaml.py:609:80: E501 line too long (91 > 79 characters)
safeyaml.py:611:80: E501 line too long (105 > 79 characters)
safeyaml.py:613:31: E261 at least two spaces before inline comment
safeyaml.py:614:1: W293 blank line contains whitespace
safeyaml.py:641:80: E501 line too long (98 > 79 characters)
safeyaml.py:654:80: E501 line too long (97 > 79 characters)
safeyaml.py:658:42: E261 at least two spaces before inline comment
safeyaml.py:666:80: E501 line too long (90 > 79 characters)
safeyaml.py:677:1: W391 blank line at end of file
tests.py:14:36: E231 missing whitespace after ':'
tests.py:15:33: E231 missing whitespace after ':'
tests.py:16:31: E231 missing whitespace after ','
tests.py:16:33: E231 missing whitespace after ','
tests.py:17:31: E231 missing whitespace after ','
tests.py:17:33: E231 missing whitespace after ','
tests.py:18:33: E231 missing whitespace after ':'
tests.py:19:33: E231 missing whitespace after ':'
tests.py:73:13: E722 do not use bare except'

Nested Keys

- name: value
- name: value

Is very popular

Space after comma

In almost all languages, including human, there's a space after a comma and for a good reason as it improves readability.

Ansible playbooks

The conventions around ansible playbooks possibly mean they are never going to be safeyaml compliant. For example, they encourage starting files with the --- separator, and make extensive use of lists-of-maps.

CLI design

High-level thoughts, based on a quick skim over gofmt, pep8/flake8/autopep8, eslint and a few language interpreters:

  • gofmt fixes by default and prints to stdout. It has a -w option to fix in place, and a -d option to print a diff instead of the fixed output.
  • eslintcomplains by default and fixes with --fix.
  • pep8 and flake8 have no fixing functionality; autopep8 seems to be the canonical fixer. Not much help.

I'm leaning towards fixing as the default behaviour and checking as a flag (ruby and node have -c|--check).

Additionally, I think checking should by default print nothing to stdout.

Should we accept directories as arguments and recursively scan them for *.{yml,yaml} files? Every linter mentioned above does, so I think the answer is yes.

Can this be packaged as a python package?

Well, sure it can -- the real question is are you open to that approach? :)

The advantages I see include:

  • easier to install & update (even with pipsi)
  • works on windows

Happy to do the first bit if you're open to the idea.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.