Allow parsing of new Looker 7.4 filter syntax about lkml HOT 6 CLOSED

Hi! I ran into this problem myself and threw together a fix. I went to create a Github issue, then noticed this one already exists.

Here's the complete workaround I'm currently using:

import lkml.tokens as tokens
from lkml.lexer import Lexer
from lkml.parser import DELIMITER, Parser as BaseParser
from typing import Optional, Union

def _pair_to_field_value(pair) -> dict:
    key, value = next(iter(pair.items()))
    return {
        "field": key,
        "value": value
    }

class Parser(BaseParser):
    """ Patches the LKML parser to support key/value pairs inside lists, e.g.

        filters: [key: "value", key2: "value2"]
    """

    @BaseParser.backtrack_if_none
    def parse_csv(self) -> Optional[list]:
        if self.log_debug:
            grammar = '[csv] = (literal / quoted_literal) ("," (literal / quoted_literal))* ","?'
            self.logger.debug("%sTry to parse %s", self.depth * DELIMITER, grammar)
        values = []

        pair = self.parse_pair()  # Fix is here: try to parse the value as a pair
        if pair: 
            values.append(_pair_to_field_value(pair))
        elif self.check(tokens.LiteralToken, tokens.QuotedLiteralToken):
            values.append(self.consume_token_value())
        else:
            return None
        

        while not self.check(tokens.ListEndToken):
            if self.check(tokens.CommaToken):
                self.advance()
            else:
                return None

            pair = self.parse_pair()  # Same fix reapplied here
            if pair:
                values.append(_pair_to_field_value(pair))
            elif self.check(tokens.LiteralToken, tokens.QuotedLiteralToken):
                values.append(self.consume_token_value())
            elif self.check(tokens.ListEndToken):
                break
            else:
                return None

        if self.log_debug:
            self.logger.debug(
                "%sSuccessfully parsed comma-separated values.", self.depth * DELIMITER
            )
        return values

Happy to turn this into a PR if you like the look of it.

Edit:

I'll note though, I'm not sure if this solution is completely correct. For example:

1. It assumes that all instances of [foo: "bar"] are for filters, and so hardcodes the transformation to [{"field": "foo", "value": "bar"}]
2. It assumes no one will mix both syntaxes in a single block.

(I don't actually need the filters, I just need the files to parse successfully, so those issues - if they are issues - aren't a problem for my use-case)

from lkml.

I'm having the same problem with this, as the new syntax broke our tests that use the lkml package...

My solution for now was to ask people to use the old syntax. I agree with @joshtemple that this new syntax looks VERY weird. I've commented on it here https://discourse.looker.com/t/looker-7-4-release-notes/16767

from lkml.

FYI, I've just added support to my node parser (v 6.3)

fabio-looker/node-lookml-parser#8

I also personally would have liked a more traditional LookML syntax here. (Unfortunately I don't have direct visibility into the reasons for this specific syntax)

from lkml.

Thanks for bringing this to my attention Tom! This requires a fairly significant change to the parser to support this syntax since the current parser grammar assumes anything in square brackets is a comma separated list. Hopefully I'll have some time to tackle this in the near future.

from lkml.

No problem! I forked your repo to give it a go myself, then realised quite quickly how hard it would be so I gave up 😅- good luck!

from lkml.

@tpakeman, actually curious to know why this syntax was chosen instead of {key: value} with braces. Everything else in Looker is key value pairs with braces, except for the set-type syntax with brackets. I don't see a reason why an array of key value pairs makes sense within LookML. Do you have any insight or know anyone at Looker who does?

from lkml.