Excellent effort, would be a great replacement for our inhouse python based system. Ho

This issue has been migrated to <a class="issue-link js-issue-link" data-error-text="F

Request for supporting documentation of fields as first class citizens about cue HOT 6 CLOSED

cuelang commented on September 23, 2024 1

Request for supporting documentation of fields as first class citizens

from cue.

Comments (6)

mpvl commented on September 23, 2024

The parser considers comments as first-class citizens, exactly for the purpose you describe, and they are accessible through the ast API.

This is a bit low level, so maybe you referring to the being able to access doc comments in the higher level API as well?

The only tricky bit is what to do if multiple fields are documented. Go does the same thing for package documentation, and we could probably copy that strategy.

The trickier part is individual fields. In CUE, a field can be defined many times, each time with different documentation. So the question is which one to pick. My best guess is to provide a slice with all unique comment groups, starting with the most high-level declarations (imported, then merged from parent dirs, then templates over individual fields).

Let me know if the ast approach suffices, or whether having doc comments accessible through cuelang.org/go/cue.

from cue.

pavlos-christoforou commented on September 23, 2024

Thanks Marcel for the prompt response much appreciated. Yes, we would like to access the doc comments from a higher level API. Nevertheless we have not yet decided to switch to cue , mentioning it so you gauge priorities etc. I would not want to sidetrack other priorities with a feature request that is not important for current users. I think for individual field docs a possible approach would be to follow the same logic cue follows for multiple values for consistency. Perhaps an example of how we currently define data models may be helpful to you (it's in python but I hope is easy to follow): class LegalEntityFormat(Spec): """ which legal entities and currency pairs to include in the calculation. """ version = 1 fields = ( Field('FundLegalEntity', S, "The fund legal entity to include in the calculation", mandatory=True), Field('CalculationCurrency', S, "The target currency for the calculation. For feeder type funds this is the share class currency. For master or standalone type funds this is usually the non base currency involved in the calculation", domain=CURRENCY_DOMAIN_SET, mandatory=True), Field('RiskCurrency', S, "The risk currency the particular share class, master fund or standalone fund is exposed to", domain=CURRENCY_DOMAIN_SET, mandatory=True), Field('TradeCurrency', S, "The trade currency on which we will peg the hedging trade", domain=CURRENCY_DOMAIN_SET, mandatory=True), Field('IncludeHedgePL', I, "If set to 1/true then include anyhedge PL in hedging calculations.", mandatory=True), Field('IncludeSubscriptions', I, "If set to 1/true then include subscriptions in hedging calculations.", mandatory=True), Field('IncludeRedemptions', I, "If set to 1/true then include redemptions in the calculation.", mandatory=True), Field('CashBufferAmount', F, "Cash buffer amount. Set to zero if not applicable.", ), Field('FundBaseCurrency', S, "The fund base currency", domain=CURRENCY_DOMAIN_SET, mandatory=True), Field('FundBaseCurrencySettlementAccount', S, "The settlement account associated with the fund base currency", ), Field('RiskCurrencySettlementAccount', S, "The settlement account associated with the risk currency", ), Field('ClientLegalEntity', S, "The client legal entity if available", ), Field('FundType', S, "The fund type if available", domain=FUND_TYPE_DOMAIN, ), Field("RedemptionBufferRatio", F, "Redemption buffer ratio to apply for cash holdback"), Field("MishedgeThreshold", F, "Mishedge threshold before hedging is adviced"), Field("HedgeTargetRatio", F, "The ratio of the total assets that must be kept hedged at all times"), Field("ExecutionMethod", S, "Trade execution method", domain=EXECUTION_METHOD_DOMAIN), Field("ExecutionFeeBasisPoints", F, "The execution fee in basis points to be levied on the transaction"), ) Where s= String, F=Float etc and the base class 'spec' provides utilities and not relevant for the data definition part. What we find very useful is that we use such definitions to auto generate csv parsers, SQL schemas with field docs, example data sets in csv with each field documented and domains./constrains listed as well as autogenerated validation functions for the most basic checks (mandary fields, type compatibility, domain restrictions). Now perhaps CUE is not meant for such use cases and we misunderstood the intention but certainly we really like some of the features included and its relative simplicity so we thought to ask. Thank you and your team for all the effort and for making cue available. Warm Regards

…

On Thu, 18 Apr 2019 at 12:48, Marcel van Lohuizen ***@***.***> wrote: The parser considers comments as first-class citizens, exactly for the purpose you describe, and they are accessible through the ast API. This is a bit low level, so maybe you referring to the being able to access doc comments in the higher level API as well? The only tricky bit is what to do if multiple fields are documented. Go does the same thing for package documentation, and we could probably copy that strategy. The trickier part is individual fields. In CUE, a field can be defined many times, each time with different documentation. So the question is which one to pick. My best guess is to provide a slice with all unique comment groups, starting with the most high-level declarations (imported, then merged from parent dirs, then templates over individual fields). Let me know if the ast approach suffices, or whether having doc comments accessible through cuelang.org/go/cue. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEUFFL3CQK36WRBTHR652DPRA7WLANCNFSM4HG3EJXQ> .

-- Pavlos Christoforou Point Nine Limited Mobile: +357 99 160960 [email protected]

from cue.

mpvl commented on September 23, 2024

Early on, the first two large potential customers indicated they wanted it to use for code generation of exactly the ilk you describe here. So this is certainly a main focus. In fact, the recent addition of attributes to the language was to accommodate this use case. It allows adding meta information to influence generators for particular languages.

Anyway, it is good to learn of more use cases. And yes, exposing docs in the high-level API was on my todo list.

I think for individual field docs a possible approach would be to follow
the same logic cue follows for multiple values for consistency.

In the example you gave, there won't be much ambiguity and I think my described approach works well (the first comment will be pretty much always the one you need). But I'm open to other suggestions. I'm not sure if you were suggesting something different here.

What we find very useful is that we use such definitions to auto generate
csv parsers, SQL schemas with field docs, example data sets in csv with
each field documented and domains./constrains listed as well as
autogenerated validation functions for the most basic checks (mandary
fields, type compatibility, domain restrictions).

Yes, that is the idea. There are currently some CLs going in mostly the opposite direction (e.g. generating CUE templates from Kubernetes Go code, which treats Go as the source of truth), but code generation in the opposite direction is certainly part of the plan. The idea is that it would not only allow data types but also detailed validation to be shared across languages.

So in CUE, the above first few fields would look something like:

// which legal entities and currency pairs to include in the calculation.
LegalEntityFormat: Spec & {
    version: 1

    // The fund legal entity to include in the calculation
    FundLegalEntity: string

    // currency aliases a regexp constraint of a currency type.
    // TODO: define in other package.
    currency = =~ "^[A-Z0-9]{3}$"

    // The target currency for the calculation. For feeder type
    // funds this is the share class currency. For master or standalone type funds
    // this is usually the non base currency involved in the calculation
    CalculationCurrency: currency  @domain("CURRENCY_DOMAIN_SET")

    // The risk currency the particular share class, master fund or
    // standalone fund is exposed to
    RiskCurrency: currency  @domain("CURRENCY_DOMAIN_SET")

    // The trade currency on which we will peg the hedging trade
    TradeCurrency: currency  @domain("CURRENCY_DOMAIN_SET")

    // The settlement account associated with the fund base currency
    FundBaseCurrencySettlementAccount?: string

    ...
}

Note the question mark to distinguish between optional and mandatory fields. I didn't know what domain meant, so I added an attribute as meta info. For giggles, I also restricted the currency types to be a three-letter ISO currency code.

from cue.

pavlos-christoforou commented on September 23, 2024

All clear and thank you for the example Marcel, (the docs were very clear and useful btw so we think we understand the various operators/functionality involved). We will consider CUE further, and if any issues arise we will revert with more feedback/use cases. Warm Regards

…

On Thu, 18 Apr 2019 at 14:03, Marcel van Lohuizen ***@***.***> wrote: Early on, the first two large potential customers indicated they wanted it to use for code generation of exactly the ilk you describe here. So this is certainly a main focus. In fact, the recent addition of attributes to the language was to accommodate this use case. It allows adding meta information to influence generators for particular languages. Anyway, it is good to learn of more use cases. And yes, exposing docs in the high-level API was on my todo list. I think for individual field docs a possible approach would be to follow the same logic cue follows for multiple values for consistency. In the example you gave, there won't be much ambiguity and I think my described approach works well (the first comment will be pretty much always the one you need). But I'm open to other suggestions. I'm not sure if you were suggesting something different here. What we find very useful is that we use such definitions to auto generate csv parsers, SQL schemas with field docs, example data sets in csv with each field documented and domains./constrains listed as well as autogenerated validation functions for the most basic checks (mandary fields, type compatibility, domain restrictions). Yes, that is the idea. There are currently some CLs going in mostly the opposite direction (e.g. generating CUE templates from Kubernetes Go code, which treats Go as the source of truth), but code generation in the opposite direction is certainly part of the plan. The idea is that it would not only allow data types but also detailed validation to be shared across languages. So in CUE, the above first few fields would look something like: // which legal entities and currency pairs to include in the calculation. LegalEntityFormat: Spec & { version: 1 // The fund legal entity to include in the calculation FundLegalEntity: string // currency aliases a regexp constraint of a currency type. // TODO: define in other package. currency = =~ "^[A-Z0-9]{3}$" // The target currency for the calculation. For feeder type // funds this is the share class currency. For master or standalone type funds // this is usually the non base currency involved in the calculation CalculationCurrency: currency @Domain("CURRENCY_DOMAIN_SET") // The risk currency the particular share class, master fund or // standalone fund is exposed to RiskCurrency: currency @Domain("CURRENCY_DOMAIN_SET") // The trade currency on which we will peg the hedging trade TradeCurrency: currency @Domain("CURRENCY_DOMAIN_SET") // The settlement account associated with the fund base currency FundBaseCurrencySettlementAccount?: string ... } Note the question mark to distinguish between optional and mandatory fields. I didn't know what domain meant, so I added an attribute as meta info. For giggles, I also restricted the currency types to be a three-letter ISO currency code. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEUFFJCENLULU37DFVOXW3PRBIPTANCNFSM4HG3EJXQ> .

-- Pavlos Christoforou Point Nine Limited Mobile: +357 99 160960 [email protected]

from cue.

pavlos-christoforou commented on September 23, 2024

Thank you

…

-- Pavlos Christoforou Partner Point Nine Limited +357 25 02 82 41 www.p9ft.com 135 Arch. Makarios III Ave. Emelle Building 3021, Limassol Cyprus

On Fri, 17 May 2019, 00:13 Marcel van Lohuizen, ***@***.***> wrote: Closed #36 <#36> via 079451a <079451a> . — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#36?email_source=notifications&email_token=AAEUFFIPOWNVTTKTWEJXEVDPVXFBFA5CNFSM4HG3EJX2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGORPXVSOA#event-2347718968>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEUFFMTNWG2PXBN33WCMFTPVXFBFANCNFSM4HG3EJXQ> .

from cue.

cueckoo commented on September 23, 2024

This issue has been migrated to cue-lang/cue#36.

For more details about CUE's migration to a new home, please see cue-lang/cue#1078.

from cue.

Request for supporting documentation of fields as first class citizens about cue HOT 6 CLOSED

Comments (6)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent