Allow '$' in grammar productions to indicate a vendor extension about cxx-abi HOT 18 CLOSED

We're wondering about the possibility of reserving a character such as '$' to indicate the start of a vendor extension in any production except identifier.

Would it cause problems for your use case if other vendors used the same extension character for a different purpose, with incompatible encodings?

from cxx-abi.

Would it cause problems for your use case if other vendors used the same extension character for a different purpose, with incompatible encodings?

Probably not, or at least, we would likely be in a better position regardless since the chances of a conflict would presumably be lower unless use of private extensions became very common. Unless we encountered a direct conflict, we would just get names that we don't know how to decode and we already run into that today because our decoder hasn't been kept up to date with changes to the Itanium ABI.

We currently only consume mangled names generated by EDG and Clang, so what implementers other than those two do doesn't affect us.

An alternative of course would be to specify manglings for at least some of the entities that we've added extensions for. I don't know if other implementers would have any use for them though. I can imagine that the Clang static analyzer might be able to make use of them.

from cxx-abi.

I think the Clang static analyzer doesn't currently use anything like that because it doesn't try to do cross-file analysis, but you're right that it might need to if it did. Clang does have an existing scheme that's supposed to stably identify declarations across translation units, but it's not mangling-based. Anyway, I've pointed Anna Zaks at this thread just in case she's interested in contributing.

Are you trying to encode additional "kind" information beyond just the name of the declaration? Because most of the things you listed already do have manglings, and even the things that formally don't (e.g. enumerators) have pretty obvious encodings (a nested-name). But the Itanium mangling relies on language rules and linkage to prevent collisions, which isn't necessarily good enough for what I expect a tool like Coverity to want, which is why I ask whether you're trying to encode additional information.

Broadly speaking, I would say that the ABI is committed to using only conventional identifier characters in the core "structural" mangling, so I think you have the guarantee you need as long as you're not assuming that '$' can't appear within a source-name.

from cxx-abi.

Are you trying to encode additional "kind" information beyond just the name of the declaration?

So far, no, but see later. We (seem to) handle "kind" conflicts ok. For example, we generate the same mangled name (_ZN1N1cE) for both class c and variable c for this code:

namespace N {
  class c {} c;
}

Because most of the things you listed already do have manglings,

They do, but not technically from the top level, right? <encoding> is either a <function name>, <data name>, or <special-name>. Generating names for class and enumeration types under <encoding> seems outside the spec, but "works" (We had to relax an assert in Clang to coerce it into giving us names for non-function non-variable declarations, but it was then (mostly) happy to oblige. I'm not sure how much coercion was necessary for EDG to do likewise).

and even the things that formally don't (e.g. enumerators) have pretty obvious encodings (a nested-name). But the Itanium mangling relies on language rules and linkage to prevent collisions, which isn't necessarily good enough for what I expect a tool like Coverity to want, which is why I ask whether you're trying to encode additional information.

The reason I'm raising this request now is because we now have a need to uniquely encode names for templates (including templated entities). In the past, we only needed names for template specializations, so the ABI spec sufficed in this regard.

Our current plans do require avoidance of conflicts for class names vs class template names vs class template partial specialization names, for function names vs function template names, and for variable names vs variable template names. E.g. we don't plan to have separate namespaces for templates vs the kind of entity being templated. Class/variable and class/variable templates will never conflict anyway, but function and function templates can.

For class templates, we need to ensure the following get distinct names:

template<typename T, typename Enable = void>
class CT {};                               // #1
template<typename T, int I>
class CT<T, std::enable_if_t<I < 10>> {};  // #2
template<typename T, int I>
class CT<T, std::enable_if_t<I >= 10>> {}; // #3

And for function templates, we need distinct names for:

template<typename T>
void f(); // #1
void f(); // #2

For reference (or feedback), here are the extensions I've proposed for our internal use. Additions are indicated by the leading '+'.

  <mangled-name>
    ::= _Z <encoding>

  <encoding>
    ::= <function name> <bare-function-type>
    ::= <data name>
+   ::= <template name>
    ::= <special-name>

  <name>
+   ::= <template-name>
    ...

  <prefix>
+   ::= <template-name>

+ <template-name>
+       # Class template
+   ::= $Tc <template-introducer> <source-name>
+       # Class partial specialization
+   ::= $Ts <template-introducer> <source-name> <template-args>
+       # Function template
+   ::= $Tf <template-introducer> <source-name> <bare-function-type>
+       # Variable template
+   ::= $Tv <template-introducer> <source-name>
+       # Alias template
+   ::= $Ta <template-introducer> <source-name>

+ <template-introducer>
+   ::= I <template-parameter-kind>* E

+ <template-parameter-kind>
+   ::= t                       # type template parameter.
+   ::= n <type>                # non-type template parameter.
+   ::= T <template-introducer> # template template parameter.

I haven't explored impact to substitutions in the above.

from cxx-abi.

Additional examples of function templates for which, I think, there is no unique obvious mangling with the current specification.

template<typename T> void f();
template<int N> void f();
template<template<typename> class TT> void f();

from cxx-abi.

I certainly agree that many things are not sufficiently distinguished by mangling.

Of these, I could most easily imagine an eventual language need to mangle function or variable templates as a template argument, although I think the language proposals in this area have leaned towards passing an entire overload set instead. It's harder to imagine a language need to mangle class template partial specializations — maybe if there were some way of "untemplating" part of a template pattern, so that it was shared across all instantiations? But this is a very speculative line of inquiry.

I think the most reasonable thing to do is just to continue to avoid specifying a mangling for these things; we'll extend you the promise that we won't use $ in a principal position in the mangling grammar, and you can embed whatever you need to embed there.

from cxx-abi.

Of these, I could most easily imagine an eventual language need to mangle function or variable templates as a template argument, although I think the language proposals in this area have leaned towards passing an entire overload set instead. It's harder to imagine a language need to mangle class template partial specializations — maybe if there were some way of "untemplating" part of a template pattern, so that it was shared across all instantiations? But this is a very speculative line of inquiry.

Agreed. The other (very speculative) potential I see is related to the Modules proposal; that there may be utility in storing a pre-compiled module in an ELF container.

I think the most reasonable thing to do is just to continue to avoid specifying a mangling for these things; we'll extend you the promise that we won't use $ in a principal position in the mangling grammar, and you can embed whatever you need to embed there.

Great, exactly what I asked for :)

Would you like me to draft some updates to abi.html? I expect a sentence or two in section 5.1.1 would suffice.

from cxx-abi.

Yes, that would be great, thanks.

from cxx-abi.

How does this look?

--- abi.html.orig       2017-02-28 17:56:18.192311600 -0500
+++ abi.html    2017-02-28 17:56:09.929350400 -0500
@@ -3980,6 +3980,10 @@
 or <code>Type?</code> for an unknown data type.

 </p><p>
+Grammar productions beginning with '$' are reserved for private implementation
+use.  Names produced using such extensions are inherently non-portable.
+
+</p><p>
 <a name="mangling-structure">
 </a></p><h4><a href="#mangling-structure"> 5.1.2 General Structure </a></h4>

from cxx-abi.

Looks good to me. Mind making a PR?

from cxx-abi.

Looks good to me. Mind making a PR?

Done!

from cxx-abi.

Thanks! Merged and closing.

from cxx-abi.

Thanks for working with us on this. Can you tell me when I should expect to see the updates show up at http://mentorembedded.github.io/cxx-abi/abi.html#mangling ?

from cxx-abi.

Hmm. It's already present in http://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling. It seems that the github.io thing didn't follow the repository.

from cxx-abi.

I'll follow up with GitHub.

from cxx-abi.

I'll follow up with GitHub.

Any luck with GitHub? The updates still aren't reflected at http://mentorembedded.github.io/cxx-abi/abi.html#mangling. Should the mentorembedded URL continue to remain the canonical location for the specification?

from cxx-abi.

The mentorembedded page is completely stale and not receiving any updates, and GitHub has no immediate intention to fix that. I'm talking with Mentor to see if they're willing to install a redirect there, just because there are so many existing links. If they aren't, I'll just ask GitHub to delete the mentorembedded page.

The canonical URL is http://itanium-cxx-abi.github.io/cxx-abi/abi.html and has been since we switched repository owners.

from cxx-abi.

The canonical URL is http://itanium-cxx-abi.github.io/cxx-abi/abi.html and has been since we switched repository owners.

Ok, great. I'll ignore the mentorembedded location then. I submitted a PR to update a few URLs that were still pointing there.

from cxx-abi.