Migrate parser to the new span combining scheme about rust HOT 2 OPEN

It may also be possible to introduce a smarter n-ary span combining operation to(a1, ..., aN) instead of the current binary to, but it may be more expensive and will make spans better only in very rare cases.
So combining using a chain of binary to operations should be fine for now.

from rust.

either an individual token

The "token" here is a "token tree" really, despite Rust parser working mostly with flattened token sequences at the moment.
So span(#[word]) should ideally be span(#) to span([word]) and not span(#) to span([) to span(word) to(span(]).

Also span combining for delimited groups can ignore all their internal tokens and only consider their delimiters.
Opening and closing delimiters always have the same context, and cannot be passed to macros separately in any way.
So span([word]) should be span([) to span(]) and not span([) to span(word) to span(]).

from rust.