Comments (2)
Thanks for finding this!
This is because internally the matching engine always uses a rune slice and match.Index
is the index into that rune slice, which for anything after a multi-byte rune will not be the same as the index into the original string.
If the user passes in a rune slice (e.g. FindRunesMatch
) then indexing from the runes makes sense, but if the user passes in a string (e.g. FindStringMatch
) then this is confusing. I think I should make a change to sort out this confusion and make the API easier to use with only strings.
I believe the best backward-compatible fix would be to keep match.Index
as the index into the rune slice and add a match.StringIndex
as the index of the original string. I'd also clean up the documentation associated with this to make it more clear what the Index
field actually is.
The Replace
function also could use a ReplaceRunes
version for compatibility with the FindRunesMatch
and if someone needs to call it repeatedly they could reduce the number of string
to []rune
conversions.
For now, to work around this issue you'll need to map the runes to the string index yourself:
func getRunesAndMap(in string) ([]rune, map[int]int) {
ret := make([]rune, len(in))
idxMap := make(map[int]int)
i := 0
for strIdx, r := range in {
idxMap[i] = strIdx
ret[i] = r
i++
}
return ret[:i], idxMap
}
func TestReplaceMultiByte(t *testing.T) {
var sample = "<title>错<style"
var runes, idxMap = getRunesAndMap(sample)
regex := MustCompile("<style", IgnoreCase|Singleline)
match, err := regex.FindRunesMatch(runes)
if err != nil {
panic(err)
}
if match != nil {
t, err := regex.Replace(sample, "xxx", idxMap[match.Index], -1)
if err != nil {
panic(err)
}
fmt.Printf("%s", t)
}
}
from regexp2.
I find another solution.
I think I can do some extra initialization when MatchEvaluator
of ReplaceFunc
is called first time, in this way I can do replacements from -1
from regexp2.
Related Issues (20)
- Why the replacement interface for []rune is not supported HOT 1
- Force timeout for testing? HOT 3
- Leaking go routines using `fastclock` HOT 6
- The matching results of strings containing Chinese characters are incorrect HOT 2
- No support for \p{unicode char class} that is supported by the Go stdlib regex package HOT 2
- No support for full unicode that is supported by the ECMAScript regex HOT 1
- xeger functionality HOT 1
- Question: Does this library support "categories" HOT 1
- Match loop cause high CPU usage HOT 2
- TestDeadline fails with go 1.21.3 HOT 4
- "cannot use []*syntax.regexNode as type []*struct" when using regexp2 inside traefik plugin HOT 1
- Line Terminator (Dollar sign) does not match as expected HOT 1
- Unable to see matches for a positive lookahead regex HOT 1
- MatchString() timeout stuck HOT 1
- FR: support Marshal/Unmarshal HOT 3
- FindStringMatch returns wrong index when using unicode characters HOT 2
- .Replace() wrong result with symbols $& in new content HOT 1
- Regex compilation breaks on certain grouping HOT 2
- Support dotAll flag HOT 9
- Is there any workaround for `split`? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from regexp2.