bugs in scenarios of Chinese characters or incorrect using of match.Index about regexp2 HOT 2 CLOSED

Thanks for finding this!

This is because internally the matching engine always uses a rune slice and match.Index is the index into that rune slice, which for anything after a multi-byte rune will not be the same as the index into the original string.

If the user passes in a rune slice (e.g. FindRunesMatch) then indexing from the runes makes sense, but if the user passes in a string (e.g. FindStringMatch) then this is confusing. I think I should make a change to sort out this confusion and make the API easier to use with only strings.

I believe the best backward-compatible fix would be to keep match.Index as the index into the rune slice and add a match.StringIndex as the index of the original string. I'd also clean up the documentation associated with this to make it more clear what the Index field actually is.

The Replace function also could use a ReplaceRunes version for compatibility with the FindRunesMatch and if someone needs to call it repeatedly they could reduce the number of string to []rune conversions.

For now, to work around this issue you'll need to map the runes to the string index yourself:

func getRunesAndMap(in string) ([]rune, map[int]int) {
	ret := make([]rune, len(in))
	idxMap := make(map[int]int)

	i := 0
	for strIdx, r := range in {
		idxMap[i] = strIdx
		ret[i] = r
		i++
	}
	return ret[:i], idxMap
}

func TestReplaceMultiByte(t *testing.T) {
	var sample = "<title>错<style"
	var runes, idxMap = getRunesAndMap(sample)

	regex := MustCompile("<style", IgnoreCase|Singleline)
	match, err := regex.FindRunesMatch(runes)
	if err != nil {
		panic(err)
	}
	if match != nil {
		t, err := regex.Replace(sample, "xxx", idxMap[match.Index], -1)
		if err != nil {
			panic(err)
		}
		fmt.Printf("%s", t)
	}
}

from regexp2.

I find another solution.
I think I can do some extra initialization when MatchEvaluator of ReplaceFunc is called first time, in this way I can do replacements from -1

from regexp2.