r3labs / diff Goto Github PK
View Code? Open in Web Editor NEWA library for diffing golang structures
License: Mozilla Public License 2.0
A library for diffing golang structures
License: Mozilla Public License 2.0
How do I force differ function so it would compare net.IP as a string and not as a slice?
https://go.dev/play/p/0VMlM5RQ9WO
package main
import (
"log"
"net"
"github.com/davecgh/go-spew/spew"
"github.com/r3labs/diff/v2"
)
type LoadBalancer struct {
IP []net.IP
}
func main() {
x := LoadBalancer{
IP: []net.IP{
net.ParseIP("192.0.2.1"),
net.ParseIP("192.0.2.2"),
},
}
y := LoadBalancer{
IP: []net.IP{
net.ParseIP("192.0.2.1"),
net.ParseIP("192.0.2.3"),
},
}
changelog, err := diff.Diff(x, y)
if err != nil {
log.Fatal(err)
}
spew.Dump(changelog)
}
(diff.Changelog) (len=1 cap=1) {
(diff.Change) {
Type: (string) (len=6) "update",
Path: ([]string) (len=3 cap=3) {
(string) (len=2) "IP",
(string) (len=1) "1",
(string) (len=2) "15"
},
From: (uint8) 2,
To: (uint8) 3,
parent: (interface {}) <nil>
}
}
Hello!
I'm diffing some large nested structures and running into really long run times with v3.
Interestingly v1 of this package returns in mere seconds with the correct changelog result.
However, I'm really looking for patching support that comes with v3. I've tried using options like 'FlattenEmbeddedStructs', 'DiscardComplexOrigin', and 'DisableStructValues' individually but nothing gets me to the performance that comes with v1
Is there an option or set of options in v3 to align performance with v1? What could be taking so much more time in v3?
Thanks for the help
Line 66 in 9e5baa4
In this^ fuction, a new Differ nd
is used to deal with each of the fields. This works okay, except when the user of the package defines a filter function.
One workaround is to do this:
func (d *Differ) structValues(t string, path []string, a reflect.Value) error {
var nd Differ
nd.Filter = d.Filter
// ...
for i := 0; i < a.NumField(); i++ {
// ...
if nd.Filter != nil && !nd.Filter(fpath, a.Type(), field) {
continue
}
err := nd.diff(fpath, xf, af, a.Interface())
if err != nil {
return err
}
}
// ...
}
=== RUN TestDiff/mixed-slice-map
diff_test.go:645:
Error Trace: diff_test.go:645
Error: Not equal:
expected: []string{"1", "\xa4name"}
actual : []string{"1", "\xa4type"}
Diff:
--- Expected
+++ Actual
@@ -2,3 +2,3 @@
(string) (len=1) "1",
- (string) (len=5) "\xa4name"
+ (string) (len=5) "\xa4type"
}
Test: TestDiff/mixed-slice-map
diff_test.go:647:
Error Trace: diff_test.go:647
Error: Not equal:
expected: string("name2")
actual : []string([]string{"null", "string"})
Test: TestDiff/mixed-slice-map
diff_test.go:645:
Error Trace: diff_test.go:645
Error: Not equal:
expected: []string{"1", "\xa4type"}
actual : []string{"1", "\xa4name"}
Diff:
--- Expected
+++ Actual
@@ -2,3 +2,3 @@
(string) (len=1) "1",
- (string) (len=5) "\xa4type"
+ (string) (len=5) "\xa4name"
}
Test: TestDiff/mixed-slice-map
diff_test.go:647:
Error Trace: diff_test.go:647
Error: Not equal:
expected: []string([]string{"null", "string"})
actual : string("name2")
Test: TestDiff/mixed-slice-map
It can be reproduced by run go test -count=10
github.com/vmihailenco/msgpack v4.0.4+incompatible
is an EOL .
Do you know if we can use the the latest version of this package?
Here is the list of policies vmihailenco/msgpack v4.0.4 violates.
Version/Branch EOL
Severity:Major
Category:Security
Scan Modes:Full
Description
Version or branch requested is EOL. Please update to a supported version.
Conditions
Component Version Approval StatusEQUALS Deprecated
Component UsageNOT EQUAL TO Dev. Tool / Excluded
Hi, In the following I am unable to patch the struct. To me it seems that the diff holds a string and that the reflection machinery does not consider the target pointer. I am missing something here? Thanks:)
type Test struct {
S *string `json:"s,omitempty"`
}
func TestPointerDiff(t *testing.T) {
str1 := "before"
t1 := Test{S: &str1}
str2 := "after"
t2 := Test{S: &str2}
changelog, err := diff.Diff(t1, t2)
assert.NoError(t, err)
patchLog := diff.Patch(changelog, t1)
assert.False(t, patchLog.HasErrors())
}
Hi, this might not be a bug, but an issue I´ve ran into and need to work around. Consider the following
type Test struct {
S *int `json:"s,omitempty"`
}
func TestPointerDiff(t *testing.T) {
val1 := 1
val2 := 2
t1 := Test{S: &val1}
t2 := Test{S: &val2}
changelog, err := diff.Diff(t1, t2)
assert.NoError(t, err)
js, err := json.Marshal(changelog)
assert.NoError(t, err)
assert.NoError(t, json.Unmarshal(js, &changelog))
patchLog := diff.Patch(changelog, &t1)
assert.False(t, patchLog.HasErrors())
}
This will fail, since the type of the int is lost when marshaling to JSON.
Any ideas how I could work around this?
At the moment, it seems nil/uninitialized slices are == initialized empty list slices.
type testStruct struct {
A []string
}
func Test_DiffNilList(t *testing.T) {
p := testStruct{}
assert.Nil(t, p.A)
n := testStruct{
A: []string{},
}
assert.NotNil(t, n.A)
res, err := diff.Diff(p, n)
if !assert.NoError(t, err) {
return
}
assert.Equal(t, 1, len(res))
}
reproduction:
package main
import (
"log"
"time"
"github.com/r3labs/diff"
)
type Order struct {
ID string `diff:"id"`
Time time.Time `diff:"time"`
}
func main() {
a := Order{
ID: "1234",
Time: time.Now(),
}
b := Order{
ID: "1234",
Time: time.Now(),
}
changelog, err := diff.Diff(a, b)
log.Println(changelog, err)
}
Expected result: something returned
Actual result: panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
go version go1.11.4 linux/amd64
Not sure if this is expected, but if so, you could consider this more of a question.
As the title says, when a map has an additional entry (entries are custom struct types), then instead of getting 1 Change
when comparing the maps, I get N changes (where N=number of fields in the structs).
I have the following types:
type Product struct {
ID int `diff:"id,identifier"`
Title string
Description string
BaseMaterials map[MaterialID]bool
ExtraMaterials []MaterialID
Sizes []SizeID
Weight int16
Prices map[SizeID]ProductPrice // also a struct type
Type TypeID
}
type Catalog struct {
Products map[int]Product `diff:"products"`
}
And I'm trying to compare two catalogs, of which the one has one key missing from its Catalog.Products
map. The following example demonstrates:
// assume we have catalog1 & catalog2, which are identical
// remove a single entry (Product) from catalog2.Products
delete(catalog2.Products, 2)
// and compare
changelog, _ := diff.Diff(catalog1, catalog2)
At this point, the changelog contains N create
diffs, where N is the number of fields of the Product. All diffs are of type "from nil to something", since in catalog2 the product is not present at all.
What I actually need, is to get a single, compound diff with the new entry. Is that possible given the current implementation, or do I have to iterate over the changelog and manually inspect the diffs to combine them to a single one?
Thanks in advance.
When diffing a map with pointer values, e.g.:
struct1 := tmstruct{Bar: 1, Foo: "one"}
struct2 := tmstruct{Bar: 2, Foo: "two"}
map1 := map[string]*tmstruct{
"one": &struct1,
}
map2 := map[string]*tmstruct{
"one": &struct1,
"two": &struct2,
}
...this comes back as ErrTypeMismatch due to comparing nil to a pointer.
I was able to fix this for my use case by modifying diff_pointer.go
thusly:
func (cl *Changelog) diffPtr(path []string, a, b reflect.Value) error {
if a.Kind() == reflect.Invalid {
cl.add(CREATE, path, nil, b.Interface())
return nil
}
if b.Kind() == reflect.Invalid {
cl.add(DELETE, path, a.Interface(), nil)
return nil
}
[...]
However, this change breaks the tests mismatched-values-struct-nil and mismatched-values-nil-struct.
Do you have thoughts on how this conflict could be resolved?
For testing my change, I added these cases to the case list in TestDiff:
{
"map-string-pointer-create-test",
map[string]*tmstruct{"one": &struct1},
map[string]*tmstruct{"one": &struct1, "two": &struct2},
Changelog{
Change{Type: CREATE, Path: []string{"two"}, From: nil, To: &struct2},
},
nil,
},
{
"map-string-pointer-delete-test",
map[string]*tmstruct{"one": &struct1, "two": &struct2},
map[string]*tmstruct{"one": &struct1},
Changelog{
Change{Type: DELETE, Path: []string{"two"}, From: &struct2, To: nil},
},
nil,
},
I couldn't find anything in the documentation but it would be really useful to exclude specific fields or paths in the struct from output change log.
I'm having issues when I have a struct that contains a pointer to an array of string. An example of this problem
package main
import (
"github.com/r3labs/diff/v2"
)
type MyStruct struct{
MyArr *[]string `diff:"myarr"`
}
func saptr(s []string) *[]string {
return &s
}
func main() {
orig := MyStruct{
MyArr: saptr([]string{"foobar"}),
}
new := MyStruct{
MyArr: nil,
}
changelog, err := diff.Diff(orig, new)
if err != nil {
panic(err)
}
_ = diff.Patch(changelog, &orig)
}
produces the panic error
panic: reflect: call of reflect.Value.Set on zero Value [recovered]
panic: interface conversion: interface {} is *reflect.ValueError, not string
Please let know know if I'm misusing the library or I've implemented this incorrectly.
I am trying to compare slice of maps but the results are inconsistent.
As I understand correctly the code below should show the same result for a == b and a == c
package main
import (
"fmt"
diff "github.com/r3labs/diff/v2"
)
func main() {
a := []map[string]interface{}{
{
"test": 2,
"a": "test",
},
{
"test": 12,
},
}
b := []map[string]interface{}{
{
"test": 2,
"a": "test1",
},
{
"test": 12,
},
}
c := []map[string]interface{}{
{
"test": 12,
},
{
"test": 2,
"a": "test1",
},
}
changelog1, _ := diff.Diff(a, b, diff.StructMapKeySupport(), diff.SliceOrdering(false))
changelog2, _ := diff.Diff(a, c, diff.StructMapKeySupport(), diff.SliceOrdering(false))
fmt.Println(changelog1)
fmt.Println(changelog2)
}
That would be awesome to ad an option that allows to have consistent results in this case
unsupported type: array when ID withh uuid.UUID
Hi, the following crash
func TestZeroPointerDiff(t *testing.T) {
type T struct {
Z *int
}
val := 1
t1 := T{Z: &val}
t2 := T{Z: nil}
changelog, err := diff.Diff(t1, t2)
assert.NoError(t, err)
d, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
assert.NoError(t, err)
patchLog := d.Patch(changelog, &t1)
assert.False(t, patchLog.HasErrors())
}
Hi
Test code using go 1.20:
type Test struct {
ID int64 `diff:"id,identifier"`
Bool bool `diff:"bool"`
Test string `diff:"test"`
}
a := []Test{{ID: 1, Bool: true, Test: "Bool set to true"}}
b := []Test{{ID: 2, Bool: false, Test: "Bool set to false"}}
changelog, err := diff.Diff(a, b)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%+v\n", changelog)
Gives:
{Type:create Path:[2 id] From:<nil> To:2 parent:<nil>} {Type:create Path:[2 test] From:<nil> To:Bool set to false parent:<nil>}]
Field Bool is missing from create. Should also have:
{Type:create Path:[2 bool] From:<nil> To:false parent:<nil>}
Any help is appreciated!
Thanks.
package main
import (
"fmt"
"github.com/r3labs/diff/v2"
)
type Data struct {
ID int32 `json:"id" diff:"ID"`
Value string `json:"value" diff:"Value"`
}
type Order struct {
Items []Data `diff:"Items,ID"`
}
func main() {
a := Order{
Items: []Data{Data{ID: 1, Value: "foo"}},
}
b := Order{
Items: []Data{Data{ID: 1, Value: "bar"}, Data{ID: 2, Value: "paper"}},
}
changelog, err := diff.Diff(a, b, diff.DisableStructValues())
if err != nil {
fmt.Println("ERROR", err)
return
}
fmt.Printf("%+v\n", changelog)
}
Output
[{Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}} {Type:create Path:[Items 1] From:<nil> To:{ID:2 Value:paper} parent:<nil>}]
What I would like is a way so that instead of
{Type:update Path:[Items 0 Value] From:foo To:bar parent:{ID:1 Value:foo}}
I get
{Type:update Path:[Items 0 Value] From:{ID: 1 Value: foo} To:{ID: 1 Value: bar} parent:{ID:1 Value:foo}}
Howdy!
What am I missing here: I am unable to patch my "blank" struct property.
The last assertion fails with reflect: call of reflect.Value.Type on zero Value
.
type PatchStruct struct {
Variable *int `json:"Variable"`
}
func TestPatch(t *testing.T) {
changes := []diff.Change{
{
Type: "update",
Path: []string{"Variable"},
From: nil,
To: 4,
},
}
var patchMe PatchStruct
d, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
assert.NoError(t, err)
result := d.Patch(changes, &patchMe)
assert.False(t, result.HasErrors())
}
When embeding struct within struct, there are no diffs reported
Example reporduction: master...geritol:diff:bug-reproduction
the example code
package main
import (
"github.com/r3labs/diff"
"log"
)
type int64Amount struct {
A int64
b int64
}
func main() {
changelog, err := diff.Diff(int64Amount{A: 1}, int64Amount{A: 2})
log.Println(changelog, err)
// will get panic
changelog, err = diff.Diff(int64Amount{b: 2}, int64Amount{b: 3})
log.Println(changelog, err)
}
we will get a panic
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
because the field b is a private field. can not execute 'Interface()' methods.
for example on diffInt
func (d *Differ) diffInt(path []string, a, b reflect.Value) error {
if a.Kind() == reflect.Invalid {
d.cl.add(CREATE, path, nil, b.Interface())
return nil
}
if b.Kind() == reflect.Invalid {
d.cl.add(DELETE, path, a.Interface(), nil)
return nil
}
if a.Kind() != b.Kind() {
return ErrTypeMismatch
}
if a.Int() != b.Int() {
d.cl.add(UPDATE, path, a.Interface(), b.Interface())
}
return nil
}
If we sure know the kind is Int, we can use Int() to get the value
I think do some change is a good idea
if a.Int() != b.Int() {
d.cl.add(UPDATE, path, a.Int(), b.Int())
}
Using diff between maps with uint as key type, returns a path without the key value.
Following code prints:
"[{delete [items ] 2 } {delete [items ] 3 }]"
type Order struct {
ID string `diff:"id"`
Items map[uint]uint `diff:"items"`
}
func main() {
a := Order{
ID: "1234",
Items: map[uint]uint{10: 1,
20: 2,
30: 3,
40: 4},
}
b := Order{
ID: "1234",
Items: map[uint]uint{10: 1,
40: 4},
}
changelog, err := diff.Diff(a, b)
if err != nil {
fmt.Println("error")
} else {
fmt.Println(changelog)
}
go 1.13
Go language version 1.13 is very old.
It is almost four years old.
Please, update it.
We want to compare slices of strings.
Following example code:
type TestStruct struct {
Slice []string `json:"slice"`
}
type TestData struct {
TS *TestStruct
}
func main() {
s1 := TestData{&TestStruct{Slice: []string{"1", "2", "3"}}}
s2 := TestData{&TestStruct{Slice: []string{"1", "4", "3"}}}
d, err := diff.NewDiffer(diff.SliceOrdering(false))
if err != nil {
panic(err)
}
changes, _ := d.Diff(nil, &s2)
if changes != nil { // for usage and debugger
fmt.Sprintf("")
}
changes2, _ := d.Diff(&s1, &s2)
if changes2 != nil { // for usage and debugger
fmt.Sprintf("")
}
changes3, _ := d.Diff(&s1, nil)
if changes3 != nil { // for usage and debugger
fmt.Sprintf("")
}
}
With this example there are three different Types of changes visible:
changes
contains a create
changes2
contains a update
changes3
contains a delete
The create
and delete
take the whole interface as the From/To values. So the path
is TS
.
The update
on the other side lists all individual changes by index and the path
is TS, Slice, 1
.
My expectation would be, that the lists of changes are consistent, what means
create
and delete
also go down to the index levelupdate
also shows the whole slice diffmaybe an option to make on or the other behaviour optional, would be the best case.
Do I miss something here or is this intended behaviour?
My previous concern is still value:
with the above example, the update
(example changes2
) has one record in the changes, that says
Type: update
path: TS, Slice, 1
From: 2
To: 4
what actually (from my perspective) is wrong, as here we have an create
and delete
of values, and not an update
.
Note: when changing the type of TestData.TS
to a value and remove the pointer, the diff is consistent and all items are listed with changes based on indices.
I personally prefer the diff as a whole, like given in create
and delete
.
EDIT: after some investigation, more info added
👋🏻 hey there
I'm using this library as it has been the most consistent way to compare structs that I've found so far, thanks!
one feature that I'm missing is the option to ignore unexported fields from being compared and marked as differences, I know I can strip them by marshal/unmarshal but if that functionality would be provided by the library itself that would be amazing 🙂
I'm ok with implementing it myself and submitting a pr!
I would like to know
I haven't checked the codebase yet, that's why I'm asking this 😅
in a case where there is a type defined such as
type Set map[string]struct{}
the result of diffing these types always ends up with an empty changelog, even if there are differences on the keys
after digging a bit through the codebase I've realized that the structValues
method won't add any entry to the changelog if the struct has no fields; when diffing maps, for a specific key, if any of the maps doesn't have a value for that key, there is a shortcut that simply returns the output of structValues
as either create or delete depending which map is missing the data
this, effectively makes the changelog to be empty for the following case
a := Set{"existing": struct{}}
b := Set{}
When I am trying to diff two slices with different number of items, but repetitive item values, I've got nil
as a Diff()
result, while those slices are not the same - they have different number of items.
Min example:
a := []int{1, 1}
b := []int{1}
changelog, err := diff.Diff(a, b)
will produce nil
,nil
as a result - no changes, no errors, while a
has different number of items than b
and is different
It looks reasonable to have one change log entry of Type: "delete"
in this case. Yes, it is impossible to say what is the index of deleted item - 0
or 1
, but any of them (0 or 1) will perfectly do and indicate the difference.
Can you, please, advice on how to handle this correctly?
Hi diff team,
I am working on a project that is leveraging the library and there appears to be a problem with interfaces when serializing the change logs. The types get lost in translation.
I have created a simplified reproduction of the issue at hand that exemplifies the issue. In the example there is a type that has a field which is an interface where we are adding different types that qualify for the interface. When we serialize the changelog, then de-serialize the changelog, then apply it as a patch, you will see the error. The important piece is that it does not recognize what type it was from before the change log was serialized.
type Something interface {
Name() string
}
type A struct {
First string
Second string
}
func (a A) Name() string {
return a.First + a.Second
}
type B struct {
First string
Second string
}
func (b B) Name() string {
return b.First + b.Second
}
type Example struct {
This []Something
}
func TestChangeExample(t *testing.T) {
before := Example{This: []Something{A{First: "Joe", Second: "Shmo"}}}
after := Example{This: []Something{A{First: "Joe", Second: "Shmo"}, B{First: "Jane", Second: "Doe"}}}
differ, err := diff.NewDiffer(diff.ConvertCompatibleTypes())
if err != nil {
t.Fatal(err)
}
cl, err := differ.Diff(&before, &after)
if err != nil {
t.Fatal(err)
}
b, err := json.Marshal(&cl)
if err != nil {
t.Fatal(err)
}
var newCL diff.Changelog
err = json.Unmarshal(b, &newCL)
if err != nil {
t.Fatal(err)
}
pl := diff.Patch(newCL, &before)
for _, p := range pl {
if p.Errors != nil {
t.Fatal(p.Errors)
}
}
println("success?")
}
In the Goland IDE inspection and comparison to the before after shows that it does not recognize the same type:
Hi.
Could we do not use state when diff structs?
https://github.com/r3labs/diff/blob/master/diff_struct.go#L101
patchLog.Applied()
is false when we run the following code snippet regardless the patch operation works as expected.
package main
import (
"fmt"
"github.com/r3labs/diff/v3"
)
type Order struct {
ID string `json:"id"`
OrderItems OrderItems `json:"orderItems"`
}
type OrderItems struct {
Items []string `json:"items"`
}
func main() {
a := Order{
ID: "1234",
}
b := Order{
ID: "1234",
OrderItems: OrderItems{[]string{"1", "2", "4"}},
}
d, _ := diff.NewDiffer(diff.TagName("json"))
changelog, _ := d.Diff(b, a)
patchLog := d.Patch(changelog, &b)
fmt.Printf("patchlog applied : %t \nhas errors %t \nerror count %d \n\n", patchLog.Applied(), patchLog.HasErrors(), patchLog.ErrorCount())
fmt.Printf("values \n a: %#v \n b: %#v", a, b)
}
Sometimes we want to record that a field has been changed, but not actually show the changes performed. A good example is a password hash, or a network passphrase.
We would be interested in a "obscure" field tag, that would record the change but maybe star out, or null the actual field values. Just returning null
for from
and to
would be easiest I guess?
Obscure wouldn't work with patching, so not sure if it's too niche for you?
I'm happy to make a pull request if you think it would fit in with the library, otherwise we can form and just keep the functionality in our own version.
Happy for feedback!
Will there be a support for the type primitive.ObjectID for MongoDB? Or is there a way to add types ourselves?
https://pkg.go.dev/go.mongodb.org/mongo-driver
Right now I have to deactivate all of them with diff:"-", that wouldn't cause an issue if I was only using primitive.ObjectID as main ID, but I'm also using that to link collections, so I need it in other area on the struct.
type ObjectID [12]byte
fmt.Print(reflect.ValueOf(obj.ID).Kind()) // array
unsupported type: array
It seems that you support Slice only and not Array
Besides that, it works well. Thanks for this library!
When I go get the module, lets say options.go is missing a few functions.
I have a use case where the diff to object is sent in many smaller parts, therefore i need to be able to not remove objects, but update/add - can this be accomplished?
Currently, when diffing a slice type and a nil value, the diff() method returns an error "types do not match".
Inspecting your code i found that the diffSlice() method is missing the part at the beginning where you check if a or b are of type reflect.Invalid and thus there is a CREATE/DELETE change happening here.
This should be fairly easy to fix though, as you can just copy that part of logic from any other diff-method.
Thanks in advance! :)
Hi there,
Lovely library, thanks for making it! We have started using it in Aether (https://aether.app - github.com/nehbit/aether) for the past week, and it's been working splendidly so far.
While I realise the best way to figure out suitability is to test, I would love to know if you have any guidance on the use expectations of this library from a performance context. In essence, what order of magnitude of elements would you be comfortable in diffing in, say, a for loop? 10, 100, 1000, 10000 or more? Can we rely on this as a component in a stream pipeline that handles hundreds of thousands of messages on a continuous basis, for example?
The way we're doing diffing right now is to convert the object to JSON with a fast JSON parser, get the SHA256 hash, and compare. We don't actually need to figure out in most cases what changed, just that the object has changed. Would this library offer a better, faster way to do this? Essentially, a way to stop processing once any difference is found, and return that there is a diff.
I know compared to the Go's own JSON implementation, this is likely faster, because Go's own implementation also relies on reflect. Likely this means your library is faster than it. However, I think the specific JSON library we use does not.
I'd love to know if you have a feel for what order of magnitude you'd feel comfortable using this library in re: performance. In the meanwhile, I'll probably run my own tests and post back with the results.
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method [recovered]
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
package transformer
import (
"github.com/davecgh/go-spew/spew"
"github.com/finsmart-gmbh/api/model"
"github.com/finsmart-gmbh/pointer"
"github.com/makeless/makeless-go/model"
"github.com/r3labs/diff/v2"
"github.com/stretchr/testify/assert"
"sync"
"testing"
)
// dst: database company lead
// src: company lead from company lead transformer
func TestCompanyLeadMergeTransformerTransform(t *testing.T) {
tests := []struct {
dst *model.CompanyLead
src *model.CompanyLead
}{
{
dst: &model.CompanyLead{
Model: makeless_go_model.Model{Id: 1},
CompanyLeadHouseholds: []*model.CompanyLeadHousehold{
{
Model: makeless_go_model.Model{Id: 2},
CompanyLeadPersons: []*model.CompanyLeadPerson{
{
Model: makeless_go_model.Model{Id: 3},
FirstName: pointer.StringPtr("Max"),
RWMutex: new(sync.RWMutex),
},
},
RWMutex: new(sync.RWMutex),
},
},
RWMutex: new(sync.RWMutex),
},
src: &model.CompanyLead{
Model: makeless_go_model.Model{Id: 1},
CompanyLeadHouseholds: []*model.CompanyLeadHousehold{
{
Model: makeless_go_model.Model{Id: 2},
CompanyLeadPersons: []*model.CompanyLeadPerson{
{
Model: makeless_go_model.Model{Id: 3},
FirstName: pointer.StringPtr("Max"),
Address: &model.Address{
Street: pointer.StringPtr("maxstraße"),
RWMutex: new(sync.RWMutex),
},
RWMutex: new(sync.RWMutex),
},
{
Model: makeless_go_model.Model{Id: 4},
FirstName: pointer.StringPtr("Fisch"),
Address: &model.Address{
Street: pointer.StringPtr("maxstraße"),
RWMutex: new(sync.RWMutex),
},
RWMutex: new(sync.RWMutex),
},
},
RWMutex: new(sync.RWMutex),
},
},
RWMutex: new(sync.RWMutex),
},
},
}
for i, test := range tests {
changelog, err := diff.Diff(test.dst, test.src)
}
}
I have problem when trying to diffing struct decimal
from https://github.com/shopspring/decimal . looks like it fails because of private field *big.Int
Example code:
package main
import (
"fmt"
"math/big"
"github.com/r3labs/diff"
)
type number struct {
value *big.Int
exp int32
}
func (n *number) SetValue(x int64) {
n.value = big.NewInt(x)
}
func main() {
a := number{}
b := number{}
b.SetValue(111)
diffSliceOrder, _ := diff.NewDiffer(diff.SliceOrdering(true))
changelog, err := diffSliceOrder.Diff(a, b)
fmt.Println(err)
fmt.Println(changelog)
}
we will get a panic
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
goroutine 1 [running]:
reflect.valueInterface(0x525fc0, 0xc000046280, 0xb6, 0x1, 0x0, 0x0)
c:/go/src/reflect/value.go:1014 +0x1c3
reflect.Value.Interface(...)
c:/go/src/reflect/value.go:1003
github.com/r3labs/diff.(*Differ).diffPtr(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_pointer.go:33 +0x8b9
github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0xc0000462b0, 0x1, 0x1, 0x525fc0, 0xc000046270, 0xb6, 0x525fc0, 0xc000046280, 0xb6, ...)
E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:128 +0x875
github.com/r3labs/diff.(*Differ).diffStruct(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff_struct.go:50 +0x460
github.com/r3labs/diff.(*Differ).diff(0xc0000044e0, 0x620358, 0x0, 0x0, 0x50bce0, 0xc000046270, 0x99, 0x50bce0, 0xc000046280, 0x99, ...)
E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:112 +0xdf5
github.com/r3labs/diff.(*Differ).Diff(0xc0000044e0, 0x50bce0, 0xc000046270, 0x50bce0, 0xc000046280, 0x0, 0x461b54, 0x508c80, 0xc000046260, 0xc00008def0)
E:/Go Project/pkg/mod/github.com/r3labs/[email protected]/diff.go:101 +0x163
main.main()
E:/Go Apps/raditzlawliet/xxx/xxx/experiment/differ.big.Int/main.go:25 +0x18a
exit status 2
I'm getting some unexpected behaviour when diffing and applying the changelog for byte slices. In this example I am comparing 2 structs that have a byte slice and patching the result onto an empty struct that has a nil byte slice. Is this supported behaviour?
package main
import (
"fmt"
"github.com/r3labs/diff/v3"
)
type MyType struct {
MyField []byte
}
func main() {
left := &MyType{[]byte{208, 72, 51, 52, 175, 134, 76, 84, 143, 38, 99, 184, 128, 24, 107, 163}}
right := &MyType{[]byte{91, 102, 170, 173, 254, 105, 66, 81, 177, 175, 32, 173, 173, 165, 129, 192}}
changelog, err := diff.Diff(left, right)
if err != nil {
fmt.Println(err)
}
dest := &MyType{}
_ = diff.Patch(changelog, dest)
fmt.Println(left.MyField) // [208 72 51 52 175 134 76 84 143 38 99 184 128 24 107 163]
fmt.Println(right.MyField) // [91 102 170 173 254 105 66 81 177 175 32 173 173 165 129 192]
fmt.Println(dest.MyField) // [32 173 173 165 254 192] ?
// Patchlog errors:
// [MyField 0] 208 91 Value index 0 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 1] 72 102 Value index 1 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 2] 51 170 Value index 2 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 3] 52 173 Value index 3 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 5] 134 105 Value index 5 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 6] 76 66 Value index 6 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 7] 84 81 Value index 7 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 8] 143 177 Value index 8 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 9] 38 <nil> Value index 9 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 10] 99 32 Value index 10 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 11] 184 173 Value index 11 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 12] 128 173 Value index 12 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 13] 24 165 Value index 13 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 14] 107 129 Value index 14 is invalid (cause count 1)
// scanning for Value index (cause count 0)
// [MyField 15] 163 192 Value index 15 is invalid (cause count 1)
// scanning for Value index (cause count 0)
}
When I have two slices with two changes (update and create) the changelogs returned is not what it is expected. The example is:
I have two slices like these:
a := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "string"}, }, }
b := []map[string]interface{}{ { "name": "name1", "type": []string{"null", "int"}, }, { "name": "name2", "type": []string{"null", "string"}, }, }
changelog, _ := diff.Diff(a, b)
The changelog is:
[{create [0 type 1] <nil> int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]
But the expected result is:
[{update [0 type 1] string int} {create [1 name] <nil> name2} {create [1 type] <nil> [null string]}]
I saw that in the diff_map.go file there is a function called swapChange that does the change from "update" to "create". Maybe here is the problem.
Thank you very much.
When I try to Patch map using code below, it removes details as well as attributes:
testMap := map[string]interface{}{}
testMap["firstName"] = "John"
testMap["lastName"] = "Michael"
testMap["createdBy"] = "TS"
testMap["details"] = map[string]interface{}{
"status": "active",
"attributes": map[string]interface{}{
"attrA": "A",
"attrB": "B",
},
}
diff.Patch(diff.Changelog{{
Type: "delete",
Path: []string{"details", "attributes"},
From: []interface{}{
map[string]interface{}{
"Key": "attrA",
"Value": "A",
},
map[string]interface{}{
"Key": "attrB",
"Value": "B",
},
},
To: nil,
}
}, &testMap)
Ω(testMap["details"]).NotTo(BeNil())
Result:
Expected
<nil>: nil
not to be nil
Expected to have at the end:
{
"firstName": "John",
"lastName": "Michael",
"createdBy": "TS",
"details": {
"status": "active"
}
}
Hi 👋 ! I'd like to know if it would make sense to add a new type of changelog entry for equal values, so that it's possible to detect that two structs are almost equals and in which fields, simply getting the changelog entries for that new EQUALS type.
My use case is comparing two structs and detecting how much similar they are.
Hi, i ran into an issue where i provide differ with maps where type of one of the values within map has changed - currently this result in an error since isvalid() method doesn't allow this. Would you accept a change which would optionally (through opts) change behaviour for this kind of situation and would just report a change for that key ?
func TestDiff(t *testing.T) {
type args struct {
from interface{}
to interface{}
}
type Info struct {
Qq string
Wechat string
}
type data struct {
Name string
Age int
Height int
Info
listInt []int
}
tests := []struct {
name string
args args
wantErr bool
}{
{
name: "lch",
args: args{
from: data{
Name: "lch",
Age: 1,
Height: 1,
Info: Info{
Qq: "qq",
Wechat: "wechat",
},
listInt: []int{1, 2, 3, 4},
},
to: data{
Name: "lchjczw",
Age: 1,
Height: 2,
Info: Info{
Qq: "myqq",
Wechat: "wechat",
},
listInt: []int{1, 2, 3},
},
},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
d, err := diff.NewDiffer(diff.AllowTypeMismatch(true),
diff.FlattenEmbeddedStructs())
if err != nil {
panic(err)
}
_, err = d.Diff(tt.args.from, tt.args.to)
if (err != nil) != tt.wantErr {
t.Errorf("Diff error = %v, wantErr %v", err, tt.wantErr)
return
}
})
}
}
It is recommended to be compatible with json tag or support custom tag
When trying to use the github.com/deckarep/golang-set
library, running a diff on the sets causes a panic from the reflection package.
Here is a minimum reproducible example:
package main
import (
"github.com/deckarep/golang-set"
"github.com/r3labs/diff/v2"
)
func main() {
a := mapset.NewSet("a", "b", "c")
b := mapset.NewSet("a", "c")
_, _ = diff.Diff(a, b)
}
And an example error:
panic: reflect.Value.Interface: cannot return value obtained from unexported field or method
goroutine 1 [running]:
reflect.valueInterface({0x8517c0, 0xc000088350, 0x0}, 0xc0)
C:/Program Files/Go/src/reflect/value.go:1362 +0xd9
reflect.Value.Interface(...)
C:/Program Files/Go/src/reflect/value.go:1351
github.com/r3labs/diff/v2.(*Differ).diffMap(0x854a00, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0x854a00}, {0x854a00, 0xc0000a63c0, 0x1b5})
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_map.go:27 +0x59f
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0xc000088340, 0x1, 0x1}, {0x854a00, 0xc0000a63a0, 0xc0000ac058}, {0x854a00, 0xc0000a63c0, 0x1b5}, ...)
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:182 +0xb30
github.com/r3labs/diff/v2.(*Differ).diffStruct(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x54}, {0x857020, 0xc0000a63c0, 0x199})
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_struct.go:62 +0xad3
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x857020, 0xc0000a63a0, 0x92b2d0}, {0x857020, 0xc0000a63c0, 0x199}, ...)
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:166 +0xf1e
github.com/r3labs/diff/v2.(*Differ).diffPtr(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0x2e1cf6a7000}, {0x863960, 0xc0000a63c0, 0x16}, ...)
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff_pointer.go:45 +0x5bc
github.com/r3labs/diff/v2.(*Differ).diff(0xc0000e0000, {0x99a288, 0x0, 0x0}, {0x863960, 0xc0000a63a0, 0xc0000cfe01}, {0x863960, 0xc0000a63c0, 0x16}, ...)
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:184 +0xabd
github.com/r3labs/diff/v2.(*Differ).Diff(0xc0000e0000, {0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0})
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:129 +0x1b3
github.com/r3labs/diff/v2.Diff({0x863960, 0xc0000a63a0}, {0x863960, 0xc0000a63c0}, {0x0, 0x2e1cf630598, 0x60})
C:/Users/Vilsol/go/pkg/mod/github.com/r3labs/diff/v2@v2.14.0/diff.go:72 +0x75
main.main()
C:/Users/Vilsol/go/src/awesomeProject/difftest/main.go:11 +0x11b
I am using:
github.com/r3labs/diff/v2
-> v2.14.0
github.com/deckarep/golang-set
-> v1.7.1
When I try to Patch map using code below
testMap := map[string]interface{}{}
testMap["firstName"] = "John"
testMap["lastName"] = "Michael"
testMap["createdBy"] = "TS"
patchLog := diff.Patch(diff.Changelog{{
Type: "update",
Path: []string{"createdBy"},
From: "TS",
To: "KS",
}
}, &testMap)
Ω(patchLog[0].Errors).To(BeNil())
I get the following error
reflect.Set: value of type string is not assignable to type map[string]interface {}
Hi there,
awesome lib, thank you, but sadly it only fill half our need.
We are currently implementing patch and reverse patch method for our internal need on known structures, but it seems more logical to be handle by the diff lib.
Do you have support, plan to support or the willingness to accept a PR about apply/patch and revert diff on arbitrary structures ?
I'm trying to diff two structs returned from a particular API and I want to ignore certain struct fields when computing a diff. I do not have control over the struct declarations, so I cannot add struct tags for this purpose. Instead, I would like to supply a filter function that returns true for fields that the diff should descend into, and false for fields that the diff should ignore.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.