Skip to content

Instantly share code, notes, and snippets.

@bymyslf
Last active August 13, 2024 15:49
Show Gist options
  • Save bymyslf/4d8bd7327fc09570835814f5f885fde8 to your computer and use it in GitHub Desktop.
Save bymyslf/4d8bd7327fc09570835814f5f885fde8 to your computer and use it in GitHub Desktop.
Creates a JsonPatchDocument (JSON Patch - RFC 6902) from comparing to C# objects
using Microsoft.AspNetCore.JsonPatch;
using Microsoft.AspNetCore.JsonPatch.Operations;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Linq;
//Based on this: https://blog.abodit.com/posts/2014-05-json-patch-c-implementation/
public static class JsonPatchDocumentDiff
{
public static JsonPatchDocument CalculatePatch(object left, object right)
{
var document = new JsonPatchDocument();
var operations = CalculatePatch(JToken.FromObject(left), JToken.FromObject(right));
document.Operations.AddRange(operations);
return document;
}
private static IEnumerable<Operation> CalculatePatch(JToken left, JToken right, string path = "")
{
if (left.Type != right.Type)
{
yield return Replace(path, "", right);
yield break;
}
if (left.Type == JTokenType.Array)
{
if (JToken.DeepEquals(left, right))
yield break;
using (var leftIterator = left.Children().GetEnumerator())
using (var rightIterator = right.Children().GetEnumerator())
{
var index = 0;
var leftCount = left.Children().Count();
var minCount = Math.Min(leftCount, right.Children().Count());
while (index < minCount)
{
leftIterator.MoveNext();
rightIterator.MoveNext();
foreach (var patch in CalculatePatch(leftIterator.Current, rightIterator.Current, Extend(path, (index++).ToString())))
yield return patch;
}
while (leftIterator.MoveNext())
yield return Remove(Extend(path, index.ToString()), "");
index = leftCount;
while (leftIterator.MoveNext())
yield return Remove(Extend(path, (--index).ToString()), "");
}
yield break;
}
if (left.Type == JTokenType.Object)
{
var leftProps = ((IDictionary<string, JToken>)left).OrderBy(p => p.Key);
var rightProps = ((IDictionary<string, JToken>)right).OrderBy(p => p.Key);
foreach (var removed in leftProps.Except(rightProps, KeyComparer.Instance))
yield return Remove(path, removed.Key);
foreach (var added in rightProps.Except(leftProps, KeyComparer.Instance))
yield return Add(path, added.Key, added.Value);
var matchedKeys = leftProps.Select(x => x.Key).Intersect(rightProps.Select(y => y.Key));
var zipped = matchedKeys.Select(k => new { key = k, left = left[k], right = right[k] });
foreach (var match in zipped)
foreach (var patch in CalculatePatch(match.left, match.right, Extend(path, match.key)))
yield return patch;
yield break;
}
if (left.ToString() == right.ToString())
yield break;
yield return Replace(path, "", right);
}
private static string Extend(string path, string extension)
=> NormalizePathSegment(path) + "/" + NormalizePathSegment(extension);
private static string NormalizePathSegment(string segment)
{
bool IsNumeric() => int.TryParse(segment, out int _);
string LowerCaseFirst()
{
char[] segmentArray = segment.ToCharArray();
segmentArray[0] = char.ToLower(segmentArray[0]);
return new string(segmentArray);
}
if (string.IsNullOrEmpty(segment) || IsNumeric())
return segment;
return LowerCaseFirst();
}
private static Operation CreateOperationFrom(string op, string path, string key, JToken value)
{
if (string.IsNullOrEmpty(key))
return new Operation { op = op, path = NormalizePathSegment(path), value = value };
return new Operation { op = op, path = Extend(path, key), value = value };
}
private static Operation Add(string path, string key, JToken value)
=> CreateOperationFrom("add", path, key, value);
private static Operation Remove(string path, string key)
=> CreateOperationFrom("remove", path, key, null);
private static Operation Replace(string path, string key, JToken value)
=> CreateOperationFrom("replace", path, key, value);
private class KeyComparer : IEqualityComparer<KeyValuePair<string, JToken>>
{
public static readonly KeyComparer Instance = new KeyComparer();
public bool Equals(KeyValuePair<string, JToken> x, KeyValuePair<string, JToken> y)
=> x.Key.Equals(y.Key);
public int GetHashCode(KeyValuePair<string, JToken> obj)
=> obj.Key.GetHashCode();
}
}
@bymyslf
Copy link
Author

bymyslf commented Apr 14, 2021

This is the first json diff patch library that I found that actually works in all cases I've been testing. Other ones either don't work in some cases, or the patches they produce are super simple/generic (example being I update a property inside of an object inside of an array, I'd expect a replace operation for array/3/obj/prop, but it just does array/3 and the value is the entire object). Thanks.

I'm glad it was helpful.

@MJSanfelippo
Copy link

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

@bymyslf
Copy link
Author

bymyslf commented Apr 17, 2021

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

I've faced the same bug with the JsonPatchDocument.ApplyTo and after your comment I noticed that I hadn't update it with the fix. In the meantime I did it. Let me know if it worked for you. However, feel free to share your solution and tests. I'll apreciate it ;).

@schepersk
Copy link

schepersk commented Aug 4, 2021

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

Can I find the updated code and tests somewhere? Would greatly appreciate it!

@MJSanfelippo
Copy link

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

Can I find the updated code and tests somewhere? Would greatly appreciate it!

Hi, I added it as a public gist to my profile. I have not had time to create a PR here. And unfortunately I cannot share the tests because they utilize some internal helper libraries that I cannot share without rewriting the tests, which I am not going to do.

https://gist.github.com/MJSanfelippo/963bd6691397c2cd46add2906556a99f

@schepersk
Copy link

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

Can I find the updated code and tests somewhere? Would greatly appreciate it!

Hi, I added it as a public gist to my profile. I have not had time to create a PR here. And unfortunately I cannot share the tests because they utilize some internal helper libraries that I cannot share without rewriting the tests, which I am not going to do.

https://gist.github.com/MJSanfelippo/963bd6691397c2cd46add2906556a99f

Thank you very much!

@bymyslf
Copy link
Author

bymyslf commented Aug 30, 2021

Of course! I did end up finding a bug in it - it generates remove ops on arrays in the wrong order. When your source object has an array with 3 things, and the target has that same array but with only 1 thing, it would produce a patch of 2 operations. This same thing happens when you remove more than 1 thing from the end of an array. The patch becomes [{remove path/to/array/1}, {remove path/to/array/2}}. When applying this patch (via asp.net core's ApplyTo method), it fails because after it removes the thing at index 1, there is no longer anything at index 2. Would you like me to submit a PR? I have already fixed this locally (and added around 20-25 tests locally).

Can I find the updated code and tests somewhere? Would greatly appreciate it!

The code in this gist it's also fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment