cathei / linqgen Goto Github PK

So we could do this:
int val = list.Gen().FirstOrDefault(/*predicate*/)

insead of

int val = list.Gen().Where(/*predicate/*).FirstOrDefault()

result2 variable in following code will always be an empty list:

List<Vector3> a = new(16);
for (int i = 0; i < 16; i++) {
    a.Add(new Vector3(Random.Range(0, 50), Random.Range(0, 50), Random.Range(0, 50)));
}
Vector3[] result1 = a.Gen().Where(v => v.x < 25).ToArray();
List<Vector3> result2 = a.Gen().Where(v => v.x < 25).ToList();
List<Vector3> result3 = a.Gen().Where(v => v.x < 25).AsEnumerable().ToList();

This issue raise 2 problems

• for further upvote of #11 and add Contains support, its implementation is just Any with predicate for compare, or Where + FirstOrDefaule or even FirstOrDefault(/*predicate*/) if #11 was implemented
• when looking in IDE problem i see
  
  I panick, have to go look it up, watch every thing in my project, only to realize after 15 minutes just your library report, maybe one remove report, two add something like LinqGen report: to clear thing more

It looks like Unity UnsafeUtility's Malloc doesn't like to be used outside of the main thread. I'm using UniTask's UniTask.SwitchToThreadPool(); to repro it.

Version Used: 0.2.2

ArgumentException: Could not allocate native memory. If this allocation was made from a managed thread outside of a job, you must use Allocator.Persistent or Allocator.TempJob.
Cathei.LinqGen.Hidden.UnsafeUtils.ArrayAlloc[T] (System.Int32 size) (at Library/PackageCache/com.cathei.linqgen@81661744fc/Runtime/Unity/UnsafeUtils.cs:27)
Cathei.LinqGen.Hidden.DynamicArrayNative`1[T]..ctor (System.Int32 capacity) (at Library/PackageCache/com.cathei.linqgen@81661744fc/Runtime/Core/Collections/Native/DynamicArrayNative.cs:17)
Cathei.LinqGen.Hidden.PooledListNative`1[T]..ctor (System.Int32 capacity) (at Library/PackageCache/com.cathei.linqgen@81661744fc/Runtime/Core/Collections/Native/PooledListNative.cs:20)
Cathei.LinqGen.Hidden.ThenByKey_6`3+Enumerator[TSelector_6,TSelector_7,TSelector_8].InitState () (at LinqGen.Generator/Cathei.LinqGen.Generator.LinqGenSourceGenerator/LinqGen.1.cs:758)
Cathei.LinqGen.Hidden.ThenByKey_6`3+Enumerator[TSelector_6,TSelector_7,TSelector_8].MoveNext () (at LinqGen.Generator/Cathei.LinqGen.Generator.LinqGenSourceGenerator/LinqGen.1.cs:779)
TargetPreview.Scripts.Targets.Extensions.TargetCueExtensions.AsTargetCues (System.Collections.Generic.IEnumerable`1[T] cues) (at 

I tried out to use the SourceGenerator but it fails at two points when used with types that are not List<> or an array:

• the first issue is in CodeGenUtils.cs @406:77 , simple fix, just the wrong symbol used in the lambda
• the other issue I could also track down but I am not sure how to fix it correctly: If you use a user implemented collection type (I was testing NativeArray for a Unity project, would be an awesome addtion for bursted code :)) then in LinqGenExpression.cs @49:13 the semantic model will always return an empty SymbolInfo for the member access.
  I am not quite sure why that happens but I assume its because the syntax trees are parsed seperately and the tree that contains the NativeArray has not yet been parsed?

Sorry that I don't have a pull request but SourceGenerators are still quite new to me.

Did some testing on a hobby project to see how LinqGen would perform but ended up with slower/more memory heavy results.

Version used: v0.2.1

Default Linq query.

value
.OrderBy(x => x.timeMs)
.ThenBy(x => x.behavior)
.ThenBy(x => x.handType)
.ToArray();

LinqGen query.

value
.Gen()
.OrderBy(new TargetCueSelectors.SelectCueTimeMs())
.ThenBy(new TargetCueSelectors.SelectCueBehavior())
.ThenBy(new TargetCueSelectors.SelectCueHandType())
.ToArray();

Default Linq

LinqGen

Is there something I'm doing wrong? The only thing that stands out to me right now is the multiple "ThenBy"s.

Hey there,
I wanted to test this out and get a feel for what it could bring to my project, but unfortunately, I received an error right off the bat.

I was attempting to use the following to count the number of materials I had stored in ScriptableObjects:

var matCount = terrainData.terrainArray.Specialize()
                .Select(x => x.detailMaterials.Length)
                .Sum();

Both terrainData.terrainArray and detailMaterials are standard Array[] collections, which seemed like it should work fine based on what I saw in the Readme (with Select() and Sum() being supported), but have the following error:

The method or operation is not implemented.

I am wondering if this has anything to do with how/where I am using it, as opposed to what I am using. I am using it in class derived from SystemBase (seen below), which already requires being partial due to source generation being ran on the Unity side of things, but I am using it in a method which is called by an event triggered by a state-machine I have made, so it is just a standard non-Burst compiled method.

public partial class PrepareMapSystem : SystemBase
{

}

The other thing that I am not sure which might be an issue is, all of my code is a package and contained within Assembly Definition files, and as such, in order to reference this package, I added this package as a local package and had to create an Assembly Definition file within your packages main folder. Then in my package, I had to add a reference to that newly created Assembly Definition in order to be able to gain access via using Cathei.LinqGen;

I am not sure if this is causing any issue or not as I can properly access the stubs, my method is public, the class is public, etc. The above-mentioned items are the only things I can think of off-hand which are "out of the ordinary" for a Unity project.

P.S. I second the hope for Burst compatibility in the future. I suspect that most Unity users who come across this package are looking for performance anywhere they can find it, but also a slight reprieve from the additional boilerplate that DOTS can bring. So being able to simplify code with Linq-style code while maintaining the amazing performance Burst can bring is exactly the kind of thing I seek out. 👍

Thanks,
-MH

I was looking into LinqGen to see if it would be able to replace most of my vanilla LINQ usage, and stumbled upon this. Vanilla LINQ's Max and MaxBy methods will return the type of the container. LinqGen's returns int instead. (Min and MinBy are also affected.)

Example:

var list = new List<double> { 1.0, 2.0, 4.0 };

// Vanilla LINQ:
double maxVanilla = list.Max();
double maxNegVanilla = list.MaxBy(x => -x);

// LinqGen:
double maxGen = list.Gen().Max(); // This won't compile, as Max return int
double maxNegGen = list.Gen().MaxBy(x => -x); // This won't compile, as MaxBy returns int

Would it be possible to add a ForEach method for LinqGen.

For example to be able to do:
list.Gen().Where(/*predicate*/).ForEach(/*do something*/)

instead of

foreach (Type x in list.Gen().Where(/*predicate*/)) //Do Something

I've only tested for Select and Where as delegates, but it may be more widespread. There are tests covering Skip and Take, and they seem to work fine. It will probably only affect operations that take a selector/predicate.

Here is some example user code that causes the problem:

TestData.ObjectStringArray.Gen().OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])

TestData.ObjectStringArray.Gen().OfType<string>().Where(g => string.IsNullOrEmpty(g))

The Select generates (with most excluded for brevity)

internal struct Select_uF<TOut_p2> : IInternalStub<char>
    {
        [MethodImpl(MethodImplOptions.AggressiveInlining)]
        internal Select_uF(in OfType_p2<TOut_p2> source, global::System.Func<string, char> selector_uF) : this()
        {
            this.source_n2 = source.source_n2;
            this.selector_uF = selector_uF;
        }
        ...
            [MethodImpl(MethodImplOptions.AggressiveInlining)]
            public bool MoveNext()
            {
               ...
                while (true)
                {
                    ...
                    var current_p2 = (TOut_p2)(object)current_n2;
                    var current_uF = parent.selector_uF.Invoke(current_p2); // Error here
                    ...
                }

                return false;
            }

I think the constructor should take a Func<TOut_p2, char> instead of Func<string, char> as the error is Argument type 'TOut_p2' is not assignable to parameter type 'string'

Locally I inserted these two lines at https://github.com/cathei/LinqGen/blob/main/LinqGen.Tests/OperationTests.tt#L94 to generate failing tests:

("OfTypeSelect", "OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])", "OfType<string>().Select(g => string.IsNullOrEmpty(g) ? '\0' : g[0])", castSources),
("OfTypeWhere", "OfType<string>().Where(g => string.IsNullOrEmpty(g))", "OfType<string>().Where(g => string.IsNullOrEmpty(g))", castSources),

Since the support for Microsoft.CodeAnalysis.CSharp 4.1.0 has been added to Unity 2022.2+, could you start working on incremental generator? Thanks.

i use linqgen in job file like this:
[BurstCompile(CompileSynchronously = true)]
public struct MovementJob : IJob
{
......
public void Execute()
{
....
TilePath startPath = movementRange.Gen().Where(p=>p.gridPosition == startDird).FirstOrDefault();
....
movementRange.Gen().Where(p =>
{
int __distance = AStarPathfinding.GetEstimatedPathCost(startPath.position, p.position);
if (__distance > 0 && __distance <= _distance)
return true;
return false;
}).ToList().ForEach(p =>
{
p.adjacentTiles.ForEach(q =>
{
if (q.g < 0 && p.g >= 0) { q.g = p.g + q.cost; }
});
//return true;
});
}

}

unity editor Error
"The managed function MovementJob.<>c__DisplayClass6_0.<AStarPathFinding>b__0(MovementJob.<>c__DisplayClass6_0* this, TilePath p) -> bool_eefeb7bf1a3bdcec3e8017e3deeff512 from Assembly-CSharp, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null is not supported
"
The location pointed out by the error is exactly where linqgen is used。。。。

there is something worry i miss???

Repro:

using System.Collections;
using Cathei.LinqGen;

var input = new[] { 2, 3, 5, 7, 11, 13, 17, 19, 23, 29 };

foreach (var item in GenericTest.Caster<string, ArrayList>(input, "foo"))
{
    Console.WriteLine($"{item.Item1} {(item.Item2?.ToString() ?? "(null)")}");
}

public static class GenericTest
{
    public static List<(int, TOut?)> Caster<TIn, TOut>(IEnumerable<int> source, TIn value) where TOut : class
    {
        return source
            .Gen()
            .Where(x => x >= 10)
            .Where(x => x <= 20)
            .Select(x => (Int: x, Value: value as TOut))
            .ToList();
    }
}

Errors: Error CS0246: The type or namespace name 'TOut' could not be found

1>------ Build started: Project: LinqGenTest, Configuration: Debug Any CPU ------
1>C:\z\Trash\2024-01-03\LuaTest\LinqGenTest\LinqGen.Generator\Cathei.LinqGen.Generator.LinqGenIncrementalGenerator\LinqGen.Select_2338007894.g.cs(15,61,15,65): error CS0246: The type or namespace name 'TOut' could not be found (are you missing a using directive or an assembly reference?)
1>C:\z\Trash\2024-01-03\LuaTest\LinqGenTest\LinqGen.Generator\Cathei.LinqGen.Generator.LinqGenIncrementalGenerator\LinqGen.Where_3084620607.g.cs(26,57,26,61): error CS0246: The type or namespace name 'TOut' could not be found (are you missing a using directive or an assembly reference?)
1>C:\z\Trash\2024-01-03\LuaTest\LinqGenTest\LinqGen.Generator\Cathei.LinqGen.Generator.LinqGenIncrementalGenerator\LinqGen.Select_2338007894.g.cs(27,31,27,35): error CS0246: The type or namespace name 'TOut' could not be found (are you missing a using directive or an assembly reference?)
1>C:\z\Trash\2024-01-03\LuaTest\LinqGenTest\LinqGen.Generator\Cathei.LinqGen.Generator.LinqGenIncrementalGenerator\LinqGen.Select_2338007894.g.cs(18,76,18,80): error CS0246: The type or namespace name 'TOut' could not be found (are you missing a using directive or an assembly reference?)
1>C:\z\Trash\2024-01-03\LuaTest\LinqGenTest\LinqGen.Generator\Cathei.LinqGen.Generator.LinqGenIncrementalGenerator\LinqGen.Select_2338007894.g.cs(54,38,54,42): error CS0246: The type or namespace name 'TOut' could not be found (are you missing a using directive or an assembly reference?)
1>Done building project "LinqGenTest.csproj" -- FAILED.

Love the project. I am trying it with https://github.com/yanghuan/CSharp.lua and it mostly working except for some minor bugs and shim implementations.