Expanded example for WPF about bike HOT 10 CLOSED

@snunez1 I've replaced the example with a simple package browser. Please take a look.

I'll add an Avalonia example later, after implementing several features I have in mind, namely the CLI interface for downloading dependencies, etc.

from bike.

Having some trouble with the callbacks when loading ('erroneous condition' seems to be specific to BIKE)

CL-USER> (ql:quickload :bike)
To load "bike":
  Load 1 ASDF system:
    bike
; Loading "bike"
.............
MSBuild version 17.7.3+8ec440e68 for .NET
  Determining projects to restore...
  Restored s:\src\third-party\bike\src\BikeInterop\BikeInterop.csproj (in 3.14 sec).
  BikeInterop -> s:\src\third-party\bike\src\BikeInterop\bin\netstandard2.0\BikeInterop.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.11
.....................................
..
; 
; compilation unit aborted
;   caught 1 fatal ERROR condition
; Evaluation aborted on #<BIKE:DOTNET-ERROR {10056F2783}>.
CL-USER> 

and the stack trace:

Represents an erroneous condition
   [Condition of type BIKE:DOTNET-ERROR]

Restarts:
 0: [TRY-RECOMPILING] Recompile callbacks and try loading it again
 1: [RETRY] Retry loading FASL for #<CL-SOURCE-FILE "bike" "src" "callbacks">.
 2: [ACCEPT] Continue, treating loading FASL for #<CL-SOURCE-FILE "bike" "src" "callbacks"> as having been successful.
 3: [RETRY] Retry ASDF operation.
 4: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 5: [RETRY] Retry ASDF operation.
 --more--

Backtrace:
  0: (BIKE::INITIALIZE-CALLBACKS)
  1: ((SB-C::TOP-LEVEL-FORM (SB-IMPL::%DEFUN (QUOTE BIKE::INITIALIZE-CALLBACKS) (SB-INT:NAMED-LAMBDA BIKE::INITIALIZE-CALLBACKS NIL (DECLARE (SB-C::TOP-LEVEL-FORM)) (BLOCK BIKE::INITIALIZE-CALLBACKS (CFFI:..
  2: (SB-FASL::LOAD-FASL-GROUP #S(SB-FASL::FASL-INPUT :STREAM #<SB-SYS:FD-STREAM for "file C:\\Users\\nunez\\AppData\\Local\\cache\\common-lisp\\sbcl-2.3.8-win-x64\\s\\src\\third-party\\bike\\src\\callback..
  3: ((LAMBDA NIL :IN SB-FASL::LOAD-AS-FASL))
  4: (SB-IMPL::CALL-WITH-LOADER-PACKAGE-NAMES #<FUNCTION (LAMBDA NIL :IN SB-FASL::LOAD-AS-FASL) {1002F676BB}>)
  5: (SB-FASL::LOAD-AS-FASL #<SB-SYS:FD-STREAM for "file C:\\Users\\nunez\\AppData\\Local\\cache\\common-lisp\\sbcl-2.3.8-win-x64\\s\\src\\third-party\\bike\\src\\callbacks.fasl" {1002F63983}> NIL NIL)
  6: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<SB-SYS:FD-STREAM for "file C:\\Users\\nunez\\AppData\\Local\\cache\\common-lisp\\sbcl-2.3.8-win-x64\\s\\src\\third-party\\bike\\src\\callbacks.fasl" {1002F6..
  7: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {581D71B}> #<SB-SYS:FD-STREAM for "file C:\\Users\\nunez\\AppData\\Local\\cache\\common-lisp\\sbcl-2.3.8-win-x64\\..
  8: (LOAD #P"C:/Users/nunez/AppData/Local/cache/common-lisp/sbcl-2.3.8-win-x64/s/src/third-party/bike/src/callbacks.fasl" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
  9: (UIOP/UTILITY:CALL-WITH-MUFFLED-CONDITIONS #<FUNCTION (LAMBDA NIL :IN UIOP/LISP-BUILD:LOAD*) {1002F630DB}> ("Overwriting already existing readtable ~S." #(#:FINALIZERS-OFF-WARNING :ASDF-FINALIZERS)))
 10: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "bike" "src" "callbacks">)
 11: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
 12: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS (ASDF/LISP-ACTION:LOAD-OP ASDF/LISP-ACTION:CL-SOURCE-FILE)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "bike" "src" "callbacks">) [fast-..
 13: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "bike" "src" "callbacks">) [fast-method]
 14: ((:METHOD ASDF/PLAN:PERFORM-PLAN (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1002948DB3}>) [fast-method]
 15: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
 16: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1002948DB3}>) [fast-method]
 17: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "bike"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method]
 18: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "bike"> :VERBOSE NIL)
 19: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))

from bike.

@snunez1 Can you please specify SBCL version, and output of dotnet --version and dotnet --list-sdks commands? Also, please specify SBCL version.

Please make sure that the latest .NET SDK is installed and loaded into SBCL(you can use Process Explorer program to check where coreclr.dll comes from), i.e. no incompatible coreclr.dll references are in PATH.

Technically, the function bike::initialize-callbacks is the least expected to fail, because what it does is it invokes a simple static method in the interop library which installs two static fields.

What happens, however, is that the mentioned static method fails with an exception. For reasons unknown.
However, if previous parts of the library were compiled successfully, then the mentioned instance of BIKE:DOTNET-ERROR is a condition object that holds a reference to the actual exception object from .NET.

You can query for the contents of the exception object using the internal low-level reflection API:

(handler-case (something-which-causes-an-exception) 
    (bike:dotnet-error (c)
      (let ((ex (bike:dotnet-error-object c)))
        (format t "~a~%~a" (bike::%get-property ex nil "Message")
                           (bike::%get-property ex nil "StackTrace")))))

Please show the message and stack trace here

from bike.

Also, you can actually use Visual Studio or a similar IDE and attach it to sbcl process to debug the .NET part of it. VS views the process as the .NET process after coreclr.dll and bikeinterop.dll are loaded into it.

from bike.

CL-USER> (lisp-implementation-version)
"2.3.8"

PS S:\src\third-party> dotnet --version
7.0.401

PS S:\src\third-party> dotnet --list-sdks
5.0.416 [C:\Program Files\dotnet\sdk]
7.0.401 [C:\Program Files\dotnet\sdk]

coreclr.dll is loaded into sbcl:

It occurs to me as I type this to ask if this might be related to PATH? My emacs is from MSYS2, and it has it's own PATH environment variable. I just tried the dotnet command in MSYS2 and it wasn't found, which means it's not likely to be found in emacs nor the spawned inferior lisp process.

How is all this configured on your machine?

from bike.

The stack trace from the handler:

CL-USER> (handler-case (asdf:load-system :bike)
    (bike:dotnet-error (c)
      (let ((ex (bike:dotnet-error-object c)))
        (format t "~a~%~a" (bike::%get-property ex nil "Message")
                           (bike::%get-property ex nil "StackTrace")))))
; 
; compilation unit aborted
;   caught 1 fatal ERROR condition
The type initializer for 'BikeInterop.LispObject' threw an exception.
   at BikeInterop.LispObject.InstallCallbacks(IntPtr freeLispHandleCallback, IntPtr applyCallback)

   at BikeInterop.Externals.InstallCallbacks(IntPtr freeLispHandleCallback, IntPtr applyCallback, IntPtr& exception)

from bike.

@snunez1 Yes, the issue seems to be related to PATH environment variable, but I'm not sure whether that has anything to do with emacs. That's sort of a known issue, listed in the README.md file.

It seems that your sbcl process loads coreclr.dll version from windows performance toolkit, which seems to be incompatible with the assemblies from C:\Program Files\dotnet\shared\Microsoft.NETCore.App... which my library tells it to use as trusted platform assemblies. Which probably causes static type initializer failure.

I don't have perf. toolkit installed and listed in PATH, but on my machine coreclr.dll comes from PowerShell 7. Thankfully, PS7 has correct coreclr.dll version. Sigh. I need to handle CFFI foreign library search somehow.

As a workaround, currently, you can either:

1. Remove directories which contain coreclr.dll from PATH(except maybe the one which contains the latest .NET runtime version)

2. Deploy the whole required .NET runtime to the directory that contains lisp runtime executable. Say, to the SBCL installation directory(or, which is probably better, you can save-lisp-and-die and copy the resulting executable to a separate directory(don't forget to configure emacs to use that executable)). Windows prefers loading DLLs that are in the same dir as the executable.

To do (2), create a project file, say, named DummyProject.csproj with the following contents:

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net7.0-windows</TargetFramework>
        <PlatformTarget>x64</PlatformTarget>
        <UseWpf>True</UseWpf>
    </PropertyGroup>
</Project>

Then, in the directory with this project file, execute the following command:

dotnet publish -c Release --self-contained -a x64 -o ./FullNetRuntime ./DummyProject.csproj

After that, ./FullNetRuntime directory will contain the .NET runtime and the standard library(including WPF etc) which you can copy to the SBCL dir, and/or redistribute with your app.

Btw. The static type initializer which throws an exception is probably the following:

        private static readonly MethodInfo ExpressionLambdaDefinition =
            typeof(Expression)
                .GetMethods(BindingFlags.Public | BindingFlags.Static)
                .Where(mi => mi.Name == nameof(Expression.Lambda) && mi.IsGenericMethodDefinition)
                .Single(mi =>
                {
                    var args = mi.GetParameters();
                    return args.Length == 3 &&
                           args[0].ParameterType == typeof(Expression) &&
                           args[1].ParameterType == typeof(bool) &&
                           args[2].ParameterType == typeof(ParameterExpression[]);
                });

No idea why it could have failed(but you can probably take a look into InnerException property and its contents, etc).

PS. My emacs comes from the GNU repository. I simply unzip the latest version into a directory on my PC each time I upgrade or install it. Haven't used the msys one.

from bike.

How exactly is the version of coreclr.dll choosen? I put C:\Program Files\dotnet\shared\ in the system path before the one from the performance toolkit, yet the toolkit version still seems to be the one that bike picks up. I'm confused.

from bike.

@snunez1 To circumvent $PATH mechanics, you should put the full path to a directory that contains coreclr.dll in here, say C:\Program Files\dotnet\shared\Microsoft.NETCore.App\7.0.12

TL;DR:

Technically, the search for coreclr.dll should work as follows:

1. probe the executable directory (resolving symlinks)
2. probe (uiop:lisp-implementation-directory) (e.g. SBCL install dir)
3. probe *default-pathname-default* (i.e. current directory)
4. otherwise invoke dotnet --list-runtimes command, pick the latest version, and load coreclr from there.

The resulting pathname is then put into bike::-coreclr-location- global variable.
And the directory part of the pathname is put into cffi:*foreign-library-directories* list.

But, there is a BUT.

I do not hardcode the pathname into cffi:define-foreign-library form, and only put bike-internals:+coreclr-library-file+(which evaluates to "coreclr.dll" on windows) here.

I do this because of:

1. I want to allow dumping of lisp image, and reloading it on different machines, which could have coreclr.dll installed elsewhere.

2. I was thinking that cffi:*foreign-library-directories* was enough for CFFI to find the correct DLL path.

The problem is that, without the full pathname being hardcoded, CFFI uses the aforementioned special variable only as a last resort. And it first tries to load the library directly. This means, it calls sb-alien:load-shared-object with an argument of "coreclr.dll", which in turn, calls LoadLibrary function from WinAPI, which utilizes its default DLL search mechanics, e.g. first probing the executable directory, then probing $PATH, then $WINDIR/System32, etc.

This could potentially be mitigated by writing a manifest and forcing Windows to load the library from WinSxS, BUT, unfortunately, .NET does not use WinSxS for coreclr.dll...

...Probably because it has its own internal API for selecting the .NET runtime version using the hostfxr thing, which I do not use because:

1. It requires a runtimeconfig.json, and in general, the hostfxr API is designed with an assumption that you have a pre-compiled binary for your program, say, written in C++, and Lisp just does not work that way(e.g. my library does all the probing and loading stuff dynamically).

2. It requires you to put a fixed .NET runtime version in the config. Which would prevent dynamically probing the version.

The fix for all of this, which I'm planning to make, is to circumvent CFFI library load mechanics, by loading the library directly using the full pathname. But that's a TODO.

from bike.

Success. The PATH work-around mentioned above gets things unstuck. Thank you.

from bike.