aardvarkxr / aardvark Goto Github PK

The state file should remember these gadgets even through they aren't on a hook/don't have a grabbable. There should also be a way to kill them.

Presumably with the desktop compositor APIs, however those work.

Maybe the whole desktop should be available in the same way?

Right now the user needs to run "node server/server_bundle.js", and then the server keeps running forever (or until it crashes).

Before announcement, that needs to change to include:

1. Run the server automatically
2. Stop the server when "the app" exits. That probably means avrenderer.exe.
3. Restart the server when it crashes.

Before hanging a gadget on a hook I think that there are some mouse clicks that are happening on the page.

Tested using the scratch example from PixiJS.

The desktop window in avrenderer is useful for debugging, but useless for users. And annoying if the app is set to auto-launch in SteamVR. Make it optional?

when a sphere is placed at something like 1/1/1 and you move the ball slightly, onTransformUpdated gives out a position like x:-1.7472448348999023,"y":1.4837164878845215,"z":1.706096529960632 -- where is it getting these numbers?

<AvGrabbable
    preserveDropTransform={true}
    onTransformUpdated={this.onTransformUpdated}
>
    <AvTransform
        translateX={this.state.transformValue.position!.x ? this.state.transformValue.position!.x : 2.1}
        translateY={this.state.transformValue.position!.y ? this.state.transformValue.position!.y : 2.1}
        translateZ={this.state.transformValue.position!.z ? this.state.transformValue.position!.z : 2.1}
        uniformScale={.1}
    >
        <AvModel color={this.state.color} uri="https://aardvark.install/models/sphere/sphere.glb"/>
        <AvSphereHandle radius={0.1} />
    </AvTransform>
</AvGrabbable>

To repro: press and release the B button a bunch of times. The renderer will hit an assert and then stop working.

Each scene graph change seems to be using a descriptor and not returning them to the pool.

Probably BSD 3-clause.

Need to set it:

• aardvark-react package
• main aardvark package
• Somewhere in cmake files?
• Add to repo

The NPM packages all have version numbers, but I've been hand-numbering the release builds. Maybe the build script should set up the appropriate resource files to stamp the version number on those builds, zip them up, etc. automatically?

Is there some service that can manage collecting crashdumps for projects like this? Is Microsoft's crash upload thing reasonably easy to use these days?

As normal users show up we'll need a way to collect crash dumps (and log files?) from them.

given a fresh clone and removing package.lock in websrc this is the debug output of the install comand

14833 warn [email protected] requires a peer of jss@^9.7.0 but none is installed. You must install peer dependencies yourself.
14834 warn [email protected] requires a peer of jss@^9.0.0 but none is installed. You must install peer dependencies yourself.
14835 warn [email protected] requires a peer of jss@^9.4.0 but none is installed. You must install peer dependencies yourself.
14836 warn [email protected] requires a peer of jss@^9.0.0 but none is installed. You must install peer dependencies yourself.
14837 warn [email protected] requires a peer of jss@^9.0.0 but none is installed. You must install peer dependencies yourself.
14838 warn [email protected] requires a peer of jss@^9.0.0 but none is installed. You must install peer dependencies yourself.
14839 verbose stack Error: ENOENT: no such file or directory, rename 'C:\Users\moose\src\aardvark-master\websrc\node_modules\.staging\node-bin-setup-da3abe88' -> 'C:\Users\moose\src\aardvark-master\packages\aardvark-shared\node_modules\node-bin-setup'
14840 verbose cwd C:\Users\moose\src\aardvark-master\websrc
14841 verbose Windows_NT 10.0.18362
14842 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "i"
14843 verbose node v12.13.0
14844 verbose npm  v6.12.0
14845 error code ENOENT
14846 error syscall rename
14847 error path C:\Users\moose\src\aardvark-master\websrc\node_modules\.staging\node-bin-setup-da3abe88
14848 error dest C:\Users\moose\src\aardvark-master\packages\aardvark-shared\node_modules\node-bin-setup
14849 error errno -4058
14850 error enoent ENOENT: no such file or directory, rename 'C:\Users\moose\src\aardvark-master\websrc\node_modules\.staging\node-bin-setup-da3abe88' -> 'C:\Users\moose\src\aardvark-master\packages\aardvark-shared\node_modules\node-bin-setup'
14851 error enoent This is related to npm not being able to find a file.
14852 verbose exit [ -4058, true ]

Right now the rendering is cobbled together from https://github.com/SaschaWillems/Vulkan-glTF-PBR

We'll eventually want lighting, better animation, etc. Should consider moving rendering to a full fledged engine, preferably something open source. Maybe Godot?

Need to handle how to handle textures meant to illuminate parts of an object surface.

There are many steps to getting things working in a dev environment. They should all be automated so a script can turn a fresh clone of the repo into a standalone build that would work on a user's machine.

And maybe CI should be set up to run that script on each commit.

Moving forward with troubleshooting, but this was the output

Severity	Code	Description	Project	File	Line	Suppression State
Error	MSB3073	The command "setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/openvr/bin/win64/openvr_api.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug/
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
mt.exe -nologo -manifest "C:/Users/Staff01/Desktop/aardvark/src/avrenderer/avrenderer.exe.manifest" "C:/Users/Staff01/Desktop/aardvark/src/avrenderer/compatibility.manifest" -outputresource:"C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//avrenderer.exe";#1
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/chrome_elf.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//chrome_elf.dll
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/d3dcompiler_47.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//d3dcompiler_47.dll
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/libcef.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//libcef.dll
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/libEGL.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//libEGL.dll
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/libGLESv2.dll C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//libGLESv2.dll
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/natives_blob.bin C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//natives_blob.bin
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/snapshot_blob.bin C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//snapshot_blob.bin
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/v8_context_snapshot.bin C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//v8_context_snapshot.bin
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_directory C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Debug/swiftshader C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//swiftshader
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/cef.pak C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//cef.pak
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/cef_100_percent.pak C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//cef_100_percent.pak
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/cef_200_percent.pak C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//cef_200_percent.pak
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/cef_extensions.pak C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//cef_extensions.pak
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/devtools_resources.pak C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//devtools_resources.pak
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_if_different C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/icudtl.dat C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//icudtl.dat
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
setlocal
"C:\Program Files\CMake\bin\cmake.exe" -E copy_directory C:/Users/Staff01/Desktop/aardvark/src/thirdparty/cefbinary_72/Resources/locales C:/Users/Staff01/Desktop/aardvark/src/build/avrenderer/Debug//locales
if %errorlevel% neq 0 goto :cmEnd
:cmEnd
endlocal & call :cmErrorLevel %errorlevel% & goto :cmDone
:cmErrorLevel
exit /b %1
:cmDone
if %errorlevel% neq 0 goto :VCEnd
:VCEnd" exited with code 1.	avrenderer	C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.CppCommon.targets	138	

I've only ever loaded them from FILE URLs, and the rules are somewhat different in Chromium. Before announcing, we should make sure gadgets can actually be served from the internet.

I'm not 100% sure on this one, but using the Polly GLB sample it seems like the back faces render, but not the front.

https://www.youtube.com/watch?v=NKV182hUDdg&feature=youtu.be

That means it needs:

1. A .vrmanifest file that identifies it as an overlay app, and points at the action manifest
2. A way to register that manifest file. Maybe just do this on first run if it's not already registered?
3. Do we want to point users at the "autorun" setting in SteamVR or provide a way to set that automatically?
4. Should Aardvark register a dashboard overlay so it can have a few settings of its own? Seems like it could do that in its own UI, and that could include setting the autorun setting.

aardvark://install URL handler? Websocket request from the browser to the server while Aardvark is running? little file you download and run with a known extension?

Whatever the solution is, users shouldn't be forced to run "avcmd install " to install a gadget.

The gadgets, monitor, master, and renderer are going to lose their connection to the server when this happens.

• Monitor needs to reconnect and refresh when that happens. (Pretty sure this already works.)
• Master needs to reconnect, and go through the normal startup sequence.
• Renderer should probably close itself. Master should restart it. (Or maybe that should actually be the server's job?)
• Gadgets should probably close themselves. Master or persistence should restart them when the server comes back up.

This will be important once gadgets are more widely developed. Right now we're depending on there not being bugs in the version of CEF/Chromium without the second layer of OS-enforced reduced privileges.

1. put a gadget in the toolbox that doesn't have a root grabbable
2. drag that gadget out of the toolbox to spawn it
3. master gadget exits

3 shouldn't happen. I think what's going on is that the grabbable on the gadget seed is retained as the grabbable, and then when that's dropped in the world its parent gadget is destroyed.

When I run avrenderer with the default configuration, gadget_manifest.json isn't loaded by gadgets because 'data' isn't being added to the path after avrender.

Screenshot:

Probably needs to happen after #3 .

Error: Cannot find module 'axios' at Function.Module._resolveFilename (internal/modules/cjs/loader.js:581:15) at Function.Module._load (internal/modules/cjs/loader.js:507:25) at Module.require (internal/modules/cjs/loader.js:637:17) at require (internal/modules/cjs/helpers.js:22:18) at Object.<anonymous> (C:\Users\Jared\AppData\Roaming\npm\node_modules\@aardvarkxr\aardvark-cli\lib\avcmd.js:6:41) at Module._compile (internal/modules/cjs/loader.js:689:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10) at Module.load (internal/modules/cjs/loader.js:599:32) at tryModuleLoad (internal/modules/cjs/loader.js:538:12) at Function.Module._load (internal/modules/cjs/loader.js:530:3)

Render function:

return (
       <div className="FullPage" >
		<AvGrabbable 
			preserveDropTransform={true}
			onTransformUpdated={this.onTransformUpdated}
			initialTransform={this.state.transformValue}
		>
			<AvModel uri="https://aardvark.install/models/sphere/sphere.glb"/>
			<AvSphereHandle radius={0.1} />
		</AvGrabbable>
	</div>
)

See spheresIntersect. In the face of non-uniform scaling, a single radius is insufficient, and the math will fail to do the right thing.

I guess these need to be ellipsoid-ellipsoid intersections?

Should happen right before announcement.

Will probably want to update the subscription on slack when that happens

The binaries are from "unknown publisher" at the moment. It would be good to have them registered to Aardvark Team or something.

ran into various issues setting up on a fresh machine. They were resolved by removing package-lock files and reinstalling.

Probably everything should close when master disconnects to make room for whatever master is going to bring back up. Right now we end up with two renderers, which isn't so good.

• Gadgets collide with each other
• The "I'm in range" indicator of increasing the scale is hard to see on many gadgets
• Gadget "icons" are all different sizes
• Gadget names don't appear anywhere
• There's no way to remove a gadget from the list once it appears there
• The scroll thumb (if that's even the right metaphor) is too small

Need to give credit to https://github.com/SaschaWillems/Vulkan-glTF-PBR for sure. Also, CEF, TinyGLTF, TinyJson, glm, stb_image,, and probably more. Plus all the licenses for the web stuff.

avcmd install/uninstall/reset modify this file, but the server never reloads it. When that happens everything should restart, but especially the server should restart. And doubly-especially it should restart when you're running it in nodemon to allow easy debugging.

We have some level of UI to instantiate them (which needs work #24 ) but there are other operations to perform:

• Uninstall
• Toggle autorun

Others? That UI needs to live somewhere. Maybe a tab in the gadget toolbox itself?

if there's no /gadget_manifest.json, it's not a valid gadget path.

This is especially easy to do when using avcmd with a local path.

It should come back up and get its state back from the server. Any rendering resources on the C++ side need to be freed up so they're available for the new renderer.

Probably grabs in progress will be lost, but that's ok. The main thing is that reloading the render not require restarting the whole system.

This could start as a private repo with a copy of aardvark-react in it. It should get that package from NPM before it's published.

Looks like it's writing Z and is clipping opaque stuff as a result.

Sphere asset is too big for precise interactions. Something like a cursor or stylus might work well.

There's a gadget picking UI on the gear icon on each hand, but it uses a fixed list of gadgets, all of which are included in the main repo.

There needs to be some way to configure additional gadgets to add to that list. Maybe a list of URLs in the persistence file that contain one of:

1. The gadget URL (i.e. everything up to, but not including gadget_manifest.json
2. A JSON file full of gadget URLs (so the main list can be more stable? And so somebody can maintain an online list of interesting gadgets if they want to.)

Dropping them somewhere other than a hook currently just sends them to the origin. They need to be removed from persistence, have their browser tab closed, and be removed from the renderer.

For gadgets that are only meant to be held on hooks, dropping them into empty space should probably be a "throw away" gesture. If gadgets that aren't on any hook is a thing, they'll need to have a specific gesture. Or maybe they're responsible for having a "close" button of some kind?

Downloaded a fresh aardvark repo, installed everything per README, built new gadget via README instructions, installed gadget with "avcmd install ", ran avrenderer, I can see the gadget listed (sphere shows up), click on sphere, a panel blinks on then off. Default gadgets work; freshly built gadgets do not

This went away for a while, but apparently it's back. To repro, run avrenderer and then close the window to exit it. Seems to be a browser instance teardown ordering issue with Aardvark's interaction with CEF.

What other things are needed in order to setup an externally hosted aardvark gadget?

Most PBR materials look pretty great, but ones that absorb a lot of light don't show up well because the IBL and environment map isn't very bright. Here is a video for reference:
https://youtu.be/b48Acc_0R2Q

Having a brighter environment map would be nice.

It could also be nice to have a dynamic environment map or light intensity based on scene app or headset mirror.

Their scene graphs should be discarded when they disconnect. They should come back with the same persistent ID, default hook, etc.

It's ok if the state isn't exactly the same, and that will vary from gadget to gadget. They should appear as though they were just created from a fresh load.