kayhhh / houseki Goto Github PK

Add an audio package, and support (at least partially) the KHR_audio glTF extension.

Add a button in examples to toggle the physics debug lines

Is it possible to add an Undo Recovery Manager?

I noticed that the avatar is vibrating/choppy in the "Mesh" and "Player" examples when walking around and jumping/falling.

Funnily last night it was running smooth and the issue would only happen after some time, but now it's happening deterministically on opening the example.

It's occurring locally and on the vercel demo:

• https://houseki.vercel.app/mesh
• https://houseki.vercel.app/player

I want to creating some sort of scripting system for the engine, or more specific support a glTF extension for adding sand boxed behaviors to a model. I'm thinking about it in the context of the parent project UNAVI, but it would be a generic system around glTF.

I see two options, WASM scripting, or a node-based behavior graph.

• Web Scene Graph provides an example of what WASM scripting could look like
• behave-graph is an example of what a node-based system could look like. Also can look at VRChat Udon

Am leaning towards a WASM approach but I'm still a bit unsure. Would appreciate input if anyone has ideas on this. Some pros and cons of how I see things:

WASM

• If you know how to code, WASM could be faster and easier to work with, especially for larger, more complex scripts
• Would allow for the creation of re-usable libraries (!)
• Better performance?

Nodes

• More accessible to non-coders
• Could be worked on from within the 3D environment itself like Gmod, or in VR like Neos (!)

Add some moving kinematic platforms, spinning platforms, etc to the player example. Also may need to add some functionality to the player controls to make them play nice.

Orbit controls aren't working very well on mobile. After you press down and start dragging, a pointercancel event is being sent, which stops all input from being sent to the orbit controls. Not sure why this event is being sent.

Not entirely sure what this looks like yet, but I want the player controls to have a VR mode. Maybe its a separate package, @lattice-engine/xr that shims into the @lattice-engine/player system schedules to add support for it.

VR brings a lot with it, so this may get split up into multiple issues in the future. Some ideas:

• Enter the camera into VR
  • basic VR rendering
• Support moving around with joystick
  • Could be nice to add other movement options in the future, like teleport
• Support rotating around with joystick
  • Could add option for snap rotations, or smooth rotation
• Do some avatar IK to track player hands
• Grabbable physics objects
  • Send input from controllers into ECS as events

Things aren't very documented right now, and the APIs for using lattice can be pretty ugly. I think there's a good opportunity to go through each package and add documentation (either in the readme, or create a proper docs site), and in the process think through the current APIs (components, systems, helper functions, etc) and clean things up.

2 copies of Three.js are being exported to the examples site, idk y ;-;

three is set as a peerDependency in each package.json
it is also set as a rollup external dependency in each vite config

we run a raycast to detech collision with the world, we should use a sphere instead of a ray to give so extra space so the camera doesnt get put in places where it can see through walls

On glTF import, we should store some information about the import in a component on the glTF entity.
Main thing I need atm is a mapping of glTF index -> entity ID.

Can store this in the ECS as an array, ie nodes: u64[]. Nodes index 0 would be the entity ID for node 0.

Right now, most packages don't export all their systems (because I was lazy). For scheduling within Thyseus you may need to reference these systems, so they should be exported.

Moving the camera around seems to work fine already on mobile, but there needs to be a way to move the player around, probably with a joystick on the left side of the screen, with the right side of the screen for rotating the camera.

Main question here is how to display this joystick. We could use a secondary, "UI" canvas and draw it in a 2D webgl context, but this seems annoying. I think the best approach would be rendering a UI layer after the 3D render, using an orthographic camera in Threejs. Things can be formalized into an actual UI render pass at some point if needed, but for now can keep it simple.

Requirements

• Move player around with joystick
  • Joystick on left half of screen, rotate camera with right half
• Rotate camera around while moving
• Ability to switch between third and first person
  • I guess just zooming in and out like with desktop controls?
• Ability to jump
  • Button in the bottom right of screen seems reasonable for UI
  • Should probably only be an examples thing, leaving UI up to the user of the library

Support the OMI_spawn_point extension for saving / loading player spawns to gltf