marui-plugin / blenderxr Goto Github PK

Hi,
I've managed to run my Oculus Rift DK2 on your great VR Blender version, so awesome! :)
I don't have any controllers for my DK2 though.. is there any possibility to navigate through the Blender scene using mouse and/or keyboard? Perhaps something like a Blender fly or walk mode?
Thank a lot for your great work. I'll reflect my gratitude by donating ;)
Cheers!

I have a VIVE VR headset. This working with SteamVR (1.13.10) I can play all VR games right.
When I click the 'New VR Window' menu I got "Failed to initialize VR." message.
I copied the neccessary DLLs.

What can I do?
Thanks, Gabor

Hey guys, loving this so far, but for some reason my controllers in the VR landscape are HUUUGE! They block my view often. Is there a setting to just turn off the controller models? A fix? Thanks!

How to reproduce:

• Default scene with default cube
• Open the BlenderXR new window
• Change shading mode to lookdev
• Pick the studio.exr HDRI (the problem is there with others, but it is easier to see with this one)
• Change the HDRI rotation (e.g., 25°)

The left eye updates accordingly, while the right eye is stuck with the previous shading effect.

All of Oculus SDK, OpenVR, and FoveSDK are proprietary softwares that's incompatible with GPL, so you must not use them for supporting VR HMDs and have to use open sourced VR SDK such as OSVR and OpenHMD instead.

https://developer.blender.org/T47899
https://www.gnu.org/licenses/license-list.html.en#OculusRift
https://forums.oculusvr.com/developer/discussion/487/rift-sdk-and-gpl
ValveSoftware/openvr#8

I'm asking this because I'm compiling Blender 2.80 myself, while adding little things on top of it. One thing that I'd like to add is the ability to open a VR window. For this, I'd need the changes that were done by you to the Blender master branch as a patch / DIFF. I am not talking about the MarUI plugin, I am only talking about the changes you had to make to Blender itself.

Here are a few suggestions for Blender XR

Add support to sculpt

Add support for extrusions and cuts

Add support to join and separate objects

Add support to record animations with Vive trackers

Keep up the good work!

1 Open the sln in visual studio 2017

2 build the program.
3. this shows up.

I started a VR but i cant change Focal Length, Clip start and Clip end of the camera. when I edit the value, it reset to the origin value.

I don't know wether this issue is known but whenever I try to use this plugin with ALVR the window instantly closes. Just like riftcat, ALVR makes steam think it's dealing with an HTC vive headset. All steam games work just fine but for some reason blender XR doesn't. The window instantly closes.

I use oculus quest > alvr > steam

As I am not very familiar with anything that has to do with coding I don't know what information I should include...

Please consider the possibility of using the VR input method without a HMD. The blender viewport updates much faster than the VR window, and the fidelity will also be better for most users. Obviously VR has huge advantages for conveying scale, but pancake is good too.

I would like to move, clone and rotate objects with the ease and precision I can with Oculus Touch, just behind my desk. Will you be able to support this?

Ps. you will need to keep the headset alive in the case of Oculus to read controller input.

I dramatically miss the "create edge/face" [F] function for mesh editing.
Please can you add this for example as sub-tool to "insert faces"?

The New VR Window doesn't want to open with RiftCat even though RiftCat uses the SteamVR api directly to connect to other VR programs without issue, is this a problem on your end or with the SteamVR api that RiftCat uses?

Note: RiftCat allows you to connect your phone, or a number of standalone VR headsets with your computer to use them as a headset for your computer.

These work a lot like the Oculus touch controllers of the Rift-S/Quest era, but the grip is just a button.

I use ubuntu 20.04.1 OS. I got this error when launching the Linux version of blender ./blender: error while loading shared libraries: libjemalloc.so.1: cannot open shared object file: No such file or directory.

I wanted to use it for easier posing, but there was no option on the controllers for that.

I am running on Windows 10 with a Valve Index, using the BlenderXR_Windows_20200506.zip download.

I have to move to weird (non-intuitive) positions to get the preview cuts show up:

And when I do the cuts it looks like they automatically have a 0.5 smooth applied to them with no way to modify that:

This is a lot different than the videos shown of the loop cut tool on youtube.

I am running on Windows 10 with a Valve Index, using the BlenderXR_Windows_20200506.zip download.

It seems to always want to extrude individual polygons instead of extruding them together, even when I go to tool settings and pick region.

The easiest way for me to reproduce this is to loop cut each axis of the default cube, add a subdivision surface modifier, and then in VR select 4 polys and extrude making sure region is selected and then I get this:

Hey Blender XR devs,

Just wanted to take a few minutes to let you guys know that you all are doing an awesome job and are one of the most responsive and active teams out there. Your team is making great progress, I have been blown away at how far this plugin has come in such a short time. This team is out performing major companies in VR development progress. I can't thank you enough for making this tool free, and I wish for nothing but great success for your team!

Suggestions
Recording animations with Vive Trackers
Is there any progress being made towards adding support for recording animations via motion capture using the Vive trackers? This would be an amazing feature that could be a game changer for animation creation. If there is any way I can help contribute, just let me know. Currently I have 2 Vive trackers. If you get a demo together for this, I will be more than happen to test it out and help you all work out the bugs.

Toggling the UI menu while inside my Vive Headset
I made a suggestion to add a button that makes your mouse visible while in the headset a while back and your team added it. However, I have not had my computer for a few months and since your most recent update with the new button menu I have not checked where that mouse visibility toggle button is or if there had been any progress on making it appear clearly inside the Vive headset. Last time I tried it, the mouse was distorted in the headset and was hard to see where I was clicking, it almost looked like you were seeing double when you looked at the mouse, and this made it hard to see where I was clicking. I also suggested allowing being able to toggle full UI menu visibility inside the headset so that way we would be able to fully use blenders tools while viewing and editing in the VR headset. There are a lot of features that are not available on the controls yet and probably will not be for a while just because there are limited buttons on the VR controllers, and the fact that your team is working as fast as possible but obviously can't add support for EVERYTHING overnight. Allowing a toggle button to interact and see the full blender UI as well as the mouse would allow Blender to be 100 percent functional and usable while in a VR headset. Also, some features and options in the Blender UI will be much easier to use with a mouse and keyboard for quite some time. Instead of delaying the availability to certain tools and features, you could enable the UI to be visible inside the headset. Until you can create an easier solution, at least consider allowing the ability to toggle the visibility of the UI while inside the VR Window so that we have full access to Blender while in VR!

Keep up the amazing work,

WW

Blender XR would be revolutionary if people could work together in a virtual space and build things together. I know this wouldn't be an easy task at the least, but it would be well worth your time. Imagine the things that would be possible if you allowed people to connect together in a virtual space and work on models remotely. It would be great if you could see the other persons headset or avatar and controllers in the virtual environment, and then network anything created between the devices that are connected in that space. This would change the game for VR 3D Modeling and Design.

Also,
Vive tracker support for creating animations would be awesome!

WW

Whenever autokeying is enabled, the changes made while in virtual reality do not record in blender.

Hey,
Thanks for marvellous work, BlenderXR is probably the only fleshed out full featured VR enabled editor ive found (that is free of course).

After some playing around with it Ive got some suggestions for UI :

• Configurable UI palette would be awesome, cause sometimes Id rather rearrange some buttons for my workflow
• Additionally ALT button is very much neccesary for me
• Snapping and mirror editing (and probably a special menu for modifiers, ike a floating window or sth) is a must for more precise work.
• less obstructive controllers maybe, more like only outline just like in sketchbox.
• on WMR selection using joystics is kinda unprecise, its very hard sometimest to quickly select some options because of joystic springing back and selecting an option near the one i wanted to select.
• the "no rotation other than Z" option for scaling and moving should be enabled by default.
• also I've managed to scale myself -1 on all axises making my eyes switch sides, so this is probably a bug

for now i model stuff traditionally and move stuff around in VR to get the feeling of space, but VR editor is the future.

Hi!
Excuse me:
Does it works only with Base Station + Controllers or the Head-set is needed?

Only want to trace drawig in the space . . . but looking at the regular screen.
I use GNU-Linux ( looking for 2nd hand HTC vive but headsets are expensive : /

Thanke Ü
Congrats 4 the great work
~ ~ ~

The window will briefly open then close itself.

Hello,

we just tried fresh blender-xr on our i7 computer. What happens is that (powerful) computer freezes (and blender-2.8-xr crashes) with when blender scene is used with textures in VR.

However, exported into OBJ files, what is without textures, our 3D scene works fine in VR.

Is it because of some software issue or is it just poor hardware ? Thanks.

Hello MARUI Team,

I downloaded and unzipped the Blender XR window. I added the oculus.dll file (as I have an Oculus Rift). When I open up the Blender file that is inside of the BlenderXR file and click on New VR Window (under the Window tab) it flashes for a second but never opens the new VR window, it only stays on the main blender window. I have tried restarting my computer but to no success. I have also tried opening the window while my Oculus window (to ensure it wasn't the fault of my Oculus).

What could be the cause of this issue? Is it a bug in the file?

Please let me know.

(I am very excited to start using blender in VR and what you guys are doing is paving the way for VR as a whole.

Nice work. Thanks. It's coming along beautifully.

Trying to use the Valve's Index Controllers is tricky though. They have a touch-pad, but it's just a vertical one. Barely has an X-Axis, long and thin. So it can't be used for the select-tool.

I can map the joystick to give touch-pad axis values, and that works to an extent. But the tool's final selection is made by releasing the touch-pad, and of course releasing the joystick tends to make the joystick return to (0,0).

Would be handy if a press on the stick/pad during selection also worked to select the tool instead of having to use position during un-touching.

When sculpting in VR, the point being drawn does not update as I move the controller (using SteamVR+Valve Index controllers). This means it's possible to add a blob onto an object being sculpted but not to draw a smooth curve.

Created a video to show difference from sculpting using non-vr input:
https://youtu.be/gQnZSGybfEk

Trying to get this up and running with my Rift but for some reason when i try opening the VR window it looks like it opens for a second then immediately crashes (attached video). Followed the steps in the video, not sure if i had to do anything else?

VRwindow_notworking.zip

I think the knife tool should work more like it does in the regular version of blender. Were it uses a click to start, then a click for each segment. It should also preview where it would start and highlight any edges or vertices that would be selected before any click.

Currently it's too difficult to know where the cut would start and then it instantly makes a segment as soon as it reaches another edge. This means I find myself needing to be very careful and slow to try to get a cut that's even close to what I want. I certainly can't get it to go to/from any desired vertex.

The UI with the oculus rift on the controllers is impossible to use.

Looking at this there are too many layers of text all on top of each other.

Have you considered easier and larger menu with easy selection like in Google blocks like this?

Unable to initialize VR
I can't seem to have much luck with str_ManufacturerName, str_TrackingSystemName and str_ModelNumber for my headset, I am using Virtual Desktop and the debug build is not building as win64_vc14 is no longer in use (win64_vc15)

I hope that I am providing enough information, also, is this necessary?

Please add the "spherical select" tool as it feel way more natural to select with sphere in VR than box parallel to world XYZ axis. Most likely it can be modified from "Proximity select" tool.

Start drag set sphere center. Drag change select sphere size. End drag perform select like "Proximity Select" do.

As additional feature it can be cool to diffirentiate VR controller trigger position.
The light trigger press spawn sphere with last used diameter and drag move center. If user pull trigger all the way down than start change diameter. Release trigger perform select.

I've read the code but i'm not in blender and vr development yet so there is too much specific details to implement by myself...

Is there support for a side-by-side mode, which can be used by anyone who can do the cross-their-eyes-method (3d stereograms)?

Also, I have a 3d tv (an old LG), it has support to convert side-by-side or vertical splits, in addition to a few other modes which would be more hardware-specific.

Here is an image of the 3d methods it provides (when connected via HDMI to my PC's old nvidia card -- less options are available when connected to other devices). Also, note that one of the options is animated, showing side-by-side with alternating frames I believe:

Did I install the dll's incorrectly?

While trying to open/import/save clicking on file or folder does nothing.
System and bookmarks panels works fine.

I've found i can overcome this by dragging a frame with mouse over required item in list

Win 10 x64, Oculus Rift CV1 + Touch

I have everything working correctly for oculus and I can enter vr fine. I went to change the preferences and add plugins after testing and I cannot enable cuda for rendering. Ihave an rtx 2070 and I am running with studio drivers. The other blender builds do load the cuda setting, anyone else have this issue? Should I just use this build for scene setup and then transfer to another build to render?

On Windows 10 pro 64-bit, with Oculus Rift: I followed the steps provided https://www.marui-plugin.com/blender-xr/#download and downloaded from those links. Program appears to work fine before opening a VR window including using the sculpt tool. However, crashed to desktop upon using the sculpt tool in the VR window on a sphere. (Also haven't figured out how to choose different sculpt options yet...)
In action: https://youtu.be/xoHUA8AUn5Q

Also was able to crash it when testing another tool on the sphere (knife?)

I am thrilled someone has taken on the challenge of adding support for VR in blender! I am testing it out right now with the HTC Vive and it is awesome. Thank you for your time and for making this plugin free.

Right away I saw a few things that could be done to improve this tool. I am only posting because I believe this is the future and I wanted to contribute my part to make sure that this tool is the very best it can be.

In short:

1. Make it to where I can see the mouse in VR
2. Create blender XR to support multiple tools
3. Add Vive Tracker Support as an additional tool to the controllers
4. Add Mouse/Keyboard functionality so that I can use it along side the controllers while building things. This will allow for much more complex configuration and usefulness for the Blender XR Tool

Suggestions:
Add support to integrate mouse and keyboard inside VR. If I could see my mouse inside the headset, I would be able to build models with the headset much easier. It would be great if I could sit at my computer and build stuff with the headset on and just use the mouse/keyboard. If you haven't tried an app that supports this yet, it is actually incredible useful and feels natural to use to mouse and keyboard. I just know that blender is a very complex tool and while the controllers lack buttons/functionality for some of blenders features, the mouse with the headset would work perfectly. What would be awesome is if I could use the controllers and mouse side by side. So I could make some adjustments, then pick up my controllers and use it for something else.

Vive Tracker support
Support for Vive trackers in Blender XR. You may already have support, but it would be cool if I could use and implement the vive tracker as an additional tool to the controllers and potentially the mouse.

Issues:
Using the mouse and adding a cube causes the app to crash. This ties in with implementing mouse/keyboard functionality along side the controller. This would be a major step in the right direction if you could fix this bug. Every time I try to add an object with the mouse inside the new VR window it just crashes.

I'm running a Valve Index with HTC Vive wands which are correctly detected in SteamVR. I have downloaded the Vive .dll files. My model loads inside the Valve Index correctly but it shows Index controllers instead of my Vive wands and none of the buttons work.

DetectedDevices shows:
ManufacturerName=Valve
TrackingSystemName=lighthouse
ModelNumber=Index

Hello,

I've been working on merging the final Blender v2.80 into BlenderXR. It's almost working: it builds and the VR window works, but the controllers are not visible (they work, I can interact with them, but I don't see them, nor their menu).

To make the merge I created a new repository with only the blender directory of BlenderXR and merged the v2.80 tag from the official Blender repository. So this repository contains the history of the blender part of BlenderXR, and the history of Blender.

What do you think about using such a repository instead of a plain copy in the blender directory? It could be a submodule, or just a separate repository. I think it would make future Blender merges much easier (just a pull from the official blender repository).

The repository is currently at https://github.com/sigmike/blender, with the last commit in the xr-2.8 branch being the merge. It's a big merge, there has been a lot of changes in Blender since February, including a clang-format of all the source files (I formatted the vr files too, and moved the external VR libs to /extern so that they won't be formatted by make format).

About the invisible controllers, I'm not sure I'll be able to debug that. I'm not familiar enough with the Blender source code. Do you have a suggestion on how to fix that?

Author needs to read people's webpages much more closely.

Add the ability to do texture painting from inside of the VR view.

Is it possible to change the user scale in VR?
In other words:
If I have a 1:1 city scene, how would I scale myself to the size of godzilla?

Thanks.

Neither pressing the touchpad nor pressing down on the joystick switches to edit mode. I have also tried remapping other buttons to left_joystick_press and left_touchpad_press but still can't switch to edit mode.

I am similarly unable to do a Coordinate System Switch.

pthreads needed for cycles is version 2 in this build and version 3 in the svn downloaded libs.
Solution is to either update the build to match or copy the v2 libs and rename to v3 , then copy into blender.exe directory.

I also used Cmake-gui to turn off Collada since it was not found and was causing the build to fail. The make.bat does not seem to have the feature to turn off collada.

I ran into an issue with clearing the screen and screen lag :

The screen lagged when lots of annotation verts were on the screen.

Is this using one thread to pull the data from openvr into blender or separate threads ?
It looks great. how did you send the window to the vr headset?

Hi,
Thank you very much for your work, I’m quite new to both blender and to VR, but it seems to be a paradigm shift and being able to jump inside blender models is just incredible.

I have a new Oculus Quest, connected to the PC through the beta feature Oculus Link, so that it acts like a Rift S.

It all works fine with other software, including Steam VR, but when opening a new Blender XR VR window it just gets stuck in loading ( a window of a new starting app shows up with the only message ‘loading....’ and no icon/image, it seems to have the icon of a missing referenced image actually, but when I move back to oculus home menu controller then blender icon appers in the loading window, but it still keeps hanging forever).

I tried both with the oculus dll and later changing that for the steamVR dlls and I run in the same issue (a window of startup shows up in steam and keeps loading forever).

Is it just working with the Rift and not with the linked Quest, or there is something that I might do wrong?

Thank you,

On Windows 10 pro-64-bit, with HTC Vive: I followed the steps provided https://www.marui-plugin.com/blender-xr/#download and downloaded from those links. The plugin works fine before opening a VR window and using the sculpt tool. It instantly crashes to the desktop upon using the sculpt tool in the VR window on anything more complex than a cube (default primitive). The issue appears much like the previously reported bug with the Oculus Rift, I'll link the Rift users' video for reference: https://youtu.be/xoHUA8AUn5Q

I am currently running Blender 2.8.1. The patch does not appear to have corrected this error.

--

This was originally a response to a thread, but I spent so much time typing it that I decided to re post it as a separate thread. I have typed up a list of suggestions that I hope will be able to help you see my vision for recording animations via motion capture inside Blender.

Networked 3D Modeling Idea
Ok I am going to type this up really quickly due to the fact that I am on a crunch for time, but I feel it is worth my time to pitch this. It would be amazing if I could join a virtual space with my friends or coworkers and work on a 3d model with them in real time. Obviously you could start in small steps like adding network support so you can at least see someone else build their model and be able to talk to them. Eventually though, it would be amazing if I could interact with someone else's model. This would be great for a team that is working on a project, they could visualize their ideas, show suggestions for improvements real time, and even build a large project together by all working on separate parts simultaneously. I understand this is no small task, but if you are interested in this concept you should check out possibly using Networked A-Frame or something along those lines to create this!

I have typed up a list of suggestions that I hope will be able to help you see my vision for recording animations via motion capture inside Blender.

Use cases for motion capture using the HTC Vive controllers and trackers

1. Recording animations for human avatars
   So for example lets say I have a avatar in blender, I want to make it to where the body moves with the avatars bone structure. I want to then be able to record myself doing something like waving, crouching, jumping etc... I would also love to be able to record animations with a character such as swinging a sword in a game with Vive trackers. It would save me so much time if I could just record myself swinging a sword and then saving that recording and using it as an animation.

2. Recording animations for machine parts
   Now this theoretically could be recorded with the Vive trackers too, but it would be cool if I could recording moving parts of a machine.
   a.) So for example lets say I have a robotic arm and I wanted to quickly record an animation of the arm moving from one point to the next.
   b.) Another example would be to record the movement of things like a wheel, gear, or motor by hitting the motion capture button and selecting the object and then recording myself twisting my controllers. If there were a lock grid button in the options tab for motion capture, I would be able to freeze my positions of X, Y and Z. I could then enable rotation only and record me twisting my controller or Vive tracker but only record the spinning part, not the full motion of the controller.

3. Recording a moving vehicles
   So lets say we wanted to record a vehicle moving down the street. I would start by going to the motion capture settings and binding the car to the controller. Let's say the X axis is your horizontal plane in this example. I could freeze the Z and Y axis, and that way when I record the car driving in a straight line down the street, it would one recording motion in a specific direction. If there was no option to freeze the axis then the recording would look terrible and nothing like moving a car just because the chances of you recording yourself moving your controller in a perfectly straight line are slim. With those toggle settings, all sorts of movements and motion could be quick and easy to record.

4. Recording a door opening and closing
   so for this example I would select a door as my object bound to my controller, and set the pivot point where the hinges would be. Now since I only want to record the door swinging back and forth on my X axis, I would disable movement on Y and Z. Now once I press record, the door will appear to swing from the hinges just by moving my controller left or right.

5. Recording dance moves
   By assigning avatar to all of my controllers and vive trackers via the tool button, I could record dance moves.

6. Recording emotes
   By rigging an avatar to my controllers and trackers, I would be able to record different emotes such as spinning in a circle, jumping, waving my hand, swinging my fist, picking something up from the ground, answering my cellphone and much more. This would be amazing for recording NPC characters movements, combat animations, expressions, and much more.

Settings ideas for the Motion Capture button

1. There needs to be a motion capture button on the menu of the Vive controller that is easy to toggle on and off.

2. If there were a motion capture button on the menu, it would need some options such as being able to select where the pivot point would be while in the headset. Would be cool if it gave you a little point at the tip of the controller when you hit the pivot button, so when you have the tip of you controller where you want it you could press the trigger button or something to set the new pivot point.

3. Another useful button in the options menu of the motion capture button would be the ability to select an object and bind it to your controller. Binding the object to the controller basically would mean that you could see the 3D model follow your hand movement. So if I select an object, I can have it following my controllers movements which would allow me to record the actual movement of my controller.

4. The ability to lock positions and rotations on the grid such as the X, Y and Z axis so that you can record specific animations much smoother. Check the Machine parts example for how this could be a great tool

5. A toggle button for a countdown until you record the animation. A countdown from 3 to start recording would be really helpful, that way you get a chance to position yourself. A shortcut button to record might also do the trick, but I think the countdown may be a nice option that you could toggle.

6. A button to allow you to insert a model from your folders. Then once the model is loaded it gives a separate pop up window that can be grabbed and moved around when you press your grip buttons, I suggest you make it to where you controller has to be touching the menu or window in order to grab and move the window in a different place in your virtual space.

7. A button that allows you to either bind the 3d model to your controller or to multiple controllers. So for example lets say I wanted to select an avatar that I wanted to rig to my controllers for full body tracking. I could then select the option to rig the avatar to my body instead of binding it to just one controller to record a simple animation, Now that the avatar is rigged to my controllers and trackers, This tool would allow me to record realistic and fluent character movement very quickly and easily. If you are not already aware of A-Frame's component for motion capture, you should check it out here. It is open source so you might be able to refer to that for guidance on how to setup this button inside Blender XR. There is a GIF on that website that will also give you an idea on how this concept would work. Check out A-Frame here

There are a ton of possibilities for motion capture using the HTC Vive controllers and trackers . Let me know if you have any questions or if you need one of these concepts better explained. I would be glad to help any way I can.

WW