Linux support about wickedengine HOT 38 CLOSED

I'm interested in adding Linux support. Right off the bat, I'm trying to come up a a good build solution. If we would use Scons, we could unify both the Windows and Linux build systems. Scons also supports exporting a Visual Studio project. Scons is also based on python, so it's fairly easy to use. Thoughts?

from wickedengine.

Hello, this project looks promising indeed! Hopefully the Linux+Vulkan version will take off at some point.
I'm looking for a Linux-capable open-world engine on Vulkan that could spare me from making one from scratch (still a n00b in the topic).

Hajrá! :D

Üdv

from wickedengine.

Hello
I've been dabbling with porting the W.E. to Linux, and while I've not succeeded, these are the issues I've encountered.

1. Use of windows types ( UINT/DWORD etc.) -> replaced with stdint.h ones
2. The math library (DirectXMath) -> partially replaced with glm ( sadly, I think aligned glm types are less stringent )
3. Window creation -> partially replaced with glfw
4. Input -> partially replaced with glfw
5. Networking -> partially replaced with ACE library ( which, I admit is an overkill, but "you go with what you know" 😄 )
6. Shader compilation -> likely will have to change to using SPIRV-cross tools

from wickedengine.

@Je06jm this might help microsoft/DirectXMath#89 (with luck, it could just work after removing the MSC_VER check and including that header).

from wickedengine.

I'm planning on using a Linux machine to develop and test Linux support. I think nightlark's idea is a smart one, because Visual Studio supports CMake. Visual Studio also supports building with CMake https://devblogs.microsoft.com/cppblog/cmake-support-in-visual-studio/#build-cmake

from wickedengine.

CMake supports backward compatibility by using policies

In practice dealing with backwards compatibility hasn't been hard with CMake -- basically pick the minimum version you want to support (3.0 or newer recommended) and use a cmake_policy(VERSION major.minor[.patch[.tweak]]) line to make newer versions use policies based on the minimum targeted version.

from wickedengine.

Hi, the main thing holding back the Linux port is the graphics API. There is going to be a DX12 layer first, and a Vulkan layer later, and from that point it will be feasible to port this to Linux. I have no experience with Linux development yet, but I am interested. Thanks!

from wickedengine.

Hi! I am currently in the process of writing the vulkan layer, hopefully I will to have it up and running soon, but don't expect production quality anywhere in the near future. Even the DX12 renderer is still only experimental. Anyway, thanks for the interest.

-Janos

(Üdv neked is! :))

from wickedengine.

It looks like there are separate DX12 and Vulkan layers now -- so one of the next steps might be getting a cross-platform build system in place.

from wickedengine.

Hi, I would be slightly interested in the getting the Linux version up and running, but I have no Linux machine currently, so can't promise anything. Btw the Vulkan layer still needs much work too.

from wickedengine.

Hello,
Thanks for the interest in this. Let me share my opinions of your points:

1. Good point, that is something I wanted to do anyway, replacing all UINT and DWORD with uint32_t.
2. I don't think it's viable to replace the math library, but DirectXMath is a header only open source library, that could be included for any platform. Though the one I am using is included from the Windows SDK folder. I imagine it can just be moved over to the engine sources folder.
3. I am not a fan of adding dependencies. However, I don't know how difficult is to get the windowing working on Linux. For Windows platform, it should stay in the way that doesn't use dependencies.
4. Same as point 3)
5. Networking needs a complete rewrite, preferably without using dependency, however in that case I could consider adding a dependency and making it an optionally compiled feature.
6. HLSL -> SPIRV compilation is working already. The only thing that needs to be done is improve the build script to build them automatically.

About the dependencies, I am avoiding them because I feel like the build process should be the simplest possible, just hitting F5 to build and run within Visual Studio must work out of the box. Also, the produced executable must run without any additional dll files and such. It is also important to keep the program size small as possible. At this time there are some dependencies included, but except Bullet physics, each one is a simple header only library.

Thank you for taking your time to experiment with this.

from wickedengine.

ad. 2 It seems the DirectXMath.h uses __declspec(align(x)), which gcc does not support, so to make it cross-platform WickedEngine would have to fork the lib and provide a custom version of it.
ad. 3/4 The windowing and input are not pleasant to work with on linux :)
Also using glfw likely would get you OSX support as well ( at least for those 2 subsystems :) )
ad. 6 I've mentioned SPIRV-cross, since generate_shader_buildtask_spirv.py seems to be using dxc.exe ?

from wickedengine.

2. I didn't know that it is using such an extension, but should be easily replaceable with c++ standard alignas() specifier (I think).

3/4) I quickly checked glfw and although the zlib license seems quite permissive, it is still not MIT license which this project is aiming for. For that reason, I would go against including glfw into the engine. However, a more preferable way would be to let the engine expose a native linux interface, then a linux user could use glfw, initialize his window/input/whatever, and use the engine's native interface.
However, this is only my initial feeling about, and not set in stone. (change my mind :) )

6. You are right, it is using dxc.exe, which can compile hlsl to spirv. If I understand correctly, SpirV-Cross can provide a reflection API for Spirv that can be also used to compile it to other languages. However, this project is using hlsl as a shading language and can compile it to spirv. Also, this project does not rely on shader reflection at all, everything that needs to be shared with the application is shared via shader headers and available at compile time. So I see no reason to use the Spirv-Cross tool.

from wickedengine.

ad 2. ) Agreed, although the number of changes will be non-trivial - there is a [fork of the library]
(https://github.com/VladSerhiienko/DirectXMath) that might work with gcc/clang compilers

ad 3/4) I'm unsure how external ZLIB licensed library would impact the main project's license?
As for providing 'os specific' interfaces, it might be hard to properly craft an interface that's rich enough to provide some of the more interesting features ( window activity/inactivity, mouse cursor support, multi-monitor support, high-dpi etc. )

ad 6. ) Sorry somehow I've confused spirv-cross with glslang, still dxc.exe will not be available under linux, so some kind of solution will have to be found ( maybe putting spirv versions of shaders in the repo ?)

from wickedengine.

2. I agree, it might be non-trivial, but probably doable

3/4) I am also unsure about the license. If it could be an optional external module, it would be fine, I think.

6. DirectXShaderCompiler (DXC) now supports Linux, so it should be fine. microsoft/DirectXShaderCompiler#1236 (comment)

from wickedengine.

ad 3/4) Looking at ZLIB license, there's no problem with statically linking it with any project, You could even skip mentioning the usage of such a library in docs : ..If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required..

from wickedengine.

everything still in progress for Linux?

from wickedengine.

There has never been any progress in Linux (from me), sorry. I don't have a Linux machine and currently not interested in setting one up. I am open to contribution though.

from wickedengine.

My general feeling is that CMake would be a better choice than Scons for unifying build systems -- it's more widely used, has good support for different OSes and build environments, CPack provides a pretty straightforward way to create installers, and IDE support in Visual Studio/CLion for working with CMake-based projects is pretty good (without requiring a user to manually generate a project for the IDE first).

The main complaint people tend to have about it is the language (which has some quirks that can take some getting used to), and the paradigm shift of using a build system generator (though cmake --build helps abstract the underlying build system that invokes the compiler/linker).

from wickedengine.

Before any complicated multi platform build system is set up, first the Linux build should be up and running. Besides that, I want to keep the windows build available as a Visual Studio project without any project generator. Isn't Linux development supported now through Visual Studio somehow?

from wickedengine.

Yes I know about it, cmake support has been brought up a lot. I am personally not a fan of maintaining a cmake project, since Visual Studio is the industry standard dev environment used for PC and console development (this is the area I am familiar with). My feeling is that cmake would get into the way of everyday development workflow.
Consider tweaking some project parameter in VS, and if you want to keep it in the long run, you will need to go back to cmake project to enable the feature and testing whether the generated projects still work correctly.
Not to mention all the backwards compatibility issues between different cmake versions. If cmake is really important for Linux development, it can be considered to only develop through cmake for the Linux platform.

from wickedengine.

We might have to do that. Another problem that has occurred is that DirectXMath only supports Visual Studio. We would need to A: modify the entire DirectXMath library to work with Linux, B: Only develop on Windows and cross-build, or C: choose a different 3D math library and rework the engine to use that. I don't recommend A, because that would take a lot of time on top of adding Linux support. Option B (shutters) is not helpful for Linux developers. We might be able to add C as a Linux only option, but then we would have two math libraries. What are your thoughts?

from wickedengine.

What makes you say the DirectXMath requires Visual studio? In that case, modifying DirectXMath should be the solution. It definitely should support multiple compilers with some preprocessor switches. I don't want to switch math libraries that's for sure.

from wickedengine.

As far as I am aware, you cannot download the visual c++ redistributable library for Linux. In DirectXMath.h, it has the following lines
#if defined(_MSC_VER) && (_MSC_VER < 1910) #error DirectX Math requires Visual C++ 2017 or later. #endif
I'm going to need help adding Linux support. Just porting DirectXMath might take a while. I'm going to ask around in the Discord server tomorrow.

from wickedengine.

I was also looking around, but I cannot find any windowing library that has the MIT license. Luckily, it looks like it should be pretty easy to implement our own.

from wickedengine.

Visual Studio can now build the engine for Linux. So far this is only the static library. Future plan is to implement an application such as the Editor for Linux. Cmake is also under consideration.

from wickedengine.

I was also looking around, but I cannot find any windowing library that has the MIT license. Luckily, it looks like it should be pretty easy to implement our own.

It isn't exactly a windowing library, but there's actually a quite complete set of gui controls, implemented in C++, and under the MIT license, in the Godot engine. And the latest code on the master branch has support for actual independent windows, so it nearly qualifies as a windowing system.

It might be worth at least taking a look to see how they do it, if not borrowing code to use as a starting point for your own.

Heck, maybe it would lead to some kind of cooperative, mutual-cross-pollination, synergy, with the Wicked library becoming something that can be dropped into the Godot IDE Tools on Linux. Your respective projects are both under the MIT license, and both prioritize having no/minimal external dependencies. So there's no reason you can't both already borrow at least ideas if not code from each other.

And personally, I greatly prefer the SCons build system over CMake. If you already know Python (heck, even if you don't, it's so easy to read and learn), you can understand and maintain the build scripts easily.

from wickedengine.

Small note that glm is header only too.

from wickedengine.

I'm very much looking forward for this engine to get native Linux support.
There is a need for something to compete with Godot.

from wickedengine.

Actually, Visual Studio (2017 and beyond) supports working with CMake projects out of the box, so no .sln generation step is anymore needed.

Not to mention all the backwards compatibility issues between different cmake versions.

In my opinion the backward compatibility issues between different Visual Studio versions are much, much larger. CMake supports backward compatibility by using policies, so the project files would rarely (if ever) need to be updated only for the reason of fixing "it doesn't open in the latest version of CMake".

from wickedengine.

In practice dealing with backwards compatibility hasn't been hard with CMake

Even more so. In contrast to this, upgrading VS solution to the next version can be a real pain.

from wickedengine.

@turanszkij can you please provide a how to build for linux using vscode?

from wickedengine.

@nonunknown I don't know how to use vscode to build C++. Currently, only Visual Studio is supported. As Cmake is the most requested build system, I'll look into that one for Linux.

from wickedengine.

Vscode supports Cmake, also there is a extension called Easy Cpp which create a project ready to compile with organized folders, then you just modify the MakeFile as you need

from wickedengine.

Linux support is now added, some things are still missing from the Linux build though:

• Audio
• Controller input
• Network functions
• DPI scaling

See the readme for details on how to build for Linux.
The linux automatic build is live, and the packaged version is generated for Ubuntu here: https://github.com/turanszkij/WickedEngine/actions?query=workflow%3ABuild

from wickedengine.

Hello,
I was very deeply impressed with this project so I have decided to try it out, unfortunately I am using Manjaro which sometimes has issues with CMakes find_package and due to that I could not include SDL2 so I have decided to fork this project and fix it such that it also builds under my linux distribution. If you think it's beneficial I would love to merge my changes with this repository since this could potentially save some time to others.

from wickedengine.

Definetely share what you can improve with us and I'll gladly review it as well (remeber to add me as a reviewer so I get notified). For the SDL2 problem, it seems that old distributions use one method and new distributions another one. So I was thinking of doing something like

if (target SDL2::SDL2 found)
    set(SDL2_LIBRARIES SDL2::SDL2)
endif()

and test it on both ubuntu and archlinux (which are the two platforms I know to be incompabile between each other)

The other solution is to include the cmake files of sdl2 inside the project, but I would like to avoid that for the moment.

from wickedengine.

I think the Linux version can be considered pretty well covered, all the main features are now available on Linux. Special thanks to:
@portaloffreedom
@megumumpkin
@amerkoleci
for the making this possible.

To check out how to use this on Linux, visit the readme on how to build the engine, or download a nightly build artifact from github actions for Ubuntu,

from wickedengine.