Giter VIP home page Giter VIP logo

satin's Introduction

Satin - A 3D Graphics Framework built on Apple's Metal

build status Swift Package Manager (SPM) compatible

About ๐Ÿ‘‹

Satin is a 3D graphics framework (inspired by threejs) that helps designers and developers work with Apple's Metal API. Satin provides helpful classes for creating meshes, materials, buffers, uniforms, geometries, pipelines (shaders), compute kernels, and more. Satin's API is rapidly evolving so its best to stick to a tagged version or git commit when using it in production.

Satin makes simple graphics tasks fun and easy to accomplish quickly and complex graphics tasks easier to accomplish without having to write tons of boilerplate code. It does this by providing structure, opinions, and tons of helpful abstractions on Metal to help you get up and rendering / coding in a few minutes. Satin is mostly Swift based, however when performing expensive CPU operations, Satin uses SatinCore, which is written in C (for tasks like geometry generation, triangulation, bounds & computational geometry calculations, and more) to make sure things are as fast as possible. That being said, if you are looking for the most performant way to render things with Metal, please check out Metal-cpp or Metal directly via Objective-C.

Satin is no longer in active development.

Examples โœจ

Requirements โš™๏ธ

  • macOS 10.15.
  • Xcode 11.0.
  • Swift 5.0.

Supported Platforms ๐Ÿ’ป ๐Ÿ“ฑ ๐Ÿ“บ

  • macOS 10.15.
  • iOS 13.0.
  • tvOS 13.0.

Installation

Swift Package Manager

Swift Package Manager is a tool for automating the distribution of Swift code and is integrated into the Swift compiler. Once you have your Swift package set up, adding Satin as a dependency is as easy as adding it to the dependencies value of your Package.swift.

  dependencies: [
      .package(url: "https://github.com/Hi-Rez/Satin.git", .branch("master"))
  ]

Features ๐Ÿ“‹

  • Tons of examples that show how to use the API (2D, 3D, Raycasting, Compute, Exporting, Live Coding, AR, etc).
  • Object, Mesh, InstancedMesh, Material, Shader, Geometry and Renderer classes.
  • PBR Standard & Physical Materials (Based on Disney's PBR Implementation)
  • You can live code shaders ๐Ÿ”ฅ.
  • A couple builtin Materials (BasicColor, BasicTexture, BasicDiffuse, Normal, UV Color, Skybox, MatCap, PBR Standard, PBR Physical, and more).
  • Tons of Geometries (Box, Sphere, IcoSphere, Circle, Cone, Quad, Plane, Capsule, RoundedRect, Text, and more).
  • Cameras (Orthographic, Perspective).
  • 2D & 3D Camera Controllers.
  • Run-time & Dynamic Struct creation via Parameters for Buffers and Uniforms.
  • Metal Shader Compiler (useful when live coding), used in LiveShader (which is used in LiveMaterial).
  • Buffer & Texture Compute Systems for crunching data fast and hard.
  • Generators for BRDF LUT, Image Based Lighting (HDR -> Specular & Diffuse IBL Textures)
  • Fast raycasting via Bounding Volume Hierachies (very helpful to see what you clicked or tapped on).
  • Hooks for custom Metal rendering via Mesh's preDraw, Material's onBind, Buffer & Texture Computes' preCompute, etc
  • Hooks for custom Renderable via Renderable protocol
  • FileWatcher for checking if a resource or shader file has changed.
  • Tons of examples to show how to use the API.
  • Examples that show how to use Satin & ARKit
  • Basic Directional Shadows

Usage ๐Ÿš€

Satin helps to draw things with Metal. To get up and running quickly without tons of boilerplate code and worrying about triple buffering or event (setup, update, resize, key, mouse, touch) callbacks, Satin pairs well with Forge, but can be used without it. The example below shows how to used Forge & Satin together to render a color changing box that looks at a moving point in the scene.

Simple Example:

import SwiftUI
import MetalKit

import Forge
import Satin

// Subclass Forge's Renderer to get triple buffered rendering and
// callbacks for Setup, Update, Draw, Resize and Events
class SimpleRenderer: Forge.Renderer {
    // A Context contains important information that is needed to help compile shaders
    // and ensure we are drawing with the right color and depth pixel formats and sample count

    // Forge's Renderer class provides a MTLDevice and convenience getters for the view's color pixel format,
    // depth pixel format and stencil pixel format, by default a Forge Renderer has depth
    lazy var context = Context(device, sampleCount, colorPixelFormat, depthPixelFormat, stencilPixelFormat)

    // A Satin Renderer handles setting the Content on all the objects in the scene graph
    // and drawing the scene either to a texture or on screen

    // Create a Satin Renderer by passing in a context, scene and camera
    lazy var renderer = Satin.Renderer(context: context)

    // A PerspectiveCamera is used to render the scene using perspective projection
    // All Satin Cameras inherit from Object, so it has
    lazy var camera = PerspectiveCamera(position: [3.0, 3.0, 3.0], near: 0.01, far: 100.0, fov: 45)

    // An Object is just an empty node in Satin's Scene Graph, it can have children and a parent
    // Objects have a position, orientation, scale and label
    lazy var scene: Object = Object("Scene", [boxMesh])

    // Meshes inherit from Object, so they have all the properties an object has.
    // A Mesh has unique properties like geometry, material and rendering properties
    // To create renderable object aka a Mesh, you passing it a Geometry and Material like so
    var boxMesh = Mesh(geometry: BoxGeometry(size: 1.0), material: BasicDiffuseMaterial(0.75))

    // Create a time variable so we can change things in our scene over time
    var time: Float = 0.0

    // Forge calls setup once after it has a valid MTKView (mtkView)
    override func setup() {
        camera.lookAt(.zero)
        // There are many properties you can set on the renderer, this is how to clear to white
        renderer.setClearColor(.one)
    }

    // Forge calls update whenever a new frame is ready to be updated, make scene changes here
    override func update() {
        // We increment our time variable so we can procedurally set the box mesh's orientation and material color
        time += 0.05
        let sx = sin(time)
        let sy = cos(time)

        // Setting a material property done by using the set function, this modifies the material's uniforms
        boxMesh.material?.set("Color", [abs(sx), abs(sy), abs(sx + sy), 1.0])

        // You can manually an object's position, orientation, scale, and localMatrix. Here I'm using a
        // convenience lookAt function to orient the box to face the point passed from its current position
        boxMesh.lookAt([sx, sy, 2.0])
    }

    // Forge calls draw when a new frame is ready to be encoded for drawing
    override func draw(_ view: MTKView, _ commandBuffer: MTLCommandBuffer) {
        guard let renderPassDescriptor = view.currentRenderPassDescriptor else { return }

        // To render a scene into a render pass, just call draw and pass in the render pass descriptor
        // You can also specify a render target and render to a texture instead
        renderer.draw(
            renderPassDescriptor: renderPassDescriptor,
            commandBuffer: commandBuffer,
            scene: scene,
            camera: camera
        )
    }

    // Forge calls resize whenever the view is resized
    override func resize(_ size: (width: Float, height: Float)) {
        // our camera's aspect ratio is set
        camera.aspect = size.width / size.height

        // our renderer's viewport & texture sizes are set
        renderer.resize(size)
        // if you need to render to a custom viewport, you can specify that after the resize call:
        // renderer.viewport = MTLViewport(...)
    }
}

// Using SwiftUI you can use a ForgeView to easily create a MTKView and pass in a Forge.Renderer
struct ContentView: View {
    var body: some View {
        ForgeView(renderer: SimpleRenderer())
    }
}

Credits ๐Ÿ˜…

License ๐ŸŽ“

Satin is released under the MIT license. See LICENSE for details.

satin's People

Contributors

dependabot[bot] avatar oskargroth avatar raheelahmad avatar rezaali avatar syedhali avatar wtholliday avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

satin's Issues

Lighting / Shadows

Implementing lights & shadows into the framework. Let me know if you want to delegate this work!

editor

What editor is the editor for, or is it possible to develop it later?

ARObject does not update ARAnchor transforms

From the Apple documentation on ARAnchorCopying:

Because anchor objects are immutable, ARKit must copy them to make changes from one ARFrame to the next.

Therefore, the transform values here in Satin are never getting updated (they stay the same):

fileprivate class ARObject: Object {
โ€ฆ
  override func update(camera: Camera, viewport: simd_float4) {
    guard let anchor = anchor else { return }
    self.worldMatrix = anchor.transform
    super.update(camera: camera, viewport: viewport)
  }

ARKit appears to require us to use the
ARSessionDelegate.session(_ session: ARSession, didUpdate anchors: [ARAnchor])
method to get updated transform values.

Pods error

Hey Reza

Building according to instructions (using bundler) goes well:

Downloading dependencies
Installing Forge (0.1.0)
Installing Satin (0.2.0)
Generating Pods project
Integrating client project
Pod installation complete! There are 2 dependencies from the Podfile and 2 total pods installed.

...but then on building the project using xcworkspace file, I get 'No such module 'Forge'' in renderer.swift...is that happening to you too?

Internal Error when running PBR-iOS sample

When I run the sample PBR-iOS on an iPhone 12 Pro the following error appear in the console:

Diffuse - Level: 0 - Size: 64
2022-06-07 14:59:04.084563+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.084625+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 1 - Size: 32
2022-06-07 14:59:04.119867+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.120190+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 2 - Size: 16
2022-06-07 14:59:04.181093+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.181260+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Diffuse - Level: 3 - Size: 8
Diffuse - Level: 4 - Size: 4
Diffuse - Level: 5 - Size: 2
Diffuse - Level: 6 - Size: 1

Specular - Level: 0 - Size: 512
2022-06-07 14:59:04.281598+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.281868+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 1 - Size: 256
2022-06-07 14:59:04.315465+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.315655+0200 Latest PBR[644:26983] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 2 - Size: 128
2022-06-07 14:59:04.336547+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
2022-06-07 14:59:04.336725+0200 Latest PBR[644:26981] Execution of the command buffer was aborted due to an error during execution. Internal Error (0000000e:Internal Error)
Specular - Level: 3 - Size: 64
Specular - Level: 4 - Size: 32
Specular - Level: 5 - Size: 16
Specular - Level: 6 - Size: 8
Specular - Level: 7 - Size: 4
Specular - Level: 8 - Size: 2
Specular - Level: 9 - Size: 1

The metallic sphere on the top left seems to have an issue and turns dark:
IMG_EDF29A896CE6-1

It is supposed to look like this

Simulator Screen Shot - iPhone 11 Pro - 2022-06-08 at 12 59 08

Syphon support/example

Is it possible to easily send textures around using Syphon with Satin? On apple sillicon, there are currently no easy options (besides unity) for a creative coding framework that both supports access to both compute shaders and syphon

How to use the of lookAt function?

Hi @rezaali,
I'm playing with the lookAt function and I can't get it to work as I expect it โ€“ meaning I would like the cones to point to each other by the tip. But also at the middle crossing point they "flip". Do you know how to solve this?
thx

Screenshot.2022-10-07.at.00.11.55.mov
import MetalKit
import SwiftUI

import Forge
import Satin

class MKTest1: Forge.Renderer
{
    var context: Context!
    var renderer: Satin.Renderer!
    var camera: PerspectiveCamera!
    var scene: Object = .init("Scene")

    var coneMesh1 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
                                                res: (angular: 8, radial: 1, vertical: 1)),
                         material: BasicDiffuseMaterial(0.75))

    var coneMesh2 = Mesh(geometry: ConeGeometry(size: (radius: 1, height: 3),
                                                res: (angular: 8, radial: 1, vertical: 1)),
                         material: BasicDiffuseMaterial(0.5))

    var time: Float = 0.0

    override func setup()
    {
        context = Context(device, sampleCount, colorPixelFormat, depthPixelFormat, stencilPixelFormat)
        camera = PerspectiveCamera(position: [0, 0, 35], near: 0.01, far: 100.0)
        renderer = Satin.Renderer(context: context, scene: scene, camera: camera)
        renderer.setClearColor([0, 0, 0, 1])

        coneMesh1.position = [0, 0, 0]
        coneMesh2.position = [10, 0, 0]

        scene.add(coneMesh1)
        scene.add(coneMesh2)
    }

    override func update()
    {
        time += 0.01
        let sx = sin(time)
        let sy = cos(time * 5)
        let sz = sin(time)

        coneMesh2.position = [sx * 10, sy * 10, sz * 5]

        coneMesh1.material?.set("Color", [abs(sx), abs(sy), abs(sx + sy), 1.0])

        coneMesh1.lookAt(coneMesh2.position)
        coneMesh2.lookAt(coneMesh1.position)
    }

    override func draw(_ view: MTKView, _ commandBuffer: MTLCommandBuffer)
    {
        guard let renderPassDescriptor = view.currentRenderPassDescriptor else { return }
        renderer.draw(renderPassDescriptor: renderPassDescriptor, commandBuffer: commandBuffer)
    }

    override func resize(_ size: (width: Float, height: Float))
    {
        camera.aspect = size.width / size.height
        renderer.resize(size)
    }
}

struct MKTest1ContentView: View
{
    var body: some View
    {
        ForgeView(renderer: MKTest1())
    }
}

Physical Shader Error

I tried using the default PhysicalMaterial and got this error:

Physical Shader: program_source:985:5: error: no matching function for call to 'pbrDirectLighting'
    pbrDirectLighting(pixel, lights);
    ^~~~~~~~~~~~~~~~~
program_source:673:6: note: candidate function not viable: no known conversion from 'const constant Light' to 'const constant Light *' for 2nd argument; take the address of the argument with &
void pbrDirectLighting(thread PixelInfo &pixel, constant Light *lights)
     ^

There appears to be a missing ampersand on this line in the Physical Shader.

Other calls to pbrDirectLighting within this repo use an ampersand, such as this one and this one.

I cloned Stain, added an ampersand to that line, added it as a local swift package and the error went away.

AR too bright on Satin + SceneKit + Metal fix

AR background is way too bright. Sorry, should be a pull request, but not sure it's perfect.

I think you're meant to change the shader in SatinSceneKitARRenderer to:

    // don't return ycbcrToRGBTransform * ycbcr;
    
    float4 color = ycbcrToRGBTransform * ycbcr;
    // Flatten Color Space to rgb?
    return float4(pow(color.rgb, float3(2,2,2)), color.a); 

Very cool project.

MSL resource files?

Why are MSL resource files not integrated into 'Group' files, but packaged in the form of resource drops? Are there any subsequent situations to be considered in the design? Or is the original intention of such a design in consideration of aspects?

Read Me

This is summed up in Satin.podspec, but:

  • iOS/oSX/tvOS minimum version - iOS is defaulting to 13 right now (which isn't out)
  • Cocoa Pods minimum version
  • Terminal commands to run/install
  • Localized image assets (app icons, etc are causing issues)

I still can't get the example to work (I'm unsure if you need Xcode 11+)

Triangulator sometimes error

Log: Triangulation for ่ฝ› FAILED!

let input = "่ปŠ่ฝ›"
let geo = TextGeometry(text: input, fontName: "PingFangSC-Regular", fontSize: 8)

seams little problem with new index of triangle. happens only in PingFangSC-Light and PingFangSC-Regular, but works well in PingFangSC-Medium and PingFangSC-Semibold.

image
image

PingFangSC-Semibold is right, I don't know why
image

LiveCode editor not opening

I think this might be a Catalina issue:

2020-06-26 10:59:30.385096-0400 LiveCode[18428:4915416] [cas] LaunchedApplication: failed with error -13052 (null){ "ApplicationType"="Foreground", "BundleIdentifierLowerCase"="com.apple.finder", "CFBundleExecutablePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal/Contents/MacOS/Finder", "CFBundleExecutablePathDeviceID"=16777221, "CFBundleExecutablePathINode"=1152921500312409048, "CFBundleIdentifier"="com.apple.finder", "CFBundleName"="Finder", "CFBundlePackageType"="FNDR", "CFBundleSignature"="MACS", "LSASN"=ASN:0x0-0x206206:, "LSBundlePath"="/Users/jzting/Downloads/Satin-master/Example/Source/Shared/LiveCode/Assets/Shaders.metal", "LSBundlePathDeviceID"=16777221, "LSBundlePathINode"=8645686135, "LSDisplayName"="Shaders.metal", "LSExecutableFormat"="LSExecutableMachOFormat", "LSLaunchDLabel"="com.apple.Finder", "LSLaunchedByLaunchServices"=true, "LSLaunchedWithLaunchD"=true, "LSLaunchEventRecordTime"=85367951456787, "LSLaunchTime"=now-ish 2020/06/26 10:59:30, "LSParentASN"=ASN:0x0-0x204204:, "LSWantsToComeForwardAtRegistrationTimeKey"=true, "pid"=5092 }

Issues Installing Dependencies on M1 chip

It looks like there's an issue with the ffi gem not supporting the M1 chip yet, which causes an issue when you initially setup the examples and try to run

bundle exec pod install

I get the following error:

Analyzing dependencies
Pre-downloading: `Forge` from `https://github.com/Hi-Rez/Forge.git`, commit `4d1c7760cb12e129fb027e4e250037a8bed520a3`
/Users/USER/Desktop/code/Satin/Example/vendor/bundle/ruby/2.6.0/gems/ffi-1.15.0/lib/ffi/library.rb:275: [BUG] Bus Error at 0x0000000100cb4000
ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.arm64e-darwin20]

And relevant crash report info:

Process:               ruby [1268]
Path:                  /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
Identifier:            ruby
Version:               145.100.1
Code Type:             ARM-64 (Native)
Parent Process:        zsh [1107]
Responsible:           Terminal [519]
User ID:               501

PlugIn Path:             /Users/USER/Desktop/*/ffi_c.bundle
PlugIn Identifier:       ffi_c.bundle
PlugIn Version:          ??? (0)

Date/Time:             2021-04-29 14:16:58.436 -0500
OS Version:            macOS 11.3 (20E232)
Report Version:        12
Anonymous UUID:        72AB6060-84EC-29E6-9538-A204D5EB2E7D


Time Awake Since Boot: 450 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGABRT)
Exception Codes:       KERN_PROTECTION_FAILURE at 0x0000000100cb4000
Exception Note:        EXC_CORPSE_NOTIFY

I'm going to try the suggestions on the StackOverflow thread, but wanted to document the issue here as well.

Build fails on macOS 10.15

Followed the instructions carefully, but I can't compile and run macOS2D on 10.15.5 with Xcode 11.6

any ideas?

image

@Published and @ObservableObject use

Object gained support for observability with conformance to @ObservedObject and a couple of @Published values, which is great as it opens up for supporting a SwiftUI frontend that displays information about the scene.

But the property wrappers were later changed from @Published to a custom @PublishedDidSet:

@PublishedDidSet open var id: String = UUID().uuidString

This custom property wrapper defeats the purpose of conforming to ObservableObject, as the whole purpose of this class is:

ObservableObject
A type of object with a publisher that emits before the object has changed.

If an object does not have any @Published properties, there is no longer any effect in making it an ObservableObject (and thus breaking the ability to use directly in SwiftUI).

If possible I'd suggest to revert these back to @Published. If there is need to observe @Published values in another lib (Youi?) after they have changed, you can always use an async dispatch on the next runloop to ensure the value has been set:

    cancellable = object.$position.sink { [weak self] position in
         // Position not yet set here, self?.object.position is still old
         DispatchQueue.main.async { 
             // Position has been set here, self?.object.position is new
         }
     }

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.