Export Green Screen in Swift about pixelkit HOT 27 CLOSED

Hi, you can record a video with the RecordPIX

from pixelkit.

I mean this is my code

let content = VideoPIX()
content.load(fileNamed: "superman", withExtension: "mov")

let image = ImagePIX()
image.image = UIImage(named: "city")

let key = ChromaKeyPIX()
key.input = content
key.keyColor = .green

let blend = BlendPIX()
blend.blendingMode = .over
blend.inputA = image
blend.inputB = key

let final: PIX = blend
final.view.frame = view.bounds
view.addSubview(final.view)

how it's possible to render the video and save it with blending?

from pixelkit.

So PixelKit is a realtime based framework. It dose not have a timeline.
Tho it's still possible to export a video!
Depending on the quality you are looking for it can be done in different ways.
All ways make use of the RecordPIX.

The quick and dirty way:

• Start recording: recordPix.startRec(name: "video-name")
• Start the video: content.play()
• Wait with a timer. Get the duration in seconds: content.duration
• Stop the recoding: recordPix.stopRec({ url in })

Let me know if you want a more frame accurate method.

from pixelkit.

Wait with a timer. Get the duration in seconds: content.duration

you mean that need to check video duration with the playing status so I decied to stop recording

my code

        let content = VideoPIX()
        let record = RecordPIX()
        content.load(fileNamed: "bo", withExtension: "mp4")
        let image = ImagePIX()
        image.image = UIImage(named: "bg")
        let key = ChromaKeyPIX()
        key.input = content
        key.keyColor = .green
        let blend = BlendPIX()
        blend.blendMode = .over
        blend.inputA = image
        blend.inputB = key
        let final: PIX = blend
        final.view.frame = view.bounds
        view.addSubview(final.view)
        do {
            try? record.startRec(name: "testMe")
            content.play()
            record.stopRec({ url in
                print("Finish", url)
            }, didError: { error in
                print("Error Record", error)
            })
        } catch {
            print("Not Working")
        }

from pixelkit.

Also, I got an error

PixelKit #413 ERROR [2] RecordPIX "record" >>> Rec Stop A. Writer or Input not found.
Error Record render("Rec Stop A. Writer or Input not found.")
my code

        let content = VideoPIX()
        let record = RecordPIX()
        content.load(fileNamed: "bo", withExtension: "mp4")
        let image = ImagePIX()
        image.image = UIImage(named: "bg")
        let key = ChromaKeyPIX()
        key.input = content
        key.keyColor = .green
        let blend = BlendPIX()
        blend.blendMode = .over
        blend.inputA = image
        blend.inputB = key
        let final: PIX = blend
        final.view.frame = view.bounds
        view.addSubview(final.view)
        
        try? record.startRec(name: "testMe")
        content.play()
        content.listenToDone {
            record.stopRec({ url in
                print("Finish", url)
            }, didError: { error in
                print("Error Record", error)
            })
        }

from pixelkit.

Don't forget to connect the final to the record:

record.input = final

Then setup the timer something like this:

do {
    try record.startRec(name: "testMe")
    content.play()
    RunLoop.current.add(Timer(timeInterval: content.duration!, repeats: false, block: { _ in
        record.stopRec({ url in
            print("Finish", url)
        }, didError: { error in
            print("Error Record", error)
        })
    }), forMode: .common)
} catch {
    print("Not Working")
}

from pixelkit.

The listenToDone way should work too.
Forgot I added that method. Nice find!
Let me know if it works!

from pixelkit.

Thank you for helping but I got the same error even after doing the input thing

PixelKit #413 ERROR [2] RecordPIX "record" >>> Rec Stop A. Writer or Input not found.
Error Record render("Rec Stop A. Writer or Input not found.")

Code

let content = VideoPIX()
        let record = RecordPIX()
        content.load(fileNamed: "bo", withExtension: "mp4")
        let image = ImagePIX()
        image.image = UIImage(named: "bg")
        let key = ChromaKeyPIX()
        key.input = content
        key.keyColor = .green
        let blend = BlendPIX()
        blend.blendMode = .over
        blend.inputA = image
        blend.inputB = key
        let final: PIX = blend
        final.view.frame = view.bounds
        view.addSubview(final.view)
        
        record.input = blend

        try? record.startRec(name: "testMe")
        content.play()
        content.listenToDone {
            record.stopRec({ url in
                print("Finish", url)
            }, didError: { error in
                print("Error Record", error)
            })
        }

from pixelkit.

Ok, so first make sure startRec(name:) worked by either try! or a do catch block.
Then double check listenToDone is not being called to early. That might cause the crash.
If the listenToDone method dose not work (if so I'm sorry), then try the timer method.

from pixelkit.

maybe try to start the record and playback in the video load callback, like this:

content.load(fileNamed: "bo", withExtension: "mp4", done: { resolution in
    try! record.startRec(name: "testMe")
    content.play()
    content.listenToDone {
        record.stopRec({ url in
            print("Finish", url)
        }, didError: { error in
            print("Error Record", error)
        })
    }
})

from pixelkit.

it export but the problem now with the video it's very strange

writer Optional(<AVAssetWriter: 0x600002130770, outputURL = file:///Users/ahmedsafadi/Library/Developer/CoreSimulator/Devices/837DB0F3-5156-4254-8E54-F94029FB9B3E/data/Containers/Data/Application/CBB87496-B419-472E-98B2-775E8FCD208C/Documents/pixelKit/renders/8624EE74-1295-463C-9D69-0910E38F3DCD/testMe.mov, outputFileType = com.apple.quicktime-movie>)
PixelKit #315 ERROR [2] RecordPIX "record" >>> Exported frame failed, writer status: 1.
PixelKit #323 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
Finish file:///Users/ahmedsafadi/Library/Developer/CoreSimulator/Devices/837DB0F3-5156-4254-8E54-F94029FB9B3E/data/Containers/Data/Application/CBB87496-B419-472E-98B2-775E8FCD208C/Documents/pixelKit/renders/8624EE74-1295-463C-9D69-0910E38F3DCD/testMe.mov

Preview looks like

Output looks like

so do you know what the problem could be?

I just want to say that this is amazing efforts from this library

from pixelkit.

I keep getting this error when the recording is start

ixelKit #367 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #370 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #372 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #376 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #380 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #384 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #387 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #392 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #396 WARNING [1] VideoPIX "video" Resource >>> Texture genration failed, using backup method. Error: makeTexture("Pixel Buffer to Metal Texture Converion Faied with result: -6660") LD: "The operation couldn’t be completed. (RenderKit.Texture.TextureError error 4.)"
PixelKit #402 WARNING [1] VideoPIX "video" Resource >>> Texture genr

from pixelkit.

Wow that dose not look good.
So I think you can disregard some of the errors (-6660). I'll check them.
So print double check the resolution of your blend pix with .renderResolution.
The resolution is always derived from inputA.
You can swap inputA and inputB and use .under as the blending mode.
Thanks for the kudos.

from pixelkit.

Let me know if swapping the blend inputs help.

blend.blendMode = .under
blend.inputA = key
blend.inputB = image

from pixelkit.

If you need to flip the video (some known bugs with vertical flipping are not fixed yet), use ._flipY()

from pixelkit.

@Hexagons I did the swap and I guess it's work but the frame per second looks too slow

from pixelkit.

also, I guess the green background for the video didn't go

from pixelkit.

try blend.placement = .aspectFill

from pixelkit.

@Hexagons yes the green hide but the record looks like the fps is soo slow after export the video is very slow show up it like 10 frame per second

from pixelkit.

ah ok, try recordPix.timeSync = false

from pixelkit.

or recordPix.realtime = true

from pixelkit.

recordPix.timeSync = false ( make the video too fast)
recordPix.realtime = true ( make the video slow like it was )

from pixelkit.

So this might be hard to fix with this method. PixelKit is a realtime software, so if your device can't playback in realtime it can't record in realtime. Tho there is a manual override mode. It's a bit more complex to setup. This will render frame accurate video. Let me know if you want try learn the method and I can write a post about it.

from pixelkit.

It involves turning off realtime mode in the engine, seeking the video to each frame and rendering manually by saving the .renderedImage of the PIX.

PixelKit.main.render.engine.renderMode = .manual
PixelKit.main.render.engine.manuallyRender { print("frame rendered") }

from pixelkit.

First, thank you very much for this kind of fast and helping

what I discover that I guess the export setting need to be controlled by the user since you export .mov with a very good setting for good quality, that what makes it slow my guessing, I try on a real device it was faster but not smooth, I recommend creating documentation for export since this will make this library very famous.

and again thank you

Output
test.mov.zip

from pixelkit.

I will work on writing more documentation.
I will also look at adding a quality parameter.
Thank you!

from pixelkit.

I will be watching it more and more, keep this amazing effort, and I will look to see it In future <3

from pixelkit.