Comments (1)
I tried to investigate the background compatibility a bit more.
The problem is that often the brightness of the "real frame" (let's call it foreground) is very different from the brightness of the background. As a consequence the border between foreground and background becomes very visible and it may even flicker a bit.
Adjusting the histogram of two images in order to make them "similar" is not easy nor straightforward. Some suggestions may come from this matlab page.
A tentative, naive but easier solution is to adjust both the background and foreground brightnesses in order to let them "meet" at half way. In other words, we adjust both to let them fit each other.
This solution does not adjust the whole histograms but only the overall brightness.
Here is a sniplet for the get_frame function in fake.py, that implements a sort of weighted brightness adjustment:
def get_frame(cap, background, sf):
global rem_mask
global rem
_, frame = cap.read()
cols, rows,_ = background.shape
b_brightness = np.sum(background) / (255 * cols * rows)
brightness = np.sum(frame) / (255 * cols * rows)
target_brightness = (b_brightness + 8*brightness)/9
ratio_f = brightness / target_brightness
ratio_b = b_brightness / target_brightness
#if ratio < 1:
# adjust brightness to get the target brightness
frame = cv2.convertScaleAbs(frame, alpha = 1 / ratio_f, beta = 0)
background = cv2.convertScaleAbs(background, alpha = 1 / ratio_b, beta = 0)
# fetch the mask with retries (the app needs to warmup and we're lazy)
# e v e n t u a l l y c o n s i s t e n t
mask = None
if(rem_mask is None):
rem = 0
while mask is None:
try:
mask = get_mask(frame,sf)
rem_mask = mask
except:
print("mask request failed, retrying")
else:
mask = rem_mask
rem+=1
if(rem>1):
rem_mask = None
# print(rem)
# composite the background
for c in range(frame.shape[2]):
frame[:,:,c] = frame[:,:,c] * mask + background[:,:,c] * (1 - mask)
return frame
Note the calls to convertscale.
Please keep in mind that this is just a proof-of-concept and that histogram adjustment should be performed differently (again, check the matlab page linked above for some examples).
from webcam-virtual-background.
Related Issues (10)
- a few hints... HOT 2
- Specify physical webcam
- node core dumps with illegal instruction HOT 13
- Fixing errors with N-API during installation or at first run
- Fix for slow background updating
- Why not multithreading? HOT 2
- [Suggestion] video as a background HOT 1
- New Tensorflow Runtime?
- Issue running faky.py (with python 3.9.7) / app.js HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webcam-virtual-background.