clearbuds-ios's People
clearbuds-ios's Issues
Prediction model works real-time on-device or based on cloud?
Dear ClearBuds-Team from U Washington,
thank you for creating such an awesome tool as Clearbuds! We already ordered our CB-PCB. For the time we are waiting, I was taking a closer look on your iOS app. I have a major question about how the prediction model of your prototype works (https://clearbuds.cs.washington.edu/).
In your paper you report a run-time performance of 21.4ms on an iPhone 12 (Table 2 and text). Moreover, you write about your goal of optimizing the Neural Network to run in real-time on smart devices and the benefits of such versus cloud transmission (Chatterjee et al., 2022).
However, in your iOS app, I could not find a reference to your Neural Network in a form of a .tflite model or similar (CB Neural Network is referenced here: https://github.com/vivjay30/clearbuds). Instead, the recorded audio data seems to be pushed on your servers via AWS (ViewController.swift line 526 ff., line 617). Is the filtering process of the recorded audio performed on a cloud instead of the mobile device?
- If yes, does the runtime of 21.4ms correspond to the performance including cloud transmission or another kind of implementation? Can this runtime test still be found in this repo?
- If not, could you please explain, whether and where in this Repo I can find the exact on-device execution of the prediction model?
Thank you very much for your help!
Chatterjee, I., Kim, M., Jayaram, V., Gollakota, S., Kemelmacher, I., Patel, S., & Seitz, S. M. (2022, June). ClearBuds: wireless binaural earbuds for learning-based speech enhancement. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services (pp. 384-396).
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.