Android App load local model about mobillama HOT 3 OPEN

Hi @AnswerZhao,

Thank you for interest in our work.

Please refer to llama.cpp (https://github.com/ggerganov/llama.cpp) and llama_cpp_dart (https://github.com/netdur/llama_cpp_dart) for loading local model.

Thanks,

from mobillama.

You could likely get faster inference using the native android APIs to use NPU but would be a undertaking for sure

from mobillama.

@OmkarThawakar Thank you for the references. I will give it a try.

from mobillama.