Giter VIP home page Giter VIP logo

brutalcoding / aub.ai Goto Github PK

View Code? Open in Web Editor NEW
143.0 9.0 11.0 122.2 MB

AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.

Home Page: https://pub.dev/packages/aub_ai

License: GNU Affero General Public License v3.0

Shell 7.51% Kotlin 0.05% Ruby 2.29% Swift 0.60% Objective-C 0.01% Dart 72.59% CMake 7.81% C++ 8.58% C 0.40% Objective-C++ 0.16%
android dart flutter gen-ai ios ipados linux llamacpp macos mistral-7b

aub.ai's Introduction

3..2..1.. action!

brutalcoding.mp4

Daniel Breedeveld

  • Born & raised in '95 where my squinty eyes were mesmerized by the Netherlands πŸ‡³πŸ‡±.
  • Fav tech stack to work with: Flutter (frontend), Dart (backend) & AI (NLP & CV)
  • Worked for various startups since 2013.

Find Me

Visit my website which will redirect you to my Google Meet at random times where you'll see me work on shady.ai, llama_dart or other projects. No worries, there are no expectations.

Watch me work, ask me anything or feel free to sit in silence while you work on your own things too.

Git Stats

BrutalCoding's GitHub stats

aub.ai's People

Contributors

brutalcoding avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aub.ai's Issues

Phi-2

App crash after third message on phi-2.Q5_K_M.gguf model on iPhone 15 Pro Max
App version: 1.2.0

Fix support for Metal on macOS systems

By default, llama.cpp uses Metal GPU acceleration when running on macOS.

This works fine when I used llama.cpp on its own, but things get complicated when trying to bundle this in a native Mac app.

When bundling the native dynamic library for macOS, and leaving Metal support turned on, I would get the following error when compiling the macOS project:

"ggml_metal_init: found device: Apple M1 Pro
ggml_metal_init: picking default device: Apple M1 Pro
ggml_metal_init: default.metallib not found, loading from source
ggml_metal_init: error: could not use bundle path to find ggml-metal.metal, falling back to trying cwd
ggml_metal_init: loading 'ggml-metal.metal'
ggml_metal_init: error: Error Domain=NSCocoaErrorDomain Code=260 "The file β€œggml-metal.metal” couldn’t be opened because there is no such file." UserInfo={NSFilePath=ggml-metal.metal, NSUnderlyingError=0x1572d8b70 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
llama_new_context_with_model: ggml_metal_init() failed"

I've tried bundling the .metal file within the native macOS project directory, in various locations, to no avail. I do think I can solve this, I just haven't gotten the time to spend more hours on this issue.

As for the time being, the current macOS library doesn't make use of Metal and instead relies on CPU entirely.

DartWorker (46): EXC_BAD_ACCESS

Hello, I've been trying to run the example app on my phone but I seem to be getting an error. I've tried 2 different models to ensure it's not an issue with the model and also attached a screenshot of the line of code that's resulting in an error.

iPhone details:
iPhone 13 Pro
iOS Version 17.2.1

Recordings:
https://github.com/BrutalCoding/aub.ai/assets/146910627/979e497e-fdf3-49fa-b4cb-c310a40044ae
https://github.com/BrutalCoding/aub.ai/assets/146910627/99a20707-3d50-402b-8c06-f5872d925325

Screenshot:

If you require any further information then please let know πŸ™‚

Feedback

Hey I have officially run a 7b model on my Android using your app.

system
You are a helpful assistant. Be concise and helpful. If you don't know the answer to a question, please don't share false information.
user
What's up?
assistant
Not much, just here to assist you with any questions or information you may need. How can I help you today?

It is slower and has less functions (like sampling settings, changing picture, prompt templates) than https://github.com/Mobile-Artificial-Intelligence/maid however I like that you plan to release a standalone plugin that can be used in other projects. For now I will stick with maid but will have an eye on this project.

Andoird libllama.so error

Ran the app on an android device. It seemed to cmake build the llama.cpp. Downloaded GGUF file and loaded it.

When I asked a question I got:

ArgumentError (Invalid argument(s): Failed to load dynamic library 'libllama.so': dlopen failed: library "libllama.so" not found)

Build Failures on Windows: CMake Version Conflicts and Missing prompt_template.dart When Running Example App

Hi there! This plugin looks really cool, seems like you've done a lot of work on this. Very impressive.

I'm trying to test out the example app on Android from my Windows 10 machine. I'm very inexperienced with flutter, but I've been able to test out previous flutter projects on my own devices before which worked fine, but when I try to run flutter run for the example app I get an error related to the cmake version installed.

flutter run
Launching lib\main.dart on Pixel 8 in debug mode...

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':aub_ai:configureCMakeDebug[arm64-v8a]'.
> [CXX1405] error when building with cmake using [Project Path]\src\CMakeLists.txt: -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: [Android SDK Path]\ndk\23.1.7779620\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: [Android SDK Path]\ndk\23.1.7779620\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Configuring incomplete, errors occurred!
  See also "[Project Path]\android\.cxx\Debug\[Build ID]\arm64-v8a\CMakeFiles\CMakeOutput.log".

  C++ build system [configure] failed while executing:
      @echo off
      "[Android SDK Path]\cmake\3.18.1\bin\cmake.exe" ^
        "-H[Project Path]\src" ^
        "-DCMAKE_SYSTEM_NAME=Android" ^
        "-DCMAKE_EXPORT_COMPILE_COMMANDS=ON" ^
        "-DCMAKE_SYSTEM_VERSION=28" ^
        "-DANDROID_PLATFORM=android-28" ^
        "-DANDROID_ABI=arm64-v8a" ^
        "-DCMAKE_ANDROID_ARCH_ABI=arm64-v8a" ^
        "-DANDROID_NDK=[Android SDK Path]\ndk\23.1.7779620" ^
        "-DCMAKE_ANDROID_NDK=[Android SDK Path]\ndk\23.1.7779620" ^
        "-DCMAKE_TOOLCHAIN_FILE=[Android SDK Path]\ndk\23.1.7779620\build\cmake\android.toolchain.cmake" ^
        "-DCMAKE_MAKE_PROGRAM=[Android SDK Path]\cmake\3.18.1\bin\ninja.exe" ^
        "-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=[Project Path]\example\build\aub_ai\intermediates\cxx\Debug\[Build ID]\obj\arm64-v8a" ^
        "-DCMAKE_RUNTIME_OUTPUT_DIRECTORY=[Project Path]\example\build\aub_ai\intermediates\cxx\Debug\[Build ID]\obj\arm64-v8a" ^
        "-DCMAKE_BUILD_TYPE=Debug" ^
        "-B[Project Path]\android\.cxx\Debug\[Build ID]\arm64-v8a" ^
        -GNinja
    from [Project Path]\android
  CMake Warning at [Android SDK Path]\ndk\23.1.7779620\build\cmake\android-legacy.toolchain.cmake:416 (message):
    An old version of CMake is being used that cannot automatically detect
    compiler attributes.  Compiler identification is being bypassed.  Some
    values may be wrong or missing.  Update to CMake 3.19 or newer to use
    CMake's built-in compiler identification.
  Call Stack (most recent call first):
    [Android SDK Path]\ndk\23.1.7779620\build\cmake\android.toolchain.cmake:55 (include)
    [Android SDK Path]\cmake\3.18.1\share\cmake-3.18\Modules\CMakeDetermineSystem.cmake:93 (include)
    CMakeLists.txt:6 (project)


  CMake Error at CMakeLists.txt:9 (add_subdirectory):
    The source directory

      [Project Path]\src\llama.cpp

    does not contain a CMakeLists.txt file.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 14s
Running Gradle task 'assembleDebug'...                             15.5s
Error: Gradle task assembleDebug failed with exit code 1

I've tried installing a newer version of cmake from Android Studio's SDK Manager but even when I uninstall cmake v3.18.1 and installed 3.22.1 it always tries to use 3.18. I figured it's tied to the NDK version then so I tried installing NDK 26.1 and uninstalling 23.1 but then when I run flutter run it automatically installs 23.1 and runs into the same error as above.

I then tried pointing aub_ai in pubspec.yaml to the version of aub_ai to use rather than using the locally cloned repo aub_ai: ^1.0.3 instead of aub_ai \n path: ../ but this gets me to the following error, where it seems prompt_template.dart is missing

flutter run
Launching lib\main.dart on Pixel 8 in debug mode...
lib/main.dart:4:8: Error: Error when reading '[...]/AppData/Local/Pub/Cache/hosted/pub.dev/aub_ai-1.0.3/lib/data/prompt_template.dart': The system cannot find the path specified.

import 'package:aub_ai/data/prompt_template.dart';
       ^
lib/main.dart:41:3: Error: Type 'PromptTemplate' not found.
  PromptTemplate promptTemplate = PromptTemplate.chatML();
  ^^^^^^^^^^^^^^
lib/main.dart:41:3: Error: 'PromptTemplate' isn't a type.
  PromptTemplate promptTemplate = PromptTemplate.chatML();
  ^^^^^^^^^^^^^^
lib/main.dart:41:35: Error: Undefined name 'PromptTemplate'.
  PromptTemplate promptTemplate = PromptTemplate.chatML();
                                  ^^^^^^^^^^^^^^
lib/main.dart:313:5: Error: 'PromptTemplate' isn't a type.
    PromptTemplate? tmpPromptTemplate;
    ^^^^^^^^^^^^^^
lib/main.dart:322:27: Error: The method 'PromptTemplate' isn't defined for the class '_MyAppState'.
 - '_MyAppState' is from 'package:aub_ai_example/main.dart' ('lib/main.dart').
Try correcting the name to the name of an existing method, or defining a method named 'PromptTemplate'.
      tmpPromptTemplate = PromptTemplate(
                          ^^^^^^^^^^^^^^
Target kernel_snapshot failed: Exception

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:compileFlutterBuildDebug'.
> Process 'command '[Flutter SDK Path]\bin\flutter.bat'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 11s
Running Gradle task 'assembleDebug'...                             11.6s
Error: Gradle task assembleDebug failed with exit code 1

Not sure if this problem is related to my flutter setup, so here is the output of flutter doctor -v just in case

flutter doctor -v
[√] Flutter (Channel master, 3.17.0-9.0.pre.13, on Microsoft Windows [Version 10.0.19045.3803], locale en-NL)
    β€’ Flutter version 3.17.0-9.0.pre.13 on channel master at C:\Users\KJW\flutter
    β€’ Upstream repository https://github.com/flutter/flutter.git
    β€’ Framework revision dfa694ec8f (5 weeks ago), 2023-11-14 16:10:17 -0500
    β€’ Engine revision f15b259fe9
    β€’ Dart version 3.3.0 (build 3.3.0-127.0.dev)
    β€’ DevTools version 2.30.0-dev.4

[√] Windows Version (Installed version of Windows is version 10 or higher)

[√] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    β€’ Android SDK at C:\Users\KJW\AppData\Local\Android\Sdk
    β€’ Platform android-34, build-tools 34.0.0
    β€’ ANDROID_HOME = C:\Users\KJW\AppData\Local\Android\Sdk
    β€’ Java binary at: C:\Program Files\Android\Android Studio\jbr\bin\java
    β€’ Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)
    β€’ All Android licenses accepted.

[√] Chrome - develop for the web
    β€’ Chrome at C:\Program Files\Google\Chrome\Application\chrome.exe

[√] Visual Studio - develop Windows apps (Visual Studio Build Tools 2019 16.11.30)
    β€’ Visual Studio at C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools
    β€’ Visual Studio Build Tools 2019 version 16.11.34031.81
    β€’ Windows 10 SDK version 10.0.19041.0

[√] Android Studio (version 2022.3)
    β€’ Android Studio at C:\Program Files\Android\Android Studio
    β€’ Flutter plugin can be installed from:
       https://plugins.jetbrains.com/plugin/9212-flutter
    β€’ Dart plugin can be installed from:
       https://plugins.jetbrains.com/plugin/6351-dart
    β€’ Java version OpenJDK Runtime Environment (build 17.0.6+0-b2043.56-10027231)

[√] VS Code (version 1.85.0)
    β€’ VS Code at C:\Users\KJW\AppData\Local\Programs\Microsoft VS Code
    β€’ Flutter extension version 3.78.0

[√] Connected device (4 available)
    β€’ Pixel 8 (mobile)  β€’ 38031FDJH00MTA β€’ android-arm64  β€’ Android 14 (API 34)
    β€’ Windows (desktop) β€’ windows        β€’ windows-x64    β€’ Microsoft Windows [Version 10.0.19045.3803]
    β€’ Chrome (web)      β€’ chrome         β€’ web-javascript β€’ Google Chrome 119.0.6045.200
    β€’ Edge (web)        β€’ edge           β€’ web-javascript β€’ Microsoft Edge 120.0.2210.77

[√] Network resources
    β€’ All expected network resources are available.

β€’ No issues found!

Thanks for this plugin btw, I really hope I can get this working and can start playing around with it.

Groetjes uit het moederland πŸ‡³πŸ‡±

Android 8.0, better 7.0

Hello, can you make compatibility for lower androids? I havr tried your app on Xiaomi Redmi Note 8 Pro, but woild like to try it on my own phone.

Thank you for understanding and especially for your great job! It is awesome app!

Example app error

cloned repo and did pub get. flutter run

Lexical or Preprocessor Issue (Xcode): 'llama.cpp/llama.h' file not found

Is this something we need to download?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.