Giter VIP home page Giter VIP logo

swift-llm-kit's Introduction

swift-llm-kit

This library is a Work in Progress

Motivation

LLMKit is a library focussing on providing a highly composable tool to interact with Large Language Models.

a LLMKit value can be seen as a function that takes a CompletionChain and returns a new CompletionChain

a CompletionChain is a list of ChatLog each one being the result of a completion call to the LLM

Usage

// Create a openAI llmkit
let llm: LLMKit = .openAI(apiKey:"<your api token>")

// Initialize a completion chain
let initialChain: CompletionChain = .init(systemPrompt: "")
// Get a new completion by calling openAI api
let chain: CompletionChain = try await llm(chain: initialChain, message"Hello!")

// Get the last state of the chat
let chatLog: ChatLog = chain.output
// get the last message
let lastMessage: Model.MessageContent? = chatLog.messages.last

Constructing a llmKit elements

The library is heavily composability oriented. It provides a lot of ways to construct and compose llmKit elements but you can easily implement your own.

Creation

let llm: LLMKit = .openAI()
let llm: LLMKit = .openAI(url: "...")

// misc examples
let llm: LLMKit = .echoing // just repeats the last user's message
let llm: LLMKit = .jsEvaluating // returns the result of the last user's message evaluated as a js expression

// you can even implement your own computation by using the poor man's plugin
let llm: LLMKit = .computing { chatLog in 
    let computedString = ...
    return computedString
}

Modify and Compose

let filtering = llm.filter { message in 
    message.user == .agent
} 
let debugged = llm.debug
let pipe = llm.pipe(llm2) (or llm | llm2)
let fallingback = llm.fallback(llm2)

implementing your own LLMKit

Extensing the library is straightforward.

extension LLMKit { 
    static var myLLMKit: Self { 
        .init { chatLog in
            let computedChatLog = ...
            return computedChatLog
        }
    }
}
...
// getting an instance
let myLLM: LLMKit = .myLLMKit // you can also use all modification / composition tools like .debug, .pipe, etc..

// using it...
let completion = try await myLLM.complete(chatLog, message: "Hello!")

You can take a look at how LLMKit.openAI or LLMKit.echo are implemented. Those are good sources of information.

Streaming

WIP...

// get a stream of chatlog deltas
let chatLogDeltas: AsyncThrowingStream<ChatLog.Delta> = try await llm.stream(chatLog, message: "hello!")

Window management

WIP...

let strat: LLMWindowStrategy = .trim(maxToken: 4096, position: .older, tokenizer: someTokenizer)
let strat: LLMWindowStrategy = .trim(tokenizer: tokenizer) { chatLog, tokenizer in 
    ...
    return computedChatLog
}


let llmWithWindowManagement = llm.withWindowStrategy(strat)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.