https://tidb.ai/c/VlARoUrD2E8E

Code Ref:
https://github.com/shadcn-ui/ui/tree/main/apps/www

Style:
https://ui.shadcn.com/docs

menu items: Docs(https://tidb.ai/docs), Demo(https://tidb.ai)
add social media icon/github, mode switcher to top-right, and remove the same one on the bottom-left.

Entrance of /docs should be put at the bottom-center(sticky to bottom with a suitable margin)

We paste images into this GitHub Issue, then retrieve the image URL for use in displaying images in the README.

https://tidb.ai/c/4wo1YnvkJtaR

As there are many stronger LLM that chatgpt4 created, we should have ability to add more embedding method.

favicon

• For self-hosted
• Vercel

https://tidb.ai/conversations/HWgaKxMYl4to -> https://tidb.ai/c/HWgaKxMYl4to

Shorter is better! 🤞

Background

For our next milestone, we'll focus on making our app easier to deploy and use:

Product Refinement:

• Dedicate efforts to polishing the product, ensuring it meets user expectations in usability.

Deployment:

• Add support for deploying on Vercel(maybe and Cloudflare).
• Make it possible to deploy locally using tools like npx for quick testing.

Content Updates:

• Create a welcoming landing page for new visitors.
• Write usage docs/api docs to help users get started and integrate.

Todo list

• #80
• product
  • api & api token mgmt
  • more LLM support(maybe need to re-split/re-index/re-embedding the content of docs)
  • support more data source: word etc.
  • Role-based access control & mgmt(user, admin)
• deployment
  • manually deployment(with npx etc.)
  • docker
  • Vercel
  • Cloudflare(optional)
  • fly.io(optional)
  • LLM/tokens usage statistic
• security
  • cross-domain white list
• content
  • landing page
    • /
    • /showcases
    • /docs/get-started
    • /docs/api
  • README.md

For title&subtitle, we can make the width shorter and wrap the text.

For docs in footer, we can make it one line per document, like this:

Line1: Docs
Line2: Another Link
Line3: Linkeeee4

We should provide a copy button of js scrip somewhere in /settings page, to allow user to copy js and then paste it to his/her blogs, webiste to provide the bottom-right chatbot service.

• Move code from @shczhen to pingcap/tidb.ai
• Add JavaScript tab in /settings
• Make this widget configurable, e.g.: icon, button-text, dialog title, subtitle, example questions etc.
  

3 related(next) questions according to the previous context.

https://tidb.ai/c/dSHuudxqQXUY

Examples Questions wlll also be used in embeddable js scripts

Maybe a tab

src/core/interface.ts

export namespace rag {
  export interface Content<ContentMetadata> {
    content: string[];
    digest: string;
+  lastModifiedAt: Date;
    metadata: ContentMetadata;
  }

  export type ImportSourceTaskResult = {
    enqueue?: Array<{ type: string, url: string }>
    content?: {
      buffer: Buffer
      mime: string
    }
+   incrementalState?: unknown
  }

  export abstract class ImportSourceTaskProcessor<Options> extends Base<Options> {
    abstract support (taskType: string, url: string): boolean;
    abstract process (task: { url: string }): Promise<ImportSourceTaskResult>

+   abstract supportIncremental (taskType: string, url: string): boolean;
+   abstract processIncremental (previousState: unknown): Promise<ImportSourceTaskResult>
  }
}

src/core/db/importSource.ts

export interface ImportSource {
  created_at: Date;
  filter: string | null;
  filter_runtime: string | null;
  id: string;
  type: string;
  url: string;
+ incremental_state: JSON | null;
+ last_scheduled_at: Date | null;
}

Description

Bitdeer is a cloud computing platform that provides computing power for cryptocurrency mining.

In the AI age, Bitdeer also provides multiple managed AI models (For example, llama2, mistral, etc.) API services.

Tasks

• #97
• #107
• #114
• API support streaming mode

Calculate the cost of RAG in calling various third-party services, like:

• Serverless Function (Vercel)
• Storage
• Embedding
• Rerank
• LLM Completion
• Database Query (TiDB Serverless)

Description

After the PR run-llama/LlamaIndexTS#734 merged. LlamaindexTS will support embedding and reranker of Jina AI.

Tasks

• #78
• Create a new "Index" with embedding & reranker of Jina AI for test

Background

As we develop Vector Search in TiDB Serverless, we'll build an app to test how easy it is to use. This app will provide answers to TiDB usage questions on our official websites using our in-progress vector storage in TiDB Serverless.

Todo list

• core rag logic
  • data source management
    • upload pdf/markdown/csv etc.
    • crawl sitemap.xml of a domain
  • ui/ux
    • conversational search
      • chat history mgmt after login
    • embeddable js
• system settings
  • overview: statistics of chats & docs
  • basic info settings
    • logo
    • site name
    • search title & subtitle
    • example questions, max to 4
    • footer links
    • GitHub / Discord etc. social media links top-right
  • oauth configurations
    • GitHub
    • Gmail
  • (pre|post| prompt settings
  • rag loader & spliter configruations: chunk size, overlap etc.
• Use llamaindex-ts as RAG engine
• add widget on the bottom-right of (www|ask).pingcap.com to answer questions about tidb usage / use cases

Tasks

• #98
• Support metadata extraction from HTML label
• Add post filters to filter the ANN search result by metadata #122
• Add config page for the metadata extraction rule
• Experiment: Embedding text combined with the metadata extracted by document

Add footer links to /settings to be configurable too:

https://tidb.ai/c, need to add some apis maybe:

• GET /c, to list all conversations, with paging
• DELETE /c/, with a simple alert

• support multiple indexing config (splitter, embedding, ...) for same document
• UI to compare conversation quality

Langfuse does not provide TypeScript version of Llamaindex support (reference: langfuse/langfuse#2519), so we need to realize it ourself.

• Work around
• Add LangfuseContext
• Add CallbackHandler, reference https://github.com/langfuse/langfuse-js/blob/main/langfuse-langchain/src/callback.ts
• Add @observe decorator

• Social Media of RAG
• Title & Sub Title
• Example Questions( just leave 5 there in /settgings)

It doesn't work currently.

no idea...