Comments (3)
I turned this into a package https://www.npmjs.com/package/mistral-edge
from client-js.
Fixed thanks to https://github.com/haverstack/axios-fetch-adapter/
Instructions
- Install mistralai js client:
npm install @mistralai/mistralai
- Install axios-fetch-adapter:
npm install @haverstack/axios-fetch-adapter
- Edit the file
node_modules/@mistralai/mistralai/src/client.js
:
Import the module:
import fetchAdapter from "@haverstack/axios-fetch-adapter"; //import fetchAdapter (line 3)
Specify the adapter:
...
const response = await axios({
adapter: fetchAdapter, //Use the fetchAdapter (line 47)
method: method,
...
from client-js.
I was also facing this issue and couldn't work out how to get the adapter to work so I just wrote my own client for the rest API.
Might be useful for some people:
interface Message {
role: "user" | "system" | "assistant";
content: string;
}
interface MistralConfig {
model: "mistral-tiny" | "mistral-small" | "mistral-medium";
temperature?: number;
maxTokens?: number;
topP?: number;
randomSeed?: number;
safeMode?: boolean;
}
export async function* streamMistralChat(
messages: Message[],
config: MistralConfig,
apiKey: string | undefined = undefined,
): AsyncGenerator<string, void, void> {
const r = await fetch("https://api.mistral.ai/v1/chat/completions", {
method: "post",
headers: {
Authorization: `Bearer ${apiKey ?? process.env.MISTRAL_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
model: config.model,
messages: messages,
temperature: config.temperature,
max_tokens: config.maxTokens,
top_p: config.topP,
random_seed: config.randomSeed,
stream: true,
safe_prompt: config.safeMode,
}),
});
if (!r.ok) {
console.error(`Error fetching from Mistral API ${r.status}`);
console.error(await r.text());
throw new Error(`Mistral API error, status: ${r.status}`);
}
const reader = r.body.getReader();
const decoder = new TextDecoder();
let buffer: string[] = [];
while (true) {
const { done, value: bytes } = await reader.read();
if (done) {
break;
}
const chunk = decoder.decode(bytes, { stream: true });
for (let i = 0, len = chunk.length; i < len; ++i) {
// We've got newline delimited JSON which has a double newline to separate chunks
const isChunkSeparator = chunk[i] === "\n" && buffer[buffer.length - 1] === "\n";
// Keep buffering unless we've hit the end of a data chunk
if (!isChunkSeparator) {
buffer.push(chunk[i]);
continue;
}
const chunkLine = buffer.join("");
if (chunkLine.trim() === "data: [DONE]") {
break;
}
if (chunkLine.startsWith("data:")) {
const chunkData = chunkLine.substring(6).trim();
if (chunkData !== "[DONE]") {
const chunkObject = JSON.parse(chunkData);
// We just stream the completion text
yield chunkObject.choices[0].delta.content ?? "";
}
} else {
throw Error(`Invalid chunk line encountered: ${chunkLine}`);
}
buffer = [];
}
}
}
Just put your API key in the MISTRAL_API_KEY
env var and run
const response = streamMistralChat([{ role: "user", content: "Hello!" }], {
model: "mistral-tiny",
temperature: 0.7,
// etc
});
for await (const chunk of response) {
// Do whatever you want with the streamed text
console.log(chunk);
}
from client-js.
Related Issues (18)
- Repo name generic when standing alone HOT 3
- SSE Parsing Robustness HOT 1
- Streaming example throws `JSON SyntaxError` HOT 2
- Context Window size HOT 2
- CORS issues with frontend applications HOT 8
- Add test suite for client HOT 2
- TypeScript typings are not picked up
- Missing Typescript definitions HOT 3
- Top-level await
- Client is not working on NodeJS HOT 3
- Typescript types pointing to wrong file HOT 2
- Streaming doesn't work in the browser HOT 3
- Version 001 Refactor for Typescript & Microbundle HOT 1
- Add support for aborting (preferably via AbortSignal) HOT 2
- Contribs welcome? HOT 1
- Feature Request: Usage/Billing API HOT 2
- No stopping conditions HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from client-js.