Giter VIP home page Giter VIP logo

replicate-javascript's Introduction

Replicate Node.js client

A Node.js client for Replicate. It lets you run models from your Node.js code, and everything else you can do with Replicate's HTTP API.

Important

This library can't interact with Replicate's API directly from a browser. For more information about how to build a web application check out our "Build a website with Next.js" guide.

Supported platforms

You can also use this client library on most serverless platforms, including Cloudflare Workers, Vercel functions, and AWS Lambda.

Installation

Install it from npm:

npm install replicate

Usage

Import or require the package:

// CommonJS (default or using .cjs extension)
const Replicate = require("replicate");

// ESM (where `"module": true` in package.json or using .mjs extension)
import Replicate from "replicate";

Instantiate the client:

const replicate = new Replicate({
  // get your token from https://replicate.com/account/api-tokens
  auth: "my api token", // defaults to process.env.REPLICATE_API_TOKEN
});

Run a model and await the result:

const model = "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478";
const input = {
  prompt: "a 19th century portrait of a raccoon gentleman wearing a suit",
};
const output = await replicate.run(model, { input });
// ['https://replicate.delivery/pbxt/GtQb3Sgve42ZZyVnt8xjquFk9EX5LP0fF68NTIWlgBMUpguQA/out-0.png']

You can also run a model in the background:

let prediction = await replicate.predictions.create({
  version: "27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
  input: {
    prompt: "painting of a cat by andy warhol",
  },
});

Then fetch the prediction result later:

prediction = await replicate.predictions.get(prediction.id);

Or wait for the prediction to finish:

prediction = await replicate.wait(prediction);
console.log(prediction.output);
// ['https://replicate.delivery/pbxt/RoaxeXqhL0xaYyLm6w3bpGwF5RaNBjADukfFnMbhOyeoWBdhA/out-0.png']

To run a model that takes a file input you can pass either a URL to a publicly accessible file on the Internet or a handle to a file on your local device.

const fs = require("node:fs/promises");

// Or when using ESM.
// import fs from "node:fs/promises";

const model = "nightmareai/real-esrgan:42fed1c4974146d4d2414e2be2c5277c7fcf05fcc3a73abf41610695738c1d7b";
const input = {
  image: await fs.readFile("path/to/image.png"),
};
const output = await replicate.run(model, { input });
// ['https://replicate.delivery/mgxm/e7b0e122-9daa-410e-8cde-006c7308ff4d/output.png']

Note

File handle inputs are automatically uploaded to Replicate. See replicate.files.create for more information. The maximum size for uploaded files is 100MiB. To run a model with a larger file as an input, upload the file to your own storage provider and pass a publicly accessible URL.

Webhooks

Webhooks provide real-time updates about your prediction. Specify an endpoint when you create a prediction, and Replicate will send HTTP POST requests to that URL when the prediction is created, updated, and finished.

It is possible to provide a URL to the predictions.create() function that will be requested by Replicate when the prediction status changes. This is an alternative to polling.

To receive webhooks you’ll need a web server. The following example uses Hono, a web standards based server, but this pattern applies to most frameworks.

See example 
import { serve } from '@hono/node-server';
import { Hono } from 'hono';

const app = new Hono();
app.get('/webhooks/replicate', async (c) => {
  // Get the prediction from the request.
  const prediction = await c.req.json();
  console.log(prediction);
  //=> {"id": "xyz", "status": "successful", ... }

  // Acknowledge the webhook.
  c.status(200);
  c.json({ok: true});
}));

serve(app, (info) => {
  console.log(`Listening on http://localhost:${info.port}`)
  //=> Listening on http://localhost:3000
});

Create the prediction passing in the webhook URL to webhook and specify which events you want to receive in webhook_events_filter out of "start", "output", ”logs” and "completed":

const Replicate = require("replicate");
const replicate = new Replicate();

const input = {
    image: "https://replicate.delivery/pbxt/KWDkejqLfER3jrroDTUsSvBWFaHtapPxfg4xxZIqYmfh3zXm/Screenshot%202024-02-28%20at%2022.14.00.png",
    denoising_strength: 0.5,
    instant_id_strength: 0.8
};

const callbackURL = `https://my.app/webhooks/replicate`;
await replicate.predictions.create({
  version: "19deaef633fd44776c82edf39fd60e95a7250b8ececf11a725229dc75a81f9ca",
  input: input,
  webhook: callbackURL,
  webhook_events_filter: ["completed"],
});

// The server will now handle the event and log:
// => {"id": "xyz", "status": "successful", ... }

Verifying webhooks

To prevent unauthorized requests, Replicate signs every webhook and its metadata with a unique key for each user or organization. You can use this signature to verify the webhook indeed comes from Replicate before you process it.

This client includes a validateWebhook convenience function that you can use to validate webhooks.

To validate webhooks:

  1. Check out the webhooks guide to get started.
  2. Retrieve your webhook signing secret and store it in your enviroment.
  3. Update your webhook handler to call validateWebhook(request, secret), where request is an instance of a [web-standard Request object](https://developer.mozilla.org/en-US/docs/Web/API/object, and secret is the signing secret for your environment.

Here's an example of how to validate webhooks using Next.js:

import { NextResponse } from 'next/server';
import { validateWebhook } from 'replicate';

export async function POST(request) {
  const secret = process.env.REPLICATE_WEBHOOK_SIGNING_SECRET;

  if (!secret) {
    console.log("Skipping webhook validation. To validate webhooks, set REPLICATE_WEBHOOK_SIGNING_SECRET")
    const body = await request.json();
    console.log(body);
    return NextResponse.json({ detail: "Webhook received (but not validated)" }, { status: 200 });
  }

  const webhookIsValid = await validateWebhook(request.clone(), secret);

  if (!webhookIsValid) {
    return NextResponse.json({ detail: "Webhook is invalid" }, { status: 401 });
  }

  // process validated webhook here...
  console.log("Webhook is valid!");
  const body = await request.json();
  console.log(body);

  return NextResponse.json({ detail: "Webhook is valid" }, { status: 200 });
}

If your environment doesn't support Request objects, you can pass the required information to validateWebhook directly:

const requestData = {
  id: "123",            // the `Webhook-Id` header
  timestamp: "0123456", // the `Webhook-Timestamp` header
  signature: "xyz",     // the `Webhook-Signature` header
  body: "{...}",        // the request body as a string, ArrayBuffer or ReadableStream
  secret: "shhh",       // the webhook secret, obtained from the `replicate.webhooks.defaul.secret` endpoint
};
const webhookIsValid = await validateWebhook(requestData);

TypeScript

The Replicate constructor and all replicate.* methods are fully typed.

Currently in order to support the module format used by replicate you'll need to set esModuleInterop to true in your tsconfig.json.

API

Constructor

const replicate = new Replicate(options);
name type description
options.auth string Required. API access token
options.userAgent string Identifier of your app. Defaults to replicate-javascript/${packageJSON.version}
options.baseUrl string Defaults to https://api.replicate.com/v1
options.fetch function Fetch function to use. Defaults to globalThis.fetch
options.fileEncodingStrategy string Determines the file encoding strategy to use. Possible values: "default", "upload", or "data-uri". Defaults to "default"

The client makes requests to Replicate's API using fetch. By default, the globalThis.fetch function is used, which is available on Node.js 18 and later, as well as Cloudflare Workers, Vercel Functions, and other environments.

On earlier versions of Node.js and other environments where global fetch isn't available, you can install a fetch function from an external package like cross-fetch and pass it to the fetch option in the constructor.

const Replicate = require("replicate");
const fetch = require("fetch");

// Using ESM:
// import Replicate from "replicate";
// import fetch from "cross-fetch";

const replicate = new Replicate({ fetch });

You can also use the fetch option to add custom behavior to client requests, such as injecting headers or adding log statements.

const customFetch = (url, options) => {
  const headers = options && options.headers ? { ...options.headers } : {};
  headers["X-Custom-Header"] = "some value";

  console.log("fetch", { url, ...options, headers });

  return fetch(url, { ...options, headers });
};

const replicate = new Replicate({ fetch: customFetch });

replicate.run

Run a model and await the result. Unlike replicate.prediction.create, this method returns only the prediction output rather than the entire prediction object.

const output = await replicate.run(identifier, options, progress);
name type description
identifier string Required. The model version identifier in the format {owner}/{name}:{version}, for example stability-ai/sdxl:8beff3369e81422112d93b89ca01426147de542cd4684c244b673b105188fe5f
options.input object Required. An object with the model inputs.
options.wait object Options for waiting for the prediction to finish
options.wait.interval number Polling interval in milliseconds. Defaults to 500
options.webhook string An HTTPS URL for receiving a webhook when the prediction has new output
options.webhook_events_filter string[] An array of events which should trigger webhooks. Allowable values are start, output, logs, and completed
options.signal object An AbortSignal to cancel the prediction
progress function Callback function that receives the prediction object as it's updated. The function is called when the prediction is created, each time it's updated while polling for completion, and when it's completed.

Throws Error if the prediction failed.

Returns Promise<object> which resolves with the output of running the model.

Example:

const model = "stability-ai/sdxl:8beff3369e81422112d93b89ca01426147de542cd4684c244b673b105188fe5f";
const input = { prompt: "a 19th century portrait of a raccoon gentleman wearing a suit" };
const output = await replicate.run(model, { input });

Example that logs progress as the model is running:

const model = "stability-ai/sdxl:8beff3369e81422112d93b89ca01426147de542cd4684c244b673b105188fe5f";
const input = { prompt: "a 19th century portrait of a raccoon gentleman wearing a suit" };
const onProgress = (prediction) => {
   const last_log_line = prediction.logs.split("\n").pop()
   console.log({id: prediction.id, log: last_log_line})
}
const output = await replicate.run(model, { input }, onProgress)

replicate.stream

Run a model and stream its output. Unlike replicate.prediction.create, this method returns only the prediction output rather than the entire prediction object.

for await (const event of replicate.stream(identifier, options)) { /* ... */ }
name type description
identifier string Required. The model version identifier in the format {owner}/{name} or {owner}/{name}:{version}, for example meta/llama-2-70b-chat
options.input object Required. An object with the model inputs.
options.webhook string An HTTPS URL for receiving a webhook when the prediction has new output
options.webhook_events_filter string[] An array of events which should trigger webhooks. Allowable values are start, output, logs, and completed
options.signal object An AbortSignal to cancel the prediction

Throws Error if the prediction failed.

Returns AsyncGenerator<ServerSentEvent> which yields the events of running the model.

Example:

const model = "meta/llama-2-70b-chat";
const options = {
  input: {
    prompt: "Write a poem about machine learning in the style of Mary Oliver.",
  },
  // webhook: "https://smee.io/dMUlmOMkzeyRGjW" // optional
};
const output = [];

for await (const { event, data } of replicate.stream(model, options)) {
  if (event === "output") {
    output.push(data);
  }
}

console.log(output.join("").trim());

Server-sent events

A stream generates server-sent events with the following properties:

name type description
event string The type of event. Possible values are output, logs, error, and done
data string The event data
id string The event id
retry number The number of milliseconds to wait before reconnecting to the server

As the prediction runs, the generator yields output and logs events. If an error occurs, the generator yields an error event with a JSON object containing the error message set to the data property. When the prediction is done, the generator yields a done event with an empty JSON object set to the data property.

Events with the output event type have their toString() method overridden to return the event data as a string. Other event types return an empty string.

replicate.models.get

Get metadata for a public model or a private model that you own.

const response = await replicate.models.get(model_owner, model_name);
name type description
model_owner string Required. The name of the user or organization that owns the model.
model_name string Required. The name of the model. 
{
  "url": "https://replicate.com/replicate/hello-world",
  "owner": "replicate",
  "name": "hello-world",
  "description": "A tiny model that says hello",
  "visibility": "public",
  "github_url": "https://github.com/replicate/cog-examples",
  "paper_url": null,
  "license_url": null,
  "latest_version": {
    /* ... */
  }
}

replicate.models.list

Get a paginated list of all public models.

const response = await replicate.models.list();
{
  "next": null,
  "previous": null,
  "results": [
    {
      "url": "https://replicate.com/replicate/hello-world",
      "owner": "replicate",
      "name": "hello-world",
      "description": "A tiny model that says hello",
      "visibility": "public",
      "github_url": "https://github.com/replicate/cog-examples",
      "paper_url": null,
      "license_url": null,
      "run_count": 5681081,
      "cover_image_url": "...",
      "default_example": {
        /* ... */
      },
      "latest_version": {
        /* ... */
      }
    }
  ]
}

replicate.models.search

Search for public models on Replicate.

const response = await replicate.models.search(query);
name type description
query string Required. The search query string.

replicate.models.create

Create a new public or private model.

const response = await replicate.models.create(model_owner, model_name, options);
name type description
model_owner string Required. The name of the user or organization that will own the model. This must be the same as the user or organization that is making the API request. In other words, the API token used in the request must belong to this user or organization.
model_name string Required. The name of the model. This must be unique among all models owned by the user or organization.
options.visibility string Required. Whether the model should be public or private. A public model can be viewed and run by anyone, whereas a private model can be viewed and run only by the user or organization members that own the model.
options.hardware string Required. The SKU for the hardware used to run the model. Possible values can be found by calling replicate.hardware.list().
options.description string A description of the model.
options.github_url string A URL for the model's source code on GitHub.
options.paper_url string A URL for the model's paper.
options.license_url string A URL for the model's license.
options.cover_image_url string A URL for the model's cover image. This should be an image file.

replicate.hardware.list

List available hardware for running models on Replicate.

const response = await replicate.hardware.list()
[
  {"name": "CPU", "sku": "cpu" },
  {"name": "Nvidia T4 GPU", "sku": "gpu-t4" },
  {"name": "Nvidia A40 GPU", "sku": "gpu-a40-small" },
  {"name": "Nvidia A40 (Large) GPU", "sku": "gpu-a40-large" },
]

replicate.models.versions.list

Get a list of all published versions of a model, including input and output schemas for each version.

const response = await replicate.models.versions.list(model_owner, model_name);
name type description
model_owner string Required. The name of the user or organization that owns the model.
model_name string Required. The name of the model. 
{
  "previous": null,
  "next": null,
  "results": [
    {
      "id": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
      "created_at": "2022-04-26T19:29:04.418669Z",
      "cog_version": "0.3.0",
      "openapi_schema": {
        /* ... */
      }
    },
    {
      "id": "e2e8c39e0f77177381177ba8c4025421ec2d7e7d3c389a9b3d364f8de560024f",
      "created_at": "2022-03-21T13:01:04.418669Z",
      "cog_version": "0.3.0",
      "openapi_schema": {
        /* ... */
      }
    }
  ]
}

replicate.models.versions.get

Get metatadata for a specific version of a model.

const response = await replicate.models.versions.get(model_owner, model_name, version_id);
name type description
model_owner string Required. The name of the user or organization that owns the model.
model_name string Required. The name of the model.
version_id string Required. The model version 
{
  "id": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "created_at": "2022-04-26T19:29:04.418669Z",
  "cog_version": "0.3.0",
  "openapi_schema": {
    /* ... */
  }
}

replicate.collections.get

Get a list of curated model collections. See replicate.com/collections.

const response = await replicate.collections.get(collection_slug);
name type description
collection_slug string Required. The slug of the collection. See http://replicate.com/collections

replicate.predictions.create

Run a model with inputs you provide.

const response = await replicate.predictions.create(options);
name type description
options.version string Required. The model version
options.input object Required. An object with the model's inputs
options.stream boolean Requests a URL for streaming output output
options.webhook string An HTTPS URL for receiving a webhook when the prediction has new output
options.webhook_events_filter string[] You can change which events trigger webhook requests by specifying webhook events (start | output | logs | completed) 
{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "status": "succeeded",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": null,
  "completed_at": null,
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel",
    "stream": "https://streaming.api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq" // Present only if `options.stream` is `true`
  }
}

Streaming

Specify the stream option when creating a prediction to request a URL to receive streaming output using server-sent events (SSE).

If the requested model version supports streaming, then the returned prediction will have a stream entry in its urls property with a URL that you can use to construct an EventSource.

if (prediction && prediction.urls && prediction.urls.stream) {
  const source = new EventSource(prediction.urls.stream, { withCredentials: true });

  source.addEventListener("output", (e) => {
    console.log("output", e.data);
  });

  source.addEventListener("error", (e) => {
    console.error("error", JSON.parse(e.data));
  });

  source.addEventListener("done", (e) => {
    source.close();
    console.log("done", JSON.parse(e.data));
  });
}

A prediction's event stream consists of the following event types:

event format description
output plain text Emitted when the prediction returns new output
error JSON Emitted when the prediction returns an error
done JSON Emitted when the prediction finishes

A done event is emitted when a prediction finishes successfully, is cancelled, or produces an error.

replicate.predictions.get

const response = await replicate.predictions.get(prediction_id);
name type description
prediction_id number Required. The prediction id 
{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel"
  },
  "status": "starting",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": null,
  "completed_at": null
}

replicate.predictions.cancel

Stop a running prediction before it finishes.

const response = await replicate.predictions.cancel(prediction_id);
name type description
prediction_id number Required. The prediction id 
{
  "id": "ufawqhfynnddngldkgtslldrkq",
  "version": "5c7d5dc6dd8bf75c1acaa8565735e7986bc5b66206b55cca93cb72c9bf15ccaa",
  "urls": {
    "get": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq",
    "cancel": "https://api.replicate.com/v1/predictions/ufawqhfynnddngldkgtslldrkq/cancel"
  },
  "status": "canceled",
  "input": {
    "text": "Alice"
  },
  "output": null,
  "error": null,
  "logs": null,
  "metrics": {},
  "created_at": "2022-04-26T22:13:06.224088Z",
  "started_at": "2022-04-26T22:13:06.224088Z",
  "completed_at": "2022-04-26T22:13:06.224088Z"
}

replicate.predictions.list

Get a paginated list of all the predictions you've created.

const response = await replicate.predictions.list();

replicate.predictions.list() takes no arguments.

{
  "previous": null,
  "next": "https://api.replicate.com/v1/predictions?cursor=cD0yMDIyLTAxLTIxKzIzJTNBMTglM0EyNC41MzAzNTclMkIwMCUzQTAw",
  "results": [
    {
      "id": "jpzd7hm5gfcapbfyt4mqytarku",
      "version": "b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05",
      "urls": {
        "get": "https://api.replicate.com/v1/predictions/jpzd7hm5gfcapbfyt4mqytarku",
        "cancel": "https://api.replicate.com/v1/predictions/jpzd7hm5gfcapbfyt4mqytarku/cancel"
      },
      "source": "web",
      "status": "succeeded",
      "created_at": "2022-04-26T20:00:40.658234Z",
      "started_at": "2022-04-26T20:00:84.583803Z",
      "completed_at": "2022-04-26T20:02:27.648305Z"
    }
    /* ... */
  ]
}

replicate.trainings.create

Use the training API to fine-tune language models to make them better at a particular task. To see what language models currently support fine-tuning, check out Replicate's collection of trainable language models.

If you're looking to fine-tune image models, check out Replicate's guide to fine-tuning image models.

const response = await replicate.trainings.create(model_owner, model_name, version_id, options);
name type description
model_owner string Required. The name of the user or organization that owns the model.
model_name string Required. The name of the model.
version string Required. The model version
options.destination string Required. The destination for the trained version in the form {username}/{model_name}
options.input object Required. An object with the model's inputs
options.webhook string An HTTPS URL for receiving a webhook when the training has new output
options.webhook_events_filter string[] You can change which events trigger webhook requests by specifying webhook events (start | output | logs | completed) 
{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "starting",
  "input": {
    "text": "..."
  },
  "output": null,
  "error": null,
  "logs": null,
  "started_at": null,
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": null
}

Warning If you try to fine-tune a model that doesn't support training, you'll get a 400 Bad Request response from the server.

replicate.trainings.get

Get metadata and status of a training.

const response = await replicate.trainings.get(training_id);
name type description
training_id number Required. The training id 
{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "succeeded",
  "input": {
    "data": "..."
    "param1": "..."
  },
  "output": {
    "version": "..."
  },
  "error": null,
  "logs": null,
  "webhook_completed": null,
  "started_at": "2023-03-28T21:48:02.402755Z",
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": "2023-03-28T02:49:48.492023Z"
}

replicate.trainings.cancel

Stop a running training job before it finishes.

const response = await replicate.trainings.cancel(training_id);
name type description
training_id number Required. The training id 
{
  "id": "zz4ibbonubfz7carwiefibzgga",
  "version": "3ae0799123a1fe11f8c89fd99632f843fc5f7a761630160521c4253149754523",
  "status": "canceled",
  "input": {
    "data": "..."
    "param1": "..."
  },
  "output": {
    "version": "..."
  },
  "error": null,
  "logs": null,
  "webhook_completed": null,
  "started_at": "2023-03-28T21:48:02.402755Z",
  "created_at": "2023-03-28T21:47:58.566434Z",
  "completed_at": "2023-03-28T02:49:48.492023Z"
}

replicate.trainings.list

Get a paginated list of all the trainings you've run.

const response = await replicate.trainings.list();

replicate.trainings.list() takes no arguments.

{
  "previous": null,
  "next": "https://api.replicate.com/v1/trainings?cursor=cD0yMDIyLTAxLTIxKzIzJTNBMTglM0EyNC41MzAzNTclMkIwMCUzQTAw",
  "results": [
    {
      "id": "jpzd7hm5gfcapbfyt4mqytarku",
      "version": "b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05",
      "urls": {
        "get": "https://api.replicate.com/v1/trainings/jpzd7hm5gfcapbfyt4mqytarku",
        "cancel": "https://api.replicate.com/v1/trainings/jpzd7hm5gfcapbfyt4mqytarku/cancel"
      },
      "source": "web",
      "status": "succeeded",
      "created_at": "2022-04-26T20:00:40.658234Z",
      "started_at": "2022-04-26T20:00:84.583803Z",
      "completed_at": "2022-04-26T20:02:27.648305Z"
    }
    /* ... */
  ]
}

replicate.deployments.predictions.create

Run a model using your own custom deployment.

Deployments allow you to run a model with a private, fixed API endpoint. You can configure the version of the model, the hardware it runs on, and how it scales. See the deployments guide to learn more and get started.

const response = await replicate.deployments.predictions.create(deployment_owner, deployment_name, options);
name type description
deployment_owner string Required. The name of the user or organization that owns the deployment
deployment_name string Required. The name of the deployment
options.input object Required. An object with the model's inputs
options.webhook string An HTTPS URL for receiving a webhook when the prediction has new output
options.webhook_events_filter string[] You can change which events trigger webhook requests by specifying webhook events (start | output | logs | completed)

Use replicate.wait to wait for a prediction to finish, or replicate.predictions.cancel to cancel a prediction before it finishes.

replicate.deployments.list

List your deployments.

const response = await replicate.deployments.list();
{
  "next": null,
  "previous": null,
  "results": [
    {
      "owner": "acme",
      "name": "my-app-image-generator",
      "current_release": { /* ... */ }
    }
    /* ... */
  ]
}

replicate.deployments.create

Create a new deployment.

const response = await replicate.deployments.create(options);
name type description
options.name string Required. Name of the new deployment
options.model string Required. Name of the model in the format {username}/{model_name}
options.version string Required. ID of the model version
options.hardware string Required. SKU of the hardware to run the deployment on (cpu, gpu-a100, etc.)
options.min_instances number Minimum number of instances to run. Defaults to 0
options.max_instances number Maximum number of instances to scale up to based on traffic. Defaults to 1 
{
  "owner": "acme",
  "name": "my-app-image-generator",
  "current_release": {
    "number": 1,
    "model": "stability-ai/sdxl",
    "version": "da77bc59ee60423279fd632efb4795ab731d9e3ca9705ef3341091fb989b7eaf",
    "created_at": "2024-03-14T11:43:32.049157Z",
    "created_by": {
       "type": "organization",
       "username": "acme",
       "name": "Acme, Inc.",
       "github_url": "https://github.com/replicate"
    },
    "configuration": {
      "hardware": "gpu-a100",
      "min_instances": 1,
      "max_instances": 0
    }
  }
}

replicate.deployments.update

Update an existing deployment.

const response = await replicate.deployments.update(deploymentOwner, deploymentName, options);
name type description
deploymentOwner string Required. Owner of the deployment
deploymentName string Required. Name of the deployment to update
options.model string Name of the model in the format {username}/{model_name}
options.version string ID of the model version
options.hardware string Required. SKU of the hardware to run the deployment on (cpu, gpu-a100, etc.)
options.min_instances number Minimum number of instances to run
options.max_instances number Maximum number of instances to scale up to 
{
  "owner": "acme",
  "name": "my-app-image-generator",
  "current_release": {
    "number": 2,
    "model": "stability-ai/sdxl",
    "version": "39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
    "created_at": "2024-03-14T11:43:32.049157Z",
    "created_by": {
       "type": "organization",
       "username": "acme",
       "name": "Acme, Inc.",
       "github_url": "https://github.com/replicate"
    },
    "configuration": {
      "hardware": "gpu-a100",
      "min_instances": 1,
      "max_instances": 0
    }
  }
}

replicate.files.create

Upload a file to Replicate.

Tip

The client library calls this endpoint automatically to upload the contents of file handles provided as prediction and training inputs. You don't need to call this method directly unless you want more control. For example, you might want to reuse a file across multiple predictions without re-uploading it each time, or you may want to set custom metadata on the file resource.

You can configure how a client handles file handle inputs by setting the fileEncodingStrategy option in the client constructor.

const response = await replicate.files.create(file, metadata);
name type description
file Blob, File, or Buffer Required. The file to upload.
metadata object Optional. User-provided metadata associated with the file. 
{
    "id": "MTQzODcyMDct0YjZkLWE1ZGYtMmRjZTViNWIwOGEyNjNhNS0",
    "name": "photo.webp",
    "content_type": "image/webp",
    "size": 96936,
    "etag": "f211779ff7502705bbf42e9874a17ab3",
    "checksums": {
        "sha256": "7282eb6991fa4f38d80c312dc207d938c156d714c94681623aedac846488e7d3",
        "md5": "f211779ff7502705bbf42e9874a17ab3"
    },
    "metadata": {
        "customer_reference_id": "123"
    },
    "created_at": "2024-06-28T10:16:04.062Z",
    "expires_at": "2024-06-29T10:16:04.062Z",
    "urls": {
        "get": "https://api.replicate.com/v1/files/MTQzODcyMDct0YjZkLWE1ZGYtMmRjZTViNWIwOGEyNjNhNS0"
    }
}

Files uploaded to Replicate using this endpoint expire after 24 hours.

Pass the urls.get property of a file resource to use it as an input when running a model on Replicate. The value of urls.get is opaque, and shouldn't be inferred from other attributes.

The contents of a file are only made accessible to a model running on Replicate, and only when passed as a prediction or training input by the user or organization who created the file.

replicate.files.list

List all files you've uploaded.

const response = await replicate.files.list();

replicate.files.get

Get metadata for a specific file.

const response = await replicate.files.get(file_id);
name type description
file_id string Required. The ID of the file.

replicate.files.delete

Delete a file.

Files uploaded using the replicate.files.create method expire after 24 hours. You can use this method to delete them sooner.

const response = await replicate.files.delete(file_id);
name type description
file_id string Required. The ID of the file.

replicate.paginate

Pass another method as an argument to iterate over results that are spread across multiple pages.

This method is implemented as an async generator function, which you can use in a for loop or iterate over manually.

// iterate over paginated results in a for loop
for await (const page of replicate.paginate(replicate.predictions.list)) {
  /* do something with page of results */
}

// iterate over paginated results one at a time
let paginator = replicate.paginate(replicate.predictions.list);
const page1 = await paginator.next();
const page2 = await paginator.next();
// etc.

replicate.request

Low-level method used by the Replicate client to interact with API endpoints.

const response = await replicate.request(route, parameters);
name type description
options.route string Required. REST API endpoint path.
options.parameters object URL, query, and request body parameters for the given route.

The replicate.request() method is used by the other methods to interact with the Replicate API. You can call this method directly to make other requests to the API.

Troubleshooting

Predictions hanging in Next.js

Next.js App Router adds some extensions to fetch to make it cache responses. To disable this behavior, set the cache option to "no-store" on the Replicate client's fetch object:

replicate = new Replicate({/*...*/})
replicate.fetch = (url, options) => {
  return fetch(url, { ...options, cache: "no-store" });
};

Alternatively you can use Next.js noStore to opt out of caching for your component.

replicate-javascript's People

Contributors

akivab avatar aron avatar bfirsh avatar dependabot[bot] avatar dkundel avatar erbridge avatar fofr avatar gr2m avatar jemgold avatar laoshu133 avatar mattrothenberg avatar mattt avatar nathanstitt avatar picsoung avatar plustokens avatar pwntus avatar robertlong avatar zeke avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

biggeye digitalbore baronrustamov nfcampos jemgold ketianwen kushwaha12345689 viviana31henao usermonk jessrosica plustokens danielrhodes hjb1243 picsoung scrande1 anodromedan1 durand007 warmwinter2 zecreh ethicalsecurity-agency gijomon12 martin258123692 gu-byte rodeanroozbehani linhuiying layandexiaoche ddontouch sherali2398 cryp7oboss synthica-ai thanhxuan842 evagrud hnykda ragesh1292 pauloparfum guzzy971 mena1999 berrgarmorrta parisa-azali wangshanbo gopi8086411 harrison001 cbh123 gsududv poop-jpg synthica-ai vickydhenowal guguzaiaaaa sunilgreengold 74rs nitin-9526 zvalentz cubestds rakhithjk egoist-lugia laoshu133 rdriaz nitingalaxystudio jach777 pawansillu34 aainastudios0783 corbibraga naikashh krphotoexpress mal-stak snilan novan363 alfie71 shishirkumarsarkar hazarcivan ilounzz bijikadal411 ian-nocta akashbagh888 sanxiaguren iqbalhussainme sachidanandas nathanstitt sshader vikasgurjarking siberianjew edbailly smr2023santoquema mdmasum42 serega44kos tetany65 sohelrafiqul yilmazelifnur wahyu52 jordanxavi1418 albasheer2002 vikram4857 johnforte vipguy7 akivab jebiga77 jackandjill23 imiracle moadam16 pengyb2001

replicate-javascript's Issues

this.fetch is not a function

When trying to generate an image, I get this error:

May 11 20:37:11 lemmygrad-art-bot pnpm[42126]: TypeError: this.fetch is not a function
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Replicate.request (/home/insomnia/lemmy-art-bot/node_modules/.pnpm/[email protected]/node_modules/replicate/index.js:147:33)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Replicate.createPrediction (/home/insomnia/lemmy-art-bot/node_modules/.pnpm/[email protected]/node_modules/replicate/lib/pred>
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Replicate.run (/home/insomnia/lemmy-art-bot/node_modules/.pnpm/[email protected]/node_modules/replicate/index.js:103:47)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at generateArt (/home/insomnia/lemmy-art-bot/bot.ts:26:13)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Object.<anonymous> (/home/insomnia/lemmy-art-bot/bot.ts:83:29)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Generator.next (<anonymous>)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at /home/insomnia/lemmy-art-bot/bot.ts:31:71
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at new Promise (<anonymous>)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at __awaiter (/home/insomnia/lemmy-art-bot/bot.ts:27:12)
May 11 20:37:11 lemmygrad-art-bot pnpm[42126]:     at Object.mention [as handle] (/home/insomnia/lemmy-art-bot/bot.ts:78:20)

I have no idea why this is happening. I'm running on Node 18.16.0, so fetch should be defined. I also made sure that my API key is correct. Here is how I am using the library:

const replicate = new Replicate({
  auth: API_KEY,
});

const generateArt = (prompt: string) =>
  replicate.run(
    'ai-forever/kandinsky-2:601eea49d49003e6ea75a11527209c4f510a93e2112c969d548fbb45b9c4f19f',
    {
      input: {
        prompt,
        width: 768,
        height: 768,
        batch_size: 3,
      },
    }
  );

I tried passing in an option for fetch to the constructor, but it didn't show up in the typescript types.

Bug: await replicate.run hangs forever

In the example below if shouldRun = true then the code will hang forever on output = await replicate.run. If you kill the process and set shouldRun = false then the replicate.predictions.list will likely show the prediction as succeeded.

Edit: I just checked the dashboard and not only does it hang it seems to hang on the server side as well. The backend was stuck starting until I killed the process. As soon as I killed the process the dashboard switched to running

Edit2: This seems to be a specific problem with lucataco/vicuna-33b-v1.3:373b7701a9993286eaa9b892277a81be71a1dd981d4f02da876d5864749f81ef I tested the replicate/vicuna-13b:6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b without issue.

Edit3: I passed bad data (model string) to replicate.run and it had the exact same behavior on the client side. No observable server-side effect.

Also, just wondering why vicuna-33b doesn't support streaming?

require("dotenv").config();
const Replicate =  require("replicate");

const replicate = new Replicate({
    auth: process.env.REPLICATE_API_TOKEN',
});

const prompt = `What is the meaning of life?.`

const shouldRun = true

const main = async () => {

    if (shouldRun) {
        const output = await replicate.run(
            "lucataco/vicuna-33b-v1.3:373b7701a9993286eaa9b892277a81be71a1dd981d4f02da876d5864749f81ef",
            {
                input: {
                    prompt,
                    max_new_tokens: 256
                }
            }
        );

        console.log(output);
    }

    const response = await replicate.predictions.list()
    console.log(response)
    
    response.results.map(async (prediction) => {
        if (prediction.status !== "succeeded") {
            return
        }
        await replicate.predictions.get(prediction.id)
            .then((predictionResult) => {
                console.log(prediction.id, predictionResult);
            })
            .catch((err) => {
                console.log(err)
            })
    })

};

main().catch(console.error);

Error "TypeError: Cannot destructure 'options' as it is undefined" thrown when attempting to create a training call to a private model

Description

The trainModel function is throwing a "TypeError: Cannot destructure 'options' as it is undefined" error, even though the options object is passed as an argument to the function.

Code Sample

trainModel = async (filePath, instancePrompt, classPrompt, ...maxTrainSteps) => {
    // Create the training options for the training call
    let inputOptions = this.#createTrainingOptions(filePath, instancePrompt, classPrompt, ...maxTrainSteps);

    // Create a string to store the username of the model
    let callDestination = `${this.#username}/${this.#model}`;

    let options = {
        version: this.#trainerVersion,
        destination: callDestination,
        input: inputOptions,
    };

    // Create a training call to the model
    const response = await this.#replicate.trainings.create(options);

    console.log(response);

} // End trainModel()

The input options arg looks like this:

{
            "instance_prompt": instancePrompt,
            "class_prompt": classPrompt,
            "instance_data": "{file url for training data}",
            "max_train_steps": 500
}

Expected Behavior

The options object should be successfully passed to the trainings.create method without any errors.

Additional Context

The code snippet lacks clear examples or instructions on what parameters should be passed to the trainings.create function. As a result, the API call here is being used as a reference for the required parameters.

Error: API request failed: Not Found

Following the guide from https://replicate.com/docs/get-started/nodejs

Error:

node index.js
/Users/alexandar/Desktop/github/replicate-nodejs/node_modules/replicate/index.js:155
      throw new Error(`API request failed: ${response.statusText}`);
            ^

Error: API request failed: Not Found
    at Replicate.request (/Users/alexandar/Desktop/github/replicate-nodejs/node_modules/replicate/index.js:155:13)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async Replicate.createPrediction (/Users/alexandar/Desktop/github/replicate-nodejs/node_modules/replicate/lib/predictions.js:24:22)
    at async Replicate.run (/Users/alexandar/Desktop/github/replicate-nodejs/node_modules/replicate/index.js:104:24)
    at async file:///Users/alexandar/Desktop/github/replicate-nodejs/index.js:24:16

index.js

import Replicate from "replicate";
import fetch from "cross-fetch"

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
  fetch: fetch
});

// const model = "stability-ai/stable-diffusion:db21e45d3f7023abc2a46ee38a23973f6dce16bb082a930b0c49861f96d1e5bf";
// const input = { prompt: "an astronaut riding a horse on mars, hd, dramatic lighting, detailed" };
// const output = await replicate.run(model, { input });

// const model = "andreasjansson/blip-2:4b32258c42e9efd4288bb9910bc532a69727f9acd26aa08e175713a0a857a608";
// const input = {
//   image: "https://replicate.delivery/pbxt/IJEPmgAlL2zNBNDoRRKFegTEcxnlRhoQxlNjPHSZEy0pSIKn/gg_bridge.jpeg",
//   question: "what body of water does this bridge cross?"
// };
// const output = await replicate.run(model, { input });

const model = "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478";
const input = {
  prompt: "a 19th century portrait of a raccoon gentleman wearing a suit",
};
const output = await replicate.run(model, { input });
// ['https://replicate.delivery/pbxt/GtQb3Sgve42ZZyVnt8xjquFk9EX5LP0fF68NTIWlgBMUpguQA/out-0.png']

// Output:
// san francisco bay

console.log(output);
// ['https://replicate.com/api/models/stability-ai/stable-diffusion/files/50fcac81-865d-499e-81ac-49de0cb79264/out-0.png']`

package.json

{
  "type": "module",
  "dependencies": {
    "cross-fetch": "^3.1.6",
    "replicate": "^0.12.1"
  }
}

num_outputs: is not working

hey, i would like to use the variations of num_outputs. unfortunately i do not get it work. that's the code:

const output = await replicate.run(
"prompthero/openjourney:9936c2001faa2194a261c01381f90e65261879985476014a0a37a334593a05eb",
  {
    input: {
      prompt: "mdjrny-v4 style a highly detailed image of a man on a hill watching a rocket launch in the distance by artgerm, by wlop, by greg rutkowski, volumetric lighting, octane render",
      width: 768,
      height: 512,
      num_outputs: 3
   }
}
);

Format of Uploaded Files

Hi,

I couldn't find any docs anywhere telling what kind of file format cog (and Replicate API) expects to receive in the input of the prompt. The NodeJS example only includes ... and I don't if python adds any extra headers with open function.

const output = await replicate.run(
  "meronym/speaker-transcription:12483517b558629508e03bed77e9c46c6f6e5756715af2b3a4f741f70be45575",
  {
    input: {
      audio: "..."
    }
  }
);
output = replicate.run(
    "meronym/speaker-transcription:12483517b558629508e03bed77e9c46c6f6e5756715af2b3a4f741f70be45575",
    input={"audio": open("path/to/file", "rb")}
)

Could you help me figure out what's the expected way to upload files using this library?

How to catch replicate errors without crashing?

I'm trying to call replicate from a node.js environment. If I have a malformed model input, the replicate SDK throws an error that crashes my server. I'm unable to catch the error and handle it. Here's my code:

 try {
        console.log("try")
        const output = await replicate.run(model, { modelInput })
  } catch (error) {
        console.log("catch")
  }
 console.log('after')

Here's my logs with the error:

  try
(node:15518) ExperimentalWarning: The Fetch API is an experimental feature. This feature could change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
/Users/pseagren/Documents/TITLES/dev/backend/wallflower/api/wallflower-backend/node_modules/replicate/index.js:159
      throw new Error(`API request failed: ${response.statusText}`);
            ^

Error: API request failed: Unprocessable Entity
    at Replicate.request (/Users/pseagren/Documents/TITLES/dev/backend/wallflower/api/wallflower-backend/node_modules/replicate/index.js:159:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.7.0

Oddly, sometimes (but infrequently) my catch block actually catches the error, but usually it doesn't and the server crashes. Am I missing something?

Path to v1.0

  • Unless you have strong reasons to use .ts files, I’d recommend to use .js + .d.ts to have full control over types and source code. I found that it saves a lot of head aches for libraries that are used by other people

    Here is how I would approach the implementation of the JS SDK

  • I’d start out by implementing replicate.request(route, parameters) and its types. It’s the core piece, all other API methods will build upon it. This is the DX we are aiming for (typescript playground)

    screenshot

    • In order to turn the generated OpenAPI types from @replicate/openapi-types into a map of { "GET /v1/collections/{collection_slug}": { /* parameters */ }, ...} you can use the Operation generic type from the upcoming Octokit. That will allow us to build types like this. We can generate this in future but for now I’d just write it out by hand, there are only a few endpoints anyway right now.
    • A reference implementation of the request method is here: https://github.com/octokit/octokit-next.js/tree/main/packages/request
    • Octokit publishes the request method standalone, but I think that’s overkill for replicate right now. I’d build it as part of the initial replicate API client, similar to Octokit’s core package: https://github.com/octokit/octokit-next.js/tree/main/packages/core. Ignore the withDefaults and withPlugins class methods for now, they make things only complicated. Also ignore all the authentication strategy code, replicate only has token authentiation right now.

  • Once the replicate.request() method is fully working, including authentication and types for all known routes, implement the REST API methods for all operations specified in the OpenAPI spec. Again this can be generated in future, but I’d probably just type it out as a first step.

  • Then implement the replicate.paginate() method. Prior art: https://github.com/octokit/plugin-paginate-rest.js/

  • Finally implement the replicate.createAndAwaitPrediction() method.

  • For http mocking in tests, make the Replicate constructor accept a fetch option, then defaults to the system fetch. In tests it can be set to fetch-mock instance instead (example)

  • For Type testing, use tsd. (example)

Cannot find module 'node_modules/replicate/lib/errors.js'

Seeing this error in the server logs from replicate/scribble-diffusion#46

I thought this was maybe a files thing in package.json, but the file is there: https://unpkg.com/browse/[email protected]/lib/

2023-03-13T19:06:25.472Z  2023-03-13T19:06:26.632Z	4c7e6aa0-c7c4-4f91-b02c-e0f56b9f7e38	ERROR	Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/var/task/node_modules/replicate/lib/errors.js' imported from /var/task/node_modules/replicate/lib/ReplicateClient.js
2023-03-13T19:06:25.472Z      at new NodeError (node:internal/errors:400:5)
2023-03-13T19:06:25.472Z      at finalizeResolution (node:internal/modules/esm/resolve:331:11)
2023-03-13T19:06:25.472Z      at moduleResolve (node:internal/modules/esm/resolve:994:10)
2023-03-13T19:06:25.472Z      at moduleResolveWithNodePath (node:internal/modules/esm/resolve:938:12)
2023-03-13T19:06:25.472Z      at defaultResolve (node:internal/modules/esm/resolve:1202:79)
2023-03-13T19:06:25.472Z      at nextResolve (node:internal/modules/esm/loader:163:28)
2023-03-13T19:06:25.472Z      at ESMLoader.resolve (node:internal/modules/esm/loader:842:30)
2023-03-13T19:06:25.472Z      at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
2023-03-13T19:06:25.472Z      at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:77:40)
2023-03-13T19:06:25.472Z      at link (node:internal/modules/esm/module_job:76:36) {
2023-03-13T19:06:25.472Z    code: 'ERR_MODULE_NOT_FOUND'
2023-03-13T19:06:25.472Z  }

webhook support

I would like to be able to specify a webhook URL and webhook_events_filter when creating predictions with the predict() or createPrediction() methods.

Based on the current API, maybe that could work something like this:

const prediction = await replicate
  .model("owner/model:version")
  .createPrediction(inputs, {webhook, webhook_eventsFilter});

// ☝🏼 Don't actually know if this is the signature we want
// Just want to illustrate that we need to pass more than `inputs`.

cc @erbridge @gr2m

prediction failing in the code but finishing in the browser

When I create a prediciton with the code below it returns an erorr in the CLI but the prediciton goes through in the dashboard without isses

Code:

async generate(prompt, negativePrompt, width, height) {

        //make sure that the height and the width fit in the specified restricitons
        if(!(height <= 1024 && width <= 768) || !(width <= 1024 && height <= 768)) throw new Error("please enter a resolution below the maximum resolution of 768x1024");

        //store the location for what model to create a prediction from
        const modelLocation = `${this.#username}/${this.#model}:${this.#modelVersion}`;

        //store the input options for the creation call
        const inputOptions = {
            prompt: prompt,
            negative_prompt: negativePrompt,
            width: width,
            height: height
        }

        //call the model and generate 
        const output = await this.#replicate.run(
            modelLocation,
            {
                input: inputOptions
            }
        );

        //create an object with the image info
        const imageInfo = {
            imageLink: output[0],
            prompt: prompt,
            negativePrompt: negativePrompt,
            height: height,
            width: width
        }

        return imageInfo

    } //end generate()

EDIT: Forgot to add, this is the error I'm getting

FetchError: request to https://api.replicate.com/v1/predictions/wqljaprbzn5q4aiqh6ooy3wieu failed, reason: 
    at ClientRequest.<anonymous> (file:///home/alexa/forknife-v5/node_modules/node-fetch/src/index.js:108:11)
    at ClientRequest.emit (node:events:511:28)
    at TLSSocket.socketErrorListener (node:_http_client:495:9)
    at TLSSocket.emit (node:events:511:28)
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  type: 'system',
  errno: 'ENETUNREACH',
  code: 'ENETUNREACH',
  erroredSysCall: undefined

Documentation and actual implementation differ

I'm trying to use replicate lib to use https://replicate.com/yuval-alaluf/sam/api but I get 422 error. After some time I figured the error:
image
That's from docs
image
And this is from actual request in the browser.
(Spoiler, it's not target_age). Docs and library say it { input: ... } but for some reason official website (and working example) does { inputS: ... }

What am I missing? And is it an error?

Thanks!

Providing an invalid webhook URL to `predictions.create` doesn't return any feedback

How to reproduce:

  await replicate.predictions.create({
    version: "6282abe6a492de4145d7bb601023762212f9ddbbe78278bd6771c8b3b2f2a13b", //vicuna
    input: {
      prompt: "this is definitely developer error  but it would be a nice quality of life improvement to surface the mistake",
    },
    webhook: `htp\\someinvalid,url.corn/webhawks`,
    webhook_events_filter: ['completed', 'logs', 'output', 'start'],
  })

The prediction will successfully run and can be verified in one's dashboard – it just never surfaces any errors or notifications that it couldn't deliver the webhook event.

I think in an ideal world predictions.create would probably:

  • error when provided a structurally invalid URL
  • still function if given a structurally valid but possibly non-responsive endpoint –– but then surface an error/warning somewhere for the user (either at an errors event endpoint or in the dashboard)

Not urgent just a nice to have that I ran in to today :)

Headers is not defined

my code
const replicate = new Replicate({
      auth: process.env.REPLICATE_API_TOKEN,
    });
    const output = await replicate.run(
  "stability-ai/sdxl:a00d0b7dcbb9c3fbb34ba87d2d5b46c56969c84a628bf778a7fdaec30b1b99c5",
  {
    input: {
      prompt: "An astronaut riding a rainbow unicorn"
    }
  }

output always return Headers is not defined

Error when using within React Native

The following example code works fine when ran standalone but throws an error when ran within a React Native project.

import Replicate from "replicate";

const replicate = new Replicate({
  auth: *************
});

const output = await replicate.run(
  "andreasjansson/blip-2:4b32258c42e9efd4288bb9910bc532a69727f9acd26aa08e175713a0a857a608",
  {
    input: {
      image: "https://upload.wikimedia.org/wikipedia/commons/thumb/b/b6/Image_created_with_a_mobile_phone.png/1200px-Image_created_with_a_mobile_phone.png"
    }
  }
);

I have a very simple React Native app, that simply calls this code when you click a button, can provide an example if necessary. You will get the following error:

Possible Unhandled Promise Rejection (id: 1):
TypeError: undefined is not a function

This error is thrown when calling the constructor. The unhandled promise rejection can be dealt with by using try{} catch{} but the TypeError remains and makes the library unusable.

Error when using within React Demo 

import Replicate from "replicate";

const REPLICATE_API_TOKEN = "**************";

async function App() {
  const replicate = new Replicate({
    auth: REPLICATE_API_TOKEN,
  });
  const output = await replicate.run(
    "tstramer/midjourney-diffusion:436b051ebd8f68d23e83d22de5e198e0995357afef113768c20f0b6fcef23c8b",
    {
      input: {
        prompt: "a photo of an astronaut riding a horse on mars",
      },
    }
  );
  console.log(output);
  return (
    <div className="App">

    </div>
  );
}

export default App;

image

`run` is not documented

run is not in the API list in the README. input and it's return value are typed as object, so it's not obvious what they are.

It might be nice to have somewhere in the docs what the model inputs are (they are visible on each model's page).

I'd recommend putting the API information into the typescript source code and generate your API docs from the source code. You can use api-extractor (it's not perfect but a good start).

@zeke told me to file an issue 😇

Negative Prompt

``# Input "Negative Prompt" prediction creation

When running a prediction using the replicate-javascript library using the following code

const output = await replicate.run(
  "alexstan0/stable-diffusion-4:187d2ae3532a4991def03d0abadfe2dab60f8f07e8fe1a9cd13be58300a5d878",
  {
    input: {
      prompt: "a photo of cjw"
    }
  }
);

How would I pass through a negative prompt as seen on the Demo page of the model interface on replicate.com?

Error: Prediction failed: CUDA out of memory. Tried to allocate 5.71 GiB (GPU 0; 14.61 GiB total capacity; 8.08 GiB already allocated; 2.04 GiB free; 11.60 GiB reserved in total by PyTorch) If reserved memory is >>

I am using this library to scale images but when I use this code I get error:

Error: Prediction failed: CUDA out of memory. Tried to allocate 5.71 GiB (GPU 0; 14.61 GiB total capacity; 8.08 GiB already allocated; 2.04 GiB free; 11.60 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

this is my code :

module.exports = async function replicateImage(imagePath) {
    function imageToBase64(imagePath) {
        try {
            const imageBuffer = fs.readFileSync(imagePath);
            const base64 = imageBuffer.toString("base64");
            const mimeType = "image/png";
            return `data:${mimeType};base64,${base64}`;
        } catch (error) {
            console.error('Error converting image to base64:', error);
            return null;
        }
    }
    async function scaleImage(imagePath) {
        const replicate = new Replicate({
            auth: process.env.REPLICATE_API_TOKEN
        });
        const model = 'nightmareai/real-esrgan:42fed1c4974146d4d2414e2be2c5277c7fcf05fcc3a73abf41610695738c1d7b';
        const input = {
            image: imageToBase64(imagePath),
            scale: 8
        };
        try {
            console.log('Start scaling image...');
            const result = await replicate.run(model, { input });
            const outputImagePath = path.join(path.dirname(imagePath), `scaled-${path.basename(imagePath)}.png`);
            const response = await axios.get(result, { responseType: 'arraybuffer' });
            fs.writeFileSync(outputImagePath, response.data);
            console.log('Done scaling image.')
            return outputImagePath;
        } catch (error) {
            console.error('Error scaling image:', error);
        }
    }
    try {
        return await scaleImage(imagePath);
    } catch (error) {
        throw error;
    }
}

Can you help me ?

(node:45154) UnhandledPromiseRejectionWarning: TypeError: this.fetch is not a function

Library bug please fix it or merge #90 maybe will solve it. Thanks

(node:45154) UnhandledPromiseRejectionWarning: TypeError: this.fetch is not a function
    at Replicate.request (/Users/sangvo/Desktop/tool-video/node_modules/replicate/index.js:152:33)
    at Replicate.createPrediction (/Users/sangvo/Desktop/tool-video/node_modules/replicate/lib/predictions.js:26:27)
    at Replicate.run (/Users/sangvo/Desktop/tool-video/node_modules/replicate/index.js:108:47)
    at replicateImage (/Users/sangvo/Desktop/tool-video/helper/ReplicateImage.js:27:40)
    at async /Users/sangvo/Desktop/tool-video/createVideoV2.js:455:40
(Use `node --trace-warnings ...` to show where the warning was created)
(node:45154) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:45154) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Headers is not defined

I tried to run the example code and give this error, but i don't know to fix

node index.js 
/mnt/c/Pitang/StableDiffusion/node_modules/replicate/index.js:189
const headers = new Headers();

ReferenceError: Headers is not defined
    at Replicate.request (/mnt/c/Pitang/StableDiffusion/node_modules/replicate/index.js:189:21)
    at Replicate.createPrediction (/mnt/c/Pitang/StableDiffusion/node_modules/replicate/lib/predictions.js:24:31)
    at Replicate.run (/mnt/c/Pitang/StableDiffusion/node_modules/replicate/index.js:116:45)
    at file:///mnt/c/Pitang/StableDiffusion/index.js:15:32
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:533:24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)```

TypeError: Illegal invocation | Cloudflare workers

Trying to run replicate on cloudflare workers but can't seem to run any model:

const model =
    "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478";
  const input = {
    prompt: "a 19th century portrait of a raccoon gentleman wearing a suit",
  };
  const output = await replicate.run(model, { input });

I keep getting this error:

A promise rejection was handled asynchronously. This warning occurs when attaching a catch handler to a promise after it rejected. (rejection #6)
Trace: TypeError: Illegal invocation
    at Object.apply (index.js:58:24)
    at withAutomaticRetries.shouldRetry.shouldRetry (index.js:507:70)
    at withAutomaticRetries (index.js:129:14)
    at async Replicate2.request (index.js:507:26)
    at async Replicate2.createPrediction (index.js:227:24)
    at async Replicate2.run (index.js:433:26)
    at async index.js:4629:14
    at async index.js:2745:7
    at async index.js:2072:62
    at async jsonError (index.js:4652:12) {
  stack: TypeError: Illegal invocation
    at Object.apply …2072:62
    at async jsonError (index.js:4652:12),
  message: Illegal invocation
}

I've tried overiding fetch but that doesn't seem to solve the issue

Documentation: make it clearer about the node.js version to use

Hello guys :)

I got an issue while trying to work with the replicate library due to the fact I was running my project with node.js 16 without the nodejs experimental flag

I think it should be cool to add something in the library documentation in order to explain it or maybe update the engines in the package.json to force the end user using node.js version > 18

I can create a PR in order to fix that :)

Publish to npm

For now, the package can be installed directly from GitHub because it doesn't require a build:

npm install replicate/replicate-js

We have the replicate package name now on npm. Let's publish it there!

Proposed process:

  • Manually publish with npx np major to publish using https://github.com/sindresorhus/np which takes care of running tests, making GitHub tags, etc. The major argument will bump it to 1.0.0 as a SemVer courtesy to the (probably nonexistent) users of the old replicate package, and to draw a line in the sand.
  • Figure out a plan for doing Semantic Release, etc.

@erbridge @gr2m what do you think?

TypeError: Failed to execute 'fetch' on 'Window': Illegal invocation

I just created a brand new Next.js project that should integrate with "replicate": "^0.11.0", but I keep getting this error when calling the run function.

TypeError: Failed to execute 'fetch' on 'Window': Illegal invocation

This is my function, as simple as it gets:

import Replicate from "replicate";

export async function getOutput(input) {
  const replicate = new Replicate({ auth: process.env.REPLICATE_AUTH_KEY });

  // Tried these, neither worked
  // replicate.fetch = fetch;
  // replicate.fetch = global.fetch;
  // replicate.fetch = globalThis.fetch;
  
  return await replicate.run(process.env.REPLICATE_IDENTIFIER, { input });
}

Does it mean Replicate is supposed to be used server-side only (e.g.: Node.js) and can't be called from the browser?

Edit: I managed to make it work only on Node.js, but only if I set the node-fetch (note: must be version 2 if using CommonJS instead of ES modules) as the fetch option, otherwise I get a ReferenceError: fetch is not defined.

import fetch from 'node-fetch';
import Replicate from "replicate";

export async function getOutput(input) {
  const replicate = new Replicate({
    auth: process.env.REPLICATE_AUTH_KEY,
    fetch: fetch,
  });
  
  return await replicate.run(process.env.REPLICATE_IDENTIFIER, { input });
}

Is there anything in the documentation that I might have missed that explains the right way to go about it?

Also, would appreciate it if you could have the options.fetch documented, as I only found out about it after reading the source code.

"Error: API request failed: Unprocessable Entity" when running with lora-training

Hi! I am trying to train a model by providing my own .zip input using the API. I'm following the instructions here (https://replicate.com/cloneofsimo/lora-training/api) but getting an error:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

const output = await replicate.run(
  "cloneofsimo/lora-training:b2a308762e36ac48d16bfadc03a65493fe6e799f429f7941639a6acec5b276cc",
  {
    input: {
      instance_data: "..."
    }
  }
);

It results with

> node index.js

/node_modules/replicate/index.js:143
      throw new Error(`API request failed: ${response.statusText}`);
            ^

Error: API request failed: Unprocessable Entity
    at Replicate.request (/node_modules/replicate/index.js:143:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Replicate.createPrediction (/node_modules/replicate/lib/predictions.js:24:22)
    at async Replicate.run (/node_modules/replicate/index.js:93:24)
    at async trainModel (file:///index.js:25:20)

Node.js v18.15.0

What I tested

I thought maybe using predictions for lora-training is not the right function. So, I created a new function by changing the this.predictions to this.trainings

  async runTraining(identifier, options) {
    const pattern =
      /^(?<owner>[a-zA-Z0-9-_]+?)\/(?<name>[a-zA-Z0-9-_]+?):(?<version>[0-9a-fA-F]+)$/;
    const match = identifier.match(pattern);

    if (!match) {
      throw new Error(
        'Invalid version. It must be in the format "owner/name:version"'
      );
    }

    const { owner, name, version } = match.groups;
    const training = await this.trainings.create(
      owner,
      name,
      version,
      options,
    );

    if (training.status === 'failed') {
      throw new Error(`Training failed: ${training.error}`);
    }

    return training.output;
  }

Now I am getting Error: API request failed: Bad Request and I guess I am missing options.destination here.

Should {username}/{model_name} be my username or the person owning the model? And does this mean we are uploading our .zip to their account? Thanks!

How does linting work in this project?

When I make changes to the files in this project in VSCode, it makes a bunch of formatting changes that I end up having to revert. I don't care what settings we're using here, I just don't want my editor to trample on them.

This project uses eslint and editorconfig, and it has .vscode/settings.json file.

When I run npm run lint on the main branch, all is well.

When I open a file like index.js or index.test.ts in VSCode and hit Save, it formats a bunch of the code, leading to a huge diff. npm run lint still passes after those changes, so apparently eslint is indifferent about all those formatting preferences that my editor is applying on save?

Forgive my ignorance on the matter, but what am I doing wrong here?

throw an error if the API token is undefined

I was doing this:

import Replicate from "replicate";

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

But REPLICATE_API_TOKEN wasn't actually set in the env. So I was getting "Not found" errors. We should check for this and blow up with an error ahead of time, before making the request.

How can I input an image into the API?

I noticed that the image input has to be in the io.IOBase type. I am currently using node js for the API but I'm not sure how to convert my data into that file type. I have the data's URI (not sure if this is useful). Thanks in advance!

`replicate.run` "sometimes" returns `Not found`

Hi.

I am not sure why and cannot find out a reason, but the following code:

import Replicate from 'replicate';

const MODEL = 'cjwbw/clip-vit-large-patch14';
const VERSION = '566ab1f111e526640c5154e712d4d54961414278f89d36590f1425badc763ecb';

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

export async function predict(input_image: string, target: string): Promise<number> {
  console.log(`Prediction for ${target} ...${input_image.slice(-50)} with ${MODEL}:${VERSION}`);
  const input = { text: `${target} | `, image: input_image };
  const output = await replicate.run(`${MODEL}:${VERSION}`, { input });
  console.log(`Prediction for ...${input_image.slice(-50)}: ${output[0]}`);
  return output[0];
}

and then we do something like:

const promises = [predict(...), predict(...)]
...
const results = await Promise.all(promises)

worked just fine for about 10k of requests. But this night, we started to sometimes get

2023-05-10T09:00:49.435Z  2023-05-10T09:00:49.364Z      8b9a495a-5846-4938-abe2-8f40c9d54985    ERROR   Error while predicting images:  Error: API request failed: Not Found
2023-05-10T09:00:49.435Z      at Replicate.request (/var/task/.next/server/chunks/205.js:133:19)
2023-05-10T09:00:49.435Z      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
2023-05-10T09:00:49.435Z      at async Replicate.wait (/var/task/.next/server/chunks/205.js:180:33)
2023-05-10T09:00:49.435Z      at async Replicate.run (/var/task/.next/server/chunks/205.js:95:28)
2023-05-10T09:00:49.435Z      at async predict (/var/task/.next/server/chunks/512.js:29:20)
2023-05-10T09:00:49.435Z      at async Promise.all (index 1)
2023-05-10T09:00:49.435Z      at async POST (/var/task/.next/server/app/assignments/imagery/api/route.js:618:23)

which is pretty bad... The supper annoying thing is that it works intermittently now - some requests go through, some don't. Also, it seems that the predictions are actually done.

Document how to use modules in your app

I saw somebody today stumble because they didn't have "type": "module" in their package.json.

Problem is the example isn't going to work if you have an existing project that doesn't use modules, so maybe we need to document how to use require() too.

Support process.env.REPLICATE_API_TOKEN for implicit auth?

To use the client today, you have to do this:

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
});

What if you could just do:

const replicate = new Replicate()

..and have it default to process.env.REPLICATE_API_TOKEN, if present?

This is similar to how the Python client works, and I kinda like it. Also makes code examples a lot cleaner.

CORS error when using Replicate package in NextJS app

Description

I'm trying to use the replicate package in my NextJS app, but I am getting a CORS error when making API requests.

Code example

import Replicate from "replicate"

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
})

const predict = async (text: string) => {
  const response = await replicate.run(
    "model_id",
    {
      input: {
        text: text,
      },
    }
  )

  return response
}

predict("example_text")
  .then((result) => console.log(result))
  .catch((error) => console.error(error))

Error from console:

axios.cjs?d475:2337 Refused to set unsafe header "User-Agent"
setRequestHeader @ axios.cjs?d475:2337

Access to XMLHttpRequest at 'https://api.replicate.com/v1/predictions' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

This error occurs on both localhost and production environments.
Please advise on how to resolve this issue.

I am getting axios.create is not a function Error

Maybe somebody else has seen this, hope somebody has some advice

import Replicate from "replicate";

const replicateToken = process.env.REPLICATE_API_TOKEN || "";

const replicate = new Replicate({
  auth: replicateToken,
});

const imageToText = async (imageFile: File) => {
  const output = await replicate.run(
    "methexis-inc/img2prompt:50adaf2d3ad20a6f911a8a9e3ccf777b263b8596fbd2c8fc26e8888f8a0edbb5",
    {
      input: {
        image: imageFile,
      },
    }
  );
  return output;
};

export default imageToText;

I am using a freshly minted CRA application

replicate_1.default is not a constructor

When I run my replicate code like below, I get this error: replicate_1.default is not a constructor.

const options = {
    auth: process.env.REPLICATE_API_KEY
}
const replicate = new Replicate(options)

The code is a basic endpoint function that defines replicate object and returns a string.

I'm using Typescript.

Node.js version: 18.15.11

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.