Now with ChatGPT API support! See Use with ChatGPT API. (Whisper coming soon!)
This library returns OpenAI API responses as streams only. Non-stream endpoints
like edits
etc. are simply a stream with only one chunk update.
OPENAI_API_KEY
from process.env
.Uses ReadableStream
by default for browser, Edge Runtime, and Node 18+, with
a NodeJS.Readable
version available at openai-streams/node
.
yarn add openai-streams
# -or-
npm i --save openai-streams
await OpenAI(
/** 'completions', 'chat', etc. */
ENDPOINT,
/** max_tokens, temperature, messages, etc. */
PARAMS,
/** apiBase, apiKey, mode, controller, etc */
OPTIONS
);
Set the OPENAI_API_KEY
env variable (or pass the { apiKey }
option).
The library will throw if it cannot find an API key. Your program will load
this at runtime from process.env.OPENAI_API_KEY
by default, but you may
override this with the { apiKey }
option.
IMPORTANT: For security, you should only load this from a process.env
variable.
await OpenAI(
"completions",
{
/* endpoint params */
},
{ apiKey: process.env.MY_SECRET_API_KEY }
);
Call the API via await OpenAI(endpoint, params, options?)
.
The params
type will be inferred based on the endpoint
you provide, i.e.
for the "edits"
endpoint, import('openai').CreateEditRequest
will be
enforced.
Example with raw
streaming mode:
await OpenAI(
"chat",
{
messages: [
/* ... */
],
},
{ mode: "raw" }
);
This will also work in the browser, but you'll need users to paste their OpenAI
key and pass it in via the { apiKey }
option.
import { OpenAI } from "openai-streams";
export default async function handler() {
const stream = await OpenAI("completions", {
model: "text-davinci-003",
prompt: "Write a happy sentence.\n\n",
max_tokens: 100,
});
return new Response(stream);
}
export const config = {
runtime: "edge",
};
If you cannot use an Edge runtime or want to consume Node.js streams for another
reason, use openai-streams/node
:
import type { NextApiRequest, NextApiResponse } from "next";
import { OpenAI } from "openai-streams/node";
export default async function test(_: NextApiRequest, res: NextApiResponse) {
const stream = await OpenAI("completions", {
model: "text-davinci-003",
prompt: "Write a happy sentence.\n\n",
max_tokens: 25,
});
stream.pipe(res);
}
See the example in
example/src/pages/api/hello.ts
.
By default, with mode = "tokens"
, you will receive just the message deltas.
For full events, use mode = "raw"
.
See: https://platform.openai.com/docs/guides/chat/introduction
const stream = await OpenAI("chat", {
model: "gpt-3.5-turbo",
messages: [
{
role: "system",
content: "You are a helpful assistant that translates English to French.",
},
{
role: "user",
content: 'Translate the following English text to French: "Hello world!"',
},
],
});
In tokens
mode, you will just receive the response chunks, which look like this
(separated with newlines for illustration):
Hello
!
How
can
I
assist
you
today
?
Use mode = "raw"
for access to raw events.
for await (const chunk of yieldStream(stream)) { ... }
. We recommend following this
pattern if you find it intuitive.Generated using TypeDoc