"
```
Now, make a request to the OpenAI [Chat Completions API](https://platform.openai.com/docs/guides/gpt/chat-completions-api):
```js
export default {
async fetch(request, env, ctx) {
const openai = new OpenAI({
apiKey: env.OPENAI_API_KEY,
});
const url = new URL(request.url);
const message = url.searchParams.get("message");
const messages = [
{
role: "user",
content: message ? message : "What's in the news today?",
},
];
const tools = [
{
type: "function",
function: {
name: "read_website_content",
description: "Read the content on a given website",
parameters: {
type: "object",
properties: {
url: {
type: "string",
description: "The URL to the website to read",
},
},
required: ["url"],
},
},
},
];
const chatCompletion = await openai.chat.completions.create({
model: "gpt-4o-mini",
messages: messages,
tools: tools,
tool_choice: "auto",
});
const assistantMessage = chatCompletion.choices[0].message;
console.log(assistantMessage);
//Later you will continue handling the assistant's response here
return new Response(assistantMessage.content);
},
};
```
Review the arguments you are passing to OpenAI:
- **model**: This is the model you want OpenAI to use for your request. In this case, you are using `gpt-4o-mini`.
- **messages**: This is an array containing all messages that are part of the conversation. Initially you provide a message from the user, and we later add the response from the model. The content of the user message is either the `message` query parameter from the request URL or the default "What's in the news today?".
- **tools**: An array containing the actions available to the AI model. In this example you only have one tool, `read_website_content`, which reads the content on a given website.
- **name**: The name of your function. In this case, it is `read_website_content`.
- **description**: A short description that lets the model know the purpose of the function. This is optional but helps the model know when to select the tool.
- **parameters**: A JSON Schema object which describes the function. In this case we request a response containing an object with the required property `url`.
- **tool_choice**: This argument is technically optional as `auto` is the default. This argument indicates that either a function call or a normal message response can be returned by OpenAI.
## 3. Building your `read_website_content()` function
You will now need to define the `read_website_content` function, which is referenced in the `tools` array. The `read_website_content` function fetches the content of a given URL and extracts the text from `` tags using the `cheerio` library:
Add this code above the `export default` block in your `index.js` file:
```js
async function read_website_content(url) {
console.log("reading website content");
const response = await fetch(url);
const body = await response.text();
let cheerioBody = cheerio.load(body);
const resp = {
website_body: cheerioBody("p").text(),
url: url,
};
return JSON.stringify(resp);
}
```
In this function, you take the URL that you received from OpenAI and use JavaScript's [`Fetch API`](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch) to pull the content of the website and extract the paragraph text. Now we need to determine when to call this function.
## 4. Process the Assistant's Messages
Next, we need to process the response from the OpenAI API to check if it includes any function calls. If a function call is present, you should execute the corresponding function in your Worker. Note that the assistant may request multiple function calls.
Modify the fetch method within the `export default` block as follows:
```js
// ... your previous code ...
if (assistantMessage.tool_calls) {
for (const toolCall of assistantMessage.tool_calls) {
if (toolCall.function.name === "read_website_content") {
const url = JSON.parse(toolCall.function.arguments).url;
const websiteContent = await read_website_content(url);
messages.push({
role: "tool",
tool_call_id: toolCall.id,
name: toolCall.function.name,
content: websiteContent,
});
}
}
const secondChatCompletion = await openai.chat.completions.create({
model: "gpt-4o-mini",
messages: messages,
});
return new Response(secondChatCompletion.choices[0].message.content);
} else {
// this is your existing return statement
return new Response(assistantMessage.content);
}
```
Check if the assistant message contains any function calls by checking for the `tool_calls` property. Because the AI model can call multiple functions by default, you need to loop through any potential function calls and add them to the `messages` array. Each `read_website_content` call will invoke the `read_website_content` function you defined earlier and pass the URL generated by OpenAI as an argument. \`
The `secondChatCompletion` is needed to provide a response informed by the data you retrieved from each function call. Now, the last step is to deploy your Worker.
Test your code by running `npx wrangler dev` and open the provided url in your browser. This will now show you OpenAI’s response using real-time information from the retrieved web data.
## 5. Deploy your Worker application
To deploy your application, run the `npx wrangler deploy` command to deploy your Worker application:
```sh
npx wrangler deploy
```
You can now preview your Worker at `..workers.dev`. Going to this URL will display the response from OpenAI. Optionally, add the `message` URL parameter to write a custom message: for example, `https://..workers.dev/?message=What is the weather in NYC today?`.
## 6. Next steps
Reference the [finished code for this tutorial on GitHub](https://github.com/LoganGrasby/Cloudflare-OpenAI-Functions-Demo/blob/main/src/worker.js).
To continue working with Workers and AI, refer to [the guide on using LangChain and Cloudflare Workers together](https://blog.cloudflare.com/langchain-and-cloudflare/) or [how to build a ChatGPT plugin with Cloudflare Workers](https://blog.cloudflare.com/magic-in-minutes-how-to-build-a-chatgpt-plugin-with-cloudflare-workers/).
If you have any questions, need assistance, or would like to share your project, join the Cloudflare Developer community on [Discord](https://discord.cloudflare.com) to connect with fellow developers and the Cloudflare team.
---
# Connect to a PostgreSQL database with Cloudflare Workers
URL: https://developers.cloudflare.com/workers/tutorials/postgres/
import { Render, PackageManagers, WranglerConfig } from "~/components";
In this tutorial, you will learn how to create a Cloudflare Workers application and connect it to a PostgreSQL database using [TCP Sockets](/workers/runtime-apis/tcp-sockets/) and [Hyperdrive](/hyperdrive/). The Workers application you create in this tutorial will interact with a product database inside of PostgreSQL.
## Prerequisites
To continue:
1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already.
2. Install [`npm`](https://docs.npmjs.com/getting-started).
3. Install [`Node.js`](https://nodejs.org/en/). Use a Node version manager like [Volta](https://volta.sh/) or [nvm](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. [Wrangler](/workers/wrangler/install-and-update/) requires a Node version of `16.17.0` or later.
4. Make sure you have access to a PostgreSQL database.
## 1. Create a Worker application
First, use the [`create-cloudflare` CLI](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare) to create a new Worker application. To do this, open a terminal window and run the following command:
This will prompt you to install the [`create-cloudflare`](https://www.npmjs.com/package/create-cloudflare) package and lead you through a setup wizard.
If you choose to deploy, you will be asked to authenticate (if not logged in already), and your project will be deployed. If you deploy, you can still modify your Worker code and deploy again at the end of this tutorial.
Now, move into the newly created directory:
```sh
cd postgres-tutorial
```
### Enable Node.js compatibility
[Node.js compatibility](/workers/runtime-apis/nodejs/) is required for database drivers, including Postgres.js, and needs to be configured for your Workers project.
## 2. Add the PostgreSQL connection library
To connect to a PostgreSQL database, you will need the `postgres` library. In your Worker application directory, run the following command to install the library:
Make sure you are using `postgres` (`Postgres.js`) version `3.4.4` or higher. `Postgres.js` is compatible with both Pages and Workers.
## 3. Configure the connection to the PostgreSQL database
Choose one of the two methods to connect to your PostgreSQL database:
1. [Use a connection string](#use-a-connection-string).
2. [Set explicit parameters](#set-explicit-parameters).
### Use a connection string
A connection string contains all the information needed to connect to a database. It is a URL that contains the following information:
```
postgresql://username:password@host:port/database
```
Replace `username`, `password`, `host`, `port`, and `database` with the appropriate values for your PostgreSQL database.
Set your connection string as a [secret](/workers/configuration/secrets/) so that it is not stored as plain text. Use [`wrangler secret put`](/workers/wrangler/commands/#secret) with the example variable name `DB_URL`:
```sh
npx wrangler secret put DB_URL
```
```sh output
➜ wrangler secret put DB_URL
-------------------------------------------------------
? Enter a secret value: › ********************
✨ Success! Uploaded secret DB_URL
```
Set your `DB_URL` secret locally in a `.dev.vars` file as documented in [Local Development with Secrets](/workers/configuration/secrets/).
```toml title='.dev.vars'
DB_URL=""
```
### Set explicit parameters
Configure each database parameter as an [environment variable](/workers/configuration/environment-variables/) via the [Cloudflare dashboard](/workers/configuration/environment-variables/#add-environment-variables-via-the-dashboard) or in your Wrangler file. Refer to an example of aWrangler file configuration:
```toml
[vars]
DB_USERNAME = "postgres"
# Set your password by creating a secret so it is not stored as plain text
DB_HOST = "ep-aged-sound-175961.us-east-2.aws.neon.tech"
DB_PORT = "5432"
DB_NAME = "productsdb"
```
To set your password as a [secret](/workers/configuration/secrets/) so that it is not stored as plain text, use [`wrangler secret put`](/workers/wrangler/commands/#secret). `DB_PASSWORD` is an example variable name for this secret to be accessed in your Worker:
```sh
npx wrangler secret put DB_PASSWORD
```
```sh output
-------------------------------------------------------
? Enter a secret value: › ********************
✨ Success! Uploaded secret DB_PASSWORD
```
## 4. Connect to the PostgreSQL database in the Worker
Open your Worker's main file (for example, `worker.ts`) and import the `Client` class from the `pg` library:
```typescript
import postgres from "postgres";
```
In the `fetch` event handler, connect to the PostgreSQL database using your chosen method, either the connection string or the explicit parameters.
### Use a connection string
```typescript
const sql = postgres(env.DB_URL);
```
### Set explicit parameters
```typescript
const sql = postgres({
username: env.DB_USERNAME,
password: env.DB_PASSWORD,
host: env.DB_HOST,
port: env.DB_PORT,
database: env.DB_NAME,
});
```
## 5. Interact with the products database
To demonstrate how to interact with the products database, you will fetch data from the `products` table by querying the table when a request is received.
:::note
If you are following along in your own PostgreSQL instance, set up the `products` using the following SQL `CREATE TABLE` statement. This statement defines the columns and their respective data types for the `products` table:
```sql
CREATE TABLE products (
id SERIAL PRIMARY KEY,
name VARCHAR(255) NOT NULL,
description TEXT,
price DECIMAL(10, 2) NOT NULL
);
```
:::
Replace the existing code in your `worker.ts` file with the following code:
```typescript
import postgres from "postgres";
export default {
async fetch(request, env, ctx): Promise {
const sql = postgres(env.DB_URL, {
// Workers limit the number of concurrent external connections, so be sure to limit
// the size of the local connection pool that postgres.js may establish.
max: 5,
// If you are using array types in your Postgres schema, it is necessary to fetch
// type information to correctly de/serialize them. However, if you are not using
// those, disabling this will save you an extra round-trip every time you connect.
fetch_types: false,
});
// Query the products table
const result = await sql`SELECT * FROM products;`;
// Return the result as JSON
const resp = new Response(JSON.stringify(result), {
headers: { "Content-Type": "application/json" },
});
return resp;
},
} satisfies ExportedHandler;
```
This code establishes a connection to the PostgreSQL database within your Worker application and queries the `products` table, returning the results as a JSON response.
## 6. Deploy your Worker
Run the following command to deploy your Worker:
```sh
npx wrangler deploy
```
Your application is now live and accessible at `..workers.dev`.
After deploying, you can interact with your PostgreSQL products database using your Cloudflare Worker. Whenever a request is made to your Worker's URL, it will fetch data from the `products` table and return it as a JSON response. You can modify the query as needed to retrieve the desired data from your products database.
## 7. Insert a new row into the products database
To insert a new row into the `products` table, create a new API endpoint in your Worker that handles a `POST` request. When a `POST` request is received with a JSON payload, the Worker will insert a new row into the `products` table with the provided data.
Assume the `products` table has the following columns: `id`, `name`, `description`, and `price`.
Add the following code snippet inside the `fetch` event handler in your `worker.ts` file, before the existing query code:
```typescript {9-32}
import postgres from "postgres";
export default {
async fetch(request, env, ctx): Promise {
const sql = postgres(env.DB_URL);
const url = new URL(request.url);
if (request.method === "POST" && url.pathname === "/products") {
// Parse the request's JSON payload
const productData = await request.json();
// Insert the new product into the database
const values = {
name: productData.name,
description: productData.description,
price: productData.price,
};
const insertResult = await sql`
INSERT INTO products ${sql(values)}
RETURNING *
`;
// Return the inserted row as JSON
const insertResp = new Response(JSON.stringify(insertResult), {
headers: { "Content-Type": "application/json" },
});
// Clean up the client
return insertResp;
}
// Query the products table
const result = await sql`SELECT * FROM products;`;
// Return the result as JSON
const resp = new Response(JSON.stringify(result), {
headers: { "Content-Type": "application/json" },
});
return resp;
},
} satisfies ExportedHandler;
```
This code snippet does the following:
1. Checks if the request is a `POST` request and the URL path is `/products`.
2. Parses the JSON payload from the request.
3. Constructs an `INSERT` SQL query using the provided product data.
4. Executes the query, inserting the new row into the `products` table.
5. Returns the inserted row as a JSON response.
Now, when you send a `POST` request to your Worker's URL with the `/products` path and a JSON payload, the Worker will insert a new row into the `products` table with the provided data. When a request to `/` is made, the Worker will return all products in the database.
After making these changes, deploy the Worker again by running:
```sh
npx wrangler deploy
```
You can now use your Cloudflare Worker to insert new rows into the `products` table. To test this functionality, send a `POST` request to your Worker's URL with the `/products` path, along with a JSON payload containing the new product data:
```json
{
"name": "Sample Product",
"description": "This is a sample product",
"price": 19.99
}
```
You have successfully created a Cloudflare Worker that connects to a PostgreSQL database and handles fetching data and inserting new rows into a products table.
## 8. Use Hyperdrive to accelerate queries
Create a Hyperdrive configuration using the connection string for your PostgreSQL database.
```bash
npx wrangler hyperdrive create --connection-string="postgres://user:password@HOSTNAME_OR_IP_ADDRESS:PORT/database_name"
```
This command outputs the Hyperdrive configuration `id` that will be used for your Hyperdrive [binding](/workers/runtime-apis/bindings/). Set up your binding by specifying the `id` in the Wrangler file.
```toml {7-9}
name = "hyperdrive-example"
main = "src/index.ts"
compatibility_date = "2024-08-21"
compatibility_flags = ["nodejs_compat"]
# Pasted from the output of `wrangler hyperdrive create --connection-string=[...]` above.
[[hyperdrive]]
binding = "HYPERDRIVE"
id = ""
```
Create the types for your Hyperdrive binding using the following command:
```bash
npx wrangler types
```
Replace your existing connection string in your Worker code with the Hyperdrive connection string.
```js {3-3}
export default {
async fetch(request, env, ctx): Promise {
const sql = postgres(env.HYPERDRIVE.connectionString)
const url = new URL(request.url);
//rest of the routes and database queries
},
} satisfies ExportedHandler;
```
## 9. Redeploy your Worker
Run the following command to deploy your Worker:
```sh
npx wrangler deploy
```
Your Worker application is now live and accessible at `..workers.dev`, using Hyperdrive. Hyperdrive accelerates database queries by pooling your connections and caching your requests across the globe.
## Next steps
To build more with databases and Workers, refer to [Tutorials](/workers/tutorials) and explore the [Databases documentation](/workers/databases).
If you have any questions, need assistance, or would like to share your project, join the Cloudflare Developer community on [Discord](https://discord.cloudflare.com) to connect with fellow developers and the Cloudflare team.
---
# Send Emails With Postmark
URL: https://developers.cloudflare.com/workers/tutorials/send-emails-with-postmark/
In this tutorial, you will learn how to send transactional emails from Workers using [Postmark](https://postmarkapp.com/). At the end of this tutorial, you’ll be able to:
- Create a Worker to send emails.
- Sign up and add a Cloudflare domain to Postmark.
- Send emails from your Worker using Postmark.
- Store API keys securely with secrets.
## Prerequisites
To continue with this tutorial, you’ll need:
- A [Cloudflare account](https://dash.cloudflare.com/sign-up/workers-and-pages), if you don’t already have one.
- A [registered](/registrar/get-started/register-domain/) domain.
- Installed [npm](https://docs.npmjs.com/getting-started).
- A [Postmark account](https://account.postmarkapp.com/sign_up).
## Create a Worker project
Start by using [C3](/pages/get-started/c3/) to create a Worker project in the command line, then, answer the prompts:
```sh
npm create cloudflare@latest
```
Alternatively, you can use CLI arguments to speed things up:
```sh
npm create cloudflare@latest email-with-postmark -- --type=hello-world --ts=false --git=true --deploy=false
```
This creates a simple hello-world Worker having the following content:
```js
export default {
async fetch(request, env, ctx) {
return new Response("Hello World!");
},
};
```
## Add your domain to Postmark
If you don’t already have a Postmark account, you can sign up for a [free account here](https://account.postmarkapp.com/sign_up). After signing up, check your inbox for a link to confirm your sender signature. This verifies and enables you to send emails from your registered email address.
To enable email sending from other addresses on your domain, navigate to `Sender Signatures` on the Postmark dashboard, `Add Domain or Signature` > `Add Domain`, then type in your domain and click on `Verify Domain`.
Next, you’re presented with a list of DNS records to add to your Cloudflare domain. On your Cloudflare dashboard, select the domain you entered earlier and navigate to `DNS` > `Records`. Copy/paste the DNS records (DKIM, and Return-Path) from Postmark to your Cloudflare domain.
:::note
If you need more help adding DNS records in Cloudflare, refer to [Manage DNS records](/dns/manage-dns-records/how-to/create-dns-records/).
:::
When that’s done, head back to Postmark and click on the `Verify` buttons. If all records are properly configured, your domain status should be updated to `Verified`.
To grab your API token, navigate to the `Servers` tab, then `My First Server` > `API Tokens`, then copy your API key to a safe place.
## Send emails from your Worker
The final step is putting it all together in a Worker. In your Worker, make a post request with `fetch` to Postmark’s email API and include your token and message body:
:::note
[Postmark’s JavaScript library](https://www.npmjs.com/package/postmark) is currently not supported on Workers. Use the [email API](https://postmarkapp.com/developer/user-guide/send-email-with-api) instead.
:::
```jsx
export default {
async fetch(request, env, ctx) {
return await fetch("https://api.postmarkapp.com/email", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-Postmark-Server-Token": "your_postmark_api_token_here",
},
body: JSON.stringify({
From: "hello@example.com",
To: "someone@example.com",
Subject: "Hello World",
HtmlBody: "Hello from Workers
",
}),
});
},
};
```
To test your code locally, run the following command and navigate to [http://localhost:8787/](http://localhost:8787/) in a browser:
```sh
npm start
```
Deploy your Worker with `npm run deploy`.
## Move API token to Secrets
Sensitive information such as API keys and token should always be stored in secrets. All secrets are encrypted to add an extra layer of protection. That said, it’s a good idea to move your API token to a secret and access it from the environment of your Worker.
To add secrets for local development, create a `.dev.vars` file which works exactly like a `.env` file:
```txt
POSTMARK_API_TOKEN=your_postmark_api_token_here
```
Also ensure the secret is added to your deployed worker by running:
```sh title="Add secret to deployed Worker"
npx wrangler secret put POSTMARK_API_TOKEN
```
The added secret can be accessed on via the `env` parameter passed to your Worker’s fetch event handler:
```jsx
export default {
async fetch(request, env, ctx) {
return await fetch("https://api.postmarkapp.com/email", {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-Postmark-Server-Token": env.POSTMARK_API_TOKEN,
},
body: JSON.stringify({
From: "hello@example.com",
To: "someone@example.com",
Subject: "Hello World",
HtmlBody: "Hello from Workers
",
}),
});
},
};
```
And finally, deploy this update with `npm run deploy`.
## Related resources
- [Storing API keys and tokens with Secrets](/workers/configuration/secrets/).
- [Transferring your domain to Cloudflare](/registrar/get-started/transfer-domain-to-cloudflare/).
- [Send emails from Workers](/email-routing/email-workers/send-email-workers/)
---
# Send Emails With Resend
URL: https://developers.cloudflare.com/workers/tutorials/send-emails-with-resend/
In this tutorial, you will learn how to send transactional emails from Workers using [Resend](https://resend.com/). At the end of this tutorial, you’ll be able to:
- Create a Worker to send emails.
- Sign up and add a Cloudflare domain to Resend.
- Send emails from your Worker using Resend.
- Store API keys securely with secrets.
## Prerequisites
To continue with this tutorial, you’ll need:
- A [Cloudflare account](https://dash.cloudflare.com/sign-up/workers-and-pages), if you don’t already have one.
- A [registered](/registrar/get-started/register-domain/) domain.
- Installed [npm](https://docs.npmjs.com/getting-started).
- A [Resend account](https://resend.com/signup).
## Create a Worker project
Start by using [C3](/pages/get-started/c3/) to create a Worker project in the command line, then, answer the prompts:
```sh
npm create cloudflare@latest
```
Alternatively, you can use CLI arguments to speed things up:
```sh
npm create cloudflare@latest email-with-resend -- --type=hello-world --ts=false --git=true --deploy=false
```
This creates a simple hello-world Worker having the following content:
```js
export default {
async fetch(request, env, ctx) {
return new Response("Hello World!");
},
};
```
## Add your domain to Resend
If you don’t already have a Resend account, you can sign up for a [free account here](https://resend.com/signup). After signing up, go to `Domains` using the side menu, and click the button to add a new domain. On the modal, enter the domain you want to add and then select a region.
Next, you’re presented with a list of DNS records to add to your Cloudflare domain. On your Cloudflare dashboard, select the domain you entered earlier and navigate to `DNS` > `Records`. Copy/paste the DNS records (DKIM, SPF, and DMARC records) from Resend to your Cloudflare domain.
:::note
If you need more help adding DNS records in Cloudflare, refer to [Manage DNS records](/dns/manage-dns-records/how-to/create-dns-records/).
:::
When that’s done, head back to Resend and click on the `Verify DNS Records` button. If all records are properly configured, your domain status should be updated to `Verified`.
Lastly, navigate to `API Keys` with the side menu, to create an API key. Give your key a descriptive name and the appropriate permissions. Click the button to add your key and then copy your API key to a safe location.
## Send emails from your Worker
The final step is putting it all together in a Worker. Open up a terminal in the directory of the Worker you created earlier. Then, install the Resend SDK:
```sh
npm i resend
```
In your Worker, import and use the Resend library like so:
```jsx
import { Resend } from "resend";
export default {
async fetch(request, env, ctx) {
const resend = new Resend("your_resend_api_key");
const { data, error } = await resend.emails.send({
from: "hello@example.com",
to: "someone@example.com",
subject: "Hello World",
html: "Hello from Workers
",
});
return Response.json({ data, error });
},
};
```
To test your code locally, run the following command and navigate to [http://localhost:8787/](http://localhost:8787/) in a browser:
```sh
npm start
```
Deploy your Worker with `npm run deploy`.
## Move API keys to Secrets
Sensitive information such as API keys and token should always be stored in secrets. All secrets are encrypted to add an extra layer of protection. That said, it’s a good idea to move your API key to a secret and access it from the environment of your Worker.
To add secrets for local development, create a `.dev.vars` file which works exactly like a `.env` file:
```txt
RESEND_API_KEY=your_resend_api_key
```
Also ensure the secret is added to your deployed worker by running:
```sh title="Add secret to deployed Worker"
npx wrangler secret put RESEND_API_KEY
```
The added secret can be accessed on via the `env` parameter passed to your Worker’s fetch event handler:
```jsx
import { Resend } from "resend";
export default {
async fetch(request, env, ctx) {
const resend = new Resend(env.RESEND_API_KEY);
const { data, error } = await resend.emails.send({
from: "hello@example.com",
to: "someone@example.com",
subject: "Hello World",
html: "Hello from Workers
",
});
return Response.json({ data, error });
},
};
```
And finally, deploy this update with `npm run deploy`.
## Related resources
- [Storing API keys and tokens with Secrets](/workers/configuration/secrets/).
- [Transferring your domain to Cloudflare](/registrar/get-started/transfer-domain-to-cloudflare/).
- [Send emails from Workers](/email-routing/email-workers/send-email-workers/)
---
# Set up and use a Prisma Postgres database
URL: https://developers.cloudflare.com/workers/tutorials/using-prisma-postgres-with-workers/
import { PackageManagers } from "~/components";
[Prisma Postgres](https://www.prisma.io/postgres) is a managed, serverless PostgreSQL database. It supports features like connection pooling, caching, real-time subscriptions, and query optimization recommendations.
In this tutorial, you will learn how to:
- Set up a Cloudflare Workers project with [Prisma ORM](https://www.prisma.io/docs).
- Create a Prisma Postgres instance from the Prisma CLI.
- Model data and run migrations with Prisma Postgres.
- Query the database from Workers.
- Deploy the Worker to Cloudflare.
## Prerequisites
To follow this guide, ensure you have the following:
- Node.js `v18.18` or higher installed.
- An active [Cloudflare account](https://dash.cloudflare.com/).
- A basic familiarity with installing and using command-line interface (CLI) applications.
## 1. Create a new Worker project
Begin by using [C3](/pages/get-started/c3/) to create a Worker project in the command line:
```sh
npm create cloudflare@latest prisma-postgres-worker -- --type=hello-world --ts=true --git=true --deploy=false
```
Then navigate into your project:
```sh
cd ./prisma-postgres-worker
```
Your initial `src/index.ts` file currently contains a simple request handler:
```ts title="src/index.ts"
export default {
async fetch(request, env, ctx): Promise {
return new Response("Hello World!");
},
} satisfies ExportedHandler;
```
## 2. Setup Prisma in your project
In this step, you will set up Prisma ORM with a Prisma Postgres database using the CLI. Then you will create and execute helper scripts to create tables in the database and generate a Prisma client to query it.
### 2.1. Install required dependencies
Install Prisma CLI as a dev dependency:
Install the [Prisma Accelerate client extension](https://www.npmjs.com/package/@prisma/extension-accelerate) as it is required for Prisma Postgres:
Install the [`dotenv-cli` package](https://www.npmjs.com/package/dotenv-cli) to load environment variables from `.dev.vars`:
### 2.2. Create a Prisma Postgres database and initialize Prisma
Initialize Prisma in your application:
If you do not have a [Prisma Data Platform](https://console.prisma.io/) account yet, or if you are not logged in, the command will prompt you to log in using one of the available authentication providers. A browser window will open so you can log in or create an account. Return to the CLI after you have completed this step.
Once logged in (or if you were already logged in), the CLI will prompt you to select a project name and a database region.
Once the command has terminated, it will have created:
- A project in your [Platform Console](https://console.prisma.io/) containing a Prisma Postgres database instance.
- A `prisma` folder containing `schema.prisma`, where you will define your database schema.
- An `.env` file in the project root, which will contain the Prisma Postgres database url `DATABASE_URL=`.
Note that Cloudflare Workers do not support `.env` files. You will use a file called `.dev.vars` instead of the `.env` file that was just created.
### 2.3. Prepare environment variables
Rename the `.env` file in the root of your application to `.dev.vars` file:
```sh
mv .env .dev.vars
```
### 2.4. Apply database schema changes
Open the `schema.prisma` file in the `prisma` folder and add the following `User` model to your database:
```prisma title="prisma/schema.prisma"
generator client {
provider = "prisma-client-js"
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
model User {
id Int @id @default(autoincrement())
email String
name String
}
```
Next, add the following helper scripts to the `scripts` section of your `package.json`:
```json title="package.json"
"scripts": {
"migrate": "dotenv -e .dev.vars -- npx prisma migrate dev",
"generate": "dotenv -e .dev.vars -- npx prisma generate --no-engine",
"studio": "dotenv -e .dev.vars -- npx prisma studio",
// Additional worker scripts...
}
```
Run the migration script to apply changes to the database:
```sh
npm run migrate
```
When prompted, provide a name for the migration (for example, `init`).
After these steps are complete, Prisma ORM is fully set up and connected to your Prisma Postgres database.
## 3. Develop the application
Modify the `src/index.ts` file and replace its contents with the following code:
```ts title="src/index.ts"
import { PrismaClient } from "@prisma/client/edge";
import { withAccelerate } from "@prisma/extension-accelerate";
export interface Env {
DATABASE_URL: string;
}
export default {
async fetch(request, env, ctx): Promise {
const path = new URL(request.url).pathname;
if (path === "/favicon.ico")
return new Response("Resource not found", {
status: 404,
headers: {
"Content-Type": "text/plain",
},
});
const prisma = new PrismaClient({
datasourceUrl: env.DATABASE_URL,
}).$extends(withAccelerate());
const user = await prisma.user.create({
data: {
email: `Jon${Math.ceil(Math.random() * 1000)}@gmail.com`,
name: "Jon Doe",
},
});
const userCount = await prisma.user.count();
return new Response(`\
Created new user: ${user.name} (${user.email}).
Number of users in the database: ${userCount}.
`);
},
} satisfies ExportedHandler;
```
Run the development server:
```sh
npm run dev
```
Visit [`https://localhost:8787`](https://localhost:8787) to see your app display the following output:
```sh
Number of users in the database: 1
```
Every time you refresh the page, a new user is created. The number displayed will increment by `1` with each refresh as it returns the total number of users in your database.
## 4. Deploy the application to Cloudflare
When the application is deployed to Cloudflare, it needs access to the `DATABASE_URL` environment variable that is defined locally in `.dev.vars`. You can use the [`npx wrangler secret put`](/workers/configuration/secrets/#adding-secrets-to-your-project) command to upload the `DATABASE_URL` to the deployment environment:
```sh
npx wrangler secret put DATABASE_URL
```
When prompted, paste the `DATABASE_URL` value (from `.dev.vars`). If you are logged in via the Wrangler CLI, you will see a prompt asking if you'd like to create a new Worker. Confirm by choosing "yes":
```sh
✔ There doesn't seem to be a Worker called "prisma-postgres-worker". Do you want to create a new Worker with that name and add secrets to it? … yes
```
Then execute the following command to deploy your project to Cloudflare Workers:
```sh
npm run deploy
```
The `wrangler` CLI will bundle and upload your application.
If you are not already logged in, the `wrangler` CLI will open a browser window prompting you to log in to the [Cloudflare dashboard](https://dash.cloudflare.com/).
:::note
If you belong to multiple accounts, select the account where you want to deploy the project.
:::
Once the deployment completes, verify the deployment by visiting the live URL provided in the deployment output, such as `https://{PROJECT_NAME}.workers.dev`. If you encounter any issues, ensure the secrets were added correctly and check the deployment logs for errors.
## Next steps
Congratulations on building and deploying a simple application with Prisma Postgres and Cloudflare Workers!
To enhance your application further:
- Add [caching](https://www.prisma.io/docs/postgres/caching) to your queries.
- Explore the [Prisma Postgres documentation](https://www.prisma.io/docs/postgres/getting-started).
To see how to build a real-time application with Cloudflare Workers and Prisma Postgres, read [this](https://www.prisma.io/docs/guides/prisma-postgres-realtime-on-cloudflare) guide.
---
# Securely access and upload assets with Cloudflare R2
URL: https://developers.cloudflare.com/workers/tutorials/upload-assets-with-r2/
import { Render, PackageManagers, WranglerConfig } from "~/components";
This tutorial explains how to create a TypeScript-based Cloudflare Workers project that can securely access files from and upload files to a [Cloudflare R2](/r2/) bucket. Cloudflare R2 allows developers to store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services.
## Prerequisites
To continue:
1. Sign up for a [Cloudflare account](https://dash.cloudflare.com/sign-up/workers-and-pages) if you have not already.
2. Install [`npm`](https://docs.npmjs.com/getting-started).
3. Install [`Node.js`](https://nodejs.org/en/). Use a Node version manager like [Volta](https://volta.sh/) or [nvm](https://github.com/nvm-sh/nvm) to avoid permission issues and change Node.js versions. [Wrangler](/workers/wrangler/install-and-update/) requires a Node version of `16.17.0` or later.
## Create a Worker application
First, use the [`create-cloudflare` CLI](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare) to create a new Worker. To do this, open a terminal window and run the following command:
Move into your newly created directory:
```sh
cd upload-r2-assets
```
## Create an R2 bucket
Before you integrate R2 bucket access into your Worker application, an R2 bucket must be created:
```sh
npx wrangler r2 bucket create
```
Replace `` with the name you want to assign to your bucket. List your account's R2 buckets to verify that a new bucket has been added:
```sh
npx wrangler r2 bucket list
```
## Configure access to an R2 bucket
After your new R2 bucket is ready, use it inside your Worker application.
Use your R2 bucket inside your Worker project by modifying the [Wrangler configuration file](/workers/wrangler/configuration/) to include an R2 bucket [binding](/workers/runtime-apis/bindings/). Add the following R2 bucket binding to your Wrangler file:
```toml
[[r2_buckets]]
binding = 'MY_BUCKET'
bucket_name = ''
```
Give your R2 bucket binding name. Replace `` with the name of the R2 bucket you created earlier.
Your Worker application can now access your R2 bucket using the `MY_BUCKET` variable. You can now perform CRUD (Create, Read, Update, Delete) operations on the contents of the bucket.
## Fetch from an R2 bucket
After setting up an R2 bucket binding, you will implement the functionalities for the Worker to interact with the R2 bucket, such as, fetching files from the bucket and uploading files to the bucket.
To fetch files from the R2 bucket, use the `BINDING.get` function. In the below example, the R2 bucket binding is called `MY_BUCKET`. Using `.get(key)`, you can retrieve an asset based on the URL pathname as the key. In this example, the URL pathname is `/image.png`, and the asset key is `image.png`.
```ts
interface Env {
MY_BUCKET: R2Bucket;
}
export default {
async fetch(request, env): Promise {
// For example, the request URL my-worker.account.workers.dev/image.png
const url = new URL(request.url);
const key = url.pathname.slice(1);
// Retrieve the key "image.png"
const object = await env.MY_BUCKET.get(key);
if (object === null) {
return new Response("Object Not Found", { status: 404 });
}
const headers = new Headers();
object.writeHttpMetadata(headers);
headers.set("etag", object.httpEtag);
return new Response(object.body, {
headers,
});
},
} satisfies ExportedHandler;
```
The code written above fetches and returns data from the R2 bucket when a `GET` request is made to the Worker application using a specific URL path.
## Upload securely to an R2 bucket
Next, you will add the ability to upload to your R2 bucket using authentication. To securely authenticate your upload requests, use [Wrangler's secret capability](/workers/wrangler/commands/#secret). Wrangler was installed when you ran the `create cloudflare@latest` command.
Create a secret value of your choice -- for instance, a random string or password. Using the Wrangler CLI, add the secret to your project as `AUTH_SECRET`:
```sh
npx wrangler secret put AUTH_SECRET
```
Now, add a new code path that handles a `PUT` HTTP request. This new code will check that the previously uploaded secret is correctly used for authentication, and then upload to R2 using `MY_BUCKET.put(key, data)`:
```ts
interface Env {
MY_BUCKET: R2Bucket;
AUTH_SECRET: string;
}
export default {
async fetch(request, env): Promise {
if (request.method === "PUT") {
// Note that you could require authentication for all requests
// by moving this code to the top of the fetch function.
const auth = request.headers.get("Authorization");
const expectedAuth = `Bearer ${env.AUTH_SECRET}`;
if (!auth || auth !== expectedAuth) {
return new Response("Unauthorized", { status: 401 });
}
const url = new URL(request.url);
const key = url.pathname.slice(1);
await env.MY_BUCKET.put(key, request.body);
return new Response(`Object ${key} uploaded successfully!`);
}
// include the previous code here...
},
} satisfies ExportedHandler;
```
This approach ensures that only clients who provide a valid bearer token, via the `Authorization` header equal to the `AUTH_SECRET` value, will be permitted to upload to the R2 bucket. If you used a different binding name than `AUTH_SECRET`, replace it in the code above.
## Deploy your Worker application
After completing your Cloudflare Worker project, deploy it to Cloudflare. Make sure you are in your Worker application directory that you created for this tutorial, then run:
```sh
npx wrangler deploy
```
Your application is now live and accessible at `..workers.dev`.
You have successfully created a Cloudflare Worker that allows you to interact with an R2 bucket to accomplish tasks such as uploading and downloading files. You can now use this as a starting point for your own projects.
## Next steps
To build more with R2 and Workers, refer to [Tutorials](/workers/tutorials/) and the [R2 documentation](/r2/).
If you have any questions, need assistance, or would like to share your project, join the Cloudflare Developer community on [Discord](https://discord.cloudflare.com) to connect with fellow developers and the Cloudflare team.
---
# Use Workers KV directly from Rust
URL: https://developers.cloudflare.com/workers/tutorials/workers-kv-from-rust/
import { Render, WranglerConfig } from "~/components";
This tutorial will teach you how to read and write to KV directly from Rust
using [workers-rs](https://github.com/cloudflare/workers-rs).
## Prerequisites
To complete this tutorial, you will need:
- [Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git).
- [Wrangler](/workers/wrangler/) CLI.
- The [Rust](https://www.rust-lang.org/tools/install) toolchain.
- And `cargo-generate` sub-command by running:
```sh
cargo install cargo-generate
```
## 1. Create your Worker project in Rust
Open a terminal window, and run the following command to generate a Worker project template in Rust:
```sh
cargo generate cloudflare/workers-rs
```
Then select `template/hello-world-http` template, give your project a descriptive name and select enter. A new project should be created in your directory. Open the project in your editor and run `npx wrangler dev` to compile and run your project.
In this tutorial, you will use Workers KV from Rust to build an app to store and retrieve cities by a given country name.
## 2. Create a KV namespace
In the terminal, use Wrangler to create a KV namespace for `cities`. This generates a configuration to be added to the project:
```sh
npx wrangler kv namespace create cities
```
To add this configuration to your project, open the Wrangler file and create an entry for `kv_namespaces` above the build command:
```toml
kv_namespaces = [
{ binding = "cities", id = "e29b263ab50e42ce9b637fa8370175e8" }
]
# build command...
```
With this configured, you can access the KV namespace with the binding `"cities"` from Rust.
## 3. Write data to KV
For this app, you will create two routes: A `POST` route to receive and store the city in KV, and a `GET` route to retrieve the city of a given country. For example, a `POST` request to `/France` with a body of `{"city": "Paris"}` should create an entry of Paris as a city in France. A `GET` request to `/France` should retrieve from KV and respond with Paris.
Install [Serde](https://serde.rs/) as a project dependency to handle JSON `cargo add serde`. Then create an app router and a struct for `Country` in `src/lib.rs`:
```rust null {1,6,8,9,10,11,15,17}
use serde::{Deserialize, Serialize};
use worker::*;
#[event(fetch)]
async fn fetch(req: Request, env: Env, _ctx: Context) -> Result {
let router = Router::new();
#[derive(Serialize, Deserialize, Debug)]
struct Country {
city: String,
}
router
// TODO:
.post_async("/:country", |_, _| async move { Response::empty() })
// TODO:
.get_async("/:country", |_, _| async move { Response::empty() })
.run(req, env)
.await
}
```
For the post handler, you will retrieve the country name from the path and the city name from the request body. Then, you will save this in KV with the country as key and the city as value. Finally, the app will respond with the city name:
```rust
.post_async("/:country", |mut req, ctx| async move {
let country = ctx.param("country").unwrap();
let city = match req.json::().await {
Ok(c) => c.city,
Err(_) => String::from(""),
};
if city.is_empty() {
return Response::error("Bad Request", 400);
};
return match ctx.kv("cities")?.put(country, &city)?.execute().await {
Ok(_) => Response::ok(city),
Err(_) => Response::error("Bad Request", 400),
};
})
```
Save the file and make a `POST` request to test this endpoint:
```sh
curl --json '{"city": "Paris"}' http://localhost:8787/France
```
## 4. Read data from KV
To retrieve cities stored in KV, write a `GET` route that pulls the country name from the path and searches KV. You also need some error handling if the country is not found:
```rust
.get_async("/:country", |_req, ctx| async move {
if let Some(country) = ctx.param("country") {
return match ctx.kv("cities")?.get(country).text().await? {
Some(city) => Response::ok(city),
None => Response::error("Country not found", 404),
};
}
Response::error("Bad Request", 400)
})
```
Save and make a curl request to test the endpoint:
```sh
curl http://localhost:8787/France
```
## 5. Deploy your project
The source code for the completed app should include the following:
```rust
use serde::{Deserialize, Serialize};
use worker::*;
#[event(fetch)]
async fn fetch(req: Request, env: Env, _ctx: Context) -> Result {
let router = Router::new();
#[derive(Serialize, Deserialize, Debug)]
struct Country {
city: String,
}
router
.post_async("/:country", |mut req, ctx| async move {
let country = ctx.param("country").unwrap();
let city = match req.json::().await {
Ok(c) => c.city,
Err(_) => String::from(""),
};
if city.is_empty() {
return Response::error("Bad Request", 400);
};
return match ctx.kv("cities")?.put(country, &city)?.execute().await {
Ok(_) => Response::ok(city),
Err(_) => Response::error("Bad Request", 400),
};
})
.get_async("/:country", |_req, ctx| async move {
if let Some(country) = ctx.param("country") {
return match ctx.kv("cities")?.get(country).text().await? {
Some(city) => Response::ok(city),
None => Response::error("Country not found", 404),
};
}
Response::error("Bad Request", 400)
})
.run(req, env)
.await
}
```
To deploy your Worker, run the following command:
```sh
npx wrangler deploy
```
## Related resources
- [Rust support in Workers](/workers/languages/rust/).
- [Using KV in Workers](/kv/get-started/).
---
# Asynchronous Batch API
URL: https://developers.cloudflare.com/workers-ai/features/batch-api/
import { Render, PackageManagers, WranglerConfig, CURL } from "~/components";
Asynchronous batch processing lets you send a collection (batch) of inference requests in a single call. Instead of expecting immediate responses for every request, the system queues them for processing and returns the results later.
Batch processing is useful for large workloads such as summarization or embeddings when there is no human interaction. Using the batch API will guarantee that your requests are fulfilled eventually, rather than erroring out if Cloudflare does have enough capacity at a given time.
When you send a batch request, the API immediately acknowledges receipt with a status like `queued` and provides a unique `request_id`. This ID is later used to poll for the final responses once the processing is complete.
You can use the Batch API by either creating and deploying a Cloudflare Worker that leverages the [Batch API with the AI binding](/workers-ai/features/batch-api/workers-binding/), using the [REST API](/workers-ai/features/batch-api/rest-api/) directly or by starting from a [template](https://github.com/craigsdennis/batch-please-workers-ai).
:::note[Note]
Ensure that the total payload is under 10 MB.
:::
## Demo application
If you want to get started quickly, click the button below:
[](https://deploy.workers.cloudflare.com/?url=https://github.com/craigsdennis/batch-please-workers-ai)
This will create a repository in your GitHub account and deploy a ready-to-use Worker that demonstrates how to use Cloudflare's Asynchronous Batch API. The template includes preconfigured AI bindings, and examples for sending and retrieving batch requests with and without external references. Once deployed, you can visit the live Worker and start experimenting with the Batch API immediately.
## Supported Models
Refer to our [model catalog](/workers-ai/models/?capabilities=Batch) for supported models.
---
# REST API
URL: https://developers.cloudflare.com/workers-ai/features/batch-api/rest-api/
If you prefer to work directly with the REST API instead of a [Cloudflare Worker](/workers-ai/features/batch-api/workers-binding/), below are the steps on how to do it:
## 1. Sending a Batch Request
Make a POST request using the following pattern. You can pass `external_reference` as a unique ID per-prompt that will be returned in the response.
```bash title="Sending a batch request" {11,15,19}
curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \
--header "Authorization: Bearer $API_TOKEN" \
--header 'Content-Type: application/json' \
--json '{
"requests": [
{
"query": "This is a story about Cloudflare",
"contexts": [
{
"text": "This is a story about an orange cloud",
"external_reference": "story1"
},
{
"text": "This is a story about a llama",
"external_reference": "story2"
},
{
"text": "This is a story about a hugging emoji",
"external_reference": "story3"
}
]
}
]
}'
```
```json output {4}
{
"result": {
"status": "queued",
"request_id": "768f15b7-4fd6-4498-906e-ad94ffc7f8d2",
"model": "@cf/baai/bge-m3"
},
"success": true,
"errors": [],
"messages": []
}
```
## 2. Retrieving the Batch Response
After receiving a `request_id` from your initial POST, you can poll for or retrieve the results with another POST request:
```bash title="Retrieving a response"
curl "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/ai/run/@cf/baai/bge-m3?queueRequest=true" \
--header "Authorization: Bearer $API_TOKEN" \
--header 'Content-Type: application/json' \
--json '{
"request_id": ""
}'
```
```json output
{
"result": {
"responses": [
{
"id": 0,
"result": {
"response": [
{ "id": 0, "score": 0.73974609375 },
{ "id": 1, "score": 0.642578125 },
{ "id": 2, "score": 0.6220703125 }
]
},
"success": true,
"external_reference": null
}
],
"usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 }
},
"success": true,
"errors": [],
"messages": []
}
```
---
# Workers Binding
URL: https://developers.cloudflare.com/workers-ai/features/batch-api/workers-binding/
import {
Render,
PackageManagers,
TypeScriptExample,
WranglerConfig,
CURL,
} from "~/components";
You can use Workers Bindings to interact with the Batch API.
## Send a Batch request
Send your initial batch inference request by composing a JSON payload containing an array of individual inference requests and the `queueRequest: true` property (which is what controlls queueing behavior).
:::note[Note]
Ensure that the total payload is under 10 MB.
:::
```ts {26} title="src/index.ts"
export interface Env {
AI: Ai;
}
export default {
async fetch(request, env): Promise {
const embeddings = await env.AI.run(
"@cf/baai/bge-m3",
{
requests: [
{
query: "This is a story about Cloudflare",
contexts: [
{
text: "This is a story about an orange cloud",
},
{
text: "This is a story about a llama",
},
{
text: "This is a story about a hugging emoji",
},
],
},
],
},
{ queueRequest: true },
);
return Response.json(embeddings);
},
} satisfies ExportedHandler;
```
```json output {4}
{
"status": "queued",
"model": "@cf/baai/bge-m3",
"request_id": "000-000-000"
}
```
You will get a response with the following values:
- **`status`**: Indicates that your request is queued.
- **`request_id`**: A unique identifier for the batch request.
- **`model`**: The model used for the batch inference.
Of these, the `request_id` is important for when you need to [poll the batch status](#poll-batch-status).
### Poll batch status
Once your batch request is queued, use the `request_id` to poll for its status. During processing, the API returns a status `queued` or `running` indicating that the request is still in the queue or being processed.
```typescript title=src/index.ts
export interface Env {
AI: Ai;
}
export default {
async fetch(request, env): Promise {
const status = await env.AI.run("@cf/baai/bge-m3", {
request_id: "000-000-000",
});
return Response.json(status);
},
} satisfies ExportedHandler;
```
```json output
{
"responses": [
{
"id": 0,
"result": {
"response": [
{ "id": 0, "score": 0.73974609375 },
{ "id": 1, "score": 0.642578125 },
{ "id": 2, "score": 0.6220703125 }
]
},
"success": true,
"external_reference": null
}
],
"usage": { "prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12 }
}
```
When the inference is complete, the API returns a final HTTP status code of `200` along with an array of responses. Each response object corresponds to an individual input prompt, identified by an `id` that maps to the index of the prompt in your original request.
---
# Fine-tunes
URL: https://developers.cloudflare.com/workers-ai/features/fine-tunes/
import { Feature } from "~/components";
Learn how to use Workers AI to get fine-tuned inference.
Upload a LoRA adapter and run fine-tuned inference with one of our base models.
---
## What is fine-tuning?
Fine-tuning is a general term for modifying an AI model by continuing to train it with additional data. The goal of fine-tuning is to increase the probability that a generation is similar to your dataset. Training a model from scratch is not practical for many use cases given how expensive and time consuming they can be to train. By fine-tuning an existing pre-trained model, you benefit from its capabilities while also accomplishing your desired task.
[Low-Rank Adaptation](https://arxiv.org/abs/2106.09685) (LoRA) is a specific fine-tuning method that can be applied to various model architectures, not just LLMs. It is common that the pre-trained model weights are directly modified or fused with additional fine-tune weights in traditional fine-tuning methods. LoRA, on the other hand, allows for the fine-tune weights and pre-trained model to remain separate, and for the pre-trained model to remain unchanged. The end result is that you can train models to be more accurate at specific tasks, such as generating code, having a specific personality, or generating images in a specific style.
---
# Using LoRA adapters
URL: https://developers.cloudflare.com/workers-ai/features/fine-tunes/loras/
import { APIRequest, TabItem, Tabs } from "~/components";
Workers AI supports fine-tuned inference with adapters trained with [Low-Rank Adaptation](https://blog.cloudflare.com/fine-tuned-inference-with-loras). This feature is in open beta and free during this period.
## Limitations
- We only support LoRAs for a [variety of models](/workers-ai/models/?capabilities=LoRA) (must not be quantized)
- Adapter must be trained with rank `r <=8` as well as larger ranks if up to 32. You can check the rank of a pre-trained LoRA adapter through the adapter's `config.json` file
- LoRA adapter file must be < 300MB
- LoRA adapter files must be named `adapter_config.json` and `adapter_model.safetensors` exactly
- You can test up to 30 LoRA adapters per account
---
## Choosing compatible LoRA adapters
### Finding open-source LoRA adapters
We have started a [Hugging Face Collection](https://huggingface.co/collections/Cloudflare/workers-ai-compatible-loras-6608dd9f8d305a46e355746e) that lists a few LoRA adapters that are compatible with Workers AI. Generally, any LoRA adapter that fits our limitations above should work.
### Training your own LoRA adapters
To train your own LoRA adapter, follow the [tutorial](/workers-ai/guides/tutorials/fine-tune-models-with-autotrain/).
---
## Uploading LoRA adapters
In order to run inference with LoRAs on Workers AI, you'll need to create a new fine tune on your account and upload your adapter files. You should have a `adapter_model.safetensors` file with model weights and `adapter_config.json` with your config information. _Note that we only accept adapter files in these types._
Right now, you can't edit a fine tune's asset files after you upload it. We will support this soon, but for now you will need to create a new fine tune and upload files again if you would like to use a new LoRA.
Before you upload your LoRA adapter, you'll need to edit your `adapter_config.json` file to include `model_type` as one of `mistral`, `gemma` or `llama` like below.
```json null {10}
{
"alpha_pattern": {},
"auto_mapping": null,
...
"target_modules": [
"q_proj",
"v_proj"
],
"task_type": "CAUSAL_LM",
"model_type": "mistral",
}
```
### Wrangler
You can create a finetune and upload your LoRA adapter via wrangler with the following commands:
```bash title="wrangler CLI" {1,7}
npx wrangler ai finetune create
#🌀 Creating new finetune "test-lora" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"...
#🌀 Uploading file "/Users/abcd/Downloads/adapter_config.json" to "test-lora"...
#🌀 Uploading file "/Users/abcd/Downloads/adapter_model.safetensors" to "test-lora"...
#✅ Assets uploaded, finetune "test-lora" is ready to use.
npx wrangler ai finetune list
┌──────────────────────────────────────┬─────────────────┬─────────────┐
│ finetune_id │ name │ description │
├──────────────────────────────────────┼─────────────────┼─────────────┤
│ 00000000-0000-0000-0000-000000000000 │ test-lora │ │
└──────────────────────────────────────┴─────────────────┴─────────────┘
```
### REST API
Alternatively, you can use our REST API to create a finetune and upload your adapter files. You will need a Cloudflare API Token with `Workers AI: Edit` permissions to make calls to our REST API, which you can generate via the Cloudflare Dashboard.
#### Creating a fine-tune on your account
#### Uploading your adapter weights and config
You have to call the upload endpoint each time you want to upload a new file, so you usually run this once for `adapter_model.safetensors` and once for `adapter_config.json`. Make sure you include the `@` before your path to files.
You can either use the finetune `name` or `id` that you used when you created the fine tune.
```bash title="cURL"
## Input: finetune_id, adapter_model.safetensors, then adapter_config.json
## Output: success true/false
curl -X POST https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/finetunes/{FINETUNE_ID}/finetune-assets/ \
-H 'Authorization: Bearer {API_TOKEN}' \
-H 'Content-Type: multipart/form-data' \
-F 'file_name=adapter_model.safetensors' \
-F 'file=@{PATH/TO/adapter_model.safetensors}'
```
#### List fine-tunes in your account
You can call this method to confirm what fine-tunes you have created in your account
```json output
{
"success": true,
"result": [
[
{
"id": "00000000-0000-0000-0000-000000000",
"model": "@cf/meta-llama/llama-2-7b-chat-hf-lora",
"name": "llama2-finetune",
"description": "test"
},
{
"id": "00000000-0000-0000-0000-000000000",
"model": "@cf/mistralai/mistral-7b-instruct-v0.2-lora",
"name": "mistral-finetune",
"description": "test"
}
]
]
}
```
---
## Running inference with LoRAs
To make inference requests and apply the LoRA adapter, you will need your model and finetune `name` or `id`. You should use the chat template that your LoRA was trained on, but you can try running it with `raw: true` and the messages template like below.
```javascript null {5-6}
const response = await env.AI.run(
"@cf/mistralai/mistral-7b-instruct-v0.2-lora", //the model supporting LoRAs
{
messages: [{ role: "user", content: "Hello world" }],
raw: true, //skip applying the default chat template
lora: "00000000-0000-0000-0000-000000000", //the finetune id OR name
},
);
```
```bash null {5-6}
curl https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora \
-H 'Authorization: Bearer {API_TOKEN}' \
-d '{
"messages": [{"role": "user", "content": "Hello world"}],
"raw": "true",
"lora": "00000000-0000-0000-0000-000000000"
}'
```
---
# Public LoRA adapters
URL: https://developers.cloudflare.com/workers-ai/features/fine-tunes/public-loras/
import { APIRequest } from "~/components";
Cloudflare offers a few public LoRA adapters that can immediately be used for fine-tuned inference. You can try them out immediately via our [playground](https://playground.ai.cloudflare.com).
Public LoRAs will have the name `cf-public-x`, and the prefix will be reserved for Cloudflare.
:::note
Have more LoRAs you would like to see? Let us know on [Discord](https://discord.cloudflare.com).
:::
| Name | Description | Compatible with |
| -------------------------------------------------------------------------- | ---------------------------------- | ----------------------------------------------------------------------------------- |
| [cf-public-magicoder](https://huggingface.co/predibase/magicoder) | Coding tasks in multiple languages | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` |
| [cf-public-jigsaw-classification](https://huggingface.co/predibase/jigsaw) | Toxic comment classification | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` |
| [cf-public-cnn-summarization](https://huggingface.co/predibase/cnn) | Article summarization | `@cf/mistral/mistral-7b-instruct-v0.1`
`@hf/mistral/mistral-7b-instruct-v0.2` |
You can also list these public LoRAs with an API call:
## Running inference with public LoRAs
To run inference with public LoRAs, you just need to define the LoRA name in the request.
We recommend that you use the prompt template that the LoRA was trained on. You can find this in the HuggingFace repos linked above for each adapter.
### cURL
```bash null {10}
curl https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1 \
--header 'Authorization: Bearer {cf_token}' \
--data '{
"messages": [
{
"role": "user",
"content": "Write a python program to check if a number is even or odd."
}
],
"lora": "cf-public-magicoder"
}'
```
### JavaScript
```js null {11}
const answer = await env.AI.run("@cf/mistral/mistral-7b-instruct-v0.1", {
stream: true,
raw: true,
messages: [
{
role: "user",
content:
"Summarize the following: Some newspapers, TV channels and well-known companies publish false news stories to fool people on 1 April. One of the earliest examples of this was in 1957 when a programme on the BBC, the UKs national TV channel, broadcast a report on how spaghetti grew on trees. The film showed a family in Switzerland collecting spaghetti from trees and many people were fooled into believing it, as in the 1950s British people didnt eat much pasta and many didnt know how it was made! Most British people wouldnt fall for the spaghetti trick today, but in 2008 the BBC managed to fool their audience again with their Miracles of Evolution trailer, which appeared to show some special penguins that had regained the ability to fly. Two major UK newspapers, The Daily Telegraph and the Daily Mirror, published the important story on their front pages.",
},
],
lora: "cf-public-cnn-summarization",
});
```
---
# Function calling
URL: https://developers.cloudflare.com/workers-ai/features/function-calling/
import { Stream, TabItem, Tabs } from "~/components";
Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call `tools`. The model then intelligently understands when it needs to do a tool call, and it returns a JSON output which the user needs to feed to another function or API.
In essence, function calling allows you to perform actions with LLMs by executing code or making additional API calls.
## How can I use function calling?
Workers AI has [embedded function calling](/workers-ai/features/function-calling/embedded/) which allows you to execute function code alongside your inference calls. We have a package called [`@cloudflare/ai-utils`](https://www.npmjs.com/package/@cloudflare/ai-utils) to help facilitate this, which we have open-sourced on [Github](https://github.com/cloudflare/ai-utils).
For industry-standard function calling, take a look at the documentation on [Traditional Function Calling](/workers-ai/features/function-calling/traditional/).
To show you the value of embedded function calling, take a look at the example below that compares traditional function calling with embedded function calling. Embedded function calling allowed us to cut down the lines of code from 77 to 31.
```sh
# The ai-utils package enables embedded function calling
npm i @cloudflare/ai-utils
```
```js title="Embedded function calling example"
import {
createToolsFromOpenAPISpec,
runWithTools,
autoTrimTools,
} from "@cloudflare/ai-utils";
export default {
async fetch(request, env, ctx) {
const response = await runWithTools(
env.AI,
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [{ role: "user", content: "Who is Cloudflare on github?" }],
tools: [
// You can pass the OpenAPI spec link or contents directly
...(await createToolsFromOpenAPISpec(
"https://gist.githubusercontent.com/mchenco/fd8f20c8f06d50af40b94b0671273dc1/raw/f9d4b5cd5944cc32d6b34cad0406d96fd3acaca6/partial_api.github.com.json",
{
overrides: [
{
// for all requests on *.github.com, we'll need to add a User-Agent.
matcher: ({ url, method }) => {
return url.hostname === "api.github.com";
},
values: {
headers: {
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
},
},
},
],
},
)),
],
},
).then((response) => {
return response;
});
return new Response(JSON.stringify(response));
},
};
```
```js title="Traditional function calling example"
export default {
async fetch(request, env, ctx) {
const response = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [{ role: "user", content: "Who is Cloudflare on GitHub?" }],
tools: [
{
name: "getGithubUser",
description:
"Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
},
],
},
);
const selected_tool = response.tool_calls[0];
let res;
if (selected_tool.name == "getGithubUser") {
try {
const username = selected_tool.arguments.username;
const url = `https://api.github.com/users/${username}`;
res = await fetch(url, {
headers: {
// Github API requires a User-Agent header
"User-Agent":
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
},
}).then((res) => res.json());
} catch (error) {
return error;
}
}
const finalResponse = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b",
{
messages: [
{
role: "user",
content: "Who is Cloudflare on GitHub?",
},
{
role: "assistant",
content: JSON.stringify(selected_tool),
},
{
role: "tool",
content: JSON.stringify(res),
},
],
tools: [
{
name: "getGithubUser",
description:
"Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
},
],
},
);
return new Response(JSON.stringify(finalResponse));
},
};
```
## What models support function calling?
There are open-source models which have been fine-tuned to do function calling. When browsing our [model catalog](/workers-ai/models/), look for models with the function calling property beside it. For example, [@hf/nousresearch/hermes-2-pro-mistral-7b](/workers-ai/models/hermes-2-pro-mistral-7b/) is a fine-tuned variant of Mistral 7B that you can use for function calling.
---
# Traditional
URL: https://developers.cloudflare.com/workers-ai/features/function-calling/traditional/
This page shows how you can do traditional function calling, as defined by industry standards. Workers AI also offers [embedded function calling](/workers-ai/features/function-calling/embedded/), which is drastically easier than traditional function calling.
With traditional function calling, you define an array of tools with the name, description, and tool arguments. The example below shows how you would pass a tool called `getWeather` in an inference request to a model.
```js title="Traditional function calling example"
const response = await env.AI.run("@hf/nousresearch/hermes-2-pro-mistral-7b", {
messages: [
{
role: "user",
content: "what is the weather in london?",
},
],
tools: [
{
name: "getWeather",
description: "Return the weather for a latitude and longitude",
parameters: {
type: "object",
properties: {
latitude: {
type: "string",
description: "The latitude for the given location",
},
longitude: {
type: "string",
description: "The longitude for the given location",
},
},
required: ["latitude", "longitude"],
},
},
],
});
return new Response(JSON.stringify(response.tool_calls));
```
The LLM will then return a JSON object with the required arguments and the name of the tool that was called. You can then pass this JSON object to make an API call.
```json
[
{
"arguments": { "latitude": "51.5074", "longitude": "-0.1278" },
"name": "getWeather"
}
]
```
For a working example on how to do function calling, take a look at our [demo app](https://github.com/craigsdennis/lightbulb-moment-tool-calling/blob/main/src/index.ts).
---
# Build a Retrieval Augmented Generation (RAG) AI
URL: https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-retrieval-augmented-generation-ai/
import { Details, Render, PackageManagers, WranglerConfig } from "~/components";
This guide will instruct you through setting up and deploying your first application with Cloudflare AI. You will build a fully-featured AI-powered application, using tools like Workers AI, Vectorize, D1, and Cloudflare Workers.
:::note[Looking for a managed option?]
[AutoRAG](/autorag) offers a fully managed way to build RAG pipelines on Cloudflare, handling ingestion, indexing, and querying out of the box. [Get started](/autorag/get-started/).
:::
At the end of this tutorial, you will have built an AI tool that allows you to store information and query it using a Large Language Model. This pattern, known as Retrieval Augmented Generation, or RAG, is a useful project you can build by combining multiple aspects of Cloudflare's AI toolkit. You do not need to have experience working with AI tools to build this application.
You will also need access to [Vectorize](/vectorize/platform/pricing/). During this tutorial, we will show how you can optionally integrate with [Anthropic Claude](http://anthropic.com) as well. You will need an [Anthropic API key](https://docs.anthropic.com/en/api/getting-started) to do so.
## 1. Create a new Worker project
C3 (`create-cloudflare-cli`) is a command-line tool designed to help you setup and deploy Workers to Cloudflare as fast as possible.
Open a terminal window and run C3 to create your Worker project:
In your project directory, C3 has generated several files.
1. `wrangler.jsonc`: Your [Wrangler](/workers/wrangler/configuration/#sample-wrangler-configuration) configuration file.
2. `worker.js` (in `/src`): A minimal `'Hello World!'` Worker written in [ES module](/workers/reference/migrate-to-module-workers/) syntax.
3. `package.json`: A minimal Node dependencies configuration file.
4. `package-lock.json`: Refer to [`npm` documentation on `package-lock.json`](https://docs.npmjs.com/cli/v9/configuring-npm/package-lock-json).
5. `node_modules`: Refer to [`npm` documentation `node_modules`](https://docs.npmjs.com/cli/v7/configuring-npm/folders#node-modules).
Now, move into your newly created directory:
```sh
cd rag-ai-tutorial
```
## 2. Develop with Wrangler CLI
The Workers command-line interface, [Wrangler](/workers/wrangler/install-and-update/), allows you to [create](/workers/wrangler/commands/#init), [test](/workers/wrangler/commands/#dev), and [deploy](/workers/wrangler/commands/#deploy) your Workers projects. C3 will install Wrangler in projects by default.
After you have created your first Worker, run the [`wrangler dev`](/workers/wrangler/commands/#dev) command in the project directory to start a local server for developing your Worker. This will allow you to test your Worker locally during development.
```sh
npx wrangler dev --remote
```
:::note
If you have not used Wrangler before, it will try to open your web browser to login with your Cloudflare account.
If you have issues with this step or you do not have access to a browser interface, refer to the [`wrangler login`](/workers/wrangler/commands/#login) documentation for more information.
:::
You will now be able to go to [http://localhost:8787](http://localhost:8787) to see your Worker running. Any changes you make to your code will trigger a rebuild, and reloading the page will show you the up-to-date output of your Worker.
## 3. Adding the AI binding
To begin using Cloudflare's AI products, you can add the `ai` block to the [Wrangler configuration file](/workers/wrangler/configuration/). This will set up a binding to Cloudflare's AI models in your code that you can use to interact with the available AI models on the platform.
This example features the [`@cf/meta/llama-3-8b-instruct` model](/workers-ai/models/llama-3-8b-instruct/), which generates text.
```toml
[ai]
binding = "AI"
```
Now, find the `src/index.js` file. Inside the `fetch` handler, you can query the `AI` binding:
```js
export default {
async fetch(request, env, ctx) {
const answer = await env.AI.run("@cf/meta/llama-3-8b-instruct", {
messages: [{ role: "user", content: `What is the square root of 9?` }],
});
return new Response(JSON.stringify(answer));
},
};
```
By querying the LLM through the `AI` binding, we can interact directly with Cloudflare AI's large language models directly in our code. In this example, we are using the [`@cf/meta/llama-3-8b-instruct` model](/workers-ai/models/llama-3-8b-instruct/), which generates text.
You can deploy your Worker using `wrangler`:
```sh
npx wrangler deploy
```
Making a request to your Worker will now generate a text response from the LLM, and return it as a JSON object.
```sh
curl https://example.username.workers.dev
```
```sh output
{"response":"Answer: The square root of 9 is 3."}
```
## 4. Adding embeddings using Cloudflare D1 and Vectorize
Embeddings allow you to add additional capabilities to the language models you can use in your Cloudflare AI projects. This is done via **Vectorize**, Cloudflare's vector database.
To begin using Vectorize, create a new embeddings index using `wrangler`. This index will store vectors with 768 dimensions, and will use cosine similarity to determine which vectors are most similar to each other:
```sh
npx wrangler vectorize create vector-index --dimensions=768 --metric=cosine
```
Then, add the configuration details for your new Vectorize index to the [Wrangler configuration file](/workers/wrangler/configuration/):
```toml
# ... existing wrangler configuration
[[vectorize]]
binding = "VECTOR_INDEX"
index_name = "vector-index"
```
A vector index allows you to store a collection of dimensions, which are floating point numbers used to represent your data. When you want to query the vector database, you can also convert your query into dimensions. **Vectorize** is designed to efficiently determine which stored vectors are most similar to your query.
To implement the searching feature, you must set up a D1 database from Cloudflare. In D1, you can store your app's data. Then, you change this data into a vector format. When someone searches and it matches the vector, you can show them the matching data.
Create a new D1 database using `wrangler`:
```sh
npx wrangler d1 create database
```
Then, paste the configuration details output from the previous command into the [Wrangler configuration file](/workers/wrangler/configuration/):
```toml
# ... existing wrangler configuration
[[d1_databases]]
binding = "DB" # available in your Worker on env.DB
database_name = "database"
database_id = "abc-def-geh" # replace this with a real database_id (UUID)
```
In this application, we'll create a `notes` table in D1, which will allow us to store notes and later retrieve them in Vectorize. To create this table, run a SQL command using `wrangler d1 execute`:
```sh
npx wrangler d1 execute database --remote --command "CREATE TABLE IF NOT EXISTS notes (id INTEGER PRIMARY KEY, text TEXT NOT NULL)"
```
Now, we can add a new note to our database using `wrangler d1 execute`:
```sh
npx wrangler d1 execute database --remote --command "INSERT INTO notes (text) VALUES ('The best pizza topping is pepperoni')"
```
## 5. Creating a workflow
Before we begin creating notes, we will introduce a [Cloudflare Workflow](/workflows). This will allow us to define a durable workflow that can safely and robustly execute all the steps of the RAG process.
To begin, add a new `[[workflows]]` block to your [Wrangler configuration file](/workers/wrangler/configuration/):
```toml
# ... existing wrangler configuration
[[workflows]]
name = "rag"
binding = "RAG_WORKFLOW"
class_name = "RAGWorkflow"
```
In `src/index.js`, add a new class called `RAGWorkflow` that extends `WorkflowEntrypoint`:
```js
import { WorkflowEntrypoint } from "cloudflare:workers";
export class RAGWorkflow extends WorkflowEntrypoint {
async run(event, step) {
await step.do("example step", async () => {
console.log("Hello World!");
});
}
}
```
This class will define a single workflow step that will log "Hello World!" to the console. You can add as many steps as you need to your workflow.
On its own, this workflow will not do anything. To execute the workflow, we will call the `RAG_WORKFLOW` binding, passing in any parameters that the workflow needs to properly complete. Here is an example of how we can call the workflow:
```js
env.RAG_WORKFLOW.create({ params: { text } });
```
## 6. Creating notes and adding them to Vectorize
To expand on your Workers function in order to handle multiple routes, we will add `hono`, a routing library for Workers. This will allow us to create a new route for adding notes to our database. Install `hono` using `npm`:
Then, import `hono` into your `src/index.js` file. You should also update the `fetch` handler to use `hono`:
```js
import { Hono } from "hono";
const app = new Hono();
app.get("/", async (c) => {
const answer = await c.env.AI.run("@cf/meta/llama-3-8b-instruct", {
messages: [{ role: "user", content: `What is the square root of 9?` }],
});
return c.json(answer);
});
export default app;
```
This will establish a route at the root path `/` that is functionally equivalent to the previous version of your application.
Now, we can update our workflow to begin adding notes to our database, and generating the related embeddings for them.
This example features the [`@cf/baai/bge-base-en-v1.5` model](/workers-ai/models/bge-base-en-v1.5/), which can be used to create an embedding. Embeddings are stored and retrieved inside [Vectorize](/vectorize/), Cloudflare's vector database. The user query is also turned into an embedding so that it can be used for searching within Vectorize.
```js
import { WorkflowEntrypoint } from "cloudflare:workers";
export class RAGWorkflow extends WorkflowEntrypoint {
async run(event, step) {
const env = this.env;
const { text } = event.payload;
const record = await step.do(`create database record`, async () => {
const query = "INSERT INTO notes (text) VALUES (?) RETURNING *";
const { results } = await env.DB.prepare(query).bind(text).run();
const record = results[0];
if (!record) throw new Error("Failed to create note");
return record;
});
const embedding = await step.do(`generate embedding`, async () => {
const embeddings = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: text,
});
const values = embeddings.data[0];
if (!values) throw new Error("Failed to generate vector embedding");
return values;
});
await step.do(`insert vector`, async () => {
return env.VECTOR_INDEX.upsert([
{
id: record.id.toString(),
values: embedding,
},
]);
});
}
}
```
The workflow does the following things:
1. Accepts a `text` parameter.
2. Insert a new row into the `notes` table in D1, and retrieve the `id` of the new row.
3. Convert the `text` into a vector using the `embeddings` model of the LLM binding.
4. Upsert the `id` and `vectors` into the `vector-index` index in Vectorize.
By doing this, you will create a new vector representation of the note, which can be used to retrieve the note later.
To complete the code, we will add a route that allows users to submit notes to the database. This route will parse the JSON request body, get the `note` parameter, and create a new instance of the workflow, passing the parameter:
```js
app.post("/notes", async (c) => {
const { text } = await c.req.json();
if (!text) return c.text("Missing text", 400);
await c.env.RAG_WORKFLOW.create({ params: { text } });
return c.text("Created note", 201);
});
```
## 7. Querying Vectorize to retrieve notes
To complete your code, you can update the root path (`/`) to query Vectorize. You will convert the query into a vector, and then use the `vector-index` index to find the most similar vectors.
The `topK` parameter limits the number of vectors returned by the function. For instance, providing a `topK` of 1 will only return the _most similar_ vector based on the query. Setting `topK` to 5 will return the 5 most similar vectors.
Given a list of similar vectors, you can retrieve the notes that match the record IDs stored alongside those vectors. In this case, we are only retrieving a single note - but you may customize this as needed.
You can insert the text of those notes as context into the prompt for the LLM binding. This is the basis of Retrieval-Augmented Generation, or RAG: providing additional context from data outside of the LLM to enhance the text generated by the LLM.
We'll update the prompt to include the context, and to ask the LLM to use the context when responding:
```js
import { Hono } from "hono";
const app = new Hono();
// Existing post route...
// app.post('/notes', async (c) => { ... })
app.get("/", async (c) => {
const question = c.req.query("text") || "What is the square root of 9?";
const embeddings = await c.env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: question,
});
const vectors = embeddings.data[0];
const vectorQuery = await c.env.VECTOR_INDEX.query(vectors, { topK: 1 });
let vecId;
if (
vectorQuery.matches &&
vectorQuery.matches.length > 0 &&
vectorQuery.matches[0]
) {
vecId = vectorQuery.matches[0].id;
} else {
console.log("No matching vector found or vectorQuery.matches is empty");
}
let notes = [];
if (vecId) {
const query = `SELECT * FROM notes WHERE id = ?`;
const { results } = await c.env.DB.prepare(query).bind(vecId).all();
if (results) notes = results.map((vec) => vec.text);
}
const contextMessage = notes.length
? `Context:\n${notes.map((note) => `- ${note}`).join("\n")}`
: "";
const systemPrompt = `When answering the question or responding, use the context provided, if it is provided and relevant.`;
const { response: answer } = await c.env.AI.run(
"@cf/meta/llama-3-8b-instruct",
{
messages: [
...(notes.length ? [{ role: "system", content: contextMessage }] : []),
{ role: "system", content: systemPrompt },
{ role: "user", content: question },
],
},
);
return c.text(answer);
});
app.onError((err, c) => {
return c.text(err);
});
export default app;
```
## 8. Adding Anthropic Claude model (optional)
If you are working with larger documents, you have the option to use Anthropic's [Claude models](https://claude.ai/), which have large context windows and are well-suited to RAG workflows.
To begin, install the `@anthropic-ai/sdk` package:
In `src/index.js`, you can update the `GET /` route to check for the `ANTHROPIC_API_KEY` environment variable. If it's set, we can generate text using the Anthropic SDK. If it isn't set, we'll fall back to the existing Workers AI code:
```js
import Anthropic from '@anthropic-ai/sdk';
app.get('/', async (c) => {
// ... Existing code
const systemPrompt = `When answering the question or responding, use the context provided, if it is provided and relevant.`
let modelUsed: string = ""
let response = null
if (c.env.ANTHROPIC_API_KEY) {
const anthropic = new Anthropic({
apiKey: c.env.ANTHROPIC_API_KEY
})
const model = "claude-3-5-sonnet-latest"
modelUsed = model
const message = await anthropic.messages.create({
max_tokens: 1024,
model,
messages: [
{ role: 'user', content: question }
],
system: [systemPrompt, notes ? contextMessage : ''].join(" ")
})
response = {
response: message.content.map(content => content.text).join("\n")
}
} else {
const model = "@cf/meta/llama-3.1-8b-instruct"
modelUsed = model
response = await c.env.AI.run(
model,
{
messages: [
...(notes.length ? [{ role: 'system', content: contextMessage }] : []),
{ role: 'system', content: systemPrompt },
{ role: 'user', content: question }
]
}
)
}
if (response) {
c.header('x-model-used', modelUsed)
return c.text(response.response)
} else {
return c.text("We were unable to generate output", 500)
}
})
```
Finally, you'll need to set the `ANTHROPIC_API_KEY` environment variable in your Workers application. You can do this by using `wrangler secret put`:
```sh
$ npx wrangler secret put ANTHROPIC_API_KEY
```
## 9. Deleting notes and vectors
If you no longer need a note, you can delete it from the database. Any time that you delete a note, you will also need to delete the corresponding vector from Vectorize. You can implement this by building a `DELETE /notes/:id` route in your `src/index.js` file:
```js
app.delete("/notes/:id", async (c) => {
const { id } = c.req.param();
const query = `DELETE FROM notes WHERE id = ?`;
await c.env.DB.prepare(query).bind(id).run();
await c.env.VECTOR_INDEX.deleteByIds([id]);
return c.status(204);
});
```
## 10. Text splitting (optional)
For large pieces of text, it is recommended to split the text into smaller chunks. This allows LLMs to more effectively gather relevant context, without needing to retrieve large pieces of text.
To implement this, we'll add a new NPM package to our project, `@langchain/textsplitters':
The `RecursiveCharacterTextSplitter` class provided by this package will split the text into smaller chunks. It can be customized to your liking, but the default config works in most cases:
```js
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
const text = "Some long piece of text...";
const splitter = new RecursiveCharacterTextSplitter({
// These can be customized to change the chunking size
// chunkSize: 1000,
// chunkOverlap: 200,
});
const output = await splitter.createDocuments([text]);
console.log(output); // [{ pageContent: 'Some long piece of text...' }]
```
To use this splitter, we'll update the workflow to split the text into smaller chunks. We'll then iterate over the chunks and run the rest of the workflow for each chunk of text:
```js
export class RAGWorkflow extends WorkflowEntrypoint {
async run(event, step) {
const env = this.env;
const { text } = event.payload;
let texts = await step.do("split text", async () => {
const splitter = new RecursiveCharacterTextSplitter();
const output = await splitter.createDocuments([text]);
return output.map((doc) => doc.pageContent);
});
console.log(
"RecursiveCharacterTextSplitter generated ${texts.length} chunks",
);
for (const index in texts) {
const text = texts[index];
const record = await step.do(
`create database record: ${index}/${texts.length}`,
async () => {
const query = "INSERT INTO notes (text) VALUES (?) RETURNING *";
const { results } = await env.DB.prepare(query).bind(text).run();
const record = results[0];
if (!record) throw new Error("Failed to create note");
return record;
},
);
const embedding = await step.do(
`generate embedding: ${index}/${texts.length}`,
async () => {
const embeddings = await env.AI.run("@cf/baai/bge-base-en-v1.5", {
text: text,
});
const values = embeddings.data[0];
if (!values) throw new Error("Failed to generate vector embedding");
return values;
},
);
await step.do(`insert vector: ${index}/${texts.length}`, async () => {
return env.VECTOR_INDEX.upsert([
{
id: record.id.toString(),
values: embedding,
},
]);
});
}
}
}
```
Now, when large pieces of text are submitted to the `/notes` endpoint, they will be split into smaller chunks, and each chunk will be processed by the workflow.
## 11. Deploy your project
If you did not deploy your Worker during [step 1](/workers/get-started/guide/#1-create-a-new-worker-project), deploy your Worker via Wrangler, to a `*.workers.dev` subdomain, or a [Custom Domain](/workers/configuration/routing/custom-domains/), if you have one configured. If you have not configured any subdomain or domain, Wrangler will prompt you during the publish process to set one up.
```sh
npx wrangler deploy
```
Preview your Worker at `..workers.dev`.
:::note[Note]
When pushing to your `*.workers.dev` subdomain for the first time, you may see [`523` errors](/support/troubleshooting/http-status-codes/cloudflare-5xx-errors/error-523/) while DNS is propagating. These errors should resolve themselves after a minute or so.
:::
## Related resources
A full version of this codebase is available on GitHub. It includes a frontend UI for querying, adding, and deleting notes, as well as a backend API for interacting with the database and vector index. You can find it here: [github.com/kristianfreeman/cloudflare-retrieval-augmented-generation-example](https://github.com/kristianfreeman/cloudflare-retrieval-augmented-generation-example/).
To do more:
- Explore the reference diagram for a [Retrieval Augmented Generation (RAG) Architecture](/reference-architecture/diagrams/ai/ai-rag/).
- Review Cloudflare's [AI documentation](/workers-ai).
- Review [Tutorials](/workers/tutorials/) to build projects on Workers.
- Explore [Examples](/workers/examples/) to experiment with copy and paste Worker code.
- Understand how Workers works in [Reference](/workers/reference/).
- Learn about Workers features and functionality in [Platform](/workers/platform/).
- Set up [Wrangler](/workers/wrangler/install-and-update/) to programmatically create, test, and deploy your Worker projects.
---
# Build a Voice Notes App with auto transcriptions using Workers AI
URL: https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-voice-notes-app-with-auto-transcription/
import { Render, PackageManagers, Tabs, TabItem } from "~/components";
In this tutorial, you will learn how to create a Voice Notes App with automatic transcriptions of voice recordings, and optional post-processing. The following tools will be used to build the application:
- Workers AI to transcribe the voice recordings, and for the optional post processing
- D1 database to store the notes
- R2 storage to store the voice recordings
- Nuxt framework to build the full-stack application
- Workers to deploy the project
## Prerequisites
To continue, you will need:
## 1. Create a new Worker project
Create a new Worker project using the `c3` CLI with the `nuxt` framework preset.
### Install additional dependencies
Change into the newly created project directory
```sh
cd voice-notes
```
And install the following dependencies:
Then add the `@nuxt/ui` module to the `nuxt.config.ts` file:
```ts title="nuxt.config.ts"
export default defineNuxtConfig({
//..
modules: ['nitro-cloudflare-dev', '@nuxt/ui'],
//..
})
```
### [Optional] Move to Nuxt 4 compatibility mode
Moving to Nuxt 4 compatibility mode ensures that your application remains forward-compatible with upcoming updates to Nuxt.
Create a new `app` folder in the project's root directory and move the `app.vue` file to it. Also, add the following to your `nuxt.config.ts` file:
```ts title="nuxt.config.ts"
export default defineNuxtConfig({
//..
future: {
compatibilityVersion: 4,
},
//..
})
```
:::note
The rest of the tutorial will use the `app` folder for keeping the client side code. If you did not make this change, you should continue to use the project's root directory.
:::
### Start local development server
At this point you can test your application by starting a local development server using:
If everything is set up correctly, you should see a Nuxt welcome page at `http://localhost:3000`.
## 2. Create the transcribe API endpoint
This API makes use of Workers AI to transcribe the voice recordings. To use Workers AI within your project, you first need to bind it to the Worker.
Add the `AI` binding to the Wrangler file.
```toml title="wrangler.toml"
[ai]
binding = "AI"
```
Once the `AI` binding has been configured, run the `cf-typegen` command to generate the necessary Cloudflare type definitions. This makes the types definitions available in the server event contexts.
Create a transcribe `POST` endpoint by creating `transcribe.post.ts` file inside the `/server/api` directory.
```ts title="server/api/transcribe.post.ts"
export default defineEventHandler(async (event) => {
const { cloudflare } = event.context;
const form = await readFormData(event);
const blob = form.get('audio') as Blob;
if (!blob) {
throw createError({
statusCode: 400,
message: 'Missing audio blob to transcribe',
});
}
try {
const response = await cloudflare.env.AI.run('@cf/openai/whisper', {
audio: [...new Uint8Array(await blob.arrayBuffer())],
});
return response.text;
} catch (err) {
console.error('Error transcribing audio:', err);
throw createError({
statusCode: 500,
message: 'Failed to transcribe audio. Please try again.',
});
}
});
```
The above code does the following:
1. Extracts the audio blob from the event.
2. Transcribes the blob using the `@cf/openai/whisper` model and returns the transcription text as response.
## 3. Create an API endpoint for uploading audio recordings to R2
Before uploading the audio recordings to `R2`, you need to create a bucket first. You will also need to add the R2 binding to your Wrangler file and regenerate the Cloudflare type definitions.
Create an `R2` bucket.
Add the storage binding to your Wrangler file.
```toml title="wrangler.toml"
[[r2_buckets]]
binding = "R2"
bucket_name = ""
```
Finally, generate the type definitions by rerunning the `cf-typegen` script.
Now you are ready to create the upload endpoint. Create a new `upload.put.ts` file in your `server/api` directory, and add the following code to it:
```ts title="server/api/upload.put.ts"
export default defineEventHandler(async (event) => {
const { cloudflare } = event.context;
const form = await readFormData(event);
const files = form.getAll('files') as File[];
if (!files) {
throw createError({ statusCode: 400, message: 'Missing files' });
}
const uploadKeys: string[] = [];
for (const file of files) {
const obj = await cloudflare.env.R2.put(`recordings/${file.name}`, file);
if (obj) {
uploadKeys.push(obj.key);
}
}
return uploadKeys;
});
```
The above code does the following:
1. The files variable retrieves all files sent by the client using form.getAll(), which allows for multiple uploads in a single request.
2. Uploads the files to the R2 bucket using the binding (`R2`) you created earlier.
:::note
The `recordings/` prefix organizes uploaded files within a dedicated folder in your bucket. This will also come in handy when serving these recordings to the client (covered later).
:::
## 4. Create an API endpoint to save notes entries
Before creating the endpoint, you will need to perform steps similar to those for the R2 bucket, with some additional steps to prepare a notes table.
Create a `D1` database.
Add the D1 bindings to the Wrangler file. You can get the `DB_ID` from the output of the `d1 create` command.
```toml title="wrangler.toml"
[[d1_databases]]
binding = "DB"
database_name = ""
database_id = ""
```
As before, rerun the `cf-typegen` command to generate the types.
Next, create a DB migration.
"create notes table"`} />
This will create a new `migrations` folder in the project's root directory, and add an empty `0001_create_notes_table.sql` file to it. Replace the contents of this file with the code below.
```sql
CREATE TABLE IF NOT EXISTS notes (
id INTEGER PRIMARY KEY AUTOINCREMENT,
text TEXT NOT NULL,
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,
audio_urls TEXT
);
```
And then apply this migration to create the `notes` table.
:::note
The above command will create the notes table locally. To apply the migration on your remote production database, use the `--remote` flag.
:::
Now you can create the API endpoint. Create a new file `index.post.ts` in the `server/api/notes` directory, and change its content to the following:
```ts title="server/api/notes/index.post.ts"
export default defineEventHandler(async (event) => {
const { cloudflare } = event.context;
const { text, audioUrls } = await readBody(event);
if (!text) {
throw createError({
statusCode: 400,
message: 'Missing note text',
});
}
try {
await cloudflare.env.DB.prepare(
'INSERT INTO notes (text, audio_urls) VALUES (?1, ?2)'
)
.bind(text, audioUrls ? JSON.stringify(audioUrls) : null)
.run();
return setResponseStatus(event, 201);
} catch (err) {
console.error('Error creating note:', err);
throw createError({
statusCode: 500,
message: 'Failed to create note. Please try again.',
});
}
});
```
The above does the following:
1. Extracts the text, and optional audioUrls from the event.
2. Saves it to the database after converting the audioUrls to a `JSON` string.
## 5. Handle note creation on the client-side
Now you're ready to work on the client side. Let's start by tackling the note creation part first.
### Recording user audio
Create a composable to handle audio recording using the MediaRecorder API. This will be used to record notes through the user's microphone.
Create a new file `useMediaRecorder.ts` in the `app/composables` folder, and add the following code to it:
```ts title="app/composables/useMediaRecorder.ts"
interface MediaRecorderState {
isRecording: boolean;
recordingDuration: number;
audioData: Uint8Array | null;
updateTrigger: number;
}
export function useMediaRecorder() {
const state = ref({
isRecording: false,
recordingDuration: 0,
audioData: null,
updateTrigger: 0,
});
let mediaRecorder: MediaRecorder | null = null;
let audioContext: AudioContext | null = null;
let analyser: AnalyserNode | null = null;
let animationFrame: number | null = null;
let audioChunks: Blob[] | undefined = undefined;
const updateAudioData = () => {
if (!analyser || !state.value.isRecording || !state.value.audioData) {
if (animationFrame) {
cancelAnimationFrame(animationFrame);
animationFrame = null;
}
return;
}
analyser.getByteTimeDomainData(state.value.audioData);
state.value.updateTrigger += 1;
animationFrame = requestAnimationFrame(updateAudioData);
};
const startRecording = async () => {
try {
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
audioContext = new AudioContext();
analyser = audioContext.createAnalyser();
const source = audioContext.createMediaStreamSource(stream);
source.connect(analyser);
mediaRecorder = new MediaRecorder(stream);
audioChunks = [];
mediaRecorder.ondataavailable = (e: BlobEvent) => {
audioChunks?.push(e.data);
state.value.recordingDuration += 1;
};
state.value.audioData = new Uint8Array(analyser.frequencyBinCount);
state.value.isRecording = true;
state.value.recordingDuration = 0;
state.value.updateTrigger = 0;
mediaRecorder.start(1000);
updateAudioData();
} catch (err) {
console.error('Error accessing microphone:', err);
throw err;
}
};
const stopRecording = async () => {
return await new Promise((resolve) => {
if (mediaRecorder && state.value.isRecording) {
mediaRecorder.onstop = () => {
const blob = new Blob(audioChunks, { type: 'audio/webm' });
audioChunks = undefined;
state.value.recordingDuration = 0;
state.value.updateTrigger = 0;
state.value.audioData = null;
resolve(blob);
};
state.value.isRecording = false;
mediaRecorder.stop();
mediaRecorder.stream.getTracks().forEach((track) => track.stop());
if (animationFrame) {
cancelAnimationFrame(animationFrame);
animationFrame = null;
}
audioContext?.close();
audioContext = null;
}
});
};
onUnmounted(() => {
stopRecording();
});
return {
state: readonly(state),
startRecording,
stopRecording,
};
}
```
The above code does the following:
1. Exposes functions to start and stop audio recordings in a Vue application.
2. Captures audio input from the user's microphone using MediaRecorder API.
3. Processes real-time audio data for visualization using AudioContext and AnalyserNode.
4. Stores recording state including duration and recording status.
5. Maintains chunks of audio data and combines them into a final audio blob when recording stops.
6. Updates audio visualization data continuously using animation frames while recording.
7. Automatically cleans up all audio resources when recording stops or component unmounts.
8. Returns audio recordings in webm format for further processing.
### Create a component for note creation
This component allows users to create notes by either typing or recording audio. It also handles audio transcription and uploading the recordings to the server.
Create a new file named `CreateNote.vue` inside the `app/components` folder. Add the following template code to the newly created file:
```vue title="app/components/CreateNote.vue"
Note transcript
Note recordings
Transcribing...
No recordings...
Clear
Save
```
The above template results in the following:
1. A panel with a `textarea` inside to type the note manually.
2. Another panel to manage start/stop of an audio recording, and show the recordings done already.
3. A bottom panel to reset or save the note (along with the recordings).
Now, add the following code below the template code in the same file:
```vue title="app/components/CreateNote.vue"
```
The above code does the following:
1. When a recording is stopped by calling `handleRecordingStop` function, the audio blob is sent for transcribing to the transcribe API endpoint.
2. The transcription response text is appended to the existing textarea content.
3. When the note is saved by calling the `saveNote` function, the audio recordings are uploaded first to R2 by using the upload endpoint we created earlier. Then, the actual note content along with the audioUrls (the R2 object keys) are saved by calling the notes post endpoint.
### Create a new page route for showing the component
You can use this component in a Nuxt page to show it to the user. But before that you need to modify your `app.vue` file. Update the content of your `app.vue` to the following:
```vue title="/app/app.vue"
```
The above code allows for a nuxt page to be shown to the user, apart from showing an app header and a navigation sidebar.
Next, add a new file named `new.vue` inside the `app/pages` folder, add the following code to it:
```vue title="app/pages/new.vue"
Create note
```
The above code shows the `CreateNote` component inside a modal, and navigates back to the home page on successful note creation.
## 6. Showing the notes on the client side
To show the notes from the database on the client side, create an API endpoint first that will interact with the database.
### Create an API endpoint to fetch notes from the database
Create a new file named `index.get.ts` inside the `server/api/notes` directory, and add the following code to it:
```ts title="server/api/index.get.ts"
import type { Note } from '~~/types';
export default defineEventHandler(async (event) => {
const { cloudflare } = event.context;
const res = await cloudflare.env.DB.prepare(
`SELECT
id,
text,
audio_urls AS audioUrls,
created_at AS createdAt,
updated_at AS updatedAt
FROM notes
ORDER BY created_at DESC
LIMIT 50;`
).all & { audioUrls: string | null }>();
return res.results.map((note) => ({
...note,
audioUrls: note.audioUrls ? JSON.parse(note.audioUrls) : undefined,
}));
});
```
The above code fetches the last 50 notes from the database, ordered by their creation date in descending order. The `audio_urls` field is stored as a string in the database, but it's converted to an array using `JSON.parse` to handle multiple audio files seamlessly on the client side.
Next, create a page named `index.vue` inside the `app/pages` directory. This will be the home page of the application. Add the following code to it:
```vue title="app/pages/index.vue"
No notes created
Get started by creating your first note
```
The above code fetches the notes from the database by calling the `/api/notes` endpoint you created just now, and renders them as note cards.
### Serving the saved recordings from R2
To be able to play the audio recordings of these notes, you need to serve the saved recordings from the R2 storage.
Create a new file named `[...pathname].get.ts` inside the `server/routes/recordings` directory, and add the following code to it:
:::note
The `...` prefix in the file name makes it a catch all route. This allows it to receive all events that are meant for paths starting with `/recordings` prefix. This is where the `recordings` prefix that was added previously while saving the recordings becomes helpful.
:::
```ts title="server/routes/recordings/[...pathname].get.ts"
export default defineEventHandler(async (event) => {
const { cloudflare, params } = event.context;
const { pathname } = params || {};
return cloudflare.env.R2.get(`recordings/${pathname}`);
});
```
The above code extracts the path name from the event params, and serves the saved recording matching that object key from the R2 bucket.
## 7. [Optional] Post Processing the transcriptions
Even though the speech-to-text transcriptions models perform satisfactorily, sometimes you want to post process the transcriptions for various reasons. It could be to remove any discrepancy, or to change the tone/style of the final text.
### Create a settings page
Create a new file named `settings.vue` in the `app/pages` folder, and add the following code to it:
```vue title="app/pages/settings.vue"
Post Processing
Configure post-processing of recording transcriptions with AI models.
Settings changes are auto-saved locally.
```
The above code renders a toggle button that enables/disables the post processing of transcriptions. If enabled, users can change the prompt that will used while post processing the transcription with an AI model.
The transcription settings are saved using useStorageAsync, which utilizes the browser's local storage. This ensures that users' preferences are retained even after refreshing the page.
### Send the post processing prompt with recorded audio
Modify the `CreateNote` component to send the post processing prompt along with the audio blob, while calling the `transcribe` API endpoint.
```vue title="app/components/CreateNote.vue" ins={2, 6-9, 17-22}
```
The code blocks added above checks for the saved post processing setting. If enabled, and there is a defined prompt, it sends the prompt to the `transcribe` API endpoint.
### Handle post processing in the transcribe API endpoint
Modify the transcribe API endpoint, and update it to the following:
```ts title="server/api/transcribe.post.ts" ins={9-20, 22}
export default defineEventHandler(async (event) => {
// ...
try {
const response = await cloudflare.env.AI.run('@cf/openai/whisper', {
audio: [...new Uint8Array(await blob.arrayBuffer())],
});
const postProcessingPrompt = form.get('prompt') as string;
if (postProcessingPrompt && response.text) {
const postProcessResult = await cloudflare.env.AI.run(
'@cf/meta/llama-3.1-8b-instruct',
{
temperature: 0.3,
prompt: `${postProcessingPrompt}.\n\nText:\n\n${response.text}\n\nResponse:`,
}
);
return (postProcessResult as { response?: string }).response;
} else {
return response.text;
}
} catch (err) {
// ...
}
});
```
The above code does the following:
1. Extracts the post processing prompt from the event FormData.
2. If present, it calls the Workers AI API to process the transcription text using the `@cf/meta/llama-3.1-8b-instruct` model.
3. Finally, it returns the response from Workers AI to the client.
## 8. Deploy the application
Now you are ready to deploy the project to a `.workers.dev` sub-domain by running the deploy command.
You can preview your application at `..workers.dev`.
:::note
If you used `pnpm` as your package manager, you may face build errors like `"stdin" is not exported by "node_modules/.pnpm/unenv@1.10.0/node_modules/unenv/runtime/node/process/index.mjs"`. To resolve it, you can try hoisting your node modules with the [`shamefully-hoist-true`](https://pnpm.io/npmrc) option.
:::
## Conclusion
In this tutorial, you have gone through the steps of building a voice notes application using Nuxt 3, Cloudflare Workers, D1, and R2 storage. You learnt to:
- Set up the backend to store and manage notes
- Create API endpoints to fetch and display notes
- Handle audio recordings
- Implement optional post-processing for transcriptions
- Deploy the application using the Cloudflare module syntax
The complete source code of the project is available on GitHub. You can go through it to see the code for various frontend components not covered in the article. You can find it here: [github.com/ra-jeev/vnotes](https://github.com/ra-jeev/vnotes).
---
# Whisper-large-v3-turbo with Cloudflare Workers AI
URL: https://developers.cloudflare.com/workers-ai/guides/tutorials/build-a-workers-ai-whisper-with-chunking/
In this tutorial you will learn how to:
- **Transcribe large audio files:** Use the [Whisper-large-v3-turbo](/workers-ai/models/whisper-large-v3-turbo/) model from Cloudflare Workers AI to perform automatic speech recognition (ASR) or translation.
- **Handle large files:** Split large audio files into smaller chunks for processing, which helps overcome memory and execution time limitations.
- **Deploy using Cloudflare Workers:** Create a scalable, low‑latency transcription pipeline in a serverless environment.
## 1: Create a new Cloudflare Worker project
import { Render, PackageManagers, WranglerConfig } from "~/components";
You will create a new Worker project using the `create-cloudflare` CLI (C3). [C3](https://github.com/cloudflare/workers-sdk/tree/main/packages/create-cloudflare) is a command-line tool designed to help you set up and deploy new applications to Cloudflare.
Create a new project named `whisper-tutorial` by running:
Running `npm create cloudflare@latest` will prompt you to install the [`create-cloudflare` package](https://www.npmjs.com/package/create-cloudflare), and lead you through setup. C3 will also install [Wrangler](/workers/wrangler/), the Cloudflare Developer Platform CLI.
This will create a new `whisper-tutorial` directory. Your new `whisper-tutorial` directory will include:
- A `"Hello World"` [Worker](/workers/get-started/guide/#3-write-code) at `src/index.ts`.
- A [`wrangler.jsonc`](/workers/wrangler/configuration/) configuration file.
Go to your application directory:
```sh
cd whisper-tutorial
```
## 2. Connect your Worker to Workers AI
You must create an AI binding for your Worker to connect to Workers AI. [Bindings](/workers/runtime-apis/bindings/) allow your Workers to interact with resources, like Workers AI, on the Cloudflare Developer Platform.
To bind Workers AI to your Worker, add the following to the end of your `wrangler.toml` file:
```toml
[ai]
binding = "AI"
```
Your binding is [available in your Worker code](/workers/reference/migrate-to-module-workers/#bindings-in-es-modules-format) on [`env.AI`](/workers/runtime-apis/handlers/fetch/).
## 3. Configure Wrangler
In your wrangler file, add or update the following settings to enable Node.js APIs and polyfills (with a compatibility date of 2024‑09‑23 or later):
```toml title="wrangler.toml"
compatibility_flags = [ "nodejs_compat" ]
compatibility_date = "2024-09-23"
```
## 4. Handle large audio files with chunking
Replace the contents of your `src/index.ts` file with the following integrated code. This sample demonstrates how to:
(1) Extract an audio file URL from the query parameters.
(2) Fetch the audio file while explicitly following redirects.
(3) Split the audio file into smaller chunks (such as, 1 MB chunks).
(4) Transcribe each chunk using the Whisper-large-v3-turbo model via the Cloudflare AI binding.
(5) Return the aggregated transcription as plain text.
```ts
import { Buffer } from "node:buffer";
import type { Ai } from "workers-ai";
export interface Env {
AI: Ai;
// If needed, add your KV namespace for storing transcripts.
// MY_KV_NAMESPACE: KVNamespace;
}
/**
* Fetches the audio file from the provided URL and splits it into chunks.
* This function explicitly follows redirects.
*
* @param audioUrl - The URL of the audio file.
* @returns An array of ArrayBuffers, each representing a chunk of the audio.
*/
async function getAudioChunks(audioUrl: string): Promise {
const response = await fetch(audioUrl, { redirect: "follow" });
if (!response.ok) {
throw new Error(`Failed to fetch audio: ${response.status}`);
}
const arrayBuffer = await response.arrayBuffer();
// Example: Split the audio into 1MB chunks.
const chunkSize = 1024 * 1024; // 1MB
const chunks: ArrayBuffer[] = [];
for (let i = 0; i < arrayBuffer.byteLength; i += chunkSize) {
const chunk = arrayBuffer.slice(i, i + chunkSize);
chunks.push(chunk);
}
return chunks;
}
/**
* Transcribes a single audio chunk using the Whisper‑large‑v3‑turbo model.
* The function converts the audio chunk to a Base64-encoded string and
* sends it to the model via the AI binding.
*
* @param chunkBuffer - The audio chunk as an ArrayBuffer.
* @param env - The Cloudflare Worker environment, including the AI binding.
* @returns The transcription text from the model.
*/
async function transcribeChunk(
chunkBuffer: ArrayBuffer,
env: Env,
): Promise {
const base64 = Buffer.from(chunkBuffer, "binary").toString("base64");
const res = await env.AI.run("@cf/openai/whisper-large-v3-turbo", {
audio: base64,
// Optional parameters (uncomment and set if needed):
// task: "transcribe", // or "translate"
// language: "en",
// vad_filter: "false",
// initial_prompt: "Provide context if needed.",
// prefix: "Transcription:",
});
return res.text; // Assumes the transcription result includes a "text" property.
}
/**
* The main fetch handler. It extracts the 'url' query parameter, fetches the audio,
* processes it in chunks, and returns the full transcription.
*/
export default {
async fetch(
request: Request,
env: Env,
ctx: ExecutionContext,
): Promise {
// Extract the audio URL from the query parameters.
const { searchParams } = new URL(request.url);
const audioUrl = searchParams.get("url");
if (!audioUrl) {
return new Response("Missing 'url' query parameter", { status: 400 });
}
// Get the audio chunks.
const audioChunks: ArrayBuffer[] = await getAudioChunks(audioUrl);
let fullTranscript = "";
// Process each chunk and build the full transcript.
for (const chunk of audioChunks) {
try {
const transcript = await transcribeChunk(chunk, env);
fullTranscript += transcript + "\n";
} catch (error) {
fullTranscript += "[Error transcribing chunk]\n";
}
}
return new Response(fullTranscript, {
headers: { "Content-Type": "text/plain" },
});
},
} satisfies ExportedHandler;
```
## 5. Deploy your Worker
1. **Run the Worker locally:**
Use wrangler's development mode to test your Worker locally:
```sh
npx wrangler dev
```
Open your browser and go to [http://localhost:8787](http://localhost:8787), or use curl:
```sh
curl "http://localhost:8787?url=https://raw.githubusercontent.com/your-username/your-repo/main/your-audio-file.mp3"
```
Replace the URL query parameter with the direct link to your audio file. (For GitHub-hosted files, ensure you use the raw file URL.)
2. **Deploy the Worker:**
Once testing is complete, deploy your Worker with:
```sh
npx wrangler deploy
```
3. **Test the deployed Worker:**
After deployment, test your Worker by passing the audio URL as a query parameter:
```sh
curl "https://.workers.dev?url=https://raw.githubusercontent.com/your-username/your-repo/main/your-audio-file.mp3"
```
Make sure to replace ``, `your-username`, `your-repo`, and `your-audio-file.mp3` with your actual details.
If successful, the Worker will return a transcript of the audio file:
```sh
This is the transcript of the audio...
```
---
# Build an interview practice tool with Workers AI
URL: https://developers.cloudflare.com/workers-ai/guides/tutorials/build-ai-interview-practice-tool/
import { Render, PackageManagers } from "~/components";
Job interviews can be stressful, and practice is key to building confidence. While traditional mock interviews with friends or mentors are valuable, they are not always available when you need them. In this tutorial, you will learn how to build an AI-powered interview practice tool that provides real-time feedback to help improve interview skills.
By the end of this tutorial, you will have built a complete interview practice tool with the following core functionalities:
- A real-time interview simulation tool using WebSocket connections
- An AI-powered speech processing pipeline that converts audio to text
- An intelligent response system that provides interviewer-like interactions
- A persistent storage system for managing interview sessions and history using Durable Objects
### Prerequisites
This tutorial demonstrates how to use multiple Cloudflare products and while many features are available in free tiers, some components of Workers AI may incur usage-based charges. Please review the pricing documentation for Workers AI before proceeding.
## 1. Create a new Worker project
Create a Cloudflare Workers project using the Create Cloudflare CLI (C3) tool and the Hono framework.
:::note
[Hono](https://hono.dev) is a lightweight web framework that helps build API endpoints and handle HTTP requests. This tutorial uses Hono to create and manage the application's routing and middleware components.
:::
Create a new Worker project by running the following commands, using `ai-interview-tool` as the Worker name:
To develop and test your Cloudflare Workers application locally:
1. Navigate to your Workers project directory in your terminal:
```sh
cd ai-interview-tool
```
2. Start the development server by running:
```sh
npx wrangler dev
```
When you run `wrangler dev`, the command starts a local development server and provides a `localhost` URL where you can preview your application.
You can now make changes to your code and see them reflected in real-time at the provided localhost address.
## 2. Define TypeScript types for the interview system
Now that the project is set up, create the TypeScript types that will form the foundation of the interview system. These types will help you maintain type safety and provide clear interfaces for the different components of your application.
Create a new file `types.ts` that will contain essential types and enums for:
- Interview skills that can be assessed (JavaScript, React, etc.)
- Different interview positions (Junior Developer, Senior Developer, etc.)
- Interview status tracking
- Message handling between user and AI
- Core interview data structure
```typescript title="src/types.ts"
import { Context } from "hono";
// Context type for API endpoints, including environment bindings and user info
export interface ApiContext {
Bindings: CloudflareBindings;
Variables: {
username: string;
};
}
export type HonoCtx = Context;
// List of technical skills you can assess during mock interviews.
// This application focuses on popular web technologies and programming languages
// that are commonly tested in real interviews.
export enum InterviewSkill {
JavaScript = "JavaScript",
TypeScript = "TypeScript",
React = "React",
NodeJS = "NodeJS",
Python = "Python",
}
// Available interview types based on different engineering roles.
// This helps tailor the interview experience and questions to
// match the candidate's target position.
export enum InterviewTitle {
JuniorDeveloper = "Junior Developer Interview",
SeniorDeveloper = "Senior Developer Interview",
FullStackDeveloper = "Full Stack Developer Interview",
FrontendDeveloper = "Frontend Developer Interview",
BackendDeveloper = "Backend Developer Interview",
SystemArchitect = "System Architect Interview",
TechnicalLead = "Technical Lead Interview",
}
// Tracks the current state of an interview session.
// This will help you to manage the interview flow and show appropriate UI/actions
// at each stage of the process.
export enum InterviewStatus {
Created = "created", // Interview is created but not started
Pending = "pending", // Waiting for interviewer/system
InProgress = "in_progress", // Active interview session
Completed = "completed", // Interview finished successfully
Cancelled = "cancelled", // Interview terminated early
}
// Defines who sent a message in the interview chat
export type MessageRole = "user" | "assistant" | "system";
// Structure of individual messages exchanged during the interview
export interface Message {
messageId: string; // Unique identifier for the message
interviewId: string; // Links message to specific interview
role: MessageRole; // Who sent the message
content: string; // The actual message content
timestamp: number; // When the message was sent
}
// Main data structure that holds all information about an interview session.
// This includes metadata, messages exchanged, and the current status.
export interface InterviewData {
interviewId: string;
title: InterviewTitle;
skills: InterviewSkill[];
messages: Message[];
status: InterviewStatus;
createdAt: number;
updatedAt: number;
}
// Input format for creating a new interview session.
// Simplified interface that accepts basic parameters needed to start an interview.
export interface InterviewInput {
title: string;
skills: string[];
}
```
## 3. Configure error types for different services
Next, set up custom error types to handle different kinds of errors that may occur in your application. This includes:
- Database errors (for example, connection issues, query failures)
- Interview-related errors (for example, invalid input, transcription failures)
- Authentication errors (for example, invalid sessions)
Create the following `errors.ts` file:
```typescript title="src/errors.ts"
export const ErrorCodes = {
INVALID_MESSAGE: "INVALID_MESSAGE",
TRANSCRIPTION_FAILED: "TRANSCRIPTION_FAILED",
LLM_FAILED: "LLM_FAILED",
DATABASE_ERROR: "DATABASE_ERROR",
} as const;
export class AppError extends Error {
constructor(
message: string,
public statusCode: number,
) {
super(message);
this.name = this.constructor.name;
}
}
export class UnauthorizedError extends AppError {
constructor(message: string) {
super(message, 401);
}
}
export class BadRequestError extends AppError {
constructor(message: string) {
super(message, 400);
}
}
export class NotFoundError extends AppError {
constructor(message: string) {
super(message, 404);
}
}
export class InterviewError extends Error {
constructor(
message: string,
public code: string,
public statusCode: number = 500,
) {
super(message);
this.name = "InterviewError";
}
}
```
## 4. Configure authentication middleware and user routes
In this step, you will implement a basic authentication system to track and identify users interacting with your AI interview practice tool. The system uses HTTP-only cookies to store usernames, allowing you to identify both the request sender and their corresponding Durable Object. This straightforward authentication approach requires users to provide a username, which is then stored securely in a cookie. This approach allows you to:
- Identify users across requests
- Associate interview sessions with specific users
- Secure access to interview-related endpoints
### Create the Authentication Middleware
Create a middleware function that will check for the presence of a valid authentication cookie. This middleware will be used to protect routes that require authentication.
Create a new middleware file `middleware/auth.ts`:
```typescript title="src/middleware/auth.ts"
import { Context } from "hono";
import { getCookie } from "hono/cookie";
import { UnauthorizedError } from "../errors";
export const requireAuth = async (ctx: Context, next: () => Promise) => {
// Get username from cookie
const username = getCookie(ctx, "username");
if (!username) {
throw new UnauthorizedError("User is not logged in");
}
// Make username available to route handlers
ctx.set("username", username);
await next();
};
```
This middleware:
- Checks for a `username` cookie
- Throws an `Error` if the cookie is missing
- Makes the username available to downstream handlers via the context
### Create Authentication Routes
Next, create the authentication routes that will handle user login. Create a new file `routes/auth.ts`:
```typescript title="src/routes/auth.ts"
import { Context, Hono } from "hono";
import { setCookie } from "hono/cookie";
import { BadRequestError } from "../errors";
import { ApiContext } from "../types";
export const authenticateUser = async (ctx: Context) => {
// Extract username from request body
const { username } = await ctx.req.json();
// Make sure username was provided
if (!username) {
throw new BadRequestError("Username is required");
}
// Create a secure cookie to track the user's session
// This cookie will:
// - Be HTTP-only for security (no JS access)
// - Work across all routes via path="/"
// - Last for 24 hours
// - Only be sent in same-site requests to prevent CSRF
setCookie(ctx, "username", username, {
httpOnly: true,
path: "/",
maxAge: 60 * 60 * 24,
sameSite: "Strict",
});
// Let the client know login was successful
return ctx.json({ success: true });
};
// Set up authentication-related routes
export const configureAuthRoutes = () => {
const router = new Hono();
// POST /login - Authenticate user and create session
router.post("/login", authenticateUser);
return router;
};
```
Finally, update main application file to include the authentication routes. Modify `src/index.ts`:
```typescript title="src/index.ts"
import { configureAuthRoutes } from "./routes/auth";
import { Hono } from "hono";
import { logger } from "hono/logger";
import type { ApiContext } from "./types";
import { requireAuth } from "./middleware/auth";
// Create our main Hono app instance with proper typing
const app = new Hono();
// Create a separate router for API endpoints to keep things organized
const api = new Hono();
// Set up global middleware that runs on every request
// - Logger gives us visibility into what is happening
app.use("*", logger());
// Wire up all our authentication routes (login, etc)
// These will be mounted under /api/v1/auth/
api.route("/auth", configureAuthRoutes());
// Mount all API routes under the version prefix (for example, /api/v1)
// This allows us to make breaking changes in v2 without affecting v1 users
app.route("/api/v1", api);
export default app;
```
Now we have a basic authentication system that:
1. Provides a login endpoint at `/api/v1/auth/login`
2. Securely stores the username in a cookie
3. Includes middleware to protect authenticated routes
## 5. Create a Durable Object to manage interviews
Now that you have your authentication system in place, create a Durable Object to manage interview sessions. Durable Objects are perfect for this interview practice tool because they provide the following functionalities:
- Maintains states between connections, so users can reconnect without losing progress.
- Provides a SQLite database to store all interview Q&A, feedback and metrics.
- Enables smooth real-time interactions between the interviewer AI and candidate.
- Handles multiple interview sessions efficiently without performance issues.
- Creates a dedicated instance for each user, giving them their own isolated environment.
First, you will need to configure the Durable Object in Wrangler file. Add the following configuration:
```toml title="wrangler.toml"
[[durable_objects.bindings]]
name = "INTERVIEW"
class_name = "Interview"
[[migrations]]
tag = "v1"
new_sqlite_classes = ["Interview"]
```
Next, create a new file `interview.ts` to define our Interview Durable Object:
```typescript title="src/interview.ts"
import { DurableObject } from "cloudflare:workers";
export class Interview extends DurableObject {
// We will use it to keep track of all active WebSocket connections for real-time communication
private sessions: Map;
constructor(state: DurableObjectState, env: CloudflareBindings) {
super(state, env);
// Initialize empty sessions map - we will add WebSocket connections as users join
this.sessions = new Map();
}
// Entry point for all HTTP requests to this Durable Object
// This will handle both initial setup and WebSocket upgrades
async fetch(request: Request) {
// For now, just confirm the object is working
// We'll add WebSocket upgrade logic and request routing later
return new Response("Interview object initialized");
}
// Broadcasts a message to all connected WebSocket clients.
private broadcast(message: string) {
this.ctx.getWebSockets().forEach((ws) => {
try {
if (ws.readyState === WebSocket.OPEN) {
ws.send(message);
}
} catch (error) {
console.error(
"Error broadcasting message to a WebSocket client:",
error,
);
}
});
}
}
```
Now we need to export the Durable Object in our main `src/index.ts` file:
```typescript title="src/index.ts"
import { Interview } from "./interview";
// ... previous code ...
export { Interview };
export default app;
```
Since the Worker code is written in TypeScript, you should run the following command to add the necessary type definitions:
```sh
npm run cf-typegen
```
### Set up SQLite database schema to store interview data
Now you will use SQLite at the Durable Object level for data persistence. This gives each user their own isolated database instance. You will need two main tables:
- `interviews`: Stores interview session data
- `messages`: Stores all messages exchanged during interviews
Before you create these tables, create a service class to handle your database operations. This encapsulates database logic and helps you:
- Manage database schema changes
- Handle errors consistently
- Keep database queries organized
Create a new file called `services/InterviewDatabaseService.ts`:
```typescript title="src/services/InterviewDatabaseService.ts"
import {
InterviewData,
Message,
InterviewStatus,
InterviewTitle,
InterviewSkill,
} from "../types";
import { InterviewError, ErrorCodes } from "../errors";
const CONFIG = {
database: {
tables: {
interviews: "interviews",
messages: "messages",
},
indexes: {
messagesByInterview: "idx_messages_interviewId",
},
},
} as const;
export class InterviewDatabaseService {
constructor(private sql: SqlStorage) {}
/**
* Sets up the database schema by creating tables and indexes if they do not exist.
* This is called when initializing a new Durable Object instance to ensure
* we have the required database structure.
*
* The schema consists of:
* - interviews table: Stores interview metadata like title, skills, and status
* - messages table: Stores the conversation history between user and AI
* - messages index: Helps optimize queries when fetching messages for a specific interview
*/
createTables() {
try {
// Get list of existing tables to avoid recreating them
const cursor = this.sql.exec(`PRAGMA table_list`);
const existingTables = new Set([...cursor].map((table) => table.name));
// The interviews table is our main table storing interview sessions.
// We only create it if it does not exist yet.
if (!existingTables.has(CONFIG.database.tables.interviews)) {
this.sql.exec(InterviewDatabaseService.QUERIES.CREATE_INTERVIEWS_TABLE);
}
// The messages table stores the actual conversation history.
// It references interviews table via foreign key for data integrity.
if (!existingTables.has(CONFIG.database.tables.messages)) {
this.sql.exec(InterviewDatabaseService.QUERIES.CREATE_MESSAGES_TABLE);
}
// Add an index on interviewId to speed up message retrieval.
// This is important since we will frequently query messages by interview.
this.sql.exec(InterviewDatabaseService.QUERIES.CREATE_MESSAGE_INDEX);
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
throw new InterviewError(
`Failed to initialize database: ${message}`,
ErrorCodes.DATABASE_ERROR,
);
}
}
private static readonly QUERIES = {
CREATE_INTERVIEWS_TABLE: `
CREATE TABLE IF NOT EXISTS interviews (
interviewId TEXT PRIMARY KEY,
title TEXT NOT NULL,
skills TEXT NOT NULL,
createdAt INTEGER NOT NULL DEFAULT (strftime('%s','now') * 1000),
updatedAt INTEGER NOT NULL DEFAULT (strftime('%s','now') * 1000),
status TEXT NOT NULL DEFAULT 'pending'
)
`,
CREATE_MESSAGES_TABLE: `
CREATE TABLE IF NOT EXISTS messages (
messageId TEXT PRIMARY KEY,
interviewId TEXT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
timestamp INTEGER NOT NULL,
FOREIGN KEY (interviewId) REFERENCES interviews(interviewId)
)
`,
CREATE_MESSAGE_INDEX: `
CREATE INDEX IF NOT EXISTS idx_messages_interview ON messages(interviewId)
`,
};
}
```
Update the `Interview` Durable Object to use the database service by modifying `src/interview.ts`:
```typescript title="src/interview.ts"
import { InterviewDatabaseService } from "./services/InterviewDatabaseService";
export class Interview extends DurableObject {
// Database service for persistent storage of interview data and messages
private readonly db: InterviewDatabaseService;
private sessions: Map;
constructor(state: DurableObjectState, env: CloudflareBindings) {
// ... previous code ...
// Set up our database connection using the DO's built-in SQLite instance
this.db = new InterviewDatabaseService(state.storage.sql);
// First-time setup: ensure our database tables exist
// This is idempotent so safe to call on every instantiation
this.db.createTables();
}
}
```
Add methods to create and retrieve interviews in `services/InterviewDatabaseService.ts`:
```typescript title="src/services/InterviewDatabaseService.ts"
export class InterviewDatabaseService {
/**
* Creates a new interview session in the database.
*
* This is the main entry point for starting a new interview. It handles all the
* initial setup like:
* - Generating a unique ID using crypto.randomUUID() for reliable uniqueness
* - Recording the interview title and required skills
* - Setting up timestamps for tracking interview lifecycle
* - Setting the initial status to "Created"
*
*/
createInterview(title: InterviewTitle, skills: InterviewSkill[]): string {
try {
const interviewId = crypto.randomUUID();
const currentTime = Date.now();
this.sql.exec(
InterviewDatabaseService.QUERIES.INSERT_INTERVIEW,
interviewId,
title,
JSON.stringify(skills), // Store skills as JSON for flexibility
InterviewStatus.Created,
currentTime,
currentTime,
);
return interviewId;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
throw new InterviewError(
`Failed to create interview: ${message}`,
ErrorCodes.DATABASE_ERROR,
);
}
}
/**
* Fetches all interviews from the database, ordered by creation date.
*
* This is useful for displaying interview history and letting users
* resume previous sessions. We order by descending creation date since
* users typically want to see their most recent interviews first.
*
* Returns an array of InterviewData objects with full interview details
* including metadata and message history.
*/
getAllInterviews(): InterviewData[] {
try {
const cursor = this.sql.exec(
InterviewDatabaseService.QUERIES.GET_ALL_INTERVIEWS,
);
return [...cursor].map(this.parseInterviewRecord);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
throw new InterviewError(
`Failed to retrieve interviews: ${message}`,
ErrorCodes.DATABASE_ERROR,
);
}
}
// Retrieves an interview and its messages by ID
getInterview(interviewId: string): InterviewData | null {
try {
const cursor = this.sql.exec(
InterviewDatabaseService.QUERIES.GET_INTERVIEW,
interviewId,
);
const record = [...cursor][0];
if (!record) return null;
return this.parseInterviewRecord(record);
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
throw new InterviewError(
`Failed to retrieve interview: ${message}`,
ErrorCodes.DATABASE_ERROR,
);
}
}
addMessage(
interviewId: string,
role: Message["role"],
content: string,
messageId: string,
): Message {
try {
const timestamp = Date.now();
this.sql.exec(
InterviewDatabaseService.QUERIES.INSERT_MESSAGE,
messageId,
interviewId,
role,
content,
timestamp,
);
return {
messageId,
interviewId,
role,
content,
timestamp,
};
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
throw new InterviewError(
`Failed to add message: ${message}`,
ErrorCodes.DATABASE_ERROR,
);
}
}
/**
* Transforms raw database records into structured InterviewData objects.
*
* This helper does the heavy lifting of:
* - Type checking critical fields to catch database corruption early
* - Converting stored JSON strings back into proper objects
* - Filtering out any null messages that might have snuck in
* - Ensuring timestamps are proper numbers
*
* If any required data is missing or malformed, it throws an error
* rather than returning partially valid data that could cause issues
* downstream.
*/
private parseInterviewRecord(record: any): InterviewData {
const interviewId = record.interviewId as string;
const createdAt = Number(record.createdAt);
const updatedAt = Number(record.updatedAt);
if (!interviewId || !createdAt || !updatedAt) {
throw new InterviewError(
"Invalid interview data in database",
ErrorCodes.DATABASE_ERROR,
);
}
return {
interviewId,
title: record.title as InterviewTitle,
skills: JSON.parse(record.skills as string) as InterviewSkill[],
messages: record.messages
? JSON.parse(record.messages)
.filter((m: any) => m !== null)
.map((m: any) => ({
messageId: m.messageId,
role: m.role,
content: m.content,
timestamp: m.timestamp,
}))
: [],
status: record.status as InterviewStatus,
createdAt,
updatedAt,
};
}
// Add these SQL queries to the QUERIES object
private static readonly QUERIES = {
// ... previous queries ...
INSERT_INTERVIEW: `
INSERT INTO ${CONFIG.database.tables.interviews}
(interviewId, title, skills, status, createdAt, updatedAt)
VALUES (?, ?, ?, ?, ?, ?)
`,
GET_ALL_INTERVIEWS: `
SELECT
interviewId,
title,
skills,
createdAt,
updatedAt,
status
FROM ${CONFIG.database.tables.interviews}
ORDER BY createdAt DESC
`,
INSERT_MESSAGE: `
INSERT INTO ${CONFIG.database.tables.messages}
(messageId, interviewId, role, content, timestamp)
VALUES (?, ?, ?, ?, ?)
`,
GET_INTERVIEW: `
SELECT
i.interviewId,
i.title,
i.skills,
i.status,
i.createdAt,
i.updatedAt,
COALESCE(
json_group_array(
CASE WHEN m.messageId IS NOT NULL THEN
json_object(
'messageId', m.messageId,
'role', m.role,
'content', m.content,
'timestamp', m.timestamp
)
END
),
'[]'
) as messages
FROM ${CONFIG.database.tables.interviews} i
LEFT JOIN ${CONFIG.database.tables.messages} m ON i.interviewId = m.interviewId
WHERE i.interviewId = ?
GROUP BY i.interviewId
`,
};
}
```
Add RPC methods to the `Interview` Durable Object to expose database operations through API. Add this code to `src/interview.ts`:
```typescript title="src/interview.ts"
import {
InterviewData,
InterviewTitle,
InterviewSkill,
Message,
} from "./types";
export class Interview extends DurableObject {
// Creates a new interview session
createInterview(title: InterviewTitle, skills: InterviewSkill[]): string {
return this.db.createInterview(title, skills);
}
// Retrieves all interview sessions
getAllInterviews(): InterviewData[] {
return this.db.getAllInterviews();
}
// Adds a new message to the 'messages' table and broadcasts it to all connected WebSocket clients.
addMessage(
interviewId: string,
role: "user" | "assistant",
content: string,
messageId: string,
): Message {
const newMessage = this.db.addMessage(
interviewId,
role,
content,
messageId,
);
this.broadcast(
JSON.stringify({
...newMessage,
type: "message",
}),
);
return newMessage;
}
}
```
## 6. Create REST API endpoints
With your Durable Object and database service ready, create REST API endpoints to manage interviews. You will need endpoints to:
- Create new interviews
- Retrieve all interviews for a user
Create a new file for your interview routes at `routes/interview.ts`:
```typescript title="src/routes/interview.ts"
import { Hono } from "hono";
import { BadRequestError } from "../errors";
import {
InterviewInput,
ApiContext,
HonoCtx,
InterviewTitle,
InterviewSkill,
} from "../types";
import { requireAuth } from "../middleware/auth";
/**
* Gets the Interview Durable Object instance for a given user.
* We use the username as a stable identifier to ensure each user
* gets their own dedicated DO instance that persists across requests.
*/
const getInterviewDO = (ctx: HonoCtx) => {
const username = ctx.get("username");
const id = ctx.env.INTERVIEW.idFromName(username);
return ctx.env.INTERVIEW.get(id);
};
/**
* Validates the interview creation payload.
* Makes sure we have all required fields in the correct format:
* - title must be present
* - skills must be a non-empty array
* Throws an error if validation fails.
*/
const validateInterviewInput = (input: InterviewInput) => {
if (
!input.title ||
!input.skills ||
!Array.isArray(input.skills) ||
input.skills.length === 0
) {
throw new BadRequestError("Invalid input");
}
};
/**
* GET /interviews
* Retrieves all interviews for the authenticated user.
* The interviews are stored and managed by the user's DO instance.
*/
const getAllInterviews = async (ctx: HonoCtx) => {
const interviewDO = getInterviewDO(ctx);
const interviews = await interviewDO.getAllInterviews();
return ctx.json(interviews);
};
/**
* POST /interviews
* Creates a new interview session with the specified title and skills.
* Each interview gets a unique ID that can be used to reference it later.
* Returns the newly created interview ID on success.
*/
const createInterview = async (ctx: HonoCtx) => {
const body = await ctx.req.json();
validateInterviewInput(body);
const interviewDO = getInterviewDO(ctx);
const interviewId = await interviewDO.createInterview(
body.title as InterviewTitle,
body.skills as InterviewSkill[],
);
return ctx.json({ success: true, interviewId });
};
/**
* Sets up all interview-related routes.
* Currently supports:
* - GET / : List all interviews
* - POST / : Create a new interview
*/
export const configureInterviewRoutes = () => {
const router = new Hono();
router.use("*", requireAuth);
router.get("/", getAllInterviews);
router.post("/", createInterview);
return router;
};
```
The `getInterviewDO` helper function uses the username from our authentication cookie to create a unique Durable Object ID. This ensures each user has their own isolated interview state.
Update your main application file to include the routes and protect them with authentication middleware. Update `src/index.ts`:
```typescript title="src/index.ts"
import { configureAuthRoutes } from "./routes/auth";
import { configureInterviewRoutes } from "./routes/interview";
import { Hono } from "hono";
import { Interview } from "./interview";
import { logger } from "hono/logger";
import type { ApiContext } from "./types";
const app = new Hono();
const api = new Hono