Running Multimodality locally

Use this page to run either extracted Multimodality project: image generation or image understanding. Both need the same Vite key change and the same OpenAI SDK update. After that, follow the repair for the project you have. The key change is the one every extracted browser project in these courses needs; Running Chef Claude locally applies it to a React project with two provider routes.

What you need first

Install a supported LTS version of Node.js. Node 24 is recommended and includes npm. The project installs the latest Vite, which needs Node 22.12 or newer.

Check that both commands work:

$ node --version
v24.18.0
$ npm --version
11.18.0

You need an OpenAI API key with billing and access to the model you use. Image generation access can require organization verification.

Use a temporary key with a low spending limit. Both projects make paid OpenAI requests from browser code, so anyone who opens the browser's developer tools can read the key.

JunoWhat you need first Install Node.js LTS, then run node --version and npm --version. Each should print a version number.

Next, create a temporary OpenAI key with billing and a low spending limit. If you plan to run the image-generation project, check whether your organization needs verification before you start.

JunoWhat you need first Vite can serve the page long before your OpenAI project has billing or model access, so a page that loads proves nothing about the account. The first paid request is what proves the key works.

Keep that key on a low spending limit so a mistake stays cheap.

JunoWhat you need first Local bundling and OpenAI access fail independently, so treat them as separate prerequisites. The browser receives the key, which means everyone who opens the page receives it too.

Set the spending limit now, and move the request to a server before anything that resembles a deployment.

Open and prepare the project

Open a terminal in the extracted folder containing package.json:

$ cd path-to-your-downloaded-project
$ npm install
$ npm install openai@latest

The second install replaces the old course SDK with the current one before you change the model.

Check index.js to see which project you have. The image-generation project calls openai.images.generate and builds a poster form with a movie title and an art style. The vision project calls openai.chat.completions.create with gpt-4-vision-preview and has an images/ folder.

Create .env and .gitignore beside package.json:

VITE_OPENAI_API_KEY=your-openai-api-key

.env
node_modules/

The .gitignore entries keep two things out of any repository you create from this folder: your key, and the node_modules folder that npm can rebuild at any time. Ignoring files and good habits covers the wider habit.

In index.js, change the OpenAI client key from:

apiKey: process.env.OPENAI_API_KEY,

to:

apiKey: import.meta.env.VITE_OPENAI_API_KEY,

Leave the dangerouslyAllowBrowser: true line below it in place. The SDK refuses to run in the browser without it.

The key is included in the frontend

This is acceptable only for temporary local learning with a restricted key. Image requests can cost money. Never deploy or share this browser-only version; move the OpenAI request to a backend first.

JunoOpen and prepare the project Open the extracted folder containing package.json, install its packages, then install the current OpenAI SDK on top.

Put your temporary key in .env, and let .gitignore keep that file out of Git. Then change the one apiKey line in index.js and leave the line below it alone.

JunoOpen and prepare the project Both projects take the same two edits: the SDK update and the Vite key substitution. Vite reads .env at startup, so restart it after changing the file.

Every VITE_ value is sent to the browser, so the prefix marks a value as published rather than hidden.

JunoOpen and prepare the project Upgrade the SDK first. The course pins OpenAI 4.20, which predates the gpt-image models, so a new model call through the old package can fail in ways that look like mistakes in your own code.

Only then replace the retired model ID and the response handling. import.meta.env moves the key into the bundle; it does not secure it.

Repair the image-generation project

The course snapshot calls the retired dall-e-3 model and expects a hosted image URL. Following the current OpenAI image-generation guide, change its image request to use gpt-image-2:

const image = await openai.images.generate({
  model: "gpt-image-2",
  prompt,
  size: "1024x1024",
})

The model returns the image as base64 data instead of a URL. Where the code reads image.data[0].url:

const imgURL = image.data[0].url;

build a PNG data URL from the returned base64 image instead:

const imgURL = `data:image/png;base64,${image.data[0].b64_json}`;

Keep the line that places imgURL in the poster output as it is.

JunoRepair the image-generation project Point the request at gpt-image-2, then change the imgURL line so it builds a PNG data URL from b64_json.

Make both changes together. If you change the model and keep .url, the poster will not appear.

JunoRepair the image-generation project Two things changed: the model you request, and the field that carries the image back. Ask for gpt-image-2 and read b64_json instead of .url.

If the picture is broken, check the data:image/png;base64, prefix before you change anything else.

JunoRepair the image-generation project The response carries the image inside the JSON rather than pointing at a provider-hosted file, so the page has to wrap it in an explicit data URL before an img tag can show it.

The rest of the rendering code stays as the course wrote it.

Repair the vision project

In the Vision Part 2 project, replace the retired model ID:

model: "gpt-4-vision-preview",

with the image-capable gpt-4o-mini:

model: "gpt-4o-mini",

The existing message format can still send text and image inputs, so the rest of the request stays the same.

JunoRepair the vision project Replace the retired vision model with gpt-4o-mini and keep the text-and-image message exactly as it is.

Only the model ID changes here, not the shape of the request.

JunoRepair the vision project This repair replaces the model and nothing else; the response-shape change belongs to image generation, not here.

If the call still fails after the replacement, check account access before you check your own message code.

JunoRepair the vision project Chat Completions still accepts a content array that mixes a text part and an image_url part, so the course's request shape carries over to gpt-4o-mini unchanged.

Narrow down which part of the request fails before you rewrite any of it.

Run either project

$ npm start

Open the Local URL printed by Vite.

In the image-generation project, enter a movie title, choose an art style, and select Generate Poster. The page shows "Generating poster for..." while the request runs, then the poster. If the request fails, the page shows "Sorry, an error occurred while generating the poster." and the browser console shows the error.

The vision project sends its request as soon as the page loads. The page shows the image being analyzed, and the model's description appears in the browser console, not on the page. Every page load sends a new paid request, including the reload Vite triggers when you save index.js.

Restart after changing .env, and stop the project with Ctrl+C.

JunoRun either project Run npm start and open the Local URL Vite prints.

For image generation, submit a movie title and art style and wait for the poster. For vision, open the browser console: the model's description appears there. Every attempt is a paid request, so decide what you want to test before you start clicking.

JunoRun either project Vite starting proves the adapted bundle compiles; it says nothing about the account. One live request checks the key, billing, model access, and the project's response handling in a single call.

Restart Vite after any .env edit, or the old value keeps being served.

JunoRun either project The two projects signal success differently. Image generation must receive base64 and render it as a data URL; vision runs its request at module load and logs the first choice to the console.

Because the vision request runs on every load, each reload is billed. Read the console before you try again.

Troubleshooting

process is not defined: Replace the remaining process.env key read in index.js with the import.meta.env.VITE_... form.

The generated image is broken: Confirm the code reads b64_json and adds the data:image/png;base64, prefix instead of reading .url.

The model is unavailable: Confirm billing, organization verification, and model access in the OpenAI project. Provider availability can vary by account.

The vision page stays blank: The page adds the image only after the request succeeds, so a blank page means the request failed. Read the error in the browser console. If it does not point to the key or the model, send a text-only message to gpt-4o-mini first; a success there confirms your account can use the model, so the image part of the message is what to inspect next.

The request works but the page stays unchanged: Check the browser console for a rendering error and compare the returned field with the relevant repair above.

JunoTroubleshooting Check these in order: fix any leftover process.env read, confirm the model, then check b64_json for image generation.

For either project, the browser console shows the error. Most failures here are one unfinished edit.

JunoTroubleshooting Separate the Vite substitution, account access, response shape, and page rendering; each layer fails on its own. A request can succeed and still render incorrectly.

Match the symptom to one layer before you edit anything.

JunoTroubleshooting Read the provider response before you change any UI code, because it shows what the model returned and the rendered page does not.

Keep image-generation output and vision message handling separate; their failures look similar but have different causes.