Vesta

Questi contenuti non sono ancora disponibili nella tua lingua.

Vesta is a virtual wardrobe you run on your own machine. You upload a full body photo, pick a garment from the wardrobe or photograph a new one, and a virtual try-on model renders that garment onto your photo. It also estimates your skin undertone and proposes a color palette. I wrote it with Gianmattia Barone.

What it does

Try-on starting from a photo of the user.
Color analysis: skin undertone computed as ITA in CIELAB, plus a season and a palette.
A wardrobe filterable by type: tops, trousers, dresses.
Adding a garment from a photo, with background removal and category assignment done automatically.
Three generation modes: local on the Mac, a free cloud GPU, or Premium with paid models (OpenAI gpt-image-1 or Google gemini-2.5-flash-image) using your own API key entered from the Profile.
A responsive interface for desktop and phone, with light and dark theme following the system.

What it does not do

It does not run local generation outside Apple Silicon. On other systems only the cloud mode is available.
It does not guarantee the cloud mode will be there. The cloud path uses a free Hugging Face GPU with a limited quota, and when the quota runs out the app falls back to local generation.
It does not come with a commercial license for the models. CatVTON and IDM-VTON are released under non commercial licenses. The repository code is MIT. Premium mode is the path with commercial API terms.
It does not ship API keys. Premium mode works only with a key you supply yourself.
It does not tell you whether a garment fits you. The output is a generated image.

Requirements

A Mac with Apple Silicon and Python 3.11 for local generation.

Install and run

cd backend
python3.11 -m venv .venv
.venv/bin/pip install -r ../requirements.txt
git clone https://github.com/Zheng-Chong/CatVTON
.venv/bin/python download_weights.py
.venv/bin/python prep_wardrobe.py
.venv/bin/python -m uvicorn server:app --host 0.0.0.0 --port 8770

Then open http://127.0.0.1:8770, or http://IP-OF-THE-MAC:8770 from a phone on the same network.

How it works

The frontend is Preact, served by the backend, with no build step. The backend is FastAPI running on Apple Silicon through Metal and MPS. Try-on uses CatVTON locally and IDM-VTON via cloud. Segmentation uses segformer and rembg.

The ios/ folder holds a SwiftUI scaffold, a WKWebView pointed at the server.

Premium mode and keys

From the Profile you enter an OpenAI or Google Gemini API key, and generation switches to the paid models. Keys are stored in backend/.keys.json, which is excluded from git, or read from the OPENAI_API_KEY and GEMINI_API_KEY environment variables. They are never exposed through the API. The plan for App Store distribution and subscriptions is kept in PIANO.md in the repository.

What is checked

Three QA scripts sit in backend/ and run as plain Python.

.venv/bin/python qa_api.py
.venv/bin/python qa_chroma.py
.venv/bin/python qa_extract.py

qa_api.py drives the API in process: health, the static assets and their content types, /analyze returning a season, /classify sorting a garment photo into the right category, /cutout and /tryon returning valid PNGs, and the second identical try-on coming back as a cache hit. On the premium path it asserts 400 without a key and 502 with a fake one, and that backend/.keys.json is written with 600 permissions and ignored by git.

qa_chroma.py builds eight garment colors on two chroma keys and requires the cutout to leave the garment opaque and its color within 12 of the original on every channel.

qa_extract.py runs the import pipeline against simulated providers, from the inventory prompt through the retry after a failed cutout to the PNG on disk and the wardrobe record.

The check on /tryon is that the response is a valid PNG. The image itself is not scored.

nerln.github.io/vesta

nerln/vesta on GitHubSource, wardrobe preparation scripts and the iOS scaffold.