How LetsVibe grades a project

LetsVibe ranks vibe-coded projects: programs written by AI from someone's prompts, usually by people who are not programmers. Every grade is produced by programs and open AI models running on LetsVibe's own computer. Your code is never sent to an AI company. This page explains every step, including what the grades do not mean.

Current rubric: v9.3 · see what changed

Step by step

1. You submit your own GitHub repository. You sign in with GitHub so we can confirm the repository belongs to your account; you cannot submit someone else's work. If the latest commit has already been graded, nothing runs again.
2. Vibe-coded check. LetsVibe is for projects written by AI from someone's prompts. Projects whose history starts before November 2022 (before vibe coding existed) and forks of someone else's project are not graded. Strong signs of vibe coding are AI tool files (such as CLAUDE.md, AGENTS.md, GEMINI.md or Cursor and Copilot instructions), AI co-author lines in a real share of the commits, AI app-builder accounts as commit authors, a README that says so, or code with several habits typical of AI-written code. Weaker signs, such as install scripts or README badges, never decide on their own, and code examples in documentation are never counted. If the check cannot confirm a project was vibe-coded (copying code from a chat window, for example, leaves few traces), the project is still graded, and the site owner reviews it before it is published.
3. A temporary copy is made. The repository is cloned into a temporary folder on LetsVibe's server. Bundled third-party libraries and generated files are set aside so the project is judged on its own work. The copy is deleted when the evaluation ends.
4. Archetype & Classification. Build files (for example platformio.ini, an Android manifest, MCP tool descriptors, or canvas/game engines) and documentation identify the project's archetype: Developer Tool / MCP Server, Game / Interactive Simulation, Media / Audio Lounge, Mobile / IoT, Automation Pipeline, Web Productivity, or General Software. This archetype determines the specific evaluation lenses and domain exemptions applied throughout the grade so software is judged fairly for its intended purpose.
5. The App Dossier (no AI). A program reads the repository and writes down what a person experiences: what the project says it is (its README, bypassing top-of-page badge walls, donation buttons and hero images to find the true pitch, and counting visual screenshots and diagrams as proof of work), the libraries and data it uses, its screens, whether several devices can use it at once (for example a TV, a host screen and guests' phones kept in sync), the program's own features, the text people read on screen, how it is installed and launched, how much must be filled in or understood before first use, whether it saves your progress or data, and any ads, payments or required sign-in found in its code (never words in its story or on-screen text). Nothing in the project is ever run.
6. Common app types (no AI judgement). A small local embedding model compares the project with a list of app types people commonly make from a single prompt (to-do lists, stock viewers, space shooters and so on). The closest one, and how similar it is, is given to the reviewer as a plain fact.
7. The code, one topic at a time. The project's own code is cut into small sections. A 7B model (Qwen2.5-Coder 7B) reads each section separately for one topic at a time (functionality, UI/UX, user journey, creativity), and only the sections that can tell it something about that topic: screens for UI/UX, setup and launch code for the user journey, and so on. It answers the same fixed-choice checklist as below, where every question has a real "no" as well as the good and middle answers, and must copy the exact lines of code behind each answer (for a "no", the code where the app falls short, such as an error that is only written to a log). A program checks every quote against the real file and throws away answers whose quotes are not in the code. For things people see (messages, confirmations, loading feedback, screen-reader labels), praise must quote something a person actually sees, never a log or console line. The 7B is then shown each answer's quoted lines and asked whether they really show the claim; answers that fail are dropped. "Not there at all" counts only when every relevant section was read and none showed it. Questions about the whole app need evidence from at least two files, and when sections disagree the answer is the one halfway between them.
8. Checklists on the dossier, one topic at a time. A 14B model (Qwen2.5-Coder 14B) reads the dossier and answers five separate checklists: real-world value, creativity, user journey, functionality and UI/UX. The model evaluates as a discerning Hackathon Judge and power-user evaluating solo vibe-coded software: rewarding high-leverage superpowers and working utility over corporate bureaucracy. Its prompt includes the project's archetype lenses and explicit domain exemptions. Every answer is a fixed choice (for example "a common idea adapted for a niche" or "setup is heavy before first use"), and every answer must quote the dossier as its evidence; a program checks that each quote is really there.
9. The two reviews are compared. Item by item: when they agree, the answer stands; when only one found checked evidence, that evidence is used; when they disagree, the 14B rereads the exact quoted lines from both sides and says which claim the code supports, quoting the line that decides it. If neither claim holds up, the item counts as "not shown". Answers are never averaged, and every decision is kept with the report.
10. Points & Domain Exemptions (no AI). Each answer is worth fixed points, published below. The model never picks a number, so every score can be traced to the answers behind it. If an archetype specifically exempts a standard (for example, screen reader labels or technical jargon avoidance for a developer CLI or MCP tool), the project receives full credit rather than being penalized for lacking features irrelevant to its domain.
11. The written review. The 14B rewrites the checked answers in plain English, pointing to the file and line where a finding was seen. Each factor's description starts with its weakest finding. "Needs work" lists every problem and every answer that falls short (such as a heavy setup, errors people are never told about, or a main task that only mostly works), with the file and line where the code shows it; if there are none, it says so instead of inventing advice.

If any stage fails or returns incomplete results, or a grade runs longer than 3 hours, no grade is published at all. We would rather show nothing than a misleading number.

The five factors

In order of weight: real-world value, creativity, user journey, functionality and UI/UX.

When neither the dossier nor the code shows evidence either way (a scan that found nothing proves nothing), the answer is "not shown" and earns middle points: something a program cannot prove is never held against a project. Code complexity is described on every report in plain English (how many paths the average function has, the most complex functions, very large files and repeated code), for information only: it never counts toward the score. Other code-health notes (unused files, ignored errors) are shared with each project's owner only. Safety is not graded: LetsVibe is not a security tool. Size never counts on its own: lines of code and number of files are not quality signals.

Every checklist answer and its points

Each factor scores the share of its points earned (0–100). The overall score weights them: Real-world value 25%, Creativity 22%, User journey 20%, Functionality 18%, UI/UX 15%.

What the grades mean

A factor's score is the share of its checklist points the project earned, and the overall score weights the five factors (above).

Letter grades are given on a curve, often called a bell curve. A raw score says little on its own, so it is not turned into a letter directly. Your letter depends on how far your score sits above or below the average, measured in standard deviations; on a full bell that gives about 7% A, 24% B, 38% C, 24% D and 7% F.

• Two public reference projects set the ends of the curve. A small app written to satisfy every factor and one written to ignore every factor were each graded several times with exactly the pipeline your project goes through. Until your category has projects of its own, the bell's average is the midpoint between them and one standard deviation is a quarter of the gap, so the good one sits two standard deviations above average and the bad one two below.
• Your category takes over as it grows. Each project added to your category shifts the bell toward that category's own average and spread, and from 30 projects on, the bell comes entirely from your category.
• Two rules keep it honest: an A also needs a score near the top reference, so the best of a weak group is not an A just for being first, and a project about as weak as the bottom reference is an F whatever its ranking.

Reference projects never appear in the rankings. The Grading curve page shows both of them, their exact prompts, every grading run and every number used. Every report also shows its rank in its category (for example "#3 of 41 in Games"), and your raw score is always shown unchanged.

If the curve is ever unavailable (for example while it is being rebuilt after a rubric change), letters temporarily come from this table, which matches the score bands:

What a LetsVibe grade is not

• Not a security audit. A published report never certifies that software is safe to run.
• Not an endorsement of a project, its author or its fitness for any purpose.
• Not perfectly repeatable. AI models vary slightly from run to run, so the same code can score a few points differently. Treat small differences between projects as a tie.
• Not a judgement of a person. Grades describe a repository at one commit, under one rubric version.

Corrections

AI reviews can get facts wrong. A project's owner can report an error in any statement of their review, pointing to the relevant code. The statement is checked once against the code; if the check confirms the error, the statement is corrected (in live data and backups) and the report shows that it was corrected. Otherwise it is reviewed by hand. Scores are not changed by a correction; the owner can request a fresh grade after pushing a new commit.

Why everything runs locally

The models run through Ollama on a single consumer graphics card owned by LetsVibe. Nothing you submit is sent to OpenAI, Anthropic, Google or any other AI provider, and nothing is used to train models. The trade-off is speed: an evaluation takes several minutes and runs one at a time.

Questions? See the FAQ or contact us.