gitwtfhub

wtf is talking-face?

compsmart/talking-face — explained in plain English

Analysis updated 2026-05-18

1JavaScriptAudience · developerComplexity · 4/5Setup · moderate

TL;DR

A from-scratch browser project that renders a glowing wireframe head, speaks aloud through Google's Gemini Live voice API with lip sync, and can wear a mapped photo of a real face.

Mindmap

mindmap
  root((talking face))
    What it does
      Talking wireframe head
      Photo face mapping
      Live voice replies
    Tech stack
      JavaScript
      WebGPU and WGSL
      Gemini Live API
    Use cases
      Avatar prototyping
      Lip sync experiments
      Graphics learning
    Audience
      Developers
      Graphics hobbyists

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

Why would anyone build with this?

REASON 1

Prototype a talking avatar that speaks through Google's Gemini voice API.

REASON 2

Map a real face photo onto a 3D wireframe head and cross fade between the two looks.

REASON 3

Experiment with real time predictive lip sync driven by streaming audio.

REASON 4

Study a hand-built WebGPU renderer with bloom, wireframe, and post-processing effects.

What's in the stack?

JavaScriptWebGPUWGSLMediaPipeGemini Live API

How it stacks up

compsmart/talking-face0xmukesh/docusaurus-tutoriala15n/andrewscheuermann
Stars111
LanguageJavaScriptJavaScriptJavaScript
Last pushed2021-12-272015-01-11
MaintenanceDormantDormant
Setup difficultymoderateeasymoderate
Complexity4/52/51/5
Audiencedeveloperdevelopergeneral

Figures from each repo's GitHub metadata at analysis time.

How do you spin it up?

Difficulty · moderate Time to first run · 30min

Needs a WebGPU-capable browser, a bundled local server, and a free Gemini API key to talk.

The README does not state a license, so usage rights are unclear.

Wtf does this do

This project, called Face Cloner, builds a glowing wireframe head in the browser that can talk out loud using Google's Gemini Live voice API, and can also wear the photo of a real face mapped onto its shape. It is written from scratch in plain JavaScript with no frameworks or build tools, using WebGPU for the graphics and WGSL for the shaders. The wireframe is based on a standard 468 point face model from Google's MediaPipe library, which can find those same points in any uploaded photo. Because the points always land in the same places, mapping a photo onto the head is a matter of matching each point in the image to the matching point on the mesh, then cross fading between the glowing wireframe look and the photo using a slider. To use it, a person runs a small bundled server, opens the page in a WebGPU capable browser like Chrome or Edge, and pastes in a free Gemini API key. Typing a message makes the face answer out loud, with lips that move in time with the generated speech, driven by analyzing the incoming audio slightly ahead of when it plays. There is also a feature that scans an uploaded photo, asks Gemini to guess who the person or character might be, and automatically writes a matching speaking style and voice for the face to use, which the user can review or edit before starting a conversation. Under the hood, the model can also send special signals mid conversation to change the face's expression or trigger a gesture, so it can smile or raise an eyebrow while it talks. The project notes that a future version could add microphone input for a full back and forth voice conversation, since the groundwork for that is already in place. This is a hobby scale, single contributor project best suited for developers curious about real time graphics, voice APIs, or expressive avatar experiments, rather than a polished consumer product.

Yoink these prompts

Prompt 1
Explain how this project maps a photo onto the wireframe face mesh.
Prompt 2
Walk me through running the bundled dev server and connecting a Gemini API key.
Prompt 3
How does the predictive lip sync system work in src/audio/lipsync.js?
Prompt 4
Show me how the model triggers facial expressions through function calling.

Frequently asked questions

wtf is talking-face?

A from-scratch browser project that renders a glowing wireframe head, speaks aloud through Google's Gemini Live voice API with lip sync, and can wear a mapped photo of a real face.

What language is talking-face written in?

Mainly JavaScript. The stack also includes JavaScript, WebGPU, WGSL.

What license does talking-face use?

The README does not state a license, so usage rights are unclear.

How hard is talking-face to set up?

Setup difficulty is rated moderate, with roughly 30min to a first successful run.

Who is talking-face for?

Mainly developer.

View the repo → Decode another repo

This repo across BitVibe Labs

Don't trust strangers blindly. Verify against the repo.