Building a Faceless Short-Form Video Pipeline

I spent a while building an automated pipeline that turns a topic into a finished 9:16 video. Most write-ups about this stop at "call an LLM, call a TTS API, done." The interesting problems are all in the layer after that, so here are the ones that actually cost me time.

Forced alignment matters more than voice quality

The naive approach chunks captions by word count and assumes a speaking rate. This breaks immediately, because TTS duration varies by voice, by punctuation, and by how the engine handles numbers. A 140-word script came back as 38s on one voice and 44s on another.

The fix is to run forced alignment against the returned audio and derive caption timings from actual word timestamps. Some engines return word-level timings directly; otherwise a small alignment pass gets you there. Doing this after synthesis instead of predicting before it removes an entire category of bug. Any faceless reels ai pipeline that skips this will drift on longer clips.

The ffmpeg filter graph is where things get slow

A typical composite is: scale and crop background to 1080x1920, overlay caption images per segment, mix voiceover with a ducked music bed, then encode. Written as one filter_complex chain this is fine. Written as several sequential ffmpeg invocations with intermediate files, it is roughly 4x slower and burns disk.

ffmpeg -i bg.mp4 -i vo.wav -i music.m4a \
  -filter_complex "[0:v]scale=-2:1920,crop=1080:1920[v];[2:a]volume=0.15[m];[1:a][m]amix=inputs=2[a]" \
  -map "[v]" -map "[a]" -c:v libx264 -preset veryfast -crf 23 out.mp4

Two things that bit me: crop defaults to centre, which is wrong whenever the subject is off-centre, and amix normalises inputs by default, so the voiceover gets quieter as soon as you add music. Use amix=normalize=0 and duck the bed explicitly with sidechaincompress.

Caption rendering: drawtext vs pre-rendered PNG

ffmpeg's drawtext is fast but painful for anything with emoji, mixed scripts, or per-word highlighting. Pre-rendering caption frames as transparent PNGs and overlaying them costs more I/O but gives full typographic control, and it makes the caption style a data file rather than a filter string. For a pipeline that needs several visual formats, the PNG route pays off quickly. This is broadly what a faceless reels generator does internally when it offers multiple caption styles.

Concurrency limits are CPU, not API

Script and TTS calls are I/O-bound and parallelise freely. Encoding is CPU-bound and does not. Running eight renders concurrently on a 4-core box made each one slower than running them two at a time. Setting the render worker pool to roughly cores - 2 and queueing the rest was the single biggest throughput change.

Determinism is worth engineering for

If a render fails halfway you want to re-run it and get byte-identical output up to the failure point. That means fixed random seeds, pinned font versions, and no wall-clock timestamps in filenames. This sounds pedantic until you are debugging why one video in a batch of thirty looks different. Format presets — one caption spec, one margin, one ducking curve, versioned together — are what makes a faceless reels maker produce a consistent-looking channel instead of thirty slightly different videos.

What I would do differently

Start with the render contract, not the LLM. Define exactly what a finished video looks like — resolution, safe margins, caption spec, audio levels — and build backwards. I did it the other way round and ended up rewriting the assembly stage twice because the script format kept changing shape.