Introduction

"A service that uploads one portrait and one reference dance video, and then generates a video of the person in the photo dancing." From the outside, such a service looks simple.

However, behind the scenes, it involves an asynchronous workflow that takes a long time, uses GPU, and involves money (credits). I feel that the difficulty lies not in the model itself, but in the surrounding design.

In this article, we will cover the following three points.

1. Concept of Motion Transfer
2. Designing the generation job as a state machine
3. Credit management that prevents double charging or refunds errors in case of failure

1. What is Motion Transfer

The text-to-video approach creates video from text, where the model "creates" the motion from the text. On the other hand, Motion Transfer splits the input into two parts.

• Motion Input: Reference video (responsible for choreography, timing, and camera movement)
• Appearance Input: Portrait (responsible for who is dancing)

Conceptually, the process involves extracting motion information from the reference video and appearance information from the photo, then passing both as conditions to the video generation model. (The specific methods vary depending on the model, and are not clear from the product page.)

As an engineer, what is important is that the output quality strongly depends on the input quality.

• If the reference video has many scene cuts or obstructions → motion information becomes incomplete
• If the portrait is blurry or dark → appearance information becomes unstable

Therefore, input validation and preview are important not only for UX but also for system design.

2. Generation jobs are handled by a state machine

Generation can take several minutes and may fail due to reasons beyond control such as GPU interruption or timeout. Design it as an asynchronous job with an explicit state, not using request/response.

This is a minimal schema example.

The points are two.

• credit_cost is saved in the job during estimation. Even if the pricing system changes later, the amount the user sees is protected.
• UNIQUE (user_id, idempotency_key) ensures that "double-clicking a button" or "client resending after timeout" becomes harmless.

Enforce state transition in one place

Aggregate transitions to prevent workers from writing any arbitrary status.

"CREATED": {"VALIDATING", "FAILED"}, "VALIDATING": {"QUEUED", "FAILED"}, "QUEUED": {"RENDERING", "FAILED"}, "RENDERING": {"POSTPROCESSING", "FAILED", "QUEUED"}, # QUEUED is retry "POSTPROCESSING": { "COMPLETED", "FAILED"}, } def transition(db, job_id, expected, new): if new not in ALLOWED: raise ValueError(f"illegal transition {expected} -> {new}") updated = db.execute( """ UPDATE render_jobs SET status = %s, updated_at = now() WHERE id = %s AND status = %s """, (new, job_id, expected), ) if updated.rowcount == 0: raise ConcurrentUpdate(job_id) # Other workers have transitioned first WHERE status = expected is using optimistic concurrency control, and even if two workers compete, only one can win. Credit is managed through 'Reserve → Confirm or Release' pattern. Common mistakes are two. Withdrawing at completion: multiple jobs can run with the same balance. Withdrawing first but forgetting to refund: inquiries increase. Stability is achieved by reserving using an append-only ledger (ledger) and confirming or releasing. ledger.sql CREATE TABLE credit_ledger ( id BIGSERIAL PRIMARY KEY, user_id BIGINT NOT NULL, job_id UUID, kind TEXT NOT NULL, -- GRANT, RESERVE, RELEASE, SETTLE amount INT NOT NULL, -- signed. RESERVE is negative created_at TIMESTAMPTZ NOT NULL DEFAULT now()(), UNIQUE (job_id, kind) -- RESERVE and RELEASE/SETTLE are processed once per job );