EditApart — AI taste-driven editor (agent preset)
EditApart is an automation-faithful editor for the EditTogether community. It takes raw footage, builds a shot inventory, proposes an EDIT DECISION SCHEMA against a declared editorial rubric, renders it deterministically with ffmpeg/scenedetect (and ImageMagick for photos), critiques the render against the rubric, and revises until it converges.
It does not operate a video-editing GUI. It reasons about editorial judgment.
The plugin registers both video and photo tools plus the taste-model tools (15
total) under one identity; the engines are bin/edit_apart_core.py (video),
bin/photo_core.py (photo) and bin/taste_model.py (the per-creator model).
The core design
- Schema is the interlingua. The same EDIT DECISION SCHEMA is the thing you train on, the thing you render from, and the thing you critique. No gap between "AI intent" and "the render."
- General grammar is shared; taste is per-creator. One shared style-brain
(film-grammar parameters) + a small per-creator identity latent
z_uin a per-creator GGUF. Only the identity is per-creator — the artifact holdsz_uand a digest of the trunk it belongs to, nothing else. - Reward is dense + group-relative (GRPO). The critic scores a render AND
yields per-element keep/drop/why deltas;
r = overall + λ·Σ(deltas). Group relative advantageA_k = (r_k − mean(r))/std(r), then a PPO-clip step. - Muon on weights, AdamW on the latent. Muon orthogonalizes momentum for
2D weight matrices; never the identity latent (Newton-Schulz degenerates
on 1D). AdamW handles
z_uand all 1D/bias/embed params. - Self-play interleaved with history.
propose_schemaemits a group of candidate schemas from a deterministic grid; the group is logged, one candidate is selected, and the render/vision/creator outcome is logged back against it. Training consumes those groups overnight. - Three model portions, and only one is the session model. General (the
session model: reads footage, proposes, critiques) · FilmGPT style-brain
(shared film grammar) · creator taste (
z_u). A video-in/out model like GLM-5.3 (Z.ai / zai, OpenAI-compatible) replaces the general portion only — it reads the raw clip via{"type":"video_url","video_url":{"url":…}}instead of pre-extracted frames; the FilmGPT and taste portions are untouched. - Model-authored video is opt-in architecture, not a preset claim. ffmpeg
renders the final cut (
render_schema). Fully AI-generated clips come from a separate store plugin (or the harnessvideo_generatetool, e.g. grok-imagine-video), installed opt-in. When present, you may generate a clip and insert it into a regular edit as ageneratedsegment (source + AI clip are separate inputs the renderer concatenates). The preset itself never claims to generate video. - Photo editing is image-only — one fewer modality. The same taste-driven
loop edits a single image via the
edit-apartplugin (bin/photo_core.py), which needs onlyimageinput (novideomodality, no video-in/out registration), so it works with the existing image-capable general model. The ImageMagick renderer is resolved at runtime (DSH_IMAGEMAGICK/PATH) and is not a hard dependency — a web-profile in-browser renderer may supply it.
The pipeline
The editor's tools (mounted as a preset plugin at
plugins/edit-apart.mjs) are the thin facade; bin/edit_apart_core.py (video)
and bin/photo_core.py (photo) are the deterministic engines they shell out to.
source ─► inventory ─► shot inventory
│
features ─► per-shot luminance/motion/audio
│
propose_schema + rubric ─► GROUP of candidate EDIT DECISION SCHEMAS
│ │ (logged to .editapart/taste_samples.jsonl)
│ ▼
│ taste identity scores the group ─► selected schema
│
render_schema ─► deterministic ffmpeg render
│
critic_edit ─► score + per-element keep/drop/why deltas
│ │ (logged as that candidate's reward;
│ │ chosen_by=creator|agent = revealed preference)
revise_schema ─► revised schema ─► re-render (bounded loop)
│
train_identity ─► shared style-brain GGUF + per-creator z_u GGUF
Status of the taste model. The identity learning is implemented and wired into the loop, not a scaffold.
propose_schemabuilds and logs a group of candidates, selects one with the creator identity when it exists,critic_editlogs the reward for the candidate that was actually rendered, andtrain_identityfits the shared style-brain plus that creator's latentz_uand writes real GGUF artifacts. The measured claims and their limits are in Taste model below — including what it does not do yet (trained selector, not a fine-tuned language model; video only).
Photo editing (image-only)
A parallel single-image loop, mounted via the edit-apart plugin:
photo_inspect → photo_propose (rubric → photo EDIT DECISION SCHEMA) →
photo_render (ImageMagick) → photo_critic (objective + subjective) →
photo_revise. It needs one fewer modality than video (image, not
video), and its renderer is not a hard dependency (resolved at runtime;
a web-profile in-browser ImageMagick/ffmpeg may supply it).
The taste model is wired in here too, with its own photo/v1 layout, its own
group log and its own style-brain (style_brain_photo_v1.gguf), so a creator
ends up with one identity per modality. Three things make this modality harder
than video, and all three are measured rather than assumed:
- No temporal axis to lean on, so the layout synthesises spatial structure:
crop geometry plus statistics of the region the crop keeps, read from a 4×4
region grid that
photo_inspectcomputes from the same downscale as the global stats (the global numbers are unchanged). - Candidate scoring costs a render. The photo reward is defined on the
rendered image, so a K-candidate group is rendered and scored K times
(video's objective critic is pure arithmetic).
group=1with no dataset stays the legacy, render-free proposal. - A quantised reward ties the group. Exposure was a threshold, so a 6-candidate group scored 6 × −0.05, spread 0.0 — exactly zero group-relative signal. The reward is now graded (full credit inside the tolerance, linear falloff outside), which took that group to spread 0.47 and left 11/18 training groups informative.
Measured on a synthetic 1600×1200 image, two creators with opposite framing taste
(tightest vs loosest crop) sharing one frozen trunk: 10/10 held-out briefs from
the training family are picked differently, 10/10 in the predicted direction
(mean crop area 0.155 vs 0.389). On one unseen crop geometry it transfers too —
10/10 over ten distinct briefs — but with a smaller margin: mean crop area 0.30
vs 0.45, a gap of 0.15 against 0.233 in-distribution. That is the geometry/content
confound of docs/paper-findings.md (Finding 4c) narrowing the effect, not
erasing it. (An earlier "0/10, no transfer" reading came from the objective bug
described under Corrections; it was an artifact and is withdrawn.)
A four-arm follow-up (tests/experiment_photo_transfer.py: {1 image, 3 images} ×
{fixed, varied crop family}, shared probe ladder, matched steps) found the repair
is partial and along the content axis, not the geometry axis: only the
3-image/narrow-family arm transferred to an unseen image and geometry (3/3,
ratio 0.400), while both varied-geometry arms gave 0/3. A no-taste control
identity showed the ordering metric's noise floor is ±0.36 — larger than most
effects measured — so treat these results as suggestive, not established.
docs/paper-findings.md Findings 4c/4d has the tables, the power analysis and the
rules (train on the family you will use; include an out-of-family check and a
no-taste control).
Taste model (wired, measured)
bin/taste_model.py is the per-creator model; bin/train_identity.py is its CLI.
The loop in bin/edit_apart_core.py drives it.
Architecture. A shared trunk over 16 named edit features (pacing, shot selection, tempo, energy, order) feeds a hidden layer that the creator latent modulates three ways:
u = tanh-free ranking score =
(a ⊙ (1 + tanh(W_s z + b_s))) @ W2 + b2 # latent gates the hidden activations
+ (z @ W_z + b_z) # latent output offset
+ x · (z @ W_l) # per-creator linear readout over the features
z_u is the only per-creator tensor, so a creator artifact is ~1 KB while the
trunk is shared. The map is nonlinear in z (tanh gates, multiplicative terms)
and interpolatable (the latent is continuous; a midpoint latent scores between
its endpoints — asserted in the test suite).
Objective. For each clip the loop logs a group of candidates with the critic's
dense reward. GRPO turns that into group-relative advantages
A_k = (r_k − mean r)/std r, and the update is a PPO-clip surrogate on the
group-softmax selection policy, plus an auxiliary pairwise preference loss over
the pick that was actually rendered and a small entropy bonus — but only when the
pick came from an external judge (chosen_by=creator|agent|human). A choice the
system itself made (taste: the identity's selection was rendered; critic: nobody
picked, the objective critic only scored) trains neither the preference term nor the
reward bonus. It still learns through the group-relative advantage over the critic's
rewards, which measures the edit rather than claiming anything about the creator. The
loop therefore records who picked, not just what was picked; taste_status reports
how many groups were excluded for this reason. (If the group record and the later
reward record disagree, the reward record — the engine's own flow — wins.) Utilities are
standardised within the candidate group before the softmax: this keeps the
logits responsive so the latent can always reorder candidates — with an
unbounded-then-squashed output the utilities saturate under the margin objective
and the latent loses all authority over the ordering (measured: 12/12 neutral
briefs identical, versus 12/12 different after the fix).
Optimizer split. Muon (quintic Newton-Schulz, coefficients and nesterov form
from the reference Muon implementation) on 2D weights; AdamW on z_u and every
1D/bias parameter. Muon's lr is in spectral-norm units, so on these small
matrices it must be ~10× below an Adam lr — at the higher lr the model stops
fitting (measured: training accuracy stalls at ~0.5 versus ~0.85).
Artifacts. Real GGUF v3 files (F32 tensors + metadata): a shared
style_brain_video_v1.gguf and a tiny per-creator <creator>.gguf carrying
z_u, the feature layout, and the digest of the trunk it was trained against. A
digest mismatch is refused rather than silently scoring with the wrong trunk.
How the loop feeds it
# 1. propose a GROUP (logged automatically); the best candidate by the identity
# is selected, or by the objective critic reward when no identity exists yet
propose_schema inventory=<inv> rubric=<rub> group=4
# 2. render the selected candidate, then log that group's outcome
critic_edit schema=<schema> inventory=<inv> rubric=<rub> group_id=<id> \
candidate=<idx> chosen_by=creator
# 3. inspect and train
taste_status
train_identity creator=erkin # writes .editapart/identity.gguf
train_identity creator=erkin freeze_style=true # later: only z_u moves
# the photo loop is the same protocol on its own log/identity
photo_propose src=<img> inspect=<insp> rubric=<rub> group=4
photo_critic schema=<schema> rubric=<rub> result=<render> group_id=<id> \
candidate=<idx> chosen_by=agent
taste_status modality=photo
train_identity modality=photo creator=erkin
chosen_by=creator|agent matters: the pick is logged as a revealed
preference, becomes that group's top reward, and gives the trainer a preference
term. A rubric-derived reward alone cannot identify per-user taste: with no
user-dependent term in the objective, two creators with opposite tastes differ on
only 4/12 neutral briefs (p ≈ 0.21 against the unrelated-picks null), versus
8/12 (p ≈ 0.0006) once the revealed pick is in the objective (margins +3.6 s
vs +7.0…+9.4 s). The override additionally fits the creator's own corpus far
better than the auxiliary loss (0.83/1.00 vs 0.22/0.26), so both mechanisms are
kept; tests/experiment_revealed_preference.py reproduces the whole table.
Multi-input effects (the v2 layout)
The v1 feature layout describes the resulting cut sequence — shot counts and durations, cuts per minute, pace band, motion, coverage, order, luminance, audio RMS. A transition, a grade, ducking and an overlay are decisions about how inputs combine (a transition joins two clips, ducking relates speech to music, an overlay composites layers), and none of them appears in that list. So with v1 the identity could only choose which selection to use; it could not set an effect's parameters at all. That is a real limitation, not a detail: an NLE-shaped editor is mostly effects.
video/v2 adds ten effect-parameter features (transition fraction, duration,
alignment and asymmetry; grade deviation; LUT; reframe; duck depth; segment gain;
overlay fraction) under a new spec id — never an edit of v1, because the layout list
is part of an artifact's digest, so a v1 identity keeps loading and keeps refusing a v2
trunk instead of being silently reinterpreted with shifted columns. A log may hold both;
the majority layout trains and the rest are counted (groups_other_spec), since mixing
vector layouts would either crash the first forward pass or mis-align columns.
Two halves are needed, and both exist: a group that spans effects, so a preference
can be learned — propose_schema(… effect_grid=true) pairs the selection grid with
effect knobs (the default grid varies selection only, so every candidate in a legacy
group agrees on every effect feature and nothing could ever teach an effect
preference) — and an identity that can choose them:
propose_schema(… refine=N) runs a deterministic coordinate ascent over
the effect axes (transition type/duration/alignment, ducking, grade strength) scored by
the identity, and the tuned schema joins the group as one more candidate. Two
properties are deliberate:
- It needs an effect-aware identity. A v1 trunk has 16 inputs and cannot score a
26-dim vector, so the request is declined with a reason rather than silently
ignored — and the feature spec a group is logged under is explicit
(
--feature-spec), because a group logged under v1 can never teach an effect. - No identity means no effect taste.
refine_effectswith no scorer returns the schema untouched: with no identity there is no taste to apply, and maximising the objective critic there would quietly turn a rubric rule into a taste claim.
Measured (tests/test_taste_model.py::TestEffectTaste): groups are logged under v2
with a reward that rises only with the transition duration — a hidden preference
expressed solely through an effect parameter — an identity is trained on it
(train_rank_acc ≥ 0.8), and the refiner then chooses a dissolve and pushes the
duration past 0.8 s from an all-cuts seed. The control run with no identity changes
nothing. TestEffectFeatures pins the blindness of v1, the superset relation and the
mixed-spec rule.
They are now rendered, and the rest are refused. cmd_render executes transitions
— xfade between runs, acrossfade for the audio, and an outgoing tpad extension so
all three alignments are exact — which is what makes the transition features true: the
rendered duration equals the planned one to the frame, and the picture at the boundary
differs from a hard cut. grade.lut renders too: the repository ships generated
.cube LUTs (luts/warm.cube, luts/cool.cube, reproducible with
tools/make_luts.py --check), a bare name resolves to one of them and a name that
resolves to nothing is refused. The all-cut path is byte-identical to the renderer
that produced every earlier measurement, so nothing had to be re-measured. The fields the
renderer still does not execute — audio.duck (no music bed is mixed), overlay (no
overlay media), transform.position (the output canvas is the source frame) — are
declared-but-unrendered and refused by assert_renderable at propose, refine and
render time, with a reason. That makes the schema's own contract
("every field in the schema is renderable by the renderer") an enforced property
instead of a comment: an edit decision can no longer
describe a picture the renderer will not produce.
Per-effect credit
Credit is attributed to the decision that actually varied, in two forms:
# the attribution table for one group: what changed, why, and how much it weighed
python bin/edit_apart_core.py attribute <dataset.jsonl> --group-id <gid>
# the corpus-wide profile: what the rewards (and an identity) prefer, effect by effect
python bin/edit_apart_core.py attribute <dataset.jsonl> [--identity ID] [--style S]
Three properties make the table trustworthy rather than plausible. Rows are keyed by
shot, not by index — candidates of one group can select different numbers of shots,
so index 6 of one candidate is a different shot from index 6 of another and an
index-keyed diff invents changes (a real value against None) and blames them on an
effect. The axis comes from the logged knobs, so a coupled decision counts once:
transition type/duration/alignment are one decision (a duration is meaningless without
the type) and they legitimately move together, where a slot count would misreport them
as three decisions. And each row carries the critic's own judgement for that shot
(decision/why/reward_delta, now logged on the reward record) beside the
group-relative advantage, so a row states its basis and its magnitude.
attributable is the flag to check before believing any of it: exactly one knob axis
engaged AND the candidates selected the same shots. A group that varies several
decisions, or that changed its material as well as its effects, has no
single-decision credit to give — the profile counts and skips those groups rather than
folding an unattributable number into a claim. That is why --effect-grid holds the
selection fixed and varies only the effects; exploring both dimensions is two
proposals, not one. (Nothing measured depended on the previous pairing, which was
introduced and replaced inside this unreleased sequence.)
One finding this makes visible: in an effect group the objective critic's reward is
flat across transition values — it scores duration, motion and target, and cannot see
a dissolve at all. So the effect columns of the profile discriminate only through the
identity (the taste_mean column), which is precisely why an effect preference has
to be learned from an external pick rather than from the rubric. The test
test_the_critic_cannot_see_a_transition fails if that ever stops being true.
Not yet done, and worth naming: credit is still per candidate, not per effect —
the group-relative advantage is still broadcast over the whole candidate rather than
graded into the per-row table — the table records the attribution (which decision
changed, on what basis, with what weight) but the gradient is not yet distributed over
its rows, so a group that varies several decisions still trains one advantage for all of
them (the segment-local-credit design in docs/expansion-plan.md).
The effect axes are a fixed grid rather than continuous optimisation, and effects carry
no provenance of their own beyond the pick provenance described above.
The training datum (and why the raws are kept)
A taste datum is not a timeline with a score; it is a group: alternatives produced under one fixed request, materials and constraint set. Schema v3 of the group record keeps everything a replay needs, in the record itself:
| Kept | Why |
|---|---|
rubric + rubric_hash |
the constraints are what make the candidates comparable |
inventory + inventory_hash |
the evidence the features were computed from (per-shot motion/luma/rms) |
each candidate's schema — the timeline, not just its knobs |
so a feature can be re-derived, a reward re-scored, and credit attributed to the effect that changed |
source_sha256 |
the raws are identified by content, not by path |
content_hash |
deterministic identity of the proposal, so a ledger can key annotations to a revision |
selected, engine, feature_spec |
what was chosen, by which build, under which layout |
Before v3 a record kept only derived numbers — no rubric, no inventory, no timeline, no source identity — so a defect in the feature extractor or a change to the critic made the corpus un-remeasurable. This project's own corrections log is a history of needing exactly that: three conclusions flipped after defects and each had to be re-measured on the same data. Keeping the raws is the precondition for that discipline, and an NLE makes it sharper, because there the edit is a reference structure over a media pool rather than a schema over one file.
# re-derive every logged group from its own record and compare
python bin/edit_apart_core.py replay <dataset.jsonl> [--limit N]
It recomputes each candidate's feature vector and the objective reward from the
recorded timeline + inventory + rubric, verifies the hashes, and reports older records
as not replayable rather than guessing. The subjective leg is a person's or agent's
judgment: it is reported as not recomputable and never re-invented.
tests/test_taste_model.py::TestLogReplay is the acceptance test, with two negative
controls — a corrupted stored vector and a changed source file are both caught.
Why annotations stay on our side. Measured: through the FCPXML adapter nothing of
ours is even written — not the effect, not the marker, not the metadata
(tools/otio_conformance.py documents the lossless container). So the ledger carries
the judgment (why, decisions, provenance) keyed by content_hash, the timeline
carries structure, and export is one-way and lossy: anything crossing an editor
boundary returns as a new revision with a new judgment, never as recovered
metadata.
Not yet v3: the photo engine still writes pre-v3 records (a replay reports them as not replayable), and media retention is a policy here rather than an enforced garbage-collection rule.
Measured results
| Claim | Measurement |
|---|---|
| Gradients are correct | analytic vs finite differences: 1.7e-08 (model) / 1.7e-09 (full objective, incl. preference term) |
| Objective recovers a hidden preference | held-out pairwise agreement 0.82–0.84 vs 0.50 chance; argmax 0.55–0.86 vs 0.20 chance (3 seeds, 120 synthetic clips) |
| The identity is interpolatable | midpoint latent brackets the endpoints for ≥70% of candidates and moves the mean monotonically |
| Optimizer split holds | Muon touches 2D only, AdamW 1D only (asserted, with witnesses) |
| Artifacts round-trip | GGUF write→read bit-exact; digest mismatch refused |
| The identity changes the loop | two creators on the same shared trunk: 8/12 neutral briefs differed, 8/12 in the predicted direction, 21.4s vs 12.0s mean selected duration (re-measured after the objective fix below) |
| The revealed pick is what carries taste | ablation (re-measured): with no user term 4/12 briefs differ (p ≈ 0.21, indistinguishable from unrelated picks) vs 8/12 (p ≈ 0.0006) for every mechanism tried; the reward override also fits the creator's own corpus far better (0.83/1.00 vs 0.22/0.26) and widens the margin (+9.4s vs +7.0s) |
| Photo taste transfers, with a narrower margin out of family | two creators, one shared trunk: 10/10 held-out briefs picked differently (10/10 in direction, crop area 0.155 vs 0.389); on one unseen crop geometry 10/10 but with a smaller gap (0.30 vs 0.45) — the geometry/content confound narrows it |
| The repair is partial and content-driven | 4-arm transfer experiment (re-measured): only the 3-image/narrow-family arm transferred to an unseen image+geometry (tight/loose ratio 0.400) while A1/A2/B2 showed no effect (1.000); on a seen image at the same geometry three of four arms did separate. The ±0.36 ordering-metric noise floor from the no-taste control is a pre-fix measurement |
| The loop is numpy-optional | legacy propose group=1 and group logging work with numpy blocked; only the model path errors |
Reproduce with tests/test_taste_model.py (74 tests, no media needed) and
tests/test_loop_e2e.py (real footage; see Verification).
Concurrent work (2026-09)
Group-relative preference learning on edit decisions is not unprecedented, and the honest version of this project's positioning says so:
- Crayotter: GRPB — Learning Long-Horizon Video Editing Agents via
Group-Relative Preference Backpropagation (Aug
- makes the same core observation this project's Finding 1 measures: a global
scalar objective over subjective edits is ambiguous and temporally uninformative, and
fixing the request/materials/constraints turns it into an ordinal comparison among
directly comparable alternatives, whose same-task rankings become zero-sum
advantages. It goes further in two ways this repository has now partly adopted:
credit localised to semantic editing segments, and a lagged allocator / guarded
transmission so a group's own judgment never trains that group. The provenance rule
above is the local form of the guard; segment-local credit is designed in
docs/expansion-plan.mdbut not yet implemented.
- makes the same core observation this project's Finding 1 measures: a global
scalar objective over subjective edits is ambiguous and temporally uninformative, and
fixing the request/materials/constraints turns it into an ordinal comparison among
directly comparable alternatives, whose same-task rankings become zero-sum
advantages. It goes further in two ways this repository has now partly adopted:
credit localised to semantic editing segments, and a lagged allocator / guarded
transmission so a group's own judgment never trains that group. The provenance rule
above is the local form of the guard; segment-local credit is designed in
- ReelBrain (Q00/ReelBrain) reaches the same governance conclusion from the product side: a learned taste profile is a behavioural prior, never evidence about the footage, and explicit current steering always overrides stored taste.
- The agent-drivable NLE and headless-montage layers (OpenChatCut, Timeline Studio,
cutible and others) have commoditised timeline-as-data, diffs, deterministic render,
MCP surfaces, dry-runs and predicted diffs — see
docs/state-of-the-art-2026-09.mdfor the survey and the gap analysis, including the interop facts verified first-hand withtools/otio_conformance.py.
What remains distinctive here is narrower and stated as such: a per-creator latent (~1 KB, against a frozen shared trunk whose digest the artifact carries) rather than a prompt profile or a fine-tuned model; the decision object serving as render input, critique target and training datum; system-generated candidate groups as an auditable comparison set; an objective, recomputable critic; and the trained policy as the selector, not the language model.
Boundaries (honest)
- The trained policy is the selector, not the language model. Training changes which candidate the loop picks among the ones this engine generates; it does not fine-tune an LLM, and it cannot invent an edit the candidate grid does not contain.
- Video only. The design's v0 scope is pacing + shot selection + tempo, which
is what the feature layout covers. Photo taste (colour/composition) is the next
phase; the model is generic over feature specs, but
photo_proposestill emits a single rubric-derived schema and logs nothing. - It needs enough clips. Held-out accuracy is only meaningful once there are
several logged clips; with one or two the model fits them and generalises
nothing.
taste_statusreports how many trainable groups are in the log. - The trunk is shared, so per-creator training freezes it (
freeze_style=true); a normal run trains the trunk too and is meant for building the shared model from the union of several creators' logs.
Install
EditApart ships as a plugin bundle that carries an agent preset, and requires
DeepSeek Harness ≥ 0.1.7: since 0.1.7 a preset is an
@deepseek-ai/dsh-agent-preset declaration row supplied by a bundle patch, and the
older user-preset directory ($DSH_HOME/.agent-presets/<id>/) is no longer read.
dsh plugin --profile web add https://github.com/EditTogether/DSH-EditApart
dsh plugin runs the profile's package manager, and an installed package that
declares dsh.bundle is activated as a bundle automatically. Restart the harness
(or open a new session) and pick EditApart (AI taste editor) in the mode
picker.
To use the optional venv toolchain, install from a checkout instead, so the venv
and .dshenv live somewhere you control:
git clone https://github.com/EditTogether/DSH-EditApart ~/dsh-edit-apart
cd ~/dsh-edit-apart && bash setup.sh
dsh plugin --profile web add ~/dsh-edit-apart
Update with dsh plugin --profile web update dsh-edit-apart (or
git -C ~/dsh-edit-apart pull for a checkout); uninstall with
dsh plugin --profile web remove dsh-edit-apart, or through the harness's plugin
manager.
What the bundle declares. package.json declares main =
./plugins/edit-apart.mjs — a bundle is judged loadable by its declared entry
artifact (the plugin market reads main, then exports, else looks for
index.js, and reports "the declared entry artifact is missing (source-only
checkout or blocked build) — the next boot would fail" when it finds none), and
this package's loadable entry is exactly the module the preset mounts. It also
points dsh.bundle.patch at cordis.patch.yml, whose single insert row is the
preset declaration: id
ai-video-editor, the display name/description/order the picker shows, and
config.plugins — the composition that used to be agent.cordis.yml. A row's
specifier resolves against the profile directory, so this package's own plugin
is named by package subpath (dsh-edit-apart/plugins/edit-apart.mjs) and its
method skill is resolved from the profile down into
node_modules/dsh-edit-apart/skills/. No shipped file contains a machine-local
path, and tests/test_bundle_packaging.py fails if one reappears, if a relative
row specifier creeps back in, if the declared entry artifact goes missing, or if
this section stops showing the real command.
Requirements. Only what the harness already ships — the composition names
harness plugins and adds no npm dependencies — plus ffmpeg/ffprobe and
scenedetect for video, and optionally ImageMagick for photos. Toolchains resolve
DSH_* env → <preset>/.dshenv → PATH, so a machine-local install is a .dshenv
(gitignored) and never a repository edit.
Verified as installed (harness 0.1.7-rc.2, throwaway profile):
| Check | Result |
|---|---|
dsh plugin --profile bundlecheck add <repo> |
installed, and dsh.profile.bundles gains dsh-edit-apart automatically |
dsh --profile bundlecheck --dump-config |
exit 0, composed tree contains the preset-edit-apart → @deepseek-ai/dsh-agent-preset row with config.id: ai-video-editor |
| the plugin row's specifier, resolved from the profile | resolves to the package's own plugins/edit-apart.mjs, which imports and exports apply |
| the skills expression, evaluated against the profile | <profile>/node_modules/dsh-edit-apart/skills/edit-apart/SKILL.md exists |
the live registry's entryListProblem over all 17 child rows |
OK — the shipped standard preset returns the same verdict |
Not separately verified: a cold-session mount of this freshly installed preset (the declaration mechanism is the one the live profile already mounts presets with).
This project is discoverable in the DeepSeek Harness ecosystem under the
dsh-plugin topic.
How to use it
- Start a session on this preset (select
ai-video-editor/ EditApart in the mode picker). The preset mounts its ownedit-apartplugin, so the toolsinventory,features,propose_schema,render_schema,review_frames,critic_edit,revise_schema,train_identity,taste_status,identity_init,photo_inspect,photo_propose,photo_render,photo_critic, andphoto_reviseappear in that session's catalog. - The bundled
edit-apartskill drives the loop. - Toolchains are resolved at runtime, not hard-locked:
ffmpeg/ffprobe/scenedetect/Python (video) and ImageMagick (photo) come from the matchingDSH_*env (DSH_FFMPEG,DSH_FFPROBE,DSH_SCENEDETECT,DSH_EDIT_PY,DSH_IMAGEMAGICK), then.dshenv, then PATH. A web-profile in-browser ffmpeg / ImageMagick (a store plugin) can supply them instead of local binaries.
Portable install (public / another machine)
The code never hardcodes an absolute path. Toolchains resolve env DSH_* →
<preset>/.dshenv → PATH, so the preset runs on any machine with ffmpeg +
scenedetect. To provision a machine where scenedetect lives in a venv (not on
PATH):
bash setup.sh # creates ~/dsh-edit-venv, installs scenedetect, writes env.sh
source ./env.sh # makes DSH_EDIT_PY / DSH_SCENEDETECT available for this shell
# OR, if the tools are already installed:
# cp env.sh.example env.sh && source ./env.sh
# setup.sh also writes .dshenv so the plugin auto-loads paths without sourcing.
env.sh.example— portable template; copy toenv.sh(gitignored) and edit.env.sh— generated, machine-local, gitignored; exports theDSH_*vars..dshenv— machine-local paths the plugin auto-loads when the matchingDSH_*env is not already set (gitignored).requirements.txt—scenedetect(+numpy<2to avoid the opencvKeyError: 'rad2deg'import crash).numpyis a scenedetect dependency anyway and is what the taste model runs on; the legacy loop (propose group=1) works without it.setup.sh— creates the venv, installs deps, writesenv.sh/.dshenv.
Override keys: DSH_EDIT_PY, DSH_SCENEDETECT, DSH_FFMPEG, DSH_FFPROBE,
DSH_IMAGEMAGICK.
Taste-model keys: DSH_EDITAPART_IDENTITY (per-creator GGUF),
DSH_EDITAPART_DATASET (group log), DSH_EDITAPART_PHOTO_IDENTITY /
DSH_EDITAPART_PHOTO_DATASET (the photo loop's own pair),
DSH_EDITAPART_DATA (directory holding them, default <workspace>/.editapart),
DSH_EDITAPART_STYLE (shared style-brain path) and DSH_EDITAPART_GROUP
(candidates per proposal, default 4). The identity and the log are per-workspace
on purpose, so each project accumulates its own creator history.
The plugin is import-free on purpose: a preset row's specifier resolves against the profile directory, where the harness's own
@deepseek-ai/*packages are not resolvable. It hand-buildsToolDefinitions and callsctx.tools.register(), shelling out tobin/edit_apart_core.py/bin/photo_core.py(found as siblings of the plugin file, so the whole package travels together).
Trust & security
- No shell command injection. The engines launch processes with list-argv
subprocess.run([...])and the plugins withspawn(PY, [CORE, ...args])— never a shell string, nevershell=True/os.system. Paths and schema values are passed as literal arguments, so a value likex; rm -rf /cannot be interpreted by a shell. The;-joined ffmpegfilter_complexstring is a single argv element, not shell syntax. - Caption text is sanitized. ImageMagick
-annotate/-caption/-labelinterpret%[...]/%[fx:...]format escapes and a leading@as "read text from file". The photo engine escapes%/@/\to visually-identical fullwidth codepoints (_escape_annotate_text) so untrusted caption text cannot trigger an IM expression or file read — independent of the host ImageMagickpolicy.xml. (The system package policy already disables@file-reads on many images, but the code no longer relies on that.) - Paths are trusted tool args.
src,out,outdir, and generatedassetpaths are operator/model-supplied and passed straight to ffmpeg/scenedetect. Within DSH the sandbox scopes writes; if you run the engines outside DSH, scope the process so a craftedout/outdircannot write outside your intended directory. .dshenvis a trust boundary. The plugins load<preset>/.dshenvand setprocess.env(only when the matchingDSH_*env isn't already set). Whoever can write that file controlsDSH_EDIT_PY/DSH_SCENEDETECT, hence the executed interpreter. Keep it out of version control (it is gitignored) and treat it as operator-owned.
Video reference run (verified)
On better-com-ceo-roasted-by.mp4 (41.7s, 23 shots):
- Inventory: 23 shots.
- Rubric: brisk pace, shots 0.9–4.0s, target 20s.
- Schema: 11 segments (dropped a 0.58s shot). Render: valid 20s 1080p mp4.
- Objective critic: overall 0.3, reward −0.1, flagged clip_001/clip_002 as "low motion / dead air." This was wrong — a motion-luma heuristic called the set-up ("Well, when are you going") and the first punchline (Ramsay, "and start showing some") dead air, and would have dropped the two best beats.
- Vision critic (normative): overall 0.62. Keep clip_001/clip_002 (the set-up + punchline, caption-bearing and high-value); trim/cut clip_005 (a long static 3.8s hold); consider reordering the "BETTER.COM EMPLOYEES" cutaway (clip_010/011) earlier as an establishing subject card.
The lesson baked into the skill: the vision leg (frame inspection) is
normative; motion-luma is only a hint and can be confidently wrong on the
highest-value shots. Always review_frames before dropping a shot.
Photo reference run (verified)
On a synthetic 1800×1200 landscape (photo.jpg), rubric = warm punch, saturation
1.15, crop 600×500, width 1200, target_luma 0.52:
- Inspect: mean_luma 0.264, luma_std 0.19, p50 0.18/p95 0.835, saturation 0.132, mean_rgb [0.214, 0.277, 0.288].
- Schema: crop → grade → resize.
- Render: 1200×1000. Critic: overall 0.5, exposure_ok/contrast_ok true, reward 0.5, all ops kept; revise kept crop/grade/resize.
- The exposure/contrast terms are graded (see the photo taste section): full credit inside the tolerance exactly as before, linear falloff outside it, so a candidate that passes is scored identically to the pre-grading critic.
Corrections
An external review re-ran the suites and reproduced five defects. All are fixed here, and the behavioural numbers this README reports were re-measured afterwards (the earlier figures are withdrawn, not silently kept).
The GRPO ratio was not a ratio (
bin/taste_model.py).train()stored raw standardised logits as the ratio baseline, sorho = exp(log_softmax(s) - s) = 1/Z— a per-group constant. Negative-advantage candidates then always fell into the clipped branch and their downward push stopped depending on|A|(measured: an identical 0.0268 for A = −0.1, −2.0 and −8.0), so the objective had silently degenerated to positive-only REINFORCE. The telemetry had the same bug:clip_fractionwas a per-call sum divided by the batch count, reported as 5.9 (a "fraction" of 590%). Fixed by storinglog_softmax(...)and separating the two denominators;tests/test_taste_model.py::TestObjectiveFidelitypinsrho == 1,clip_fraction ∈ [0, 1], and that a −2.0 advantage pushes ~10× harder than a −0.1 one. Consequence: video per-creator separation re-measures 12/12 → 8/12 (21.4s vs 12.0s) and the photo out-of-family result 0/10 → 10/10 with a narrower margin (0.30 vs 0.45).ImageMagick 6 silently killed the photo loop's crop axis (
bin/photo_core.py). Resolvingconvertand then runningconvert identify …fails on IM6, and_dimsreturned0x0silently: every percent crop became0x0+0+0and the entire spatial feature family went to zero while inspect/propose/render/critic all still reported success. Now the identify binary is resolved separately (DSH_IDENTIFY>magick identify> siblingidentify> PATH), a dimension failure is loud,proposerefuses a dimension-less inspect or an all-identical candidate group, and both cores ship aselftestconformance check (python bin/photo_core.py selftest).A video render hard-required audio on every input. One muted clip failed the whole graph with "Stream specifier 'a' matched no streams". A silent input now gets
anullsrc, and when no input carries audio the silence is emitted withoutloudnorm— single-passloudnormon digital silence divides by zero energy and hands the AAC encoder NaN."reward_obj": nullaborted the whole training log (float(None)). Unscored candidates are tolerated and skipped, with counters surfaced bytaste_status.The video E2E logged the wrong candidate's reward: it critiqued the selected schema while labelling the reward with
--candidate <pick>. Candidate schemas are now exposed in the group metadata — which is also what the skill's "render a different candidate than the selected one" workflow requires — and the test asserts the logged reward is the picked candidate's.The effect features described a renderer that did not exist (found by a second external review).
video/v2measures transition, duck, LUT, overlay and position fields whilecmd_renderonly ever emittedconcat: an identity could train for 300 epochs to prefer a dissolve and the rendered file was still a hard cut, and the logged decision saiddissolve 0.8while the picture disagreed — precisely the failure the "auditable decision object" claim exists to prevent, on the newest fields. Fixed by rendering transitions (xfade/acrossfade, frame-exact, cut path byte-identical) and by refusing the fields the renderer still cannot execute (assert_renderable, at propose, refine and render), with an effect grid that may only span rendered fields.
Packaging correction (harness 0.1.7). The previous version of this README
told users to clone the repository into $DSH_HOME/.agent-presets/<id>/. That
layout was retired in 0.1.7, where a preset is an @deepseek-ai/dsh-agent-preset
declaration row carried by a bundle patch — so the instruction installed files
nothing reads, and the "verified as installed" check that accompanied it passed
only because it exercised the package that implemented the retired mechanism.
The repository is now a bundle (package.json → dsh.bundle.patch →
cordis.patch.yml) installed with dsh plugin add; agent.cordis.yml and
preset.yml are gone, and the check that could not have caught this has been
replaced by tests/test_bundle_packaging.py plus the real scratch-profile install
in the table under Install.
Smaller fixes from the same review: schema values that reach the ffmpeg
filtergraph are validated (_num/_ff_crop), revise no longer KeyErrors on a
schema without meta, photo temperature actually renders (it was
-fill … -colorize 0, a 0% no-op), one temp dir per propose call instead of one
per candidate, created_at/trained_at artifact metadata made consistent, and an
empty style_file no longer resolves to a directory.
Verification
# packaging contract: bundle manifest, preset declaration, specifier forms and
# the no-machine-local-paths rule (needs PyYAML; <1s)
~/dsh-edit-venv/bin/python tests/test_bundle_packaging.py -v
# model + video loop unit suite: 74 tests, no media required (~25s)
~/dsh-edit-venv/bin/python tests/test_taste_model.py -v
# environment conformance: prove THIS install's renderer/prober can support the
# loop before trusting a logged group (both cores ship one)
python bin/photo_core.py selftest
python bin/edit_apart_core.py selftest # needs scenedetect on PATH/DSH_SCENEDETECT
# photo taste suite: 35 tests (feature tests need no renderer; the grouped and
# per-creator tests need ImageMagick; ~4 min, dominated by candidate renders)
~/dsh-edit-venv/bin/python tests/test_photo_taste.py -v
# end-to-end on real footage (scenedetect + ffmpeg + a real render)
DSH_EDIT_PY=~/dsh-edit-venv/bin/python \
DSH_SCENEDETECT=~/dsh-edit-venv/bin/scenedetect \
DSH_EDITAPART_E2E_SRC=/path/to/footage.mp4 \
~/dsh-edit-venv/bin/python tests/test_loop_e2e.py -v
The video E2E suite also renders a muted clip (the audio fallback) and rejects a
hostile crop/scale value with a clear error.
The unit suite checks Newton-Schulz's bounded band and scale invariance, the
optimizer split (Muon never sees a 1D gradient, AdamW never sees a 2D one),
analytic gradients against finite differences, GGUF round-trips and digest
mismatch refusal, dataset folding (including legacy records and broken lines),
learning on a synthetic hidden preference, latent ablation/interpolation,
group=1 reproducing the legacy schema exactly, critic-reward logging, and
graceful degradation when numpy is unavailable.
The E2E suite builds an inventory from real footage, drives two creators with
different revealed preferences through the real loop, trains one shared trunk
plus two frozen-trunk identities, and asserts that the same neutral brief gets
materially different edits — then renders the taste-selected schema with real
ffmpeg and critiques/revises it. Measured on
better-com-ceo-roasted-by.mp4 (41.7s, 23 shots): 8/12 briefs differed,
long-take 21.4s vs short-take 12.0s mean selected duration; the rendered
taste-selected edit was 18.75s, critic overall −0.15, 10 → 7 segments after
revise. (These are the post-correction numbers; see Corrections below.)
See skills/edit-apart/SKILL.md for the full protocol.
No comments yet. Be the first to write one.