In the Mix 2007 keynote, Metalliq demonstrated a video editing application under the somewhat unexpected name Top Banana. This application was based on Video Table, the outcome of a few days of casual experimentation in 2006. This post gives a short description of the idea behind Video Table, and of some of the ideas in the original prototype that were not shown in Top Banana.
I’ll add a video capture of using the prototype at some point, when I get time to do it.
Consumers today capture a lot of video. However, this video is often quite different from what it might have been when all we had were tape-based camcorders: Instead of capturing long shots of video on a tape, we now capture a lot of video snippets with digital cameras and cell phones. Our video snippets are for instant communication: With video web sites to share these snippets, traditional editing with its focus on story telling is not necessarily what we need.
On a weekend with my daughter, I might capture dozens of little vide snippets. Just selecting the good ones in the normal environment of the desktop, in grid-based browsers and explorers, is slow and no fun. Video table is about exploring nicer ways to select, process and assemble little video snippets for instant communication.
As you look at the screen shots in this post, keep in mind that Video Table is just a concept study for fun. Some aspects of it are rather un-evolved because they were not central to what I was exploring.
Also, many thanks to Celso Gomes, who was so kind to take the original I wrote and create a new skin to make it presentable. The fact that Celso was able to do this without touching any code, by just changing and re-arranging visuals in Expression Blend, is a good example for designer-developer workflow in WPF.
Video Table Overview
Video Table is based on the idea of a light table.
In photography studios, these are huge tables with a built-in light that you can put a whole load of slides on and then sort and select. The nice thing about light tables is that they match a natural workflow – you can arrange things in space in a natural manner. Video Table does the same, but for video snippets.
Most of the screen is taken up by a large work surface. On that surface, you can dump as many video clips as you like, and arrange them just like you would with photos on a table. As you do, you can play as many clips concurrently as you like. You can resize the thumbnails to see more detail. You can drag out the thumbnails into little filmstrips, giving you an instant view of a clip. You can arrange clips into stacks, giving you instant editing. All this you can do on a surface that feels natural, unconstrained by much UI.
Selection and sorting of multiple videos benefits a lot form the ability to create instant, on-the-fly collections and spatial arrangements – a fast, natural and visual way of creating and keeping order.
In the remainder of this post, let’s look at some details.
On Video Table, videos appear just as little thumbnails. But these thumbnails have attitude: When you move the mouse over them, player controls appear on the surface:
There is a lot you can do right here. You can scrub over the video by just dragging the mouse over the image (this is a fast, convenient gesture that needs no precision), or by grabbing the red play cursor on the mini timeline below the thumbnail. You can play the video by hitting the play button. You can resize the thumbnail with the little round manipulator in the lower right. And finally, you can trim the video using the little in-out markers on the mini timeline for the clip.
Of course there usually are many clips on the desktop. The player UI hides itself as soon as you move the mouse away from a thumbnail. Multiple players can play video concurrently, so that it is easy to compare clips.
A big problem with thumbnails for videos is that it is really hard to get a good idea what is in a clip from a single thumbnail picture. Imagine you took 47 video snippets from your daughter’s birthday party, and the thumbnails all look more or less the same. Finding things by playing every video is just so slow.
Video Table addresses this problem by allowing you to drag out a thumbnail into a filmstrip. Just take the little resize manipulator in the player controls and drag a thumbnail to be wide, and it converts itself into a filmstrip:
If you drag the red play line, the film strip adds a player:
In this picture, you can also see that the trim widgets have been used to trim down the video a bit.
The filmstrip view is also scalable. The following screen shot has three different sizes of the same clip:
Happy Together: Snippet Editing
Video Table puts a lot of focus on fast sorting browsing through collections of video clips. But while it is important to be able to quickly find, select, trim and publish individually videos to a sharing site, it is also important to be able to assemble a couple of shots quickly and easily into a mini story.
Video Table explores two ways of doing this: Stacks and storyboards.
Both methods have one thing in common that most video editors sorely lack: The ability to work on multiple timelines at a time. When you play with snippets, you often want to start trying out how one little assembly might look, and compare that with another. In normal video editors, timelines are heavy-weight and opening and closing projects is a big deal. On the infinite artboard, you can just have as many timelines or stacks of video as you like, and compare them as you go. The result is a workflow that is fun, natural and fast.
Let’s talk about stacks first. On the video table surface, you can simply drag multiple clips into a stack. This is all you need to do to create an sequence of clips. Just drag them into a stack, trim as desired before or after doing so, and send off to a sharing site.
In the following screen shot, you see a stack of three clips to the left, and another clip to the right.
When dragging the new clip over the existing stack, an insert adorner appears over the clips that are already in the stack:
In this case, the adorner indicates that the new clip will be inserted right before the one it is over. The final stack looks like this:
The resulting snippet stack is a much more ephemeral construct than a timeline in a traditional editor, but it can be edited in much the same ways. You can have as many stacks on the artboard as you need while browsing pictures. The mechanism is very fast for short and simple stacks of a few snippets, especially because you can try a few of them side by side and see what works.
As an alternative to stacks, Video Table also has storyboards. Just like with stacks, you can have as many of them as you like on the artboard. Stacks can also be converted into storyboards at any time. Keep in mind that the storyboard mechanism in the screen shots, while working, is even more sketchy and incomplete than other parts of the concept study. Take it as a placeholder to illustrate the concepts.
The following illustration shows a storyboard on the artboard:
Any number of storyboards can be created on the artboard with a simple gesture. The storyboard widget has two parts: The story board itself on top, and a simple player below. The player acts the same as any other thumbnail player: It can scrub and play and trim, and you can drag it out into a filmstrip, as shown in the screen shot above.
You can just add clips from the artboard to the storyboard by using drag and drop. This is quite similar to commercial editors such as Pinnacle Studio, with the exception that you can have any number of storyboards on the artboard, which is really useful for fast snippet editing.
During the drag and drop operation you can perform simple insert edits quite easily. In the example below, the red insert triangle shows that if the clip is dropped right there it will split the highlighted clip and insert right on the middle of it.
You can pan and zoom over the surface to see a lot with little detail, or little with a lot of detail. A “keyhole” navigator helps you to find your way around.
The orange rectangle shows the area currently visible. You can drag the rectangle around for panning, and use the little white dot to resize the rectangle for zooming. A real implementation would of course also implement zooming and panning through gestures on the artboard.
The Video Table UI was written using C# and WPF. Blend was used for all visuals, and VS for coding. It took about 3 days to write, which I found surprisingly fast given that I had to learn a lot of new WPF techniques and that most of the UI is based on custom controls and direct manipulation, rather than standard controls. As this was just a UI study, there is no real video engine backing the prototype – it just operates on image sequences.
Video Table is a concept study for a video editor targeted at fast browsing, searching, editing and publishing of video snippets. It is based on a light table paradigm, allowing to use the computer screen just like a big family table with a shoe box of photos. There is very little explicit UI, and as much direct manipulation of content as possible. Dragging thumbnails into thumbnails greatly helps to find clips at a glance. Freeform editing using stacks and storyboards right on the artboard is a fast, fun way coming up with good clip samplers for publishing in a playful manner. Beyond this post, there are also additional concepts for applying transitions, effects and titles in line with the light table paradigm.