Planet WebMadeMovies

February 12, 2013

David Humphrey

Webmaker – Sequencing Web Media

One of the most requested new features we get for Mozilla Webmaker is web media sequencing.  Users want to be able to use multiple video or audio clips in a single experience.  This past week Scott and Kate, with help from Matt and Chris, have been building and testing a solution in Popcorn Maker.

The idea is to make it easy to pick media clips, including HTML5 audio and video, YouTube, SoundCloud, and Vimeo, and let users position each clip along the timeline, and within the video canvas.

Here two YouTube clips have been added to the gallery.  I might want to use the video from one (surf video) and the audio from the other (“Wipe Out”).  The sequencer supports video+audio, video-only, and audio-only for each clip.  Next you figure out where to place clips in the timeline:

Here a Vimeo video and an HTML5 video are positioned within the timeline so that the former will start part way through the latter.  We can also specify the position and size of each visual clip within the video canvas, which means we can do interesting parallel layouts of multi-video experiences.:

In this example, four separate YouTube videos are positioned into a common space, with one of them occupying the majority of the space, and thereby emphasizing its importance  Each clip can be resized or moved (you can see that one of the clips is selected in the editor and has resize handles).

The Sequencer feature is still in active development, but is ready to be tested.  You can already make some interesting things with it.  Here are some examples you can look at:

You can remix any of the above, or make your own using this experimental branch.  You can let us know about issues you find or make suggestions in this bug.

February 12, 2013 05:18 PM

November 13, 2012

David Humphrey

Web App JavaScript Crash Reporting

We just launched Popcorn Maker 1.0 this weekend at Mozilla Festival 2012, and while the focus of the app is obviously web video remixing, one of our favourite features as a dev team was the JavaScript Crash Reporter.  Bobby and I wrote a little bit about it in our Mozilla Hacks post, but I wanted to say some more about the potential for this technique.

Working on Firefox code, I’ve long been a fan of the Mozilla Crash Reporter, which uses Google’s crash reporting system, Breakpad, and the Soccoro server.  Typing about:crashes in Firefox’s address bar will show you any crashes you’ve logged, which are links to crash-stats.mozilla.com.  Here’s an example.  When you’re lucky you get a stack showing where things went wrong, info about bugs related to this crash, aggregated data for other users’ crashes in the same spot, etc.

For developers it’s an incredible source of information, since often these crashes occur out in the wild, and under circumstances the developers can’t reproduce.  Crash reporting in native apps isn’t new, of course.  OS and application vendors routinely use this data-driven debugging method to gather useful data about edge cases in their code, and to make corrections.

The technique isn’t used as often in JavaScript applications, though.  More often than not, a user’s contribution to the data of a web app is about marketing or tracking vs. improving development and stability.  Another reason is that many pieces of JavaScript are libraries vs. apps, and get used on more than one installation, making it undesirable that code phone-home to report an issue.

With Popcorn Maker we wanted to make it easy for our users to give feedback and report issues.  Because the project isn’t for developers, asking that they find and file bugs in our issue tracker wasn’t realistic.  So we made a Feedback button and form, and also an automated Crash Reporter to deal with top-level exceptions (i.e., those that can be caught using window.onerror).

We’ve been using these techniques now for about about a month, since the beta launch for Ryan’s TED talk on Popcorn, and through the 1.0 launch, and I can now talk about what it’s like using them.  First, and perhaps not surprisingly, people don’t tend to leave much feedback.  It takes time to write free-form comments in a form, and most people don’t want to bother.  So far we’ve gotten between 3 and 6 a day.  The quality of what people say is usually quite high, though.  When they have bothered to do the work of filling out the form, they’ve taken care to do it well, and usually give us ideas for new features.

The crash reports are another story.  This morning when I sat down to check my email, I had over 50 waiting for me.  I was able to file 8 new bugs (many of them were the same issue hit by different people).  I was also able to see that we’ve regressed one bug we thought we’d killed completely, that we’re having issues in the Sogou and Zune/Media Centre browsers, that we have a timezone related bug in some of our date parsing, and that I finally have better data about a crash we first saw last week (i.e., different browsers report the error differently, giving us a more complete picture of what is actually failing). That’s data from just a single day!

I can’t emphasize how drastically this technique has changed our development practices.  In the period between 0.9 and 1.0 we found and fixed 12 bugs that our team couldn’t reproduce (there are another 12 on file that we’re still trying to fix).  We put a lot of effort into automated and manual testing, and work hard on cross-browser testing, but there is no way for us to hit all these issues.  Furthermore, there’s no way people would file all these issues.  What would they say?  “It didn’t work” isn’t helpful for them or us.  We need to know what failed so it doesn’t happen again.  Rather than having a web app that just silently fails, we give our users the chance to become part of the development process by submitting the crash data, and a lot of them do:

You can try simulating a crash by going to http://popcorn-dev.rekambew.org/templates/basic/ and doing `Butter.app.simulateError()` in your console.  If the user wants to add extra info, they can (most don’t), and then if they click ‘Yes’ we send anonymous data about the error.  For example:

We take that data and store a JSON report on Amazon S3.  Here is an example:

{
    "date": "Tue Nov 13 2012",
    "message": "TypeError: 'undefined' is not a function (evaluating 'tracksContainer.container.getBoundingClientRect()')",
    "appUrl": "https://popcorn.webmaker.org/templates/basic/?savedDataUrl=ted.json",
    "scriptUrl": "https://popcorn.webmaker.org/src/butter.js",
    "lineno": 9805,
    "mediaUrl": [
        "https://www.youtube.com/watch?v=0g2WE1qXiKM&butteruid=1352812826605"
    ],
    "stateList": [
        "trackeventadded",
        "trackeventadded",
        "trackeventadded",
        "trackadded",
        "trackeventadded",
        "trackadded",
        "trackeventadded",
        "trackeventadded",
        "ready",
        "mediaready"
    ],
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.17 (KHTML, like Gecko) Version/6.0.2 Safari/536.26.17",
    "popcornVersion": "v1.3-44-g05ac7ef",
    "butterVersion": "v1.0.4",
    "nullDomNodes": ".media-status-container, .youtube-subscribe-embed, [data-tooltip]",
    "comments": ""
}

There are a few gotchas to be aware of:

There are also other things I’d love to figure out how to do better:

I’d also love to not have to build a full UI for working with the reports :)   I think it would be great if web app developers could use Mozilla’s Socorro infrastructure to gather and analyze crashes.  Imagine if part of doing a Mozilla Market Place app was getting access to that infrastructure?  Chris Lonnen and I discussed this idea at the Festival.

At some point I’d like to rip the crash reporter out of Popcorn Maker and turn it into a stand-alone repo others can use.  The code is here, if you’re interested in seeing how we do it.  I’d highly encourage other web app developers to go this route.  It’s been a game changer for our team and app.

November 13, 2012 05:38 PM

October 19, 2012

David Humphrey

Popcorn Maker 1.0 Beta – TED Launch

We’re about 3 weeks away from the Popcorn Maker 1.0 launch at this year’s Mozilla Festival in London.  Just to raise my blood pressure a bit, we’ve shipped a beta today.  It coincides with the release of an amazing TED talk by Mozilla’s Ryan Merkley about Popcorn.js and Popcorn Maker.  “It would be cool if we could work with this video using the tool he’s discussing, is that possible?”  It is :)

I’m really proud of the work we’ve done here, not least because so many of my former and current Seneca students have worked on it as part of class projects and NSERC funded research at Seneca’s Centre for Development of Open Technology.

A huge thanks and congratulations to:

I’ll blog with more info about Popcorn Maker when we ship 1.0.

Well done team, well done.

October 19, 2012 06:59 PM

May 31, 2012

David Humphrey

HTML5 media and non-HTML5 media

I’m spending almost all of my time these days on three Mozilla video projects: Popcorn.js, the Butter SDK, and the soon-to-be-beta Popcorn Maker tool.  Yesterday, after weeks of bug fixing and reviews, we hit code freeze for the 0.5 release of Butter and Popcorn Maker (due out in late June).  The 1.3 release of Popcorn.js is not far away either.

Among the many things I’ve been working on with these releases, one of them has gotten me sidetracked on a personal project.  Popcorn is fundamentally about using HTML5 media elements in order to dynamically alter web pages in time; or as Rick likes to say: web media does something at a given time with some information.  Popcorn, and its many plugins, allow a developer to easily script changes in a web page to times in some media resource.  Popcorn code looks like this:

// Wrap a media element in the page with Popcorn
var p = Popcorn( "#video" )
  // Add a subtitle
  .subtitle({
    start: 0,
    end: 10,
    text: "This is a subtitle"
  })
  // Add an image
  .image({
    start: 15,
    end: 45,
    src: "http://videos.mozilla.org/serv/webmademovies/popcornposter.png",
    target: "image-div"
  })
  // Get some Tweets from @popcornjs
  .twitter({
    start: 30,
    end: 55,
    src: "@popcornjs",
    target: "tweets-div"
  })
  // Start the video!
  .play();

Here’s a recent example of someone using it.  It’s ridiculously easy to make complex time-based web experiences, or to create new plugins for doing complex things with simple syntax.

The problem is that lots of media floating around the web isn’t HTML5 media.  There’s a ton of Flash-based content, or things that use other plugins.  The good news is that it’s getting better with every passing month, as more and more sites switch to HTML5 media.

Some time ago, in response to the realities of content creators, we decided to create a family of Popcorn Player plugins that would allow one to use things like YouTube, Vimeo, SoundCloud, etc. with Popcorn and its plugins.  None of the APIs these services offer are as rich as HTML5 Media Elements, but we found that we could hack our way to something approximating a Popcorn media resource.

The other day my student Robert and I were trying to debug some errors in the YouTube player plugin.  As we fixed one issue, another would crop up, and eventually I threw my hands up and decided to try something different: rather than trying to find bugs in the players via Popcorn, what if I implemented the HTML5 video spec in JS around the YouTube IFrame API?

My initial experiments with this have been very good.  I’ve managed to create a new wrapper, HTMLYouTubeVideoElement, which is a drop-in replacement for a <video> element.  With this I can steal <video> tests and make sure I’m doing what a <video> would do.  Best of all I can pass it into Popcorn without any extra work:

var p = Popcorn( new HTMLVideoElement( "#video" ) );

As far as Popcorn is concerned, this is an HTML5 media element, with all the usual events, properties, and methods.  On our Popcorn call today, David Ascher turned me onto x-tag.js and I quickly whipped up a new <x-video> element, which works identically to a <video>, but uses YouTube as the “decoder”.  Check out the demo here.  Fun, right?

I’ve spent enough time over the past few years working through the HTML5 Media Elements spec to know that I’m not done, but the work thus far shows that this is doable.  My goal is to erase the API differences between using <video> and <audio> and all the various and sundry media types on the web.  We’re trying to decide the best way to leverage this directly in Popcorn and Popcorn Maker, and future versions should include it.

For now it’s one of those fun pieces of code I had to share.

May 31, 2012 02:03 AM

May 15, 2012

David Humphrey

Focus and Select: a cross-browser bed time story

Here’s an annoying little DOM’ism that stumped me last week (don’t worry, there’s a happy ending if you keep reading).  In Popcorn Maker’s UI I wanted to fix a few places where we have textboxes that contain data from json manifests.  When you click on such a pre-populated textbox, it’s nice if the contents of the textbox are selected.  And when you click again, it’s nice if the selection is removed and you instead position the cursor.

These are nice enough that one would sort of expect them to just work, and as such, that the DOM would actually allow you to do it.  Well, it’s easy to do on Firefox: just add a focus event to your input element, and call element.select().  Done.

Now test in WebKit and it doesn’t work. Sniff around the web a bit and you’ll uncover a nasty 4-digit WebKit bug from 2008 that causes mouseup to undo your selection.

“That’s fine,” you say to yourself, “just stop the mouseup event from doing its thing and clearing the selection.” This is what everything on the web I could find said to do. The trouble is, I want to not only select the contents of the textbox on first click, but I want the opposite on subsequent clicks.

In the end, cross-browser compatible code means unnecessarily elaborate code, but here we are. I’ll leave this for the next person who hits my same edge case–aka the default way a textbox should function. Here’s a little jsfiddle demo of it running, and here’s the code:

var selectaBox = (function(){

  function __highlight( e ){
    var input = e.target;
    input.select();
    input.removeEventListener( "focus", __highlight, false );
  }

  function __ignoreMouseUp( e ){
    e.preventDefault();
    var input = e.target;
    input.removeEventListener( "mouseup", __ignoreMouseUp, false );
  }

 function __addListeners( input ){
    input.addEventListener( "focus", __highlight, false );
    input.addEventListener( "mouseup", __ignoreMouseUp, false );
 }

 return function(){
   var input = document.createElement( "input" );
   input.type = "text";
   input.addEventListener(
     "blur",
     function( e ){
       __addListeners( e.target );
     },
     false
   );

   __addListeners( input );

   return input;
 };

}());

var s = selectaBox();
s.value = "This is some text."
document.body.appendChild(s);

May 15, 2012 03:27 AM

March 14, 2012

Scott Downe

Getting started with contributing to Processing.js or Popcorn.js

So, the first two open source projects I was involved in have a fairly similar work flow, and I decided to write down the steps to get started in these projects.

Both Processing and Popcorn have fairly good intro sites, and I would start there. Popcorn also have secondary site that is useful. Also might want to check out the original Processing. Those sites provide a general introduction on the projects, and not something I will be covering here.

Both projects have four primary things, an issue tracker, a source repository, revision control, and a developer community.

An issue tracker is where bugs and new features get tracked, assigned, and reviewed. Both projects use something called lighthouse as their issue tracker. These can be found here for popcorn and here for processing. Issues can be anything from whitespace fixes, to documentation changes to complete rewrites. The pages I liked are the recent activity pages. You may want to look under milestones to get a list of potential tickets.

A source repository is a place where the code is stored and updated. Both projects use github as their source repository. These can be found here for popcorn and here for processing. Those are our release repos, in that they only get updated once for each release. If you want the unreleased code, you have to get that from a separate repo. These can be found here for popcorn and here for processing. We call these development branches.

Both projects use git for revision control. It goes hand in hand with github, but I do feel like it is a different thing. Source repository is where the code is kept, and revision control are the tools that interact with it. This is how I like to think of it, as it helps me understand it the way I need to work with it. I might not be technically correct with the definitions, though. Kinda like how the alphabet is not actually a song, but a song helps to understand it.

Both projects use irc for their developer community. This is the place you can go to get real time help from the people that have the answers. Popcorn uses the channel #popcorn and processing #processing.js. These can be found on the server irc.mozilla.org. More information about connecting to irc can be found here or here.

Now that that is out of the way, you will want to create a github account if you don’t already have one, then fork one of the projects from their github page. You will probably want to fork one of the development repos.

From there, you will need to get that github repo onto your local machine. I used this post when I started this, and I recommend it to others.

Next step is to find a bug on one of the lighthouse pages. Maybe comment in the bug that you are interested in working on it, so others know not to work on it as well.

You will want to create a branch on your local git repo named after your bug number. So, if my bug is popcorn bug #927 I will want to create a branch called t927. We add the t because code commits can be shorthanded to numbers, and there may be collisions in branch names and commits. Always good practice to but a letter in it. You can do this by running this command:

git branch t927 develop
git checkout t927

What this does is created a branch named t927 that is based off the branch develop. Then, moves you over to that branch.

Next you can go about changing the code in your local git repo. Once done, you will want to run some git commands.

1. git status: this shows the files that have changed, that you may need to commit.

2. git add: this adds those files. If you notice running git status after a git add, things will of changed.

3. git commit: this commits those added files. Again, running git status after this, you should see even more changes. I like to run this command like this:

git commit -m “[#927] usefull message about what I did”

This will add a message to the commit, so future developers can see why certain lines were changed, and even track it back to the original lighthouse ticket.

4. git push: this pushes the changes into your forked github repo. An example of this command for when I do it is:

git push git@github.com:ScottDowne/popcorn-js.git t927

This command takes the ssh url of my repo, and a name to name for the branch. After doing this, I could find my branch here: https://github.com/ScottDowne/popcorn-js/tree/t927 and my changes here: https://github.com/ScottDowne/popcorn-js/commits/t927

From here you will want to take one of those two urls and drop it back intp the lighthouse ticket, as a comment, and change the status to peer-review-requested. Both projects have a two step review process. Peer-review and super-review. Once both reviews passed, the repo manager will pull your changes into develop, and then on release day, everything including your changes will be pulled into master.


March 14, 2012 05:44 PM

March 10, 2012

Scott Downe

Popcorn.js 1.2 new features

A lot of new features landed in this release.

In no particular order:

1. Players have a _teardown function a lot like a plugin. Example:

Popcorn.player( "youtube", {
  _setup: function( options ) {},
  _teardown: function( options ) {}
});

So _setup is called when a new youtube player is created by doing Popcorn.youtube( “id”, “url” ), now _teardown will be called when destroy is called on the created player by doing p.destroy(). All track events and event listeners are cleaned before the _teardown, the idea it to just reverse what you did in _setup.

2. Popcorn.smart is a new function that tries to find the appropriate player for you. Example:

Popcorn.smart( "id", "url" );

Pretty simple, but if id is a video element, and url a valid HTML5 media, a regular old HTML5 video will be setup with popcorn. If the id is a div element, and the url a youtube, it will instead setup a youtube player.

3. Popcorn _canPlayType is a new player feature that is used by Popcorn.smart.

When you create a player, you can define a canPlayType function. Example:

Popcorn.player( "youtube", {
  _setup: function( options ) {},
  _canPlayType: function( nodeName, url ) {}
})

The _canPlayType accepts a node name that the player should be playable on. This is usually a html element, and a url, which is the url to the media to be loaded into the node. This function then return true if it can play, false otherwise. If this function does not exist on a player, undefined is returned instead.

To call this function, you do this Popcorn.youtube.canPlayType( “div”, “youtubeurl” );

4. Track event toString functions. Now, once you have a reference to a track event, you can call toString on it. This will return a string of the default data on any track event, usually the start, end and target. You can also add a toString method to the plugin when the plugin is being written, to better handle what to do with non default data. Example:

  Popcorn.plugin( "something" , {
    _setup: function( options ) {
      options.toString = function(){
        return options.start + ", " + options.end + ": " + options.foo;
    },
    start: function( event, options ){
    },
    end: function( event, options ){
    },
    _teardown: function( options ) {
    }
  });

5. We have moved players parsers and locale out of the core, and into modules. You include them much like you include a plugin, and from there on in, everything else is the same. Example:

<script src="popcorn.js"></script>
<script src="popcorn.player.js"></script>
<script src="popcorn.youtube.js"></script>
<script src="popcorn.subtitle.js"></script>

<script>
Popcorn( function() {

  Popcorn.youtube( "id", "url" ).subtitle({
    start: 10,
    end: 20,
    text: "hello world"
  });
});
</script>

6. We now have a shim to support ie8. This is only supported for the youtube player. Some plugins do work, like subtitle and footnote, but that was just a coincidence, and not something we are testing for, yet. We will improve our support for this over time. To include the shim, you have to include it before you include popcorn, like so:

<script src="popcorn.ie8.js"></script>
<script src="popcorn.js"></script>
<script src="popcorn.player.js"></script>
<script src="popcorn.youtube.js"></script>

7. All our plugins have had their manifests updated to supply options flags. This is useful for programmatically reading plugins dynamically. If someone uploads a plugin, and a program reads it in, it may want to know this information regarding options and mandatory data. Popcorn-maker is going to be using this soon. The idea will be when you click a track event to edit it, these will be a collapsible container with all the optional options. This makes a new plugin far less overwhelming, and easier to get at the important data quickly.

8. You can now supply events into a custom players option object. This is useful for listening to things that can happen during the creation process of the player. Example:

Popcorn.youtube( "id", "url", {
  events: {
    "error": function( e ) {}
  }
})

The error function will be called if for whatever reason, the player did not setup properly. This is also useful for knowing if popcorn.smart failed. Example:

Popcorn.smart( "invalidid", "nonsenseurl", {
  events: {
    "error": function( e ) {}
  }
})

So, the above id and url do not match any players, and is not a valid HTML5 media, the error function will be called. You can also put any other event in here, like canplaythrough, loadeddata, play, etc.


March 10, 2012 07:29 PM

February 13, 2012

Brett Gaylor

Popcorn and Documentary

In this post I explore how Popcorn follows the tradition of technology influencing the form of documentary. This post was created with Popcorn Maker.

February 13, 2012 09:53 PM

February 10, 2012

Scott Downe

Popcorn.js down the dark path of IE8 support

What I mean by support is that it will be possible, but it will be a “Your mileage may vary” situation. We’ll make it possible, and show how to do it, but we won’t officially support it or test against it. It will be a separate entity that the Popcorn code does not care about.

This is due to increasing demand for Popcorn.js to reach more audiences, and while I do believe this is a dark road, and one that detrimental to the fight to kill IE8, I also find it a unique challenge…

Personally, I believe the numbers in IE8′s market share to be inflated, and not really accurate. It is the enterprise environment that refuses to upgrade to the next version, and not the home user. I would say for each home computer, there is one or more users. And, for each enterprise install, there are three more sitting in the office next to it that are only used once a week. I know this, because we have a bunch of dead offices in our building, probably with Windows XP and IE8. So now this person can view Popcorn on Fox’s website one day a week, instead of doing work…

Anyway, we found a way to have our cake and eat it to.

First, we’re pulling players out of core, and into a module. We are doing this as players are mainly for youtube support, which is flash, which works on IE8. So by pulling it out, we can make the player module 100% support IE8. Then, we are going to create a shim, that when included before a clean popcorn.js file, will make it dirty. So now you include your shim, popcorn, and attach the player module, and you’re good to go. I also considered simply including the shim inside the player module, as without the player module, the shim is pretty pointless, but I am not sold on this yet; need more discussion.

Another note. This will not mean each and every one of our plugins will support IE8. Some might, some might not. But, plugins can be modified and created with IE8 in mind, and that is the idea of a plugin anyway.


February 10, 2012 01:23 AM

February 08, 2012

Brett Gaylor

Mozilla Popcorn aka The Meme Generating Machine

meme ( /ˈmiːm/) “an idea, behavior or style that spreads from person to person within a culture.”

"A meme is an idea that behaves like a virus—that moves through a population, taking hold in each person it infects." - Malcolm Gladwell”

We need Mozilla Popcorn to become a virus.

Hear us out here.

In our early iterations of Popcorn Maker, we’ve been tackling the problem of how to make it easy for non-programmers to create Popcorn experiences. It remains the central focus of the project, and we’ve fleshed out our User Stories to imagine the full experience that a user might have. Ben has described these stories in a recent blog post.

During our Popcorn Maker sprint, we put a lot of thought into imagining how our users’ creations will get shared and disseminated on the web. Ben has blogged about how Popcorn.js is a gateway drug to learning JavaScript (Also of the good sort. Stay with us). Similarly, we want Popcorn Maker to be a tool for injecting the Maker meme onto the web.

Fork my meme

To do this, we’ll need to do a few things. First, we need to make it easy for Popcorn makers to embed their creations on their own blogs, Tumblrs and websites. This means offering <iframe>embeds.

More interestingly, we want to give viewers the ability to fork others’ Popcorn productions. If you’ve watched Jonathan MacIntosh’s Buffy Vs Edward pop-up video remix, for example, wouldn’t you love the ability to easily clone his creation and add to it?

Each Popcorn creation needs a post-roll that offers viewers the ability to 1) Replay, 2) Share and embed, and 3) Fork this creation.

Popcorn Gallery

To build WordPress-like community scaffolding, we need the ability for every single creation to be made available in the Popcorn Gallery. When users choose to [Share] from Popcorn Maker they have an option to share to the Gallery that is checked by default. While the Gallery will favour our default templates, it will become a jumping off point for new creators to get started with Popcorn. It will solidify the notion that creating on the web is generative. The act of creation will start by building on someone else’s work.

We think Popcorn will be a good bug to catch. Like getting the chicken pox when you were a kid. Or maybe more like taking an interest in photography. We’re sure this is the right metaphor. Note: knock knock jokes, box stores and lolcats were all considered as alternatives while titling this post. Go meme or go home.

February 08, 2012 05:28 AM

January 14, 2012

Scott Downe

Popcorn.js xray goggles plugin.

On and off this week I have been fruitfully fumbling my way around the Hackasaurus xray goggles bookmarklet, in an attempt to use them in a popcorn plugin, while other things brew.

A tl;dr demo.

The idea is to execute the goggles between a time range, on an element defined by popcorn’s data. In the case of the above demo, I used the video element for testing purposes.

Right now, it is in a demoing state, demoing the use of the goggles at a time range. It is not really in a practical usable state, as you cannot close the goggles, nor can you control the video as they are up; quite frustrating.

I am going to allow popcorn’s data decide what to do with the timing. It can either leave the video playing, pause the video for a set number of seconds, or simply leave the video paused until the user closes and hits play again.

I also need to re add some animation that I avoided for sake of getting it working. Doing this will be good practice for me, because I’ve never been much of a front end guy, but I have the artistic abilities that I just need to dust off.

The hardest part of this was using HTML5′s postMessage. I have never used it before, and was stuck for longer than I would of liked, but long enough to build character. I was attempting to send a postMessage to an iframe before the iframe was done loading, thus the iframe’s event listener was not yet defined, thus my postMessage fell on deaf ears.


January 14, 2012 05:44 AM

January 12, 2012

Scott Downe

Firefox media fragement tickets

So, to follow through with my last post, I have done some more research and have three potential bugs.

First, It turns out media fragment is already done, which is exciting because now we can use it in Popcorn.

Second, three bugs were blocked by it, which can now be done: [498253 677121 677122].

Reading bug 498253, and it seems as of second quarter of last year, there were still on going spec changes.

677121 still interests me.

And, 677122 seems straight forward enough. A bug where it seems it is caching the media fragment start time, and a refresh is required to update the time. The steps to reproduce right from the bug:

1) Open http://www.bluishcoder.co.nz/mediafrag/AudioAPI.webm#t=50,100
2) Change the fragment portion to be “#t=30.100″ and press enter

Expected Results:

1) Start and End fragment of the video would be the new position of 30 and 100 respectively.

Actual Results:

1) The video does not change the start/end fragments. They stay at 50 and 100.

This one is still a good candidate.


January 12, 2012 07:47 PM

OSD700 and back to school

Today I had my first day back to school, and first day back in Dave’s open source class.

I’m currently working at CDOT with Dave still, which means I will still be working on Popcorn and Popcorn-Maker. But, for his class, I will need to do more work for him, but it cannot be CDOT work. It makes sense to me that I find Firefox bugs involving video spec, that way I can leverage these new tools into popcorn and popcorn-maker use.

So let’s look at bugzilla.

I made a general search under “media”. I wasn’t expecting anything to jump out, just wanted to see what I was dealing with. But, something did jump out at me.

Now, this ticket might not be the starting point for my work, but it is definitely something I am interested in.

The general interest is media fragments. Which the above bug relates to, but does not mean I am not doing the above bug. More research is required.

I also have interest in general media spec bugs, and I’ll look for more to have some concrete bugs tomorrow night.


January 12, 2012 06:00 PM

November 13, 2011

Scott Downe

Popcorn.js has a new brother… Popcorn-Hacker

Anyone familiar with Popcorn.js, Popcorn-Maker, and bookmarklets checkout my demo.

A bookmarklet is JavaScript that runs via the address bar. So, you can fill a link element in a webpage to instead run code. Click the link, runs the code. You can also drag said link into your bookmarks for later, and continual use.

What I’ve done is created a bookmarklet that lets you view and edit any popcorn.js page real time.

There are a few caveats. 1. It cannot be done on any version of Popcorn before 0.4, because I need access to Popcorn.instances, but that doesn’t matter because it also needs p.media, which was introduced in 0.5. I also ended up using Popcorn.getScript, which is another dependency, but getScript is introduced before 0.5, so if we have instances and media, we’ll have getScript. 2. We need plugins with complete manifests. Manifests are like meta-data for plugins. It tells me what to expect, thus what I can enter into the form fields. Without a manifest, it is read only Popcorn. 3. Demos made from exec alone will not work. Exec isn’t really editable without writing JavaScript anyway.

Now that the academic parts of the post are out of the way, I shall ramble about my thoughts on the project. This thing, to me, is a good example of a hack. I took parts of code from one thing, and remixed them together to do something different. I had little regard for how it looked, just that it worked. So, I guess it is more of a proof of concept. I really want to take it to the next level, though. Give it an export of some sort, and more closely tie it in to Popcorn-Maker’s build tools. Other-wise, this is essentially a fork of Popcorn-Maker. Over time, it will be too different, and become a separate entity that needs to be maintained separately. In order to avoid this, we would have to do some changes to Popcorn-Maker. It has some flaws, things I had to hack around. One of the good things was Popcorn-Makers event system. Made it easy to tie in my new functionality without having to worry about what was happening in other parts. Something interesting I noticed doing this, Popcron-Maker is close to 40k lines of code. This includes HTML, CSS and remote scripts like JQuery. Still though, that’s a lot of lines.

A few good pages to try it on are: Popcorn homepage, The demo from this fantastic tutorial on Popcorn itself, and the initial semantic video demo from way back in 0.1 of popcorn. It’ll work on others, but there is code in the wild it will not work on, yet. I’m going to deploy the “fix it for one demo at a time” technique I learned doing work in Processing.js. It’s great to have user generated content to test off of!

Now for an abrupt end to this post.


November 13, 2011 03:47 AM

November 06, 2011

Scott Downe

Popcorn.js 1.0, mozfest, Popcorn.js Universal Subtitles parser

This weekend has been a blast.

Yesterday Toronto time we released Popcorn.js 1.0 in London time.. or yesterday, I’m not sure, but anyway, it was super cool. We released with new docs made by David Seifried, a Mozilla based website for what Mozilla is doing with Popcorn, and a shiny new build tool, built by Chris De Cairos.

I’ve been spending the weekend in London at the Mozilla Festival. Been getting way more Popcorn attention than I expected or imagined! Been interesting hearing impressions and educating people on Popcorn.

I also spent some time at the festival hacking up a Popcorn.js Universal Subtitles parser, and demo. I have one work in here before. Back in 0.1 I did a internal parser for Universal Subtitles, and for a beta version of Popcorn-Maker, I implemented a parser for Universal Subtitles as well. This one though, should be the final, as it’s not longer living inside something, but lives on it’s own in modular space. I also decided to take some time to make it look nice with Brian Chirls’ pop plugin! Check it out!


November 06, 2011 01:50 AM

October 21, 2011

David Humphrey

On the joys of 1.0

In a little less than two weeks, we’re going to be releasing Popcorn.js version 1.0.  Our team is pretty excited, not least because we’ll be doing the release at the Mozilla Media Festival in London.  We’re also excited because getting to 1.0 is a really big deal, and we’re happy to have made it.

A year ago I was announcing the release of another joint Mozilla-Seneca project, Processing.js 1.0.  Shipping 1.0 means a heavy investment of time, and the effort and co-ordination of a dedicated team.  It’s not typically the kind of thing you see coming out of the classroom and from college students, since most academic projects remain academic, confined to the limits of a semester.  But it’s what we pride ourselves on at Seneca’s Centre for Development of Open Technology (CDOT), and it’s possible because of great partners like Mozilla.

I remember one of the first visits Mike Shaver made to my open source Mozilla class at Seneca many years ago.  A student asked him for advice on taking what they were doing in the classroom into the job market: “What do you look for when you’re hiring?”  Mike’s response really stuck with me; he said:

When I’m hiring I want to see what you’ve shipped.  Not what you’ve started and abandoned; show me what you’ve taken over the line.  Shipping is hard.  Do you know how to do it?

The way you ship 1.0 in an academic setting, and do it repeatedly, is to use a hybrid approach, and to form a network of support.  In order to make Popcorn.js, we’ve had 15 students working on it over the past year and a half–some as volunteers, some as class work, some as co-ops, some as research internships.  We’ve used grants from government (NSERC, OCE) and Mozilla to allow us to hire students and manage the project from CDOT.  We’ve had strong project leadership (i.e., daily involvement) from Mozilla’s Brett Gaylor, who brought everything he knew about film and film making to our development.  We’ve had amazing technical leadership from Rick Waldron of Bocoup in Boston, who helped us evolve our APIs and joined our team as a daily contributor and code reviewer.  We’ve had tremendous support from Mozilla on logistics, marketing, outreach, travel opportunities, and industry contacts.  And we’ve had strong volunteer community support, both from the Popcorn community, and also from Mozilla’s developer community.

When Mark Surman initially approached me about working on Popcorn.js at CDOT, he sold it as a technology that could bridge the worlds of film and media, design, story telling, and the web.  More than a year later, Wired writes that Popcorncould be the next big thing in internet video.”  It’s incredibly rewarding to see filmmakers, web developers, designers, and others able to come together and collaborate thanks to Popcorn.

To get ready for the Festival I decided to write a plugin for Popcorn that our users have been requesting a lot lately.  Ever since Mozilla’s pdf.js project launched, I knew I wanted to use it with Popcorn so that people could render pages of PDF documents on the web in time to media (imagine slides and videos of conference talks).  After writing the plugin I made a small demo that mixes a Justin Bieber video with an academic paper about his tweets, and the real-time #bieber tweets themselves.

The demo itself is about 25 lines of code, which speaks to the power of Popcorn now, and how little development knowledge you need to make it do cool things.

I’m looking forward to the Festival and to shipping Popcorn 1.0.  It’s been a lot of hard work by Seneca and all the other developers and project members.  I’m extremely proud of the work, and the team that did it.  I’m also thankful for the continued opportunities Mozilla and Seneca give me to participate in, lead, and help ship real world projects with students.  “Yes, we know how to ship, let us show you some examples.”

See you in London.

October 21, 2011 06:07 PM

Scott Downe

Thoughts on Popcorn.js hackathon in San Francisco

Last week I brought my JavaScript knowledge and experience to an event that put JavaScript developers and filmmakers in the same room for two days. The idea was to produce something. There were six JavaScript developers, and six films with 2-4 filmmakers per film. There were also designers patrolling the grounds, helping out. I cannot stress enough how important they were. Design really does help bring out the essence of a project, like spice in food. Flavour enhancers. Doesn’t matter how good, right, or fast a program is, if it looks bad, it will, by definition, still look bad. Not only was each group was able to produce something cool, but each produced something unique.

Wired has an article about the event, so check it out.

The actual event went down like this. Breakfast and mingling for about an hour, while everyone arrives. Then we all had a meet and greet. By 10:00 (if my memory serves me) we broke off into groups. So there I was with a group of people I never met, a film I have never seen, and a clock ticking. By lunch we had specced out what we were going to do. This is always an interesting process. You have a developer that knows nothing about the project or how things should be done, and you have designers and filmmakers that don’t know what can and cannot be done or how to measure how long something will take to develop. The problem for developers is “the easiest thing isn’t always the coolest” and for filmmakers and designers “the time it takes to develop something has nothing to do with how cool it is”. The developers don’t know what should be solved, and the fimmakers don’t know what can be solved. I think the solution is for the filmmaker and designer is to start with a big vision, then to have a back and forth discussion. It really is about communication and an open mind. Developers don’t be scared if filmmakers want it all. Filmmakers, don’t be scared to want it all. Developers don’t be scared to say no. Filmmakers don’t be scared to be told no :P. Eventually you will come to something that is “quick to cool”. Then it is lunch time.

After lunch you have to start developing. At this point huge changes to the spec can be frustrating, but still, no harm in asking. You never know, that thing you want changed might actually be easier! so keep up the communication. The rest of the day I struggled with getting Facebook’s API to work, but I did. It was an unstyled, nonsensical, boring piece of what I do best :D.

The next day we got to putting in the design and data. These are two things you need to figure out in the first half of the first day, and if possible have the data ready before the end of the first day. I kept the design pretty clean, empty, but clean, so it was quite easy to plug in some css and add some images. This is where you need the designer again. So, designer during spec phase, not after, and designer during pull it all together phase, not before. Then it’s just a matter of testing and debugging, and it’s still ok to request changes, just understand it might not happen.

Communication was key. Filmmakers teaching developers, and developers teaching filmmakers. I feel like designers have already crossed this bridge.


October 21, 2011 12:35 AM

August 24, 2011

Scott Downe

Popcorn.js player plugin revisit

I’m about to revisit Popcorn.js players. Right now we have 4 players, youtube, vimeo, soundcloud and baseplayer. All hacked up in their own way, and a maintenance nightmare. I intend to make basplayer something the other players derive from. This will reduce code surface, thus reducing bugs, keep things from getting fragmented and out of sync, and most importantly, test functionality once, reducing our testing phase and bugs in our test suites.

I’m thinking we break the current API regarding players with this patch, I think it’ll be worth it.

I think baseplayer will exist internally (it won’t be callable), and when you call:

Popcorn.player( "youtube" {});

it will create an instance of baseplayer, you can then pass in the additional features. So, if someone wants to use baseplayer without any additions, they can simply do this:

Popcorn.player( "baseplayer", {});

That will create a player emulating an HTML5 video players API that can then work in Popcorn, like this:

var p = Popcorn.baseplayer();

It will not actually display anything, but things like play, pause, currentTime will work and fire events.

To create a player, the idea is to add the needed functionality to make it work, and the Popcorn.player initialization code will look something like this:

Popcorn.player( "youtube", {
  setup: ( options ) {}, // creates the element and handles its ready state
  play: function( options ) {}, // simply start the setup element's play 
  pause: function( options ) {},
  currentTime: functipon( options ) {}, 
});

I am hoping in most cases, this is all we will need. So the internals of firing events can be handled in internal baseplayer code, because we can be smart enough to tell the overloaded play and currentTime to fire popcorn’s timeupdate.

This is all to setup a player plugin. From here, a user can then use it by doing this:

var p = Popcorn.youtube( "targetDiv", url );

The actual breakage of the API is minimal. The old way is below, the new above:

var p = Popcorn( Popcorn.youtube( "targetDiv", url ));

Positioning related code of the element will be setup as well. Not sure yet how that’ll look, but I do know the internal baseplayer won’t have a position, but it will have functions to manipulate the position, and probably have the position code look inside the element created by the setup function for the position related code.

I suspect there will be other issues while I work on this, but I’ll cross those bridges as I get to them, so I need to stay flexible to reduce the rage.


August 24, 2011 05:56 PM

August 19, 2011

Scott Downe

San Francisco sprint

This week I have been in San Francisco’s Bay Area Video Coalition doing a butter.js and popcorn-maker sprint, along with getting in direct contact with people using this product.

I am finding how important it is to get into direct contact with people using your product because these are usually the people that are not part of your workflow, and thus don’t know how to get into contact with you. When using software, it is easy to forget that it was created by someone, that there is a human (or many) behind the software. I know I have done this, and it is usually when I am angry with software, I may even go so far as to curse the software by name, as if the software itself was a living entity that created itself. In reality, there is someone responsible for the software, with the ability to change it. Seeing this person makes it easy to put a face to the shadowy living entity that is software.

The rest of my time has been spent adding in new features and requests to Popcorn-Maker, which the current unreleased version can be found here.

The main thing I’ve done is incorporating timeline ticks under the video playhead, representing what time the video currently rests at.
Also, allowing the timeline area to zoom in or out by either using shift+mousewheel, or a scrollbar. The scrollbar UI was implemented by Dave Seifried.
Finally, using arrow keys to move the events and update the video’s playhead, which was initially implemented by Dave Seifried.

Some more depth.

When using arrow keys to move events, or the playhead, you can hold shift to move them on a larger increment.
You can also hold ctrl when moving events to anchor one side, to stretch the event, instead of moving it. You can then hold shift and ctrl, to expand on a larger increment, then do this while holding ctrl, shift and left and right at the same time, and you can stretch both sides of the event, while keeping the event centered.

I initially had the zoom at 6 layers, but users were requesting a larger number of layers, so I changed it to 12.

Again, you can try it out here.


August 19, 2011 06:10 PM

July 14, 2011

Scott Downe

butter API – timeline module

We’re elbow deep in making butter app into an API, that apps can be written around.

I’m just going to start with a high level explanation of how the pieces fit together.

Layer 1: The app. This is some html, css and JavaScript that display the webpage. This is what the user will interact with to author some Popcorn.

Layer 2: The Modules. A module is a piece of butter that can be used to create a butter app. Initial modules will be a previewer, a track editor, and a timeline.

Layer 3: The API. This is the communication layer between modules and the developer. The developer uses the API to register modules, and the modules use the API to communicate to other modules.

I’ve been working on the timeline module. I have a demo up. This module is where the track event visualisation happens. Basically, you can add track events to the video, displayed in a timeline. You can then move a track around once added, to change its position in the timeline. It also has a timeline that can interact with the video’s time, seeking and scrolling around.

I explored a few ways to display the timeline. The two main things my solution needed to solve are, it had to be fast, and be able to represent a really long video’s time, without bogging down the browser. A canvas has limits on how large it can be.

So, I figured I would display the timeline in HTML, that way it wouldn’t have a canvas that is too large, and should be fast considering it’s just HTML. This had the problem of not scaling very well, as some pixels would blur, and other would be crisp, depending on the allowed space. I tried three HTML based methods. Displaying images in css using background-image, appending image elements, and even tried hacking empty divs with borders to be used as the timeline’s lines. All of these had scaling issues.

This left me with canvas. I could either render one large canvas once, or I could draw a smaller canvas, and just update the position that is currently being viewed, and as the user scrolls around, it would update the offset. The first one was no good, as a large canvas might become and issue, and the second worried me because it was a performance hit I wanted to reduce if possible. I also considered drawing an offscreen canvas, and displaying its contents in an image using a dataURI of the offscreen canvas.

I ended up with a canvas, being rendered once. I solved the problem of it being too large by allowing it to be scaled down, while retaining the video’s whole time.

I first draw a bar one pixel in width for each 1/4 of a second, with the half second and full second being 3 or six pixels longer, respectively. This wasn’t too hard once I calculate how many pixels a second is based on the video’s duration in seconds and containers with in pixels.

The next step was to draw timestamps, in SMPTE format, along the timebar. Once the timebar was scaled down, the timestamps wpuld start to overlap, so I had to account for this and make sure I only draw a timestamp for the first whole second that falls within the allotter space in pixels.

It all turned out very well, and all that is left is making a github commit out of it, some notes in the ticket, blog and to merge it in with the other two modules.


July 14, 2011 05:05 PM

June 30, 2011

Scott Downe

To a grade 6er, when is the web malleable?

Yesterday, while working with a classroom of grade 6ers on their Butter projects, as briefly mentioned in the end of a previous post. I had one of the kids ask me a pretty awesome question “which error console is the best”? He then showed me the FireFox error console, and web console. I responded to his question with “FireBug”, which he promptly responded to with “what’s that”? I said “It’s an add-on”.

Less than a minute later…

“OK, I have FireBug, now what”? I show him the very basics, like how to open it, how to move it to a new window, back to inside the browser, and where the shiny new console lives.

What was really cool, is when I showed him that you can change elements on the page, by selecting it, and changing it in the HTML tab. I showed him this by going to Google, and changing the text in the search bar to a bunch of nonsense. His eyes lit up, and he was on the path of destruction.

He hit a moment where the web was no longer a tool to get information or a toy, but a canvas that he can change. Luckily, he quickly understood that he was only changing his version of google, and not everyone else’s version of google. Few minutes later he was removing chunks of a webpage, because he could. He didn’t really understand his actions yet, until after he made the changes, and experienced the results. It’s a science.


June 30, 2011 03:06 PM

June 23, 2011

Scott Downe

Popcorn.js event effects framework landing in 0.7

We’re just finishing up Popcorn.js 0.7.

One of the major things landing in this patch, is effects.

Effects are not just limited to effects, so we also decided to use the word compose. This allows our largest use case being effects to be obvious, but we also make it known that on a more abstract level, something exists. This way, the API itself is speaking the language of the user.

You create an effect plugin almost exactly like you create a regular plugin.

You can create a plugin like so:

Popcorn.plugin( "pluginname", {
  start: function( event, options ) {},
  end: function( event, options ){}
});

So create a plugin name, and define some functionality for start and end. Note, this is a very minimal example, as I was not intending to get too deep into creating a plugin.

Anyway, you can create an instance of your plugin like so:

popcorn.pluginname({
  start: 1,
  end: 5
});

This means, between 1-5 seconds in the video, your defined start function will be called, and when it leaves the 1-5 time range, it’ll call your defined end function.

Effects work similar in creation, but to create an instance from an effect, you apply it to another plugin instance “composing functionality”.

Example:

Popcorn.compose( "effectname", {
  start: function( event, options ) {},
  end: function( event, options ){}
});

popcorn.pluginname({
  start: 1,
  end: 5,
  compose: "effectname1 effectname2" // this way, we can apply as many effects as we wish
});

So now, that single plugin instance has the functionality of the start and end functions of the effect. Something to note is currently, the original plugin’s events fire first, then any composed events will be fired in the order they appear in the compose list.

Here is an example of it all in action.

Here, I create an effect for applying a css class name to the event’s target on start, and after 250 milliseconds later, it’ll go away. Using this, the potential for using CSS transitions is huge.

A more simplified demo.

Here, I do this:

    document.addEventListener( "DOMContentLoaded", function () {

      var p = Popcorn( "#video" )
      .play() 
      .footnote({
        start: 5, // seconds
        end: 15, // seconds
        target: "footnotediv",
        text: "This video made exclusively for drumbeat.org",
        effect: "applyclass",
        applyclass: "test1, test2"
      })
      .footnote({
        start: 20, // seconds
        end: 45, // seconds
        target: "footnotediv",
        text: "Visit webmademovies.org for more details",
        effect: "applyclass",
        applyclass: "applyoverlay: .overlay, hover: parent"
      });
    }, false);

What is happening here is the first footnote is simply having the two css classes applied to their target’s on start, and removed on end. Pretty simple.

The second footnote is not so simple. Here I am using CSS selectors split by semi-colon “classname: selector, classname: selector”. classname is the name of the class to apply, and selector is the target to apply it to RELATIVE to the target.

I want to stress that compose and effect are the same things, and that applyclass is only ONE example of how to use this. This is not limited to effects.

This is all pretty new stuff, and what I’ve just written is as far as I know the only documentation for effects. So anyone attempting to build an effect right now, is a trail blazer!


June 23, 2011 09:14 PM

June 15, 2011

Scott Downe

Butterapp new release candidate

Butter app is about to release 0.3.

The above link is an older version. The release candidate can be found here.

The best of the new features are:

- Youtube support. You can now enter a youtube url as your video.
- draggable track creation. You drag a command from the bottom right menu onto the track to create a track.
- html and css bases track dragging. The old version used a canvas, and now we’re using jquery draggable and droppable.
- one preview of a track at a time.
- finally, you can now import univarsal subtitles as explained in my last post.

We also have a class of grade 6ers using and testing this software. I’m pretty excited about that, as I heard children are the best at finding bugs.


June 15, 2011 02:44 PM

June 08, 2011

Scott Downe

Popcorn.js + Butterapp + Universal Subtitles

Today, I finished up integrating Popcorn.js and butter with Universal Subtitles.

Demo.

All you do is, enter a URL that has already been Universal Sub’ed on the opening page, under “Create a New Project”. Here is a URL you can test with:

http://blip.tv/file/get/Miropcf-AboutUniversalSubtitles847.ogv

Next step is, after the video loads, click “import project” on the top menu, and click “Universal Subtitles”.

This will load in all the subtitle data associated with that URL, via Universal Subtitles.

The subtitle data is not yet being displayed in the track, but I assure you, when you play the video, you should see subtitles over the video.

What is so cool about this is, you don’t have to sub the video yourself if someone already has. Just click “import project” -> “Universal Subtitles”. Also, once youtube lands inside butter, that should just click together!!!

Next step is, obviously, to display the subtitles in the timeline, but that will be much easier after my trackline ticket lands.


June 08, 2011 07:39 PM

The less code that touches my code, the better

When developing software, always look for potential APIs. An API is an Application Programming Interface (API). That is, the interface software uses to communicate with other software. On a basic level, you’re breaking a larger problem into smaller ones.

APIs have some pretty HUGE advantages.

- Easier to test (automated). Example: When you are adding after manufacture features, you have to rely on tests to give you confidence that you are not breaking something. Right, pretty common. But, when tests are all testing separate parts, it is much easier to find where and what went wrong. Also, take for example a large web application like butterapp (I will use this as an example again). How does one test an application built around visual, UI, and interactivity with automated tests? Simulate a bunch of clicks, and compare the pixels? Or, start at the other end, the first thing that happens when that click happens, and work your way up to the actual click. Think a large engine, you can test the engine. Or, you break it into smaller engines, that sends commands to the larger engine. The smaller piece knows everything it is capable of doing, so you test everything it can do. You’re not testing end results, but functionality of the pieces. You’ll also come out with much stronger tests as a whole. That being said, no harm in visual tests of the end result as well. You can never have too many tests.

- Easier to maintain. Once you know where something is broken because a failed test led you to the broken area, you can lift it out, re wire its internals, and put it back in and none of the other parts know anything has changed, because the interface is the same. Adding new features become much easier, because an API is designed from an abstract point of view. You can easily change the way the parts interfaces with the other parts, to get different results. It is easier to read, like a book, to a point. A badly designed API is just spaghetti code. A good way to avoid this is to keep in mind that the less code that touches your code, the better. Modular, dynamic, and living in its own scope. We don’t share our Tupperware with our neighbors here.

- Distributable. Never know, someone else may have a need for the pieces in your engine, to use in something completely different. This also opens up my last point.

- Collaborative. Once people start using it for other things, to solve other problems, your community will grow exponentially.

What made me ramble about all this?

I’ve been working on something called butterapp, which is the front end registration tool for popcorn.js. Inside Butter, is a visual track, containing events attached to a timeline. So me and Bobby Richter specced out trackLiner.js. Here is an older demo I did to show while it was in mid development. And, here it is, updated, and plugged into butter.


June 08, 2011 03:10 AM

June 01, 2011

David Humphrey

Nature vs. Nurture

I spend a lot of time thinking about, and actively pursing, ways to better enable community around software projects.  My motivation is to work with people vs on certain technology.  I’m blessed to be involved with quite a few thriving communities, and I wanted to say something about community based on my experience in them.

Joel Spolsky was writing about the need for Stack Exchange to hire more community managers in the form of super-evangelists:

Our goal as a company is to incubate each of these 51 communities—to get them to critical mass. Critical mass is that magic moment when the community has enough activity that it grows by itself.

Building communities on the Internet is a new kind of profession. There are an awful lot of technology companies, founded by programmers, who think they are building communities on the Internet, but they’re really just building software and wondering why the community doesn’t magically show up.

People have to take an active role in building and fostering community and participation.  Knowing how to do it effectively is hard, because as Joel points out, an algorithmic approach won’t work.  Communities are made-up of people, and because people are so different from one another, what works in one community won’t work in another.

Much of my work for the past year has been focused on extending the Mozilla community out into the larger web community.  I firmly believe that the way to grow Mozilla is to leverage the incredible energy and talent of the web development community, to show them how their work can and does influence the web itself, and how it can affect projects as large and important as Firefox.  To that end, we’ve focused on projects like the Audio Data API, Processing.js, Popcorn.js, CubicVR.js and the demo scene.

Yesterday we released version 1.2 of Processing.js, a large release on a project that keeps getting bigger.  I love working in that community, not only because it’s such an exciting technology, but also because of the people.  The same goes for the Popcorn.js and Web Made Movies community.  We had a planning call yesterday for Popcorn 0.7, and it included participants from Toronto, Boston, Milan, Vancouver, and New York.  That community and technology is really starting to gel, too, and it’s exciting.

In addition to patches, I put a lot of effort into keeping the people side of these projects healthy.  Keeping growing communities going in the right direction, helping them expand, making them interesting and creative places where people want to spend their time and use their talents–these are not things that just happen.  You have to invest in them.

I think an important aspect of this, one that is so obvious it gets overlooked, is the need for encouragement.  The people I work with on these projects are amazing, and some of the most talented software people I’ve met.  People need to hear that, need to know that their contribution is appreciated, that they are appreciated.

For the most part, there aren’t automatic ways to do this.  Yesterday github announced the new Block User button:

GitHub has always been about collaboration: we want to make it easy for you to work with other people to build great software. Whether that’s a co-worker sitting next to you or a stranger across the globe, it doesn’t matter. Collaboration should be easy and fun.

Unfortunately, there will always be people whose idea of fun is bullying others. That’s why today we are rolling out a feature designed to help you control the people you interact with on GitHub.

Wouldn’t it be great if there was a “+1 this User” button?  It’s still possible to do this, but you have to do it manually.  Luckily, it’s easy to DM someone on Twitter, send an email, drop some love in irc, write a blog post about someone else’s work, etc.  “Building communities on the Internet” is more than just a profession, it’s the responsibility of every member of those communities.

As someone who tries to be intentional about this, let me encourage you to go encourage someone in your immediate community today.  We know those people are capable of good work, so it’s easy to not say anything when they deliver it again and again.  I promise you that no one will think it odd when you send them a note of gratitude, encouragement, and love.

June 01, 2011 04:00 PM

May 30, 2011

Scott Downe

Make an opinion, to throw it away

I’ve faced a number of challenges when it comes to software development since I started school at Seneca 3 years ago, and started work at CDOT one year ago. Syntax, APIs, style, communication, research, to name a few, all had one thing in common. They all had clear and final answers, I just had to find them. I am sure one could make the argument that they only had clear answers AFTER they were solved, but, while solving them, I still knew there was an ultimate, logical, correct destination. Once found, the answer worked for almost every case after that.

The challenge I find myself facing now, is that of my own opinion. No one opinion can be used for every problem, for every person.

I spend a lot of time, going through an iteration, or a solution to a problem, that I may not agree with, but I do anyway simply because I lack an opinion. I’m shown something, I don’t know what I think of it, so I do it, and figure I can make a better opinion of it after it’s done, which I never get a chance to, Because in order to find my opinion I will have to solve every problem once, for every solution, then, and only then, can I say what I truly think. A good example is I not-so-long-ago helped port browser mouse events in processing.js, to safari touch events on the iP(od/ad/hone). I lost the argument (I don’t think anyone was on my side) do I still think I was right? No. Do I think the solution that was used was right? No. There probably isn’t a right solution, just opinions.

The proposed solution is:

Make an opinion, then throw it away, and let logic determine what’s best. Logic must have, or is real proof, so I would need to “solve every problem once, for every solution”, only difference is, I’m able to throw it away for the next problem, because I know it won’t be viable for every problem.

My solution doesn’t really solve anything regarding opinions, I just changed my opinion of the problem.

There is an opinion for every problem, for every person. That is a lot of opinions… let’s move on.


May 30, 2011 01:13 PM

May 24, 2011

Scott Downe

Butter’s track editing user interface upgrade

So, moving forward with Popcorn, I’m directing my attention towards Popcorn’s authoring tool, Butterapp.

The two major things I’m working on:

First: The current iteration of Butter uses a canvas to edit, add, remove and display track events. I’m going to change that to be html elements using jquery draggable. I want it so if a track event is dragged outside of its track, a new track will be created. Also, if it is dragged between two tracks, a new track will be created, and inserted between the two tracks.

Second: The current iteration is using the plugin type as the term to sort the tracks on. I’m going to change it to be sorted by track location. A quick example of the idea, made by Bobby Richter.

I’m going to be attacking both of these at the same time. mainly, making the functionality of the second, but making it with HTML elements.

We’ll have a largely object oriented model, with three main components. A Tracks object, multiple Track objects, and multiple TrackEvent objects.

A Tracks object: (for multiple tracks). Only one of these will ever need to exist, so probably just an object literal with the needed functions/attributes. This would contain tools for adding/removing tracks. I also think the order these are displayed is important, which makes sense to be managed by the Tracks object, and not individual tracks. The importance of order is why I see things like splice, push and pop to be important, because I feel adding and removing tracks should be accessible via order or object, so long as you know one or the other, you can manage it. Would contain a addTrack, removeTrack, getTrack and length property.

A Track object: This would be for adding/removing TrackEvent objects. Order of TrackEvents probably wouldn’t matter to the Track object, as they would be ordered based on their times/size. Would contain a addTrackEvent, removeTrackEvent, name, getTrackEvent, and length.

A TrackEvent object: that is created with a plugin type “subtitle”, “footnote” etc. Which would also hold all the values of the track event. There would be added/removed to/from the Track object. This would probably just be a reference to the popcorn track Event object.

Next step is to find all references to the current track system in butter, and remove it. Then start adding pieces one by one, starting with making the plugins draggable. Second, Implement by objects and their APIs. Third, use my object APIs to link it all together. That is all for now… more later?


May 24, 2011 08:44 PM

May 11, 2011

Scott Downe

popcorn.js performance: It’s a science

Recently, I helped on a popcorn.js demo integration with soundcloud.

It was a lot of fun to work on, and it brought light down on a couple bugs. One stuck out, being a performance issue. In the demo we had the need to add thousands (5938) of tracks, which all needed to be processed, and it was slow.

Today and yesterday I worked with Rick Waldron to try to figure out a new solution to adding tracks.

Popcorn.js has two arrays of tracks, sorted by time to display and hide the track data. One array is for time to display, the other is time to hide. If you can trust the data is in order by time, hold a reference to the last track that was checked, and have some really good development tests, you can find the data of intrest much much faster, instead of loop through the whole array each iteration ;)

The problem with this though, is data can be added to this array dynamically, at any time, which means data must be sorted each time data is added. As you could imagine, calling any form of a sort function 5938 times, it will begin to get slow. You can hold off on calling sort, but it’s almost impossible to know when to sort again, because of how dynamic things can be. We brainstormed ideas, including a sorted or not sorted flag, which would be checked against once inside the update code. This would mean a new variable is added, and that the update code would have one more task, in what is already time sensitive, optimized code. Not an ideal solution, but a viable idea.

Another idea was sort by default, with the option of the user to override the sort, and call an explicit sort function when they are finished adding data. This is also a viable idea, but the disadvantage is, it is not very user friendly. These things “should just work”.

Third options was, which was the one we ended up going with, to never sort, and make sure tracks are placed in the right location right off the bat. Thingk of it this way, the best way to clean your room, is to never make a mess of it in the first place, and that’s what we did.

What was really cool about all this, was the collaboration and teamwork that pulled this off.

Rick setup a jsperf url, that we could both modify and compare against together. We had a case using the old popcorn event code, and a case with code we could modify, setting a benchmark to beat. Then we both went to work trying ideas, comparing, and mergin ideas into one, until we came up with a jsperf we liked.

What these numbers mean: “we have a 23,891% performance improvement” :D which is over 9000.

About half way down the jsperf link, you’ll see a “run tests” button, which will then show you operations per second. Higher is better. Below that, you’ll see the cumulative test of everyone that has run these tests, their numbers, their browser, and how many times it was run. Higher is better, again.


May 11, 2011 07:31 PM

May 06, 2011

Scott Downe

Popcorn.js jsonp request for milestones

Today for the CDOT dashboard project, I finished up a php server to serve jsonp data for popcorn.sj’s milestone data inside lighthouse.

You can access it via JavaScript like so:

$.ajax({
    url: "http://scotland.proximity.on.ca/sdowne/sqlite/jsonp.php?service=lighthouse&callback=foo",
    dataType: "JSONP",
    type: "GET",
    success: function( data ) {
        console.log( data );
    }
});

With the data argument in the success function being your shiny new json data.

It is being tested and reviewed by Dave currently, who is the one that needed the jsonp data in the first place.

The obvious reason for this existing is to allow thing like json to be requested via server side scripting (JavaScript), and to my knowledge this does not exist yet for lighthouse. Another reason for this existing, is it reduces the number of calls to lighthouse. I was initially going to setup an internal timer, which would update the data served every half hour. I ended up thinking up a much much better solution. Instead, I would hold timestamps on the last time the data was updated, then, when someone requests that data, check if the timestamp is more than 30 minutes ago, if so, update via lighthouse json request, and set timestamp as the current time. This saves unneeded calls, so I ONLY update the server side data if a request is made, instead of a blind cron job. Also is much easier, and much less code.

The internals ended up storing the data on a very very simple sqlite layout. I am actually enjoying using sqlite via php very much. I just used php’s exec function to directly call and manipulate my sqlite database. No passwords, no connections, no remote.

For now I only serve popcorn’s milestone info, because there was no other way to get it, and I figured it would be a useful hello world test, but in the future we will add more to it, as needed.

I’ll be adding the source to github later today, but there is some setup that is required to get it working, that is why I setup a link on scotland.proximity.on.ca. Also, I figure it can be used by someone interested in this data for other projects.


May 06, 2011 04:30 PM

May 03, 2011

David Humphrey

Popcorn.js and Khan Academy

Last week Mozilla’s Bobby Richter and I spent some time building a demo of what’s possible with Popcorn.js’s new YouTube support.  Steven Weerdenburg has been working with me at Seneca’s Centre for Development of Open Technology over the winter, and among other projects, he wrote a bunch of player plugins.  In Popcorn.js, a player plugin is a wrapper around an object, script, etc. that changes its interface into that of an HTML5 media element.  It allows us to shim video-like objects, be they animations, custom timelines, etc. into Popcorn.js, and have them work just like the expected HTML5 video or audio element.  Steven has built player plugins for Vimeo, SoundCloud, and YouTube.

When we shipped YouTube support we knew that we needed a demo.  Obviously there is so much we could use now that YouTube works with Popcorn.js, but my mind went instantly to the Khan Academy videos.  As a professor and homeschooler I love Khan Academy and what they are trying to do (see Sal’s TED talk if you’re not familiar with the project).  There are many educators who have reached out to us with ideas and questions about using Popcorn.js in their teaching.  How could Popcorn.js augment and support a teaching tool like Khan Academy’s videos?

The answer is to use the media as a unifying narrative for a rich web experience, one that mixes visual and textual information, and turns the experience of learning from being purely linear into something that is both linear and interactive.  We often hear about how the web enables remixes, and we wanted to demonstrate an educational remix.  In our case, we wanted to mix the following:

We intentionally tried to write as little code as possible, while still giving a rich experience.  We put the planetarium demo in an iframe behind the YouTube video, and used jQuery UI to build a slide-out information panel (mouse over the left-hand side of the window).  We use Popcorn.js to drive the experience, automatically navigating around the planetarium, dynamically adding articles to the info panel, and moving and resizing the YouTube video as necessary.  The experience is equal parts directed learning (“here’s Mercury…”) and self discovery (users can manually navigate through the planets and go to the articles for more info).

There’s a lot more a domain expert could do with this method.  Neither of us are experts on the solar system, and our intent was to show how easy it was to take existing web and learning resources and mix them in order to create a more interesting learning experience with Popcorn.js.  We built this in a day, and we know that other educators could take it much further with more time.

If you’re interested in learning more, come say hi on Mozilla’s Web Made Movies mailing list or on irc.mozilla.org/popcorn.

May 03, 2011 02:14 PM

April 19, 2011

David Humphrey

“Do it on the web”

One night last week David Ascher sent me a challenge on twitter:

@humphd challenge: do http://t.co/XImyz11 on the web ;-)

Funnily enough, I’d seen this demo earlier in the day.  It’s a fantastically simple, yet captivating video by Joerg Piringer.  He describes it like this:

i made video called “unicode”. it shows all displayable characters in the unicode range 0 – 65536 (49571 characters). one character per frame.  i was starting with adobe after effects but in the end used a custom written program that just filed out png-images. i then had ffmpeg assemble them together to the 33 minutes of video.

Luckily for me, the list of printable unicode characters was already posted with the demo.  Challenge accepted!  Here’s the final product:

http://vocamus.net/unicode/

I wanted to write the simplest approximation of this using nothing but html5.  I also added some music for good measure.  Part of why I decided to do this was to try and draw attention to the fact that while the web gets criticized for not having all the sexy Adobe-style content creation tools people say we need, in reality it often does away with the need for them.  The web, the browser–these are the new content creation tools.

Thanks to Joerg for the great idea, and David for the challenge.

April 19, 2011 08:49 PM

April 02, 2011

Brett Gaylor

Video: What is Mozilla?

I’ve been working on and off for many months on a video to explain Mozilla to the uninitiated. The home for the video is now on the Get Involved page of the mozilla.org site, and I’m excited that it will be part of a process for getting people excited about pitching in at Mozilla. You can watch it below: mega hats off to Rainer Cvillink, Mozilla’s in-house video wizard, for all the great camera work, and to Jenn Strom for editing and motion grapnics.

April 02, 2011 10:29 PM

Scott Downe

Buttercamp 0.1

Last weekend I attended Buttercamp, and created a demo.

To sum up what Buttercamp was: A 12 hour HTML5 video hackfest, using Popcorn.js and Butter for developers and filmmakers.

After some coffee, and a buffer for everyone to arrive, we had a meet and greet to see what everyone had in mind for the day. Using this info, we split into groups and started working.

There was a handful of demos created using Popcorn, all of which were great, and are covered in more detail elsewhere, so I am going to be bias and talk about the one me and my group worked on ;)

I was in a pretty large group, about 7 or 8 of us. Henrik Moltke being one of them. He had an idea to use GML and Popcorn together, and luckily, most of the pieces we needed already existed, it was just a matter of hacking them together into something that passes as a webpage ;) . One of the members in my group, Zachary Lieberman also had an intrest in GML.

Zachary Lieberman had experience with git and github, so we ended up doing a lot of our work here, which proved to be good and bad.

The good was that me and Zachary were able to collaborate quickly and easily, and also gave us a place to continue working on the demo after Buttercamp if we pleased. The bad part was that it was a sort of wall to people that didn’t know git. We had a large enough group though, that we were able to produce two demos, so it worked out.

The biggest thing I learned from this is that git might be too much of a wall working with filmmakers. The question is, should developers expect filmmakers to learn git, or should developers find something more filmmaker friendly. This is not a problem I know how to solve, and if I did have the solution, I doubt it would be an easy one to implement, but is something to think about. Also, any feedback from a filmmaker that either feels git is a wall, or has tried to learn git, would be interesting to hear.

Also, I was lucky to have a pretty good developer to filmmaker ratio, but I heard other groups felt one developer could become a bottleneck if a tricky bug was found. The ideal ratio would probably be 1:1 ratio, with groups being 4-6 people. This would be ideal, but as it was still successful and a blast the way it was, and I would do it again :)

Oh, major edit here.

I forgot something else I took from it all. I did something similar to this a while ago, after 0.1 of popcorn was released, and before Butter was even an idea, and it was amazing to see the maturity of Popcorn between now and then. Last time, I spent most of my time debugging Popcorn instead of making demos, and this time Butter was the one on that end. I have no doubt Butter got the most out of this experience. More than any of the filmmakers or developers, but that in turn, benefits the filmmakers and developers. Win win. I look forward to seeing the next step of Butter in the next Buttercamp.


April 02, 2011 05:16 PM

March 30, 2011

Brett Gaylor

Videoblog: Buttercamp

Everyone is still recovering a bit from Buttercamp. Anna Sobiepanek beat us all to it with a great recap of the event, but let me say how thankful I am for the extraordinary effort the participants put into the day. We learned a lot about how to do events like these, and definitely left wanting to do more.

As Anna mentioned in her blog, our biggest lessons were a) the need for designers to be present, and b) that we should probably do these over two days. Scott Downe benefited from teaming with Zach Liberman, and mentioned that 2 developers are crucial for a team, so that if one struggles with a blocker, the other can forge ahead. Well noted.

Demos:

Note: these demos will all require a modern browser, may not work for you at all, and are the product of one day’s work.
GML + Popcorn:

http://scotland.proximity.on.ca/sdowne/popcornGML/gml+popcorn/
Integrating the Graffiti Markup Language (GML) XML standard with timed popcorn events.

Robots:
An experiment with “node” style decision forking using popcorn
http://robothaus.org/robots/

#18daysinEgypt:
To have popcorn events appear in specific locations on a full screen video, Brian Chirls wrote a popup video plugin. Video events that appear cause the main video to lower volume and loop.
http://code.chirls.com/buttercamp/

Digital Diaspora Road Show:
Accompanying the feature documentary Through A Lens Darkly, filmmaker Thomas Allen Harris creates “Roadshows” were participants explore how photography connects them to their heritage. This demo made extensive use of the Butter application, and to facilitate multiple editors, Boaz Sender added import/export functionality to the edge version of Butter.
http://dev.webmademovies.org/lensdarkly/
note: this demo does not work in Firefox due to mp4 only source.

Kapor Center Mural Unveiling
Trevor Parham used Butter initially for data entry, but then decided to focus on CSS and layout to create this compelling use case for popcorn: an annotated version of a recent event at the Kapor Center.
http://kaporcenter.org/files/video/

TubeyLoops
A modified version of PatternSketch, which controls videos via popcorn to create a video remix machine.
http://www.tubeyloops.com/tubeyloops/
note: use keyboard keys Q W E R to control video.

In addition to these demos, other plugins, connections, and starts of new projects were forged. Thanks to ITP, the Center for Development of Open Technology, Bocoup and Eyebeam for helping facilitate this meeting of great minds.

Buttercamp was:

Michael McCarthy Henrik Moltke Yasmin Elayat Kirby Ferguson Peter Asaro Peter Kaufman Ann Bennet Thomas Allen Harris Woo Cho Anna Sobiepanek Baowen Huang Ben Moskowitz Boaz Sender Bobby Richter Brett Gaylor Brian Chirls Elad Schanin Fred Truman Greg Dorsainville Ingrid Kopp Jeff Howard Jeremy Diamond Jonathan McIntosh Matt Thompson Rick Waldron Scott Downe Todd Carter Trevor Parham Yujin Yi Zachary Leiberman

March 30, 2011 10:49 PM

March 28, 2011

David Humphrey

No Comply

On Friday Mozilla released our latest Audio API/WebGL demo on Mozilla Hacks, called No ComplyWe’ve already written a post about it, so I won’t repeat all that now.  If you haven’t seen it yet, please check it out.  We’re really proud of it.

What I wanted to mention today are some of the cool things that came out of building this demo.  We built this demo for the open web, using the open web and open tools, and that meant creating some neat new things you might want to play with:

My own work was really focused on integrating all the million bits of JS we use, and having them run fast.  Here’s what our demo’s framerate graph looked like when we first integrated all the pieces (green is good, red is…not good):

You can clearly see how much work we had to do here, both in terms of getting framerates up, killing GC/CC pauses (the vertical spikes where the main thread gets locked), and moving heavy resource loading off the main thread.  Here’s what it looks like now:

The story of how we changed one into the other is long, and it’s equal parts optimizing our code and working with Mozilla to get Firefox 4 to run faster–many bugs were killed in the making of this demo. By the way, I’m using Patrick Walton’s amazing Framerate Counter extension to get this data.

If you’re thinking about building graphically intense web apps or games, using complex audio analysis, creating web pages that are larger and more complex than anything you’ve seen up to now, we’d encourage you to do it.  The web is ready to do a lot more than people think, and Firefox 4 is leading the way on much of it.

March 28, 2011 03:50 PM

March 24, 2011

Brett Gaylor

Welcome Bobby Richter

I’m very happy to announce that Bobby Richter is joining the Web Made Movies team as Creative Technical Lead. Already a Mozilla contributor, having worked with the #audio team on a variety of demos and libraries, Bobby is joining us to, as he writes on his about page, “make Web Made Movie’s ideas tangible as quick as possible”. Bobby’s job will be diverse: contributing to the popcorn.js community, advancing the Butter platform, and helping to create magnetic productions as outlined in our 2011 goals.

Bobby brings a range of experience – from helping to create the NFB’s award winning Out My Window, to a stint at Electronic Arts, Bobby has both skill and experience that all of us on the media team are thrilled to have among our nascent group.

During our recruiting conversations, I was particularly excited about his experience with WebGL, and these discussions manifested themselves in a demo. Please check it out here – its an early peek at integration WebGL and Popcorn, via an homage to Mozilla’s JavaScript jedi master Brendan Eich.

Say hello to Bobby over at his blog, and you can find him on irc.mozilla.org in the #popcorn channel as “richter”, or sometimes under has super hero name, secretrobotron. Welcome, Bobby!

March 24, 2011 07:14 PM

March 22, 2011

David Humphrey

Firefox 4, bring the noise!

Today is the launch of Firefox 4.  If you’re using an older version of Firefox, you can simply ask it to Check for Updates, and you’ll get a shiny new browser.  There’s a ton of great stuff in this release, but three things in particular excite me.

The first is the incredible work that’s been done to improve performance: startup, graphics, JavaScript, etc.  I have personally filed dozens of bugs on performance issues in Firefox, and watched them get fixed one by one.  This new browser doesn’t just feel faster, it is faster.

The second thing is WebGL and 3D on the web.  We’ve been working on WebGL (previously Canvas3D) since 2007, and I’m thrilled that our demos and libraries will work for more than just the few people willing to download our hacked-up builds.  WebGL is already all over the web, and if you’re just getting started with it, I encourage you to do some searching–many amazing demos exist right now.

The third thing, and closest to my heart, is the new Audio API in Firefox 4.  That’s a year of my life right there, and some of the work I’m most proud of that I’ve done with Mozilla.  In celebration of the launch, and since more people can finally run them, I thought I’d link you to some of our favourite audio experiments and demos.  Enjoy them, and then please make new ones that go further.  As of today, the web is officially open for business for all audio hackers, electronic artists, and musicians!

There’s many more listed here if you haven’t had enough.  Remember to View Source and copy what you see.  If you have others, toss them in the comments.

March 22, 2011 03:46 PM

March 01, 2011

Brett Gaylor

Butter screencast

As a part of our webmademovies.org relaunch, I thought I’d create a screencast of Butter, the authoring environment for popcorn.js. What do you think?

March 01, 2011 09:28 PM

February 26, 2011

Brett Gaylor

Web Made Movies at BAVC

Two things I learned this week:

1) Exercise caution when giving 12 high school students access to an etherpad that is projected against a wall

2) The Bay Area Coalition (BAVC)’s Factory program for youth is srs bizness

Mozilla has partnered up with the Bay Area Video Coalition and Zero Divide to implement the technologies coming out of Web Made Movies into the current curriculum of The Factory, a video collective for motivated youth in the Bay.

Ben Moskowitz and I came to meet members of the collective, and were joined by our Mozilla colleagues Atul Varna and Lukas Blakk. Lukas had recently been working with teenage girls at the Dare 2 B Digital Conference, addressing the gender gap in computer science, and Atul has been working on the Hackasaurus project with Drumbeat, a set of tools to encourage kids to hack the web. The 2 days at BAVC were for us a way to test some of our thinking about “web literacy”, to test the Butter authoring tool, and to meet the youth at BAVC. We were hoping that they’d understand what we were banging on about after the two days and would be inspired to bring a hacker ethic to their projects.

Our first excercise was to wake the HACKASAURUS. Everyone checked out the X-Ray Goggles, a fantastic tool that lets you see how a web page is put together, and modify elements of it. Paired with HTMLpad.org, which renders HTML typed into an Etherpad of the same name (try it - its awesome), the Hackasaurus tools teach the user that the web really is made up of many simple parts. To help visualize things, students printed out their favourite websites, cut them up, pasted them back together, and re-created their cutups in HTML. Its a great basis on which to start playing with popcorn.js - it puts one in the mindspace to consider “how can I hack this video?”. IMG_9852.jpg IMG_9861.jpg

Fortunately, our friends at Bocoup had pushed some last minute changes to Butter, the authoring tool for popcorn, and it was ready enough to be tested by the group. We loaded up one of their productions from last year, “The List”, which was a dramatization of the military recruitment that happens when students take standardized tests. There was a lot of factual information in the video, so it was a perfect test bed for popcorn.

IMG_9870.jpg
IMG_9875.jpg

The students formed groups and each took a section of The List and pulled it into Butter. They added semantic data, some hacked with Ben a method for displaying images on top of the frame, others linked to an HTML page that let users opt-out of recruitment, others brought in photos from flickr, articles from Wikipedia and maps of their school on Google Maps. Lukas wrote a quick script that would chain the videos together, and then we had a screening at the last minute!

IMG_9889.jpg
IMG_9886.jpg

You can watch the completed movies here - keep in mind these were created in two hours by a group that had never written any HTML, so we were well impressed.

One of the great things that came out of the workshop was a huge list of bugs for popcorn and butter created by the students that we can now bring back to the development community. We’re looking forward to coming back in April and working with The Factory through till the fall! IMG_9879.jpg IMG_9904.jpg

February 26, 2011 05:41 PM

February 17, 2011

Brett Gaylor

Web Made Movies 2011

Along with my other colleagues at Mozilla, I’ve been doing a lot of work lately thinking about what the rest of 2011 holds. It’s been great to read posts by Ben Moskowitz, Phillip Smith and Nathaniel James as they describe and forecast their work on the Knight/Mozilla Technology Partnership. It’s painting a great picture of Mozilla’s commitment to transforming media and journalism - I thought I’d share my plans as well in the video above.

February 17, 2011 06:49 PM

February 07, 2011

Scott Downe

Popcorn.js 0.1 vs 0.2 XML grammar and plugin API

I recently was asked for help with the XML grammar I created for Popcorn.js. I am aware this is not documented very well, but mostly because things have changed from our 0.1 to 0.2.

I ended up creating a pretty thorough email… and figured it was a waste as *just* an email, so I am going to make a blog post about it as well.

These links are the documentation for the old 0.1 XML:

old 0.1 documentation
old 0.1 XML example

The last comment here are the changes for 0.2 that the above links do not represent.

This is an up to date example of a working 0.2 XML file.

I am also going to show you a quick abstract example here, then explain it afterwards.

<popcorn>

  <manifest>

    <resource id="resourceid1" target="containerid1"/>
    <resource id="resourceid2" target="containerid2"/>

  </manifest>

  <timeline>

    <parent1 sharedattribute="value">

      <pluginname resourceid="resourceid1" in="00:00:32:00" out="00:00:40:00">text</pluginname>
      <pluginname resourceid="resourceid2" in="00:00:32:00" out="00:00:40:00">text</pluginname>

    </parent1>
    <parent2 sharedattribute="value">

      <pluginname resourceid="resourceid1" in="00:00:32:00" out="00:00:40:00">text</pluginname>
      <pluginname resourceid="resourceid2" in="00:00:32:00" out="00:00:40:00">text</pluginname>

    </parent2>

  </timeline>

</popcorn>

As you can see you need one parent tag called popcorn.

Popcorn has two children, manifest which is optional, and timeline which is mandatory.

A plugin is referenced by name, as an XML tag, and all its options follow as XML attributes. in and out attributes in the XML are the same as start and stop options in the plugin. A resourceid attribute will inherit all attributes in the connecting id of something in the manifest.

A plugin or resource can have an unlimited number of parents that simply pass down their attributes to their children until hitting a plugin.

Any tag in the timeline that does not have children will be considered a plugin.

All plugin names, and attribute names are transparent in that they are not rigidly defined, but defined by the plugins currently loaded.

Think of the XML grammar as the syntax, and the plugins as the API.

Each plugin has documentation and an examle for it here.

The parent and resource names do not matter, you can call them whatever you think makes sense for your use case.


February 07, 2011 03:30 PM

January 31, 2011

Brett Gaylor

Yes we did!

This past week, I was fortunate to be part of a great collaboration between Mozilla and the PBS Newshour team, creating an annotated version of Barack Obama’s State of The Union Speech using popcorn.js. While the demo itself is fairly humble, its actually quite an accomplishment given that we had only met the Newsroom staff the day of the speech during a meeting at the Corporation For Public Broadcasting. I think the Newshour team experienced the power of the open web first hand - a talented group of developers were able to collaborate quickly and produce something of high quality (not unlike a tv newsroom). So much so that we’ve begun a conversation on PBS to use popcorn.js in all of their future analysis.

The whole experience was a great example of the kind of innovation that we’re trying to foster with Web Made Movies: video producers come up with new use cases for technology , and the resulting code is put in a repository for future use. It’s proof that HTML5 video is a great space to be exploring.

David Humphrey and PBS Newshour recently blogged about their experiences. I thought I’d provide a blow by blow.

January 25th, 2011
Washington, DC: Ben Moskowitz, Nicholas Reville, Geoffrey MacDougall and myself are pacing around a presentation room at the Corporation for Public Broadcasting. We’ve come to express why we think there is natural alignment between public media and the open web.

10:00
A little bleary from travel, a previous day of meetings, and bad breakfast joint coffee, we meet a room full of people involved in American public media - PBS affiliate stations, the PBS Newshour team, National Public Radio, and others.

We had created some demos over the previous 2 weeks - including this popcorn treatment of a PBS Newshour piece on Hait (only the first several minutes are annotated). We also experimented with the WGBH Open Vault, creating a parser for their metadata schema that would translate into popcorn data (due to mp4 video, works only in webkit browsers. Ben and his cousin created this example video report, based on lesson plans in the PBS Newshour piece on Haiti. Nicholas at PCF also showed the results of a translation party that Universal Subtitles had hosted with PBS Newshour.

We felt the presentation went well - the room was definitely thinking about the possibilities present in popcorn, and immediately grasped the advantage of adopting Universal Subtitles.

11:30
After the speech, PBS Newshour correspondent Hari Sreenivasan remarked how they wished they had it for the State of The Union address. “Isn’t that tonight? Someone asked” Hari nods. A wireframe is sketched out.

3:00
We’re with Travis Daub, the Newshour’s digital producer. He’s on the phone getting the run down on when the video of the president’s speech will be on Youtube. He says midnight, and analysis will come in throughout the evening These news people stay up all night, too!

We hop on the popcorn irc channel (link). I ask David Humphrey at CDOT if he is up for bringing popcorn.js to the State of The Union. He laughs, of course he is. He asks if Anna Sobiepanek and Scott Downe who are co-op students at Seneca if they are in - you bet. Rick Waldron, a javascript ninja at Bocoup picks up on what is happening and starts writing code immediately based on the spec. Ben Moskowitz and I rush to keep up, posting wireframes, and communicating with Dean Jensen at PCF / Universal Subtitles who is co-ordinating the transcript that will be instumental in timing the analysis. Travis logs into IRC from his iphone in a cab.

7:00
Robert Bole for the Corporation for Public Broadcasting orders some Chinese food.

9:00
75% of the code necessary to convert PBS’ annotation system to popcorn. The wireframes are posted. Its gonna pull in twitter, have chapter selectors, wash your car, and email Obama. This is going to be easy. Lets watch the speech.

10:30
Travis starts uploading the video.

11:00
Snag. The encoded video doesn’t have audio. Travis will have to start again, and do it twice for browser compatability. Ouch. The video is so long that the encoding process is painful.

11:15
New snag. Servers seem to be failing.

1:00
Multiple snags. Grumpiness sets in. Travis valiantly posting new versions. Popcorn hackers have the analysis working with a dummy video - just waiting for the pieces to come together.

3:00 
The pieces are not coming together. The night is called.

The next day
A snow storm, a train travel, multiple taxis and subways, a hamburger at a breakfast spot - next thing you know its midnight again. Is it working yet? No, it is not. Inconsistencies in the transcript. Hotel Wifi killing us. Must sleep.

With a final push, and additional work in the morning, the bugs are ironed out. Vanessa Dennis at PBS Newshour adds styling and pushes to their blog on the front page. My favourite quote from theirblog:

“The Web is changing, and we at the PBS Newshour are changing with it through experiments like these.”

Experiments like these can be exhausting, but the thing I love about working with Mozilla is that we make change by building things. It feels great that Web Made Movies is really starting to accomplish this.

January 31, 2011 06:02 AM

January 28, 2011

David Humphrey

PBS, Popcorn.js, POTUS

PBS is writing about our work with them this week on the President’s State of the Union Speech and Mozilla, so I thought I’d write something, too.

This week my team and I at Seneca’s CDOT were handed a great opportunity.  My colleague Brett Gaylor was in Washington, meeting with PBS and showing them some of the cool Mozilla projects within Web Made Movies.  The discussions went well.  So well that they asked if we’d like to do an experiment with Obama’s State of the Union speech later that night.  Um, yes we would!

PBS typically produces an annotated transcript, with commentary and resources linked to various paragraphs in the speech.  Luckily they have this data available already as json via a RESTful api.  The trouble with this is that it is disconnected from the performance of the speech itself.  What if you could use the video of the speech, and the timing therein, to drive the loading and display of this commentary, such that watching or seeking through the video gives you timely comments?  Wonder no longer: we did it.

We did it with HTML5 video and popcorn.js, a Mozilla Drumbeat project within Web Made Movies.  With popcorn.js we can write time line based web apps that are driven by video timing.  Where the web has traditionally been about documents, and film/video about linear narrative in time, popcorn.js bridges the two.

For this PBS demo I worked with Anna Sobiepanek and Scott Downe (researchers working with me at Seneca), and Rick Waldron (partner and developer at Bocoup in Boston).  Brett and Ben Moskowitz co-ordinated with PBS in Washington.  We also hacked some code to use the speech’s timing information from Universal Subtitles, who were translating the speech in parallel.  We worked fast, and within hours we had it working–a testament to the technologies in play here, and the skills of the people using them.

I give PBS a lot of credit for being open to working with Mozilla on this.  These are new technologies, new techniques, and the timing was crazy:

This may sound simple, but considering that we came into contact with the folks at the Mozilla Foundation on the morning of State of the Union, and we managed to get this far, this fast — despite every technical glitch imaginable and a big snowstorm — it is pretty remarkable. Our sincere thanks to all the developers who burned the midnight oil with us to make this possible!

I laughed while we were doing this work when someone pointed me to a reddit post about PBS’ coverage of the news and technology.  There is some truth in this, and it’s clear that PBS wants to use technologies like the open web to enhance the news vs. sell you something:

Part of the reason people join the PBS NewsHour for news events like the State of the Union is because we give you the primary source without too much clutter. There are no NFL-style dancing robots with whiz-bang sound effects. There are no real-time approval meters in multiple colors zig-zagging across the president’s face. We provide you the facts, the context and we let you make up your own mind.

It’s still early on for popcorn.js (we are releasing version 0.3 in a few weeks), but already we’re having some great successes with it, and there are more in the pipe.  I’d encourage you to join our growing community of developers and users in order to help us figure out what time-based video development means on the web.

I’d also like to mention just how proud I am of my students, who aren’t afraid to build things this important, or try experiments this crazy and time pressured.  It inspires me every day to work with them.

January 28, 2011 04:24 PM

January 02, 2011

Scott Downe

Been AWOL all year

First post of the new year, and it feels like I have been gone for a year, but I am here still, I just… To sum things up, I got wrapped up, stressed out, and learned a lot. I was burned out, so I took the holidays off, completely off… I barely touched my computer.

Anyway. Gearing up for what is left to do for Popcorn.js 0.2.

I still need to create a 0.1 XML parser using the parser plugin structure, which will be a good test in the parser plugin structure, which hopefully will make things easier in the long run.

Things should now be more organic and dynamic internally, less hassle when it comes to maintenance, and usable by more people.

We addressed the first issue by making tools, tools that can be used by anyone to do what they want, vs what I think they might want.

Second issue, we no longer have the headaches of trying to please everyone, and maintain everyone, we let them do that, and we maintain the tools that allow this.

Final issue, drawing a line between developer and film maker. The developer tools are familiar (Jquery) and the film maker tools are a visual graphical web app called Butter (Final Cut Pro).

Something I have noticed about learning something new. You always produce less short term when you learn more, but in the long run, it is better. The things I have been learning involve Popcorn.js’s 0.2 release, and working with the people over at Bocoup. We have a new code base for popcorn.js, better described in my last post. I would like to call it a re write of the code, but really the old code is still there, it just exists in plugins. The old code was a parser, some fancy timing, and a bunch of “commands”, which all still exists, except it is now wrapped inside something elegant, something new (to me) and something familiar to people familiar with Jquery (a lot of people). The changes were, conceptually nothing difficult. The hard part was learning some new standards, to see faults before I fall into them, and fix them before they are a problem. Something that I notice I spent a lot of my previous time fixing, bugs that, given enough experience, would not of existed. This is OK to me, I am OK with fixing bugs I could of avoided, just so long as I make sure to avoid them the second time, which is something I am trying to improve on.

Working with Popcorn brought me from the standards of a student, where learning is safe, making it work is the ultimate goal, and a school assignment is never actually going to be used, or maintained, just rigidly criticized. To something real, something used by people doing this for a living, maintained, and used. Working in open source, and especially David Humphrey’s OSD class was a perfect middle ground for me, where I can experiment on something real, I had to consider the use cases, maintenance, users, to think, and draw from all my tools to solve an unknown problem, instead of drawing the tools from one chapter to solve the problem stated in that chapter. I guess the main difference in all that rambling is I am now working with people where my academic success is not in their best interest, people that are not getting paid because I am paying them to teach me. Ugh, OK, I had a point here, but I think I lost it within other points. Might be time for bed.


January 02, 2011 07:27 AM

December 21, 2010

David Humphrey

Flight of the Navigator

Mozilla is showcasing our Flight of the Navigator (FOTN) demo, and I wanted to tell you a little bit about it.  FOTN takes advantage of many of the new and improved features of Firefox 4, and is an example of the kind of things you can do with HTML5, WebGL, web audio, and faster JavaScript.

The first thing to say is that this is a web page–HTML, CSS, and JavaScript.  Some people we’ve shown this to have assumed it was a movie until they saw the real-time Twitter and Flickr integration.  This is a regular web page, and uses the web to full effect.  However, it doesn’t look like a regular web page, and therein lies much of the power and potential of the web coming with Firefox 4.

We built the demo in order to integrate and showcase the various JavaScript libraries and browser features we were creating together.  As a result the demo exploits a number of features in HTML5, namely, <canvas>, WebGL, <audio>, and <video>.  It also uses some of our JavaScript libraries (CubicVR.js, processing.js, dsp.js, beatdetektor.js), as well as the various JavaScript speed and feature improvements in Firefox 4.  Finally, the demo is driven by the new Firefox Audio API to synchronize the audio and 2D/3D visuals.

Everything you see in the demo’s 3D world is done using <canvas> with a WebGL context.  The city and all other 3D structures, the models and animation, were created using Lightwave.  We used CubicVR.js to work with the resulting 3D XML data, to build and power the scenes.  We texture the buildings and other structures with a combination of images (regular jpg, png), videos (WebM), and 2D canvases.  For example, we create various effects (interlacing on billboards, ticker-tape displays, audio visualizations) using Processing.js on 2D canvases, and then texture those onto the 3D objects.

We wanted to build a hybrid of what people are used to seeing in the regular web with the 3D capabilities of WebGL.  We did this by pulling in live data from Twitter (see the ticker tape animations for #firefox4 tweets) and Flickr (see photo billboards).  This makes the demo different for every run–we’ve had some interesting moments at conferences running this live and pulling things from Twitter.

FOTN relies on many performance upgrades in Firefox 4 to make things run smoothly.  First Hardware Acceleration makes it possible for us to mix 3D an 2D graphics very quickly, and run at high frame rates, even when the browser is fullscreen.  Next, we use many of the new features in Firefox 4’s JavaScript engine, for example Tracing to give us a speed boost on our complex audio and graphics code, and Typed JavaScript Arrays to make our audio analysis work in real time.

As the music plays we use the audio’s signal data from the Audio Data API to calculate FFT and beat detection.  This data is used to trigger animations, transitions, and visualizations.  We’re doing this all in JavaScript–the same JavaScript that’s being used to simultaneously render our 2D and 3D canvases.  It’s amazing how far you can push JavaScript now.

Speaking of the music, we’re using an Ogg Theora version of Shpongle’s “Nothing is Something Worth Doing” (Copyright Twisted Records, UK, used with permission).  Grab headphones or turn up your speakers to enjoy this awesome track.  We listened to it hundreds of times building this, and it’s never gotten old.

We built this demo in our spare time over a 4 week period with nightly builds of what is now becoming Firefox 4.  We’ve been working with these technologies for a while now, and can tell you that the potential of Firefox 4, with all the new graphics and media features it’s shipping, is amazing.  The web platform is capable of an amazing amount, and it’s exciting to think about where people will take it next.  I know we’re working on a new, larger demo already :)

FOTN was built by the same people who have been working on the Audio Data API and demos with me, and I’ll end by naming them all.  They are an amazing group of developers to work with: Charles Cliffe, Corban Brook, Al MacDonald, Andor Salga, Maciej Adwent, Boaz Sender.

December 21, 2010 02:07 AM

December 10, 2010

Brett Gaylor

Putting some Butter on Popcorn!

I’m very excited! I was just sent some very cool wireframes from Al Macdonald (aka F1LT3R), developer extraordinaire at Bocoup. He and others at the Boston-based Javascript house are in full stride working on the next release of popcorn.js and the first release of the Graphical User Interface for developing popcorn pages that we like to call Butter.

The goal for butter is that anyone with a video anywhere on the web can build HTML5 video pages that incorporate other elements of the web – allowing non-developers to create what my fellow Mozillian Tristan Nitot has dubbed “Hypervideo”.

In the new version of popcorn (its only version 0.2, so still a ways to go), we are moving all of the functionality from the previous version into plugins. This is a timely step towards a more open infrastructure that will allow any developer to write a plugin that will work with popcorn.js, and then by extension, Butter. So while the first plugins we have developed are for popular sites and services such as twitter, flickr and wikipedia, now anyone can create a plugin to support identica, open street maps, or whatever new thing the web churns out tomorrow. We have big plans for butter, and as we progress towards a 1.0 release in 2011 there will be much more functionality.

Check out our evolving project scope on our etherpad.

We have a goal for popcorn to be part of an open media ecosystem – a great web app and development platform that creators can add to the growing list of HTML5 tools that are available to them. We’ve been very inspired by projects like processing.js and Universal Subtitles that illustrate the advantages of federated and collaborative systems for creating culture that is truly OF the web. We are very early in that process and of course could use all the help we can get!

One of our next goals is to make it more clear how contributors can get involved – the best page for this is at www.webmademovies.org, where you can join our mailing list, talk in IRC or pick up a bug on our Lighthouse issue tracker.

On a personal note, this has been a great learning experience for me, as I’ve seen the power of the open web play out in realtime – a seed of an idea (what if a video could trigger events in a web page?) gets a proof of concept demo, which is turned into a library, which is now being turned into a product that we hope a lot of people will get behind. It’s been inspiring to see students from Seneca College working side by side with seasoned professionals towards a goal that will benefit the web, and makes me happy to be part of Mozilla.

Please stay tuned, and also check out Bocoup’s post on the subject, as well as Scott Downe’s recap of the experience from his end at Seneca, as well as Anna Sobiepanek’s thoughts on the refactoring for 0.2. We’re planning a release just in time for Christmas!

December 10, 2010 07:24 PM

December 06, 2010

Scott Downe

Weekend at Bocoup

Edit: I hope my point was not misinterpreted, but in no way did I mean to say I felt the people in Boston were greater to surround myself with than Toronto. I consider Seneca to be the best thing to happen to me, and David Humphrey to be the best teacher to happen to me, and CDOT to be the accumulation of both. I am gathering as much as I can from as many sources as I can, and mixing that up is huge in getting multiple perspectives. I am already surrounded by greatness, I am just expanding that greatness and making the most of it. Greatness is non linear. It is not a simple stack that is from 1 and up, but multiple stacks of varying types which should all be explored with an open mind. Only when a full understanding is obtained can an opinion be created.

I just got back from Boston, with me and Anna spending the weekend working with Al, Boaz and Rick of Bocoup on Popcorn.js.

Before I get into the specifics of what we were working on, I have to thank them for their hospitality, and the opportunity to work with them. I was watching them pretty closely, and I specifically want to mention Rick. There are moments in every career path, when you get a glimpse of “the next level”. This is the way I thought of Rick’s code, and in order to reach that level, I must first experience it. I kept thinking “You must surround yourself with great people in order to become great.”

I also have to thank David Humphrey and Brett Gaylor for their help in putting this weekend together.

Anyway, on to Popcorn.

We were working with them on refactoring Popcorn.js into a plugin architecture so it can be used be an authoring tool they will be creating. The main advantage for this is that we now have a clear line between film maker and web developer. One of the previous problems have been where to draw this line, or how to support both equally. We tried using XML as a sort of middle ground. Something filmmakers and web developers alike can use, but we quickly found out that it might be easy for both to understand, but neither like to write XML, and no one should write XML, it should be machine generated, but if we are going to be generating it from JavaScript, we might as well generate something that IS JavaScript, which is not XML. So the parser is gone, it is not needed anymore. We are now left with four main components.

First part: We have an authoring tool which will be used by filmmakers to create plugins, and develop a page that will be used to display their content with the plugins. This is an option tool and is not needed to make things work, but also is a stand alone tool as nothing else it needed to make the tool work. The previous popcorn had nothing like this.

Second part: We have plugins. These are created by the authoring tool, or by writing JavaScript by a web developer. The previous popcorn called these “commands”. They were created internally, maintained internally, and everyone had their own idea of how each command should work. Now commands are plugins, they are maintained by the author, and held externally from the popcorn internals.

Third part: This is the plugin registration framework. This is where a plugins go form being individual templates written in JavaScript, to a cohesive unit that. The plugin framework knows how to register a plugin, create commands form it, and attaches them to a target video.

The fourth and final part: This is the internal timing mechanism, and to me, the core of what popcorn is. This is what interacts with the video and knows how to use registered commands. We are pretty much able to plug in the old popcorn timing code, into the new. Al took a close look at this code, found a small bug, and offered some collaboration to make it better.

To me this method is much cleaner, and technically, isn’t much of a library anymore, but it is still a powerful JavaScript tool worthy of the name library. It is going to be much much easier for filmmakers to use, and less of a headache for developers to maintain and more familiar for them to use.


December 06, 2010 07:14 PM

November 18, 2010

Brett Gaylor

Drumbeat: A Forcing Function

Last week, in collaboration with David Humphrey, I ran the Video Lab at the Mozilla Drumbeat festival in Barcelona. I’m still recovering, so this post is a bit late, but a week has actually given me some perspective on why events like this are necessary crucibles for innovation.

IMG_8877.jpg

Everything we’ve achieved so far within the Web Made Movies project has been the result of intense collaboration over a short period of time. The popcorn.js library was created in a span of only two weeks, with half of this time devoted to creating our first demo (screen capture embedded below).

The forcing function of this demo was the Mozilla Summit - a gathering of the entire Mozilla community in Whistler, BC. It really opened my eyes to the degree to which demos drive the developer community - showing our demo at the summit was the moment we moved from “hand waving arty types” to “people actually trying to do something” in the eyes of the developers in the room. This will be crucial as we begin to move from “demo or die” mode towards shipping software.

Our next event was the Open Video Conference in New York. In the run up to this event, we knew that we were meeting with folks from the Public Broadcasting community in the US, so we wanted to show how open video technologies have a competitive advantage in that they can be quickly iterated upon. So Scott Downe and Anna Sobiepanek created two quick demos - one of a test integration between popcorn.js and Universal Subtitles (video below), and Lev Feels Fine, a an experiment in data-driven narrative.

In the run up to the Barcelona event, we knew we needed to think about how the open video tools we were developing could be used in education. I knew that Mark Surman’s son Tristan made video tutorials for video games, so I asked him and Mark for a “video book report” around a novel he was currently reading, and then we layered popcorn.js on top of his video - see the results. It was a great penny-drop for many people, illustrating how the web could be a canvas for students to create their own multi-media essays and reports.

When David and I arrived in Barcelona, we knew we’d have to show something on the first day to get people thinking. So we recorded some video of the people who we’d shared a “space wranglers” meeting with in Plaza del Ángel and asked them where they were from. David went back to his hotel to recover from jet lag, but hacked together a demo that mashed the video with Google Maps and Wikipedia (video version below).

Popcorn.js Demo: Google Maps API and Wikipedia from David Humphrey on Vimeo.

Our mission was to create something while in Barcelona - a lofty goal for these types of events, and one that depended on a delicate mix of developers, filmmakers, designers and educators showing up. As luck would have it, they did!

Over our first day we had an overview of HTML5 video technologies, and split into several breakout groups to brainstorm what we could build in our second day. We also had a visit from Aza Raskin, the creative lead for Firefox at the Mozilla Corporation, who did a talk he called “How to prototype and influence people”. It was fun and drew a pretty big crowd. The talk is below, and was picked up by BoingBoing after the conference, which was fun.

Rapid Prototyping with Aza Raskin from Dan Braghis on Vimeo.

So after some facilitation, it was decided that we would work on two projects: A meta-data demo that overlays Dublin Core data on to videos (requires Firefox 4 beta or nighties), the creation of which was certainly influenced by the fact that so many librarians and academics at the conference were expressing the lack of tools like this.

Our second creation was a short web made movie exploring the Future Of Education. The video was shot before lunch, edited, then layered with the twitter hashtag #futureofeducation (which didn’t exist the day before), and javcascript and css was hacked to overlay images from the evolving #drumbeat flickr hashtag. It isn’t exactly what everyone imagined (when does that ever happen anyway?), but remains a great outcome and a compelling piece of content by any standard.

Drumbeat “Future of Education” Demo from David Humphrey on Vimeo.

I know both projects will continue to be improved by this talented group that came together to hack, create, play and collaborate.

And as far as forcing functions, for me it has reinforced a need to improve the popcorn.js library, to create documentation, more demos, better code, to not loose sight of the artistic component of our project, and to have more events like these. The event has encouraged me to publish more, to find new collaborators, to advance the state of Open Source Cinema, to develop new forms and new languages, and to follow through on the promise of Web Made Movies - to bring developers and filmmakers together in a collaborative environment. Stay tuned for our next forcing function: 2011.

November 18, 2010 07:19 PM

November 09, 2010

Scott Downe

video presentation builder

Few days ago I made a demo for building a presentation website using a video. It took a weekend to initially build, and a few more days of tweaking with feedback from David Humphrey to get it just right. It really is amazing how you can take something raw, add feedback, and end up with something polished. It was a good workflow for me.

Two things I want to talk about.
- What it builds
- How to build it

What it builds: It builds a presentation of websites based on a video. So you would have a video describing or explaining something, say a book report, and throughout the book report you mention key characters and events in the book.

Now, when these events and characters are mentioned, you can load a webpage of the character or event to display with the video.

How to build it: It is really quite simple and has three steps, all of which require no coding.
Step 1: Enter a video URL. This is a video you can make, describing whatever it is you are presenting, like a book report. Then put that video online, or somewhere where you can have a link to it. The key is you need a link to a video.

Step 2: Open the menu on the left by scrolling over it, and use this menu to encode your webpages and resources you want displayed in the presentation, and the times these resources should be displayed, and for how long.

Step 3: Using firefox, go to file, Save Page As… and save the webpage somewhere. Now when you open saved webpage in a browser, it should have the video playing in the bottom right corner, and the encoded resource URLs displaying along with the video, at the encoded times.

I am open to even more feedback to polish it even further.


November 09, 2010 04:22 PM

November 08, 2010

David Humphrey

Mozilla Drumbeat Festival: Open Video Lab

Last week I was in Barcelona, Spain for the Mozilla Foundation’s Drumbeat Festival.  The festival’s theme was Learning, Freedom, and the Web, and attendees came to participate in sessions and workshops on a variety of education, open source, and open web topics.  Together with Mozilla’s Brett Gaylor, I ran the Open Video Lab.

Our goals for the Open Video Lab were simple to state, harder to guarantee: show people what you can do with HTML5 and <video>, <canvas>, <audio>, CSS3, etc.; link film people with developers with storytellers with designers with educators; and to, as Mark Surman is fond of saying, “help people build cool shit using the open web.” Thanks to the amazing people who came to the festival, we did all that and more.

One of the attendees, Liz Castro, did a great job blogging about what was happening during day 1 and day 2 (Gabriel Shalom also has a great post here), so I’ll point you to her posts, and focus what I want to say on the people who came, and the things we built.

For Brett and myself, the single hardest part of preparing for this event was the fact that we didn’t know who would come.  It’s hard to program two days of content when you don’t know the make-up of your group, their interests, and backgrounds.  Further, we didn’t know if people would only want to come for an hour and then leave to attend other sessions, or stick it out with us and stay to build things.

In the first session we had a packed room, and were met with professors, filmmakers, web developers, designers, producers, students, translators, writers, artists, etc.  It was a fantastic group, and had all the energy and diversity we needed to actually build some things.  Also, there was a core of people who were determined to stay and see things get finished.

On the first day Brett shot a really quick video getting various people in the square to say where they were from.  Nicholas Reville then took the video and had people translate and subtitle it into 17 languages using the Universal Subtitles project.  Next we built a simple Google Maps tool to allow us to extract longitude and latitude info so we could get geo-data for all the speakers.  Finally we put it altogether using Popcorn.js to create a mash-up of the video, Google Maps, and Wikipedia pages (video of demo here).

After we’d built an example open web video demo together, it was time to think about what we might build on the Friday.  We started brainstorming, and the group broke into three main sub-groups, each discussing:

Mozilla Firefox’s Creative Lead, Aza Raskin, then came and gave a talk to the group on how to effectively create prototypes (video, slides, and demo from the talk are here).  The room was so packed that Brett and I literally got pushed out the door so they could handle the overflow.  It was a timely topic for our group, who were starting to talk about how we should build a “platform.”  The point we picked-up on in the next hour was one that Aza underscored in his talk: if you can’t build your prototype in a day, you need to keep cutting until you can.  Our group only had one more day, and we want to take advantage of all the talent available to us.

After much debate and discussion, we agreed to take on two projects: 1) make a film about the future of education using the open web; and 2) make it possible for librarians, archivists, and other metadata people to have their bibliographic metadata reveal itself to users in <video>.

The next day we spent an entire day building things together.  The metadata team worked with JavaScript, RDFa and Dublin Core data, and built a custom overlay UI to make it possible to see info like the video’s title, creator, date, etc.  Meanwhile, the second group took advantage of the incredible set of educational experts on-hand at the festival to conduct interviews on the future of education.  This being Europe, they shot five interviews, each in a different language (French, Catalan, German, Italian, English).  They then had these translated to English, and wrote the subtitles.  The rest of the group figured out how to get flickr and twitter content to mix with the video, and worked with the designers on an overall aesthetic.

It was an amazing feeling to work in the lab that day.  Everyone was engaged and valuable–not one of us able to do everything that had to get done.  Our main goal had been to give the attendees an authentic experience of working on and with the open web, and doing so in a highly collaborative way.  This was exactly what happened, and it was a thrill to be part of it.  As we ran to the final keynote to present our projects, one of the filmmakers said to me, “So this can’t end today, we have to do more of this.”  A lot of us had that feeling.

Here are the two things we made.  The first is the metadata demo, showing a prototype of a web-based metadata parser and video overlay UI for video bibliographic information (video demo here).  If you hover your mouse over the video, a small ‘i’ will appear in the lower-right.  Clicking it brings up the metadata (title, creator, date, license), which is copy-pastable text.  For reasons we didn’t have time to fix, it doesn’t work in Firefox 3.6, but does in Firefox 4 Beta/Nightly, Chrome, etc.  It would be good to grow this out to a proper JS lib, and extend it to work for Images too.

The second is our so-called Web Made Movie on the Future of Education (video of demo here).  It has web-based subtitles, uses the #futureofeducation twitter hashtag, and #drumbeat flickr hashtag.  It still amazes me that it was all done in one day (before 6:00 pm!), and I think it captures so much of what was going on at the festival: educators, artists, hackers from all walks of life coming together to share, learn, and build.

Photo Credits: Samuel Huron on flickr.com

November 08, 2010 08:57 PM

November 04, 2010

Scott Downe

FSOSS 2010 presentation

I have been slow with keeping up with my blog lately, but I spent a good chunk of my time focused on my FSOSS 2010 presentation, then I had to write up two demos in a relatively short period of time which will be up here shortly, but they are being used in a talk today in Spain, so I will wait until that wraps up.

So anyway, my presentation link, which is where you can view a recording of my voice played along with the video of my slides. You can also view a live demo of my slides, which were created in HTML5 as well. My presentation was on open video and the video library popcorn.js, which recently moved from the processing.js lighthouse account and into the webmademovies lighthouse account.


November 04, 2010 06:53 PM

October 31, 2010

David Humphrey

50 visualizations of Life’s Things

On Friday I gave a talk on the new Audio API in Firefox 4 at FSOSS.  I was showing my youngest daughter some of the demos I presented today, and she said, “Dad, that song is so beautiful, draw some more of it.“  I really like this song too (Life’s Things is the track), and since it’s CC licensed, I made an ogg version and put 3 basic visualizations of it online:

Here’s what I am asking, dear interwebs.  Grab a Firefox 4 Beta.  Take this song.  Take the code above, View Source.  Make something using these as a base.  Make something new.  Throw a link in the comments.

The web that’s coming when Firefox 4 launches is a web where music is a first class, scriptable citizen.  You need to know how to code it, visualize it, generate it, mutate it.  Now’s the time to start.  If you know JS or Processing, canvas, html, css, that’s all you need.  Nothing I did above is more than 50-100 lines of script.

I’ve got the first 3 up, we need 47 more.  Go.

UPDATE: awesome, some people are already doing this!  One common question coming in, “Why am I getting NS_ERROR_DOM_SECURITY_ERR?”.  If you’re working locally, you have to flip a security pref for access to file:///.  See docs here.

UPDATE 2: “What if I can’t host this?” If you don’t have a server to host this, just email me your code and I’ll host for you.

October 31, 2010 04:43 PM

October 18, 2010

Scott Downe

Popcorn.js Website and JavaScript commands

Hey, so Popcorn.js has two new commands.

website command, which will display a website in the targeted div, and a JavaScript command, which takes two parameters, a start function, which is to be fired on the events in time, and a stop, which is fired on the events out.

Example of the website command is:
<webpage in=”00:04:00:08″ out=”00:04:19:00″ target=”iframes” src=”www.google.ca” width=”100%” height=”100%”></webpage>

I just noticed it should probably be a: <webpage />

The script command is simple:
<script in=”1″ out=”10″ start=”customJS.start” stop=”customJS.stop”/>

    var customJS = (function() {
      var interval;
      var content = document.getElementById("content");

      function work() {
        var item = document.createElement("p");
        item.innerHTML = "working...";
        content.appendChild(item);
      }

      return {
        start: function() {
          var item = document.createElement("p");
          item.innerHTML = "starting...";
          content.appendChild(item);
          interval = setInterval(work, 1000);
        },
        stop: function() {
          var item = document.createElement("p");
          item.innerHTML = "done!";
          content.appendChild(item);
          clearInterval(interval);
        }
      };
    })();

Using the above as the function to call.


October 18, 2010 12:57 PM

October 09, 2010

Brett Gaylor

Lev Feels Fine demo is live

Thanks to the efforts of Anna Sobiepanek and Scott Downe, I’m happy to invite you to try out the latest experiment from Web Made Movies. As outlined in an earlier post, we wanted to pay homage to both the Kuleshov Effect and We Feel Fine. This demo is an experiment in merging the traditions of cinema with the emerging ideas of the web.

Of course, we know that this demo by itself has limited emotional and narrative impact - but as with everything in the Web Made Movies project, our idea is to experiment at the fringes, hoping that other filmmakers and developers will both build on our work, but also bring their own projects and ideas into our lab-in-progress. Please do contact us if this sparks an idea that you would like to pursue. We’re open :)

Visit the demo!

October 09, 2010 01:53 AM

October 08, 2010

Brett Gaylor

Universal Subtitles + popcorn.js integration

From early on in the popcorn.js process, we’ve been imagining BUTTER - an easy way for non-coders to be able to add metadata to their videos. Currently, popcorn.js uses an external XML file (though we’ll be switching to JSON in our next releases). Neither of these formats are reasonable for lay video creators to use, so discussion of a front end has been in the works for months.

However, looking at the amazing work done by the Participatory Culture Foundation on Universal Subtitles, we thought this interface would be great to experiment with, and so they were nice enough to add support for metadata in the language drop down on their staging site. Scott Downe and Adam Duston are responsible for this early work, which you can see a screencast of below.

October 08, 2010 05:50 AM

October 06, 2010

Scott Downe

Universal Subtitles + Popcorn.js

When we were finishing off the first demo and initial popcorn.js work. Popcorn is a library for defining all sorts of semantic data to a video keyed on time, so when the video hits this time, show this data. We kept our data in xml and quickly noticed writing all that xml was not a fun job, and not necessary. As a programmer, I feel xml is meant to be written by a machine, and not a person, so how can I expect a film maker to write it?

We talked about later going back and writing a front end gui page that could be used to create data, and spit out an xml file you can download.

We also knew of a project called Universal Subtitles which was another open source project that was very similar. I am not sure who contacted who, but we ended up collaborating with them so their interface could be used to create a popcorn file of semantic data. We started with wikipedia, google maps, and twitter, then later put in the base subtitle.

This is much better than our initial plan of downloading an xml file, because now the data is kept in a central location, instead of on someones local machine, making it easier for everyone to share their subtitle work.

The idea of how to use this is, you subtitle a video keyed by the video’s url using Universal Subtitles, and then for the popcorn data source, you just use that same link, and popcorn should take care of the rest.

This is still a very early hack, and not yet released on either Universal Subtitle’s official webpage, or in the most recent version of popcorn… yet.

Another thing I would like to mention is adding a geo location required finding the latitude and longitude of the location, then pasting that into the xml, which was also not fun. So I created a bookmarklet that would should a google map, you could pan to your location, and hit done and it would paste in the latitude and longitude for you.


October 06, 2010 05:14 PM

October 04, 2010

David Humphrey

sfxr.js: sound for open web games

Last week Mozilla Labs announced its Game On competition, a contest “all about games built, delivered and played on the open Web and the browser.”  The open web platform has got a lot to offer game developers these days.  I’m hoping that some people will enter processing.js based games (version 1.0 is coming in the next month).

I’m also hoping to see how far people will push sound effects, audio, and music.  When I think about video games (and I date myself here), I mostly remember sounds.  See if you know these games.  I could close my eyes and tell you what most of them are:

Those chirps, pops, and pew! pew! pew! sounds are a fundamental part of the video game landscape.  And if we’re going to talk about making games for the web, to me, we need to think about how to recreate this soundscape.  Luckily Ted had the solution: “Someone needs to port SFXR to the Audio Data API.”

SFXR is a little application that allows a user to generate basic video game sound effects.  Tomas Pettersson wrote it C++ in 2007, and Thomas Vian then ported it to ActionScript in 2009.  This weekend I ported it to JavaScript: sfxr.js.  Here’s a short video I made of it.

Right now sfxr.js allows you to generate various sound effects, modify them if you wish, then create a .wav file.  Eventually I might write a small library to allow you to use these sounds in a web page without needing the .wav files.  Another thing I like about it is how easy it is to collaborate with someone else as you design sounds.  Every sound can be “saved” via the URL and the sound=… parameter–just bookmark it, drop it in irc, etc..  Here are some sounds I generated:

As is usually the case with the things I build, I don’t have an immediate need for this (although my daughters and I had a great time looking for a “dragon fire” sound).  Instead, I’m interested in building the things that enable others to do interesting work.  Part of the amazing potential of the web is that it makes creators out of people who would otherwise have been consumers.  Want to build a little game and need the sound of a character jumping?  What are your options?  Buy it somewhere?  Steal it from somewhere?  Why not create it yourself?

The open web, whether it’s being used for games, sounds, video, etc. is really about giving people the control they need to build and share things.  I hope you build something cool with sfxr.js, and if you’d like to help make it do more, let me know.

October 04, 2010 03:40 PM

October 01, 2010

Scott Downe

Open Subtitles Design Summit 2010

Today I attended the Open Subtitles Design Summit 2010, and just wanted to quickly jot down some of what i took out of it.

The first discussion was on how to connect the subtitle with the video, so if someone has a video, they can search for the subtitle info without having to write it themselves. A few problems with this:
- Copyright issues. If someone makes a video, someone else cold subtitle it, put it online, without the content owner even knowing that this happened.
- Synching multiple video links to the same video with a key, a unique key, so there doesn’t exists two subtitle files for the same video. The problems with this is if a video is essentially the same by name, but one has commercials, thus is no longer the same from a technical standpoint. An idea to solve this is to get a finger print of the audio data, and as far as subtitles are concerned, that is the exact same video.
- The final main problem I took from this is where to host this subtitle data that people could look up. Do you do it in one place like a database, or do you spread it around like a torrent.

Second discussion was on video meta data. That term can be thrown around without a clear definition.
- The main thing I got from this is that there are three types of meta data. Data about the video, like the size, length, or other technical details. Data about the video’s contents, like the people in the video, the people that made it… content. The final one, which is timed meta data, like a subtitle starting at time x and ending at time y. The concept of timed meta data helped me understand it, because it made me see the object inheritance of meta data. If you look at it this way, the video is data, and the subtitle is meta data, and the start and end time is meta data about the subtitle, and not the video itself. So maybe there is only two types, with subtitles being about content, and the start and end being technical meta data about that subtitle.
- Another concept that interested me is that some of this data will never change. Like director. The director of a film will always be the director, but something like rating will fluctuate. Both are not new types of meta data as they are both data about the content, so content or technical meta data can be fluid OR concrete.

Fourth discussion was about popcorn.js and where we can go with it.
- We need to consider our xml format, and make it standard, because we met pad.ma which is doing something very very similar, in a very similar way, so we have to take a closer look at the xml and see how we can create a standard between at least the two of us, to get the ball rolling.
- Also, because of the discussion on meta data, I have a clear description of what popcorn is… timed meta data.
- Having a command to link into the api of what people are listening too. Just a new command to add to the library.
- Something else that is really interesting is a working standard, which is not yet implemented, but is a timed event listener called oncuechange… Basically, do this function at this time. It can be found here which is very exciting and could improve the foundation of popcorn.js.

Fifth discussion was on openindie service. Which is like a hub for independent video makers to get their videos out there, and send screeners of their work to people.
- They were looking into getting subtitle data much the same way popcorn.js is from universal subtitles, and I think it is important for popcorn and openindie to work together with universal subtitles to get the same data, so it only has one format. This would probably be different than the subtitle info used by pad.ma and popcorn. The difference being xml and json, and different standards for each when it comes to subtitles and semantic data.

Final discussion was the last little bit on what can be done in the next 3 months, and what can be done in the next 12 months.
- the best thing from this was an idea where we oculd use the audio api to get the raw sound data of a video, and read it, then play one sentence, then pause, so whoever is entering the subtitle can keep up with the video. Write one sentence, hit enter, write the next, etc. This would be ideal and very intuitive. Talked to David Humphrey about this, who is the lead developer of the audio build, who recently had a discussion on this, and he gave the idea of writing a library that would control and deal with when sound of a video changes in frequency, and then you can fire events, or do something else with it.


October 01, 2010 04:04 AM

September 11, 2010

Brett Gaylor

Web Made Movies hack day at Open Video Conference!

As the Open Video Conference in New York draws closer (Oct 1-3), we’re excited to announce our hack day on Sunday October 3rd.

The Web Made Movies hack day is a chance for filmmakers, developers and anyone interested in creating Open Video to turn the inspiration of OVC into action.

Are you a videomaker?
Its a great opportunity to meet with people who can help you explore the potential of HTML5 video – for instance, applying the Popcorn.js video library to your videos to pull data from flickr, twitter, wikiPedia and Google Maps into your video experiences.

Are you a software developer?
Meet with the folks who have created the Popcorn.js library and brainstorm how to improve and expand its scope. Meet with like-minded developers who need help or have some to offer. Learn and teach.

Just curious?
We’ll be featuring a series of short talks and presentations that will spark your imagination and have you leaving the OVC weekend with some concrete next steps to apply to your future work. And we will have lots of coffee during, beer afterwards, and some pioneers of internet video hanging around.

If you’re interested in coming, it would be great if you could fill in a small google spreadsheet form – it will help us in designing the day and pairing up video creators with developers.

Find out more about Web Made Movies, Mozilla’s open video lab.

Details of the event: Sunday October 3rd, 10 am - 721 Broadway

View Larger Map

Please fill out this survey if you plan on attending!

September 11, 2010 01:40 AM

September 10, 2010

Scott Downe

popcorn.js + processing.js + box2d.js Part Deux

Yesterday, I loaded up an old demo I made and blogged about a few months? back, and decided to add to it, and do some profiling and optimization.

This is what I ended up with.

First thing I did was update popcorn.js and processing.js.

Second thing was add the translation feature. Which went as expected, by that I mean, I thought it would be easy, and there was in fact an unexpected problem, which I expect at this point. The problem ended up being a typo in the div name ><. Also, a little quirk with this is the word sizes change, and, if you select Japanese as your language, you get the whole subtitle as one giant word.

The third and most important addition is optimization. The old demo was pretty slow, and I knew this, but at the time I wasn’t so worried about optimization yet.

First thing I did for optimization was profile, and found that I was spending 80%!! of my time in a processing.js function; a function that I have spent time working in called text$line. It’s just a little internal function. I then proceeded to comment out code in text$line until I started to see a difference in framerate and time spent in text$line. What was happening was every time we drew text onto the screen, we would reset the current context’s font and size, when it makes more sense to only set those when the font or size change. So I moved one line out of text$line, and into textFont() and textSize(), then filed a ticket here. This fix was able to lower my time spent in text$line from 80% of total program usage, to 30%. Still not perfect, but a HUGE improvement, and I am sure the number of times I am redrawing text in this demo has a lot to do with the high number of text calls. I also lowered my framerate from about 20 to 40.


September 10, 2010 07:34 PM

Brett Gaylor

Kuleshov Effect + We Feel Fine = Lev's all right?

Was discussing in the Web Made Movies IRC today with David Humphrey the need to create a new, and vastly different than our last, demo for the upcoming Open Video Conference.

We wanted to create something that presented a dynamically generated video based on who was watching, and that created different meaning from different juxtapositions of shots.

This reminded me of the Kuleshov effect - the famous experiment by Russian film theorist Lev Kuleshov, in which he proved that meaning was created in cinema through montage.

Kuleshov took a shot of a man staring at the camera, which was devoid of emotion. He then made 3 sequences where he spliced a bowl of soup, an attractive woman, and a child in a coffin, followed by a shot of the same man.

When showing the 3 sequences to audiences, he found that despite the expressionless face of the man, they perceived him to be hungry, lustful or sad, depending on which shot was placed in the middle. Thus, he declared, the meaning of individual shots and sequences was changed by whatever preceded or followed them. It was an extremely influential finding.

Seeing as Web Made Movies is about seeing what happens when we mix the traditions of cinema (time) and the traditions of the web (hypertext), Dave and I wanted to experiment with how this experiment would be different in the context of the web.

I though it would be fun to mash this experiment with the ethos of We Feel Fine, a human emotion experiment by Jonathan Harris and Sep Kamvar that presents web visitors with visualizations of the worlds’ emotions by gathering data from searches and blogs.

So I mocked up a wireframe that you can see below, and I’m looking for some feedback on how it could more meaningful riff of these two experiments. In the example below, the user is asked for their twitter account, as well as their location.

The demo would then build a unique sequence of 3 shots that would be tailored to the users emotions. The first image would be a shot of a face, staring similarly to Kuleshov’s actor, which would be either sad, happy or angry. Which emotion appeared on the actors face would be determined by how often the twitter user mentioned those emotions in their twitter stream of the last month (or so). The second image would be pulled from flickr - from a search of an emotion (you can see that interesting results, for instance, can be found be a flickr search of “sad”.

The third image would be the same as the first.

The music would be chosen from a sampling of 10 songs (or so) which would be categorized based on emotion. The chosen song would correspond to whatever the “world” was feeling, which would be transmitted from We Feel Fine’s API.

The background of the page playing the video would be influenced by the weather in the users location - if it was rainy, it would be dark, sunny would be bright, and so on. Potentially we could also throw in some weather effects via something like processing.js.

Finally, the user would be asked to categorize the emotion of the video. This would be added to a third page, which would provide a grouping of the emotions similar to the 9 elements demo (only works in FF 3.6, not FF4). Users can then preview the different videos that were generated for others.

So it doesn’t feel quite right yet, but we’re thinking we might start and see where it gets us. But could really use some feedback on how this experiment could feel more true and more, well, meaningful. The wireframe draft is below. Comments here most appreciated.

September 10, 2010 02:49 PM

September 09, 2010

Brett Gaylor

Web Made Movies vault

In creating our first demo for popcorn, we gathered footage from projects like Grassrootsmapping.org and The Village Telco to create the final video. However, we only used small snippets, so I took some time to edit down longer versions of each.

Grass Roots Mapping

The Village Telco

September 09, 2010 05:38 PM

September 08, 2010

Brett Gaylor

Not Found

The URL you requested could not be found.

September 08, 2010 07:20 PM

August 20, 2010

Tristan Nitot

Video+html5+Popcorn.js=hyper-video

You may have seen that Brett Gaylor is joining Mozilla (see also Mark Surman's post). For those who don't know Brett, he's particularly famous for his "Open Source documentary" Rip! A remix Manifesto[1].

One may ask why Mozilla has hired a film director[2], but it actually makes a lot of sense thanks to Mozilla Drumbeat, as Brett is working on a Drumbeat project called Web made movies.

Now Brett has been a Mozilla community member for quite some time, contributing with the good folks at CDOT / Seneca College to create popcorn.js, "a JavaScript library for merging video with semantic data". I understand that this is a bit of a mouthful, but don't close your browser window just yet! Popcorn.js is what I would describe as "hyper-video" ("hyper" as in "hypertext"): the ability to leverage data from the video and link to it, Web style. Such data include:

Screenshot of the demo

Go and see for yourself the PopCorn.js demo (in case you're stuck with an older browser that is not capable of running the demo, here is a video of the demo).

I think this is a very significant step further for video on the Web, which was until now a very TV-like, passive and linear approach, now merged with the hypertext nature of the Web (its ability to link to things in other places), so that users can click on links in order to learn more. Of course, this is just a demo. Tons of things need to be done, but I see this as a very cool way to show what HTML5 and its video element, combined with the power of JavaScript and mash-ups.

Notes

[1] I can't say how strongly I recommend watching this movie, starting with its trailer.

[2] It's actually the second one, as Henrik Moltke, co-author of Good copy, bad copy, is already working at Mozilla... on Drumbeat!

August 20, 2010 02:32 PM

August 19, 2010

Brett Gaylor

Where I've Been - Where I'm Going

As some of my friends and colleagues know, I’m actually quite late announcing some changes that have been afoot in my personal and professional life. Let me rectify!

In March I began a collaboration with Mozilla on Web Made Movies, an innovation lab bringing together software developers and filmmakers to innovate video on the web. (For folks who don’t know, Mozilla is a public benefit organization that promotes freedom and innovation on the Internet by building stuff like the Firefox web browser). Like anything worth doing, the Web Made Movies project shifted from its original conception, which was to create a documentary series about the open web using an open video platform. It became clear that the need for open video tools was too strong for only one content project to drive forward, and so the decision was made to focus on the creation of a lab where filmmakers and web developers could work collaboratively, and where storytelling and form were placed on an equal footing with technology. Since that decision, we teamed up with the Center for Development of Open Technology at Seneca College, and began our first project: Popcorn.js, a javascript library for merging video with semantic data. I have found the work challenging, rewarding, and affirming of my desire to innovate what video on the web can be. We’ve had some great early results - check out our demo of semantic video.

I’m particularly excited that Web Made Movies is part of Drumbeat, Mozilla’s initiative to involve a broad spectrum of society in its mission to build and protect the open Internet. I’ve believed for some time that in order for the internet to survive as an open platform, it can’t be only the geeks holding down the fort. Storytellers, educators, artists, journalists and activists need to get involved, feel welcome and have a stake in the internet remaining a global resource that everyone has a hand in shaping. I’m happy that Mozilla is moving in this direction – over 350 million people are touched every day by their work on Firefox, and having a public service organization in the browser space is a much needed check-and-balance system in an extremely competitive (and cutthroat) environment. Observing Mozilla over the last decade, I’ve been impressed by their pro-activism. They showed that a faster and safer internet experience was possible if they put the needs of the user before those of the board room – and they showed this was possible by BUILDING it, and continuing to improve it.

This is why I’ve made the decision to join Mozilla full time. Yup! I’ve accepted a full time position and I couldn’t be more thrilled. For the next several months I’ll be focusing on Web Made Movies – bringing in external projects, charting a course with developers, and collaborating with filmmakers to bring some storytelling innovation into open video on the web. In addition to Web Made Movies, I’ll be developing a series of documentary shorts that tell the story of Mozilla: our goal is to share the passion of the Mozilla community with the broader public – to let people know that there is a vibrant non-profit behind the browser. I want people to be as intrigued as I was to discover that Mozilla’s exists to Protect the Open Nature of the Internet - a Greenpeace for the online commons. Did you know we even have a Manifesto? We want to grow to a worldwide movement, made of millions of Internet users who act as stewards of the open web. We think storytelling is a means to this end.

Come the new year, I’ll be moving into my role as a project producer for Drumbeat. Along with an amazing team that includes my esteemed European colleague Henrik Moltke, Drumbeat ringleader Matt Thompson, Web Developer extraordinaire Paul Osman, Fundraising and outreach queen Chelsea Novak and of course our fearless leader - the Mozilla Foundation’s Executive Director Mark Surman,I will be actively seeking out promising projects that Drumbeat can help foster. Projects that invite participation from both software developers and people that aren’t normally involved in the development process: teachers, cartographers, musicians, chefs, you name it (ok, maybe no spammers). My job is to get more people involved in the cause of keeping the web open, and doing it the Mozilla way – by creating things. It’s pretty humbling to be in on the ground floor of such an important project.

Over the last several years, my wife Shelley and I have dreamed of moving to the BC coast where we come from. People often ask me “can’t you do your job from anywhere?”. The truth is, remote working is pretty tough in most situations. I’ve needed (and enjoyed) constant interaction with other filmmakers, funders, broadcasters, and a thriving urban culture to be able to survive as a working filmmaker. At Mozilla, though, working from home is a reality for a large part of the team, and the systems and tools they’ve developed over the years to facilitate the world’s largest open source project make it work (although it’s a learning curve to be sure). So we’re taking the plunge, and come this fall will be moving from the island of Montreal to Galiano Island where I grew up. It’s a fantastic place for kids, and our daughter Layla loves running around outside without her parents hovering over her 24/7. Next year come by when our garden is churning out the good stuff! (and if anyone knows how to set up a microwave mesh network between islands, give me a shout).

Of course, it is going to be tough to leave our friends in Montreal, particularly my crew at EyeSteelFilm. Since graduating University, and even before, I’ve worked with my colleagues on an amazing batch of films – assistant editing on Squeegee Punks in Traffic, and then shooting and editing on Roach’s follow up, RoachTrip. I went to the Arctic as a co-creator of Inuuvunga: I am Inuk, I Am Alive, took a tour of duty in the editing suite on Chairman George, learned drupal and humility on homelessnation.org, and for six plus years was given sage advice, tough love and incredible teamwork in the creation of Rip! A Remix Manifesto. There are so many great people working at what has become the undisputed coolest documentary production company in Canada, if not the planet earth. I wish the entire crew the brightest of futures, not that they need this nerd’s wishing to get them anywhere. The new slate of projects and new crop of never-say-die indy filmmakers will continue to slay pitching competitions, film festivals and television screens the world over.

I moved to this city in 1997 – the world wide web was in its infancy. Back then, Apple was an underdog, Google was running on pentiums at Stanford, and Netscape was kicking serious ass. The web continues to surprise us, and not a day goes by that I don’t marvel to be living in such interesting times. As I start a new chapter, with a new team and and a new family, I can’t wait to see what happens next. Won’t you join me?

August 19, 2010 01:54 AM

August 17, 2010

Scott Downe

No more exceptions!

Yesterday’s post, I was talking about an exception I was receiving while trying to change an HTML5 video’s current time. I ended up solving it by putting it into an event listener, which worked, and made me suspect it was a loading issue, as event listeneres either fire after a delay of some sort. In the case of my previous solution, I would fire during “ontimeudate”. Looked like this:

<script>
  var update = function() {
    if (document.getElementById("video").currentTime < 2)
    {
      document.getElementById("video").currentTime = 2;
    }
  };
  document.getElementById("video").setAttribute("ontimeupdate", "update();");
  document.getElementById("video").play();
</script>

This worked, but was not the best way.

After getting advice from a blog comment, a link from Dave, and a demo. The blog comment mentioned what I was suspecting, which had to do with loading. The link from Dave was the firefox source of what caused the error. This ended up saying a mDecoder was null, which would throw the error. The mDecoder was being set inside a function called “FinishDecoderSetup”, which may be called after something is loaded… ok, I will admit this code is too much to completely understand in one shot, but it does contain all my answers, so I will use it as a worst case scenario. For now, I will try the easier leads, and work my way down. Now, the demo’s source code basically was using currentTime while inside the ‘loadedmetadata’ event listener.

<script>
  document.getElementById("video").addEventListener('loadedmetadata', function() {
    this.currentTime = 3;
    this.play();
  }, false);
</script>

Oddly enough, I thought I tried this… but something external was probably mucking up the works. Lesson to be learned here is make reduced test cases to test, help you understand, and to help others understand you.


August 17, 2010 02:51 PM

August 16, 2010

Scott Downe

HTML5 mediaElement.currentTime error

Recently, while working on a html5 video demo, I ran into an exception I could not figure out.

What I am doing is simply trying to change the video’s current time, but kept receiving an INVALID_STATE_ERR exception… It’s hard to get info on this, but I found “Will throw an INVALID_STATE_ERR exception if there is no selected media resource” in this page. Not much help, as I knew for a fact the media resource existed, but whatever selected media meant, I don’t know yet.

Here is an example of the problem:

<script>
  document.getElementById("video").currentTime = 2;
  document.getElementById("video").play();
</script>

Something as simple as this causes the exception, and I still don’t know why… but I did fix it.

A fix looks like this:

<script>
  var update = function() {
    if (document.getElementById("video").currentTime < 2)
    {
      document.getElementById("video").currentTime = 2;
    }
  };
  document.getElementById("video")
    .setAttribute("ontimeupdate", "update();");
  document.getElementById("video").play();
</script>

All I do here is modify the currentTime form inside an event listener, and it works…

I think this has to do with what parts of the video are not yet loaded.


August 16, 2010 03:50 PM

July 11, 2010

Scott Downe

popcorn.js + processing.js + box2d.js

I started a little demo Friday night, which is why I didn’t do a blog post yesterday even though I said I would do one a day.

A blog a day is just a bar I set, that even if I fall short of it, every other day is still pretty good, better than none at all, and I still plan on shooting for one a day.

What this demo is, is just a simple hack of processing.js, popcorn.js and box2d.js, and a previous demo made by Andor Salga, (which is currently broken) which I stole,  and a video made by filmmaker Brett Gaylor of WebMadeMovies who is working closely with me during the creation of popcorn.js.

I had to add a small hack into popcorn.js, and do some fixing to allow subtitles to load on a custom div, instead of generating a default one, but it still generates a default if no target is specified.

Not much else to say, so check out the demo, you might need to refresh if subtitles don’t appear the first time (best to let the page fully load before hitting play, it seems)


July 11, 2010 03:30 PM

July 08, 2010

Scott Downe

Popcorn.js

A project I have been working on is an HTML video library called Popcorn.js. A demo that was put together, this is the demo that is being shown tomorrow at Mozilla summit 2010, so if you are an attendee, you might want to hold off on spoiling the demo ;)

What this is, at heart, is an XML grammar, that defines various “commands” that can be used with the video, based on the video’s current time. So you can load a google news article about whatever you define in the xml tag, at a certain time in the video. For example: &lt;googlenews in=”00:00:05:02″ out=”00:00:10:07″ target=”googlenewsdiv” topic=”Village Telco”/&gt; googlenews is the keyword for the command type, in is when the news articles can be loaded, and out is when it should leave. Target is where you want to put the article, and topic is the search topic to be used in the article. Currently we have 11 “commands”, including:

Some of my favorites are:

Some tags, like the example above is self closing &lt; /&gt and others, like subtitle, contains the text between &lt; &gt;&lt;/ &gt.

Some things I think we did wrong, concerns, things I want to add and that we plan on changing in 0.2 is:

If you want to get involved, we use lighthouse to track bugs, and you can find us on irc under irc.mozilla.org/popcorn. Come and share your ideas.

One final note, while adding subtitles to the top most demo, we got into an argument about what was being said. If you want to help the cause, you can check out 1:48 of the demo, and give me your opinion on what he’s actually saying.

Check out what another developer on this project is saying here.


July 08, 2010 01:39 AM

June 17, 2010

Brett Gaylor

Early Web Made Movies wireframes

We’ve been working with the ultra-talented Gopal Raju to design some wireframes for the demo we’ll be launching at the end of the month.  Feedback appreciated!

Version 1

Version 2

June 17, 2010 01:47 AM