Skip to content

Guide to video in p5.js

Jump to bottom Edit New page
Kevin Siwoff edited this page May 2, 2016 · 11 revisions

#Guide to video in p5.js

This document should get you started with video in p5.js. To run the examples, you'll need video files on your local machine. Feel free to use your own files, or you can download open source video from:

A note on video format: To play video on the web, we'll need the files to be in mpeg4/h264/webm/ogg formats. If you have Adobe Creative Cloud on your computer, you can use Adobe Media Encoder to transcode video formats. There are a number of open source alternatives for video transcoding as well. The most popular open source media transcoder is ffmpeg (https://www.ffmpeg.org/).

For more information regarding supported video formats on the web, please see: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats

Want to get started really quick and don't have time for video conversion? Let's use this file for now.

https://dl.dropboxusercontent.com/u/882054/anni003.mp4

Hello World Video

Let's start with a simple example using the createVideo() method. This creates a DOM element. Since we're not drawing anything to our canvas, we don't need our draw() method. Everything we need is taken care of in our setup.

/*
 * @name Hello World Video
 * @description <p>Load a video and create a DOM element for it to play inside</p>
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var space;

function setup() {
  // specify multiple formats for different browsers
  space = createVideo(['media/anni003.mp4']);
  space.loop(); // set the video to loop and start playing
  space.volume(0);//we set the volume to 0 because we don't want
                  //sound
  space.showControls();
  //a centered video element
  space.position(windowWidth/2 - (space.width/2),windowHeight/2 - (space.height/2));
}

##Video Data Let's say we want to retrieve some information from our video, like time code, or duration. To do this, we simply call: space.time(); and space.duration()

Our example now looks like this:

/*
 * @name Video Data
 * @description <p>Load a video and create a DOM element for it to play inside</p>
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var space;

function setup() {
  createCanvas(windowWidth,windowHeight);
  // specify multiple formats for different browsers
  space = createVideo(['media/anni003.mp4']);
  space.loop(); // set the video to loop and start playing
  space.volume(0);//we set the volume to 0 because we don't want
                  //sound
  space.showControls();
  //a centered video element
  space.position(windowWidth/2 - (space.width/2),windowHeight/2 - (space.height/2));

  textSize(32);
}
function draw(){
  background(255);
  text("current video time: " + space.time(), 20,32);
  text("video duration: " + space.duration(), 20,64);
  
}

##Video Cues Now that we're warmed up, how about we set some cues to pass callback events at certain breakpoints. To do that, we'll need the addCue() method like this:

space.addCue(time, callback, callbackParameter );

In a full example, let's change the background color of our sketch at 3 seconds, 4 seconds and 6.5 seconds.

/*
 * @name Video Cues
 * @description <p>Load a video and create a DOM element for it to play inside</p>
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var space;
var bgColor;

function setup() {
  bgColor = color(255,255,255);
  createCanvas(windowWidth,windowHeight);
  // specify multiple formats for different browsers
  space = createVideo(['media/anni003.mp4']);
  space.loop(); // set the video to loop and start playing
  space.volume(0);//we set the volume to 0 because we don't want
                  //sound
  space.showControls();
  //a centered video element
  space.position(windowWidth/2 - (space.width/2),windowHeight/2 - (space.height/2));

  textSize(32);
}
function draw(){
  background(bgColor);
  text("current video time: " + space.time(), 20,32);
  text("video duration: " + space.duration(), 20,64);
  
  space.addCue(3.0, changeBgColor, color(123,0,23));
  space.addCue(4.0, changeBgColor, color(0,124,23));
  space.addCue(6.5, changeBgColor, color(23,0,125));
  
}

function changeBgColor(col){
  bgColor = col;
}

Another helpful function for adding an event handler to the video completion event is onended() and works like this: ```space.onended(handler)``

##Video Canvas Example So far, we've read data from our video and added a few cues. We haven't manipulated the video pixels at all. Let's go ahead and draw our video element on our canvas.

/*
 * @name Video Canvas
 * @description <p>Load a video with multiple formats and draw it to the canvas.</p>
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var space;

function setup() {
  createCanvas(windowWidth, windowHeight);
  // specify multiple formats for different browsers
  space = createVideo(['media/anni003.mp4']);
  space.loop(); // set the video to loop and start playing
  space.hide(); // by default video shows up in separate dom
                  // element. hide it and draw it to the canvas
                  // instead
  space.volume(0);//we set the volume to 0 because we don't want
                  //sound
}

function draw() {
  background(150);
  image(space,10,10); // draw the video frame to canvas
  image(space,150,150); // draw a second copy to canvas
}

##Video Pixels Example

We can manipulate video images on a per pixel basis. Pixel manipulation can be extremely powerful, but might also be heavy on the browser if there are a lot of pixels to loop through. As such, it's generally better to loop through smaller video sizes than larger to avoid crashing the browser. Try recreating the effect Michel Gondry uses in The White Stripes' Fell in Love With a Girl

/*
 * @name Video Pixels
 * @frame 320,240
 * @description <p>Load a video, manipulate its pixels and draw to canvas.
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var fingers;

function setup() {
  createCanvas(320, 240);
  // specify multiple formats for different browsers
  fingers = createVideo(['media/anni003.mp4']);
  fingers.loop();
  fingers.hide();
  noStroke();
  fill(0);
}

function draw() {
  background(255);
  fingers.loadPixels();
  var stepSize = round(constrain(mouseX / 8, 6, 32));
  for (var y=0; y<height; y+=stepSize) {
    for (var x=0; x<width; x+=stepSize) {
      var i = y * width + x;
      var darkness = (255 - fingers.pixels[i*4]) / 255;
      var radius = stepSize * darkness;
      ellipse(x, y, radius, radius);
    }
  }
}

And here's a more complex example of per pixel manipulation:

/*
 * @name Video Pixels
 * @frame 320,240
 * @description <p>Load a video, manipulate its pixels and draw to canvas.
 * <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p>
 */
var fingers;

function setup() {
  createCanvas(windowWidth,windowHeight);
  // specify multiple formats for different browsers
  space = createVideo(['media/anni003.mp4']);
  space.loop(); // set the video to loop and start playing
  space.volume(0);//we set the volume to 0 because we don't want
                  //sound
  space.hide();
  noStroke();
  //fill(0);
}

function draw() {
  background(255);
  space.loadPixels();
  //var stepSize = round(constrain(mouseX / 8, 6, 64));
  for (var y=0; y<height; y+=25) {
    for (var x=0; x<width; x+=25) {
      var i = y * width + x;
      var darkness = (255 - space.pixels[i*4 % (space.pixels.length / 4)])/255;
      var radius = darkness*25;//stepSize * darkness;
      fill(
        space.pixels[i % space.pixels.length],
        space.pixels[i*2 % space.pixels.length],
        space.pixels[i*3 % space.pixels.length]
      );
      ellipse(x, y, radius, radius);
    }
  }
}

##Webcam video Video streams are not only limited to video files. With the createCapture() method, we're able to "capture" video from our laptop or connected webcam. Here's a simple example:

/*
 * @name Video Capture
 * @frame 710,240
 * @description <p><em><span class="small"> To run this example locally, you will need the
 * <a href="http://p5js.org/reference/#/libraries/p5.dom">p5.dom library</a>
 * at least one video file, and a running <a href="https://github.com/processing/p5.js/wiki/Local-server">local server</a>.</span></em></p><br><br>
 * Capture video from the webcam and display
 * on the canvas as well with invert filter. Note that by
 * default the capture feed shows up, too. You can hide the
 * feed by uncommenting the capture.hide() line.
 */
var capture;

function setup() {
  createCanvas(390, 240);
  capture = createCapture(VIDEO);
  capture.size(320, 240);
  //capture.hide();//uncomment this to hide the video DOM element
}

function draw() {
  background(255);
  image(capture, 0, 0, 320, 240);
  filter('INVERT');
}

##Webcam Pixels

var capture;

function setup() {
  createCanvas(720, 360);
  capture = createCapture(VIDEO);
  capture.hide();
}

function draw() {
  background(255);
  image(capture,0,0);
  capture.loadPixels();
  for(var y=0; y<capture.height;y+=10){
    for(var x=0; x<capture.width;x+=10){
      var i = y * width + x;
      var darkness = 255 - capture.pixels[i*4];
      fill(darkness);
      ellipse(x, y, 10, 10);
    }
  }
}
Add a custom footer
Add a custom sidebar
Clone this wiki locally