Video Comments – Revisited

A few years ago, John Schimmel and I worked on an in-time commenting system for video. Specifically we made a WordPress Plugin that interfaced with the built-in WordPress commenting system including user authentication, spam prevention, and so on.

Unfortunately, it no longer works out of the box because we used the QuickTime plugin for video and support for that is waning in the browser space.

Yesterday, I did a quick and dirty update to allow the plugin to use HTML5 video rather than QuickTime. To my delight, it mostly works: Video Commenting Test (try in Safari or Chrome as the video is MP4/h.264).

What still needs to be done is to update the Admin interface to allow multiple video sources and mime type selections for HTML5 video and removing the QuickTime specific portions.

Also, I would love to put an HTML5 canvas on top of this and let people make spacial in-time!

If you are interested, I put it up on GitHub (make sure you use the html5 branch). Pull requests are welcome!

Android Live Streaming courtesy of JavaCV and FFMPEG

For the last little or should I say, long while I have been working on wrangling a solution for live streaming from Android that is both decent quality and extensible. For those of you interested, the litter in my GitHub account documents various previous attempts.

As far as I can tell, most folks that are streaming live video from Android are relying upon the video captured by the MediaRecorder and underlying classes and doing a bit of trickery with the file either while it sent to the server or on the server for distribution. This is fine but it doesn’t give hooks into the actual video frames before they are encoded and sent over the network.

Recently, I came across Samuel Audet’s amazing open source JavaCV project. In that project Samuel is wrapping FFMPEG’s underlying libraries (avcodec, avformat, and so on) using his equally amazing JavaCPP project to expose their functionality to any Java application.

Finally, after a few weeks of experimentation and little (actually a LOT) of help from Samuel himself, I have something working!

Running the live streaming app on a Galaxy Camera
Running the live streaming app on a Galaxy Camera

App and resulting stream on desktop via Wowza Media Server and Flash

There is a quick example of writing a file up on the JavaCV site which provides the foundation: https://code.google.com/p/javacv/source/browse/samples/RecordActivity.java

I have the beginnings of a full blown project (which needs some updating based on the above example) up on GitHub: https://github.com/vanevery/JavaCV_0.3_stream_test

Re: Networked Video in 10 Years : Networked Video == Parseable Video | Not sLop

Interesting, I just got some comment spam on this post from January 2007: Networked Video in 10 Years : Networked Video == Parseable Video | Not sLop.

In the post, I describe the proceedings from a breakout group at that year’s Beyond Broadcast conference.  My conclusion was that online video needs to be more than just video online, that it needs to be parseable (indexed and hyper-linkable and so on).

Unfortunately, for the most part, online video now, is pretty much the same as it was then.  Typically it exists on a web server as a file, is embedded in a web page with a bit of textual information around it and that’s it.  Not a lot of interactivity or time based meta-data as part of it.  Certainly not parseable in the way described in the post.  No easy way to link to specific content or to associate content on the page with any particular point in time in the video.

Fortunately, while that is still mostly the case, it isn’t always the case.  The good folks at Mozilla have been working on an open source JavaScript library called Popcorn.js that allows any time based media (audio/video) to execute code, manipulate a page, display other content and so on.   They have even created a GUI interface so you don’t have to be a JavaScript programmer in order to take advantage.  Nice!

I spent last week, during ITP’s Teach Yourself JavaScript Together, getting familiar with Popcorn and then gave a workshop that showed it (as part of an overall HTML5/JavaScript media workshop).  If you are interested, here is the talk (jump in a little more than 2 minutes):

YouTube Link and the notes are here.

Bringing in a generic H.264 stream to Wirecast (via Wowza and Wirecast’s Generic IP Camera support)

Wirecast is truly a studio in a box. It has a great support for multiple cameras, mixing live and recorded sources, graphic overlays and so on. Recent versions even allow you to bring in live feeds from IP cameras including support for specific Axis cameras.

Since I am a big fan of IP cameras and Axis in particular this is great news. Unfortunately Wirecast doesn’t have direct support for most models and I had to dig quite a bit to get things to work using their “Generic” IP camera support.

First test was to get a straight H.264 encoded into Wowza and then out to Wirecast. To do this, I used the Flash Media Live Encoder and set it publish to “rtmp://localhost:1935/img” (I have Wowza running on my local machine and an application called “img” which is a copy of Wowza’s “live” application). I set the Stream name in FMLE to “media.sav” which is what Wirecast is looking for by default.

In Wirecast’s Source Settings, I added a new IP Camera and set it’s IP address to: “127.0.0.1:1935” and choose “Generic” as the type.

Viola! It works, the video is being captured and encoded by FMLE, sent to Wowza and pulled into Wirecast as a Generic IP camera. In this manner, I can have live cameras via FMLE from anywhere in the world brought live into my final stream.

(Big thanks to Steve McFarlin the developer of the LiveU iPhone broadcasting app for his post on Wirecast’s Forum detailing how he got his software working.)

The Listserve Responses @thelistserve

Almost a month ago, I was the lucky person who got to write The Listserve message of the day.

In my message I talked about email and I asked people to respond in the hopes that I could engage people in the medium. Well, I am writing now to say it worked, sort of.

First of all, I got more than 2000 messages in response! A number so large that I haven’t actually been able to deal with them in a meaningful way. Many of them were simply replies but many of them were much more that I feel compelled to follow up with. (I think this may represent one of the problems with email ;-))

Part of my problem is that the reply address assigned to the message by the good folks the run The Listerve (shawn.van.every at thelistserve.com) was not not the same as the address I asked people to email (thelisterve at walking-productions.com), instead it had an autoreply. A week or so later they changed it so that the messages to that address were forwarded to me. They also forwarded me all of the messages that they had to that address as well. It now appears as though I have a bunch of duplicate messages.

This is the type of hiccup I really should have expected but I didn’t really think it through. I suppose if I had, I wouldn’t have done what I did and I would have missed out on the fun (and I am serious, reading these messages has been fun). When I get a chance, I hope to pare the number of messages down to unique senders and do another tally.

In any case, if you are someone who sent me a message, I am sorry for the delay and I am still working on the stats but for now I can say that roughly 2,000+ folks out of the 20,000+ on the list replied. A solid 10%!

I will do my best to send a response with the stats.

The Secure Smart Camera App for Human Rights Video : Video For Change :: A WITNESS blog

Bryan at WITNESS put up a blog post concerning the app that I am working on along with other Guardian folks.

The Secure Smart Camera App for Human Rights Video : Video For Change :: A WITNESS blog.

It’s worth a look if you are interested in the intersection of human rights, mobile technology and citizen media. It’s an open source Android project too!

Flash Media Server Sending Images

One of my students in my live web class is developing an interesting application that sends screenshots to other people. I put together some sample code to help him along and thought this would be of general interest.

Using the Flash Media Server with Remote Shared Objects this can be built. Here is a walk through of the code:

First of all, this uses the JPGEncoder class from the AS3CoreLib so you probably want to grab that and import it.

	import com.adobe.images.JPGEncoder;

This example uses a SharedObject, a NetConnection and NetStreams for sending the video as well as the screen shots. Once the NetConnection is established, the SharedObject can be setup:

        // Listener for connection
        private function connectionHandler (e:NetStatusEvent):void 
        { 
        	// If we are connected
             if (e.info.code == "NetConnection.Connect.Success") 
             {
				// Set up the shared object
				// We'll call it SimpleSO, pass in the app url and not make it persistent
				sharedObject = SharedObject.getRemote("SimpleSO",nc.uri,false);
				
				// Tell the shared object to call methods in this class if requested
				sharedObject.client = this;				
				
				// Add a listener for when shared object is changed
			   	sharedObject.addEventListener (SyncEvent.SYNC,syncEventCallBack); 

				// Connect the shared object to our netConnection
				sharedObject.connect(nc);
				
				// All of the video streaming setup
				doEverything();
				
				// Register mouseclicks, how we'll determine when to send a frame
				stage.addEventListener(MouseEvent.MOUSE_DOWN, saveFrame);
             } 
        }

Here is the method that is called when the mouse is clicked. It creates a bitmapdata object, encodes as a JPEG and sends that as a bytearray through the shared object:

        private function saveFrame(e:MouseEvent):void
        {        
        	// Save the frame to the bitmapdata object
        	var bmpd:BitmapData = new BitmapData(320,240);
        	bmpd.draw(videoIn1);
                	
        	// First encode as JPEG so it is smaller
			var jpgEncoder:JPGEncoder = new JPGEncoder(100);
			var jpgByteArray:ByteArray = jpgEncoder.encode(bmpd);
			
			// Send it via the shared object
			sharedObject.send("newBitmap",jpgByteArray);
        }

The SharedObject.send method calls the function “newBitmap” on everyone who is connected and passes in the jpgByteArray. The newBitmap function uses the Loader to uncompress the JPG and when it is done calls “gotBitmapData”:

		public function newBitmap(jpgByteArray:ByteArray):void
		{
			var loader:Loader = new Loader();
			loader.contentLoaderInfo.addEventListener(Event.COMPLETE, gotBitmapData)
			loader.loadBytes(jpgByteArray);
		}

gotBitmap data just creates a regular Bitmap and displays it:

	private function gotBitmapData(e:Event):void
	{
		var decodedBitmapData:BitmapData = Bitmap(e.target.content).bitmapData

        	// Create the bitmap image
        	var bmp:Bitmap = new Bitmap(decodedBitmapData);
        	
        	// Add it to the stage
        	bmp.x = 0;
        	bmp.y = 240;
        	addChild(bmp);			
	}

Here is the full AS3 class (it could be improved but it works):

package
{
	// Import JPEGEncoder Class from: http://code.google.com/p/as3corelib/
	import com.adobe.images.JPGEncoder;

	import flash.display.Sprite; 
	import flash.display.MovieClip; 
	import flash.events.NetStatusEvent; 
	import flash.net.NetConnection; 
	import flash.net.NetStream; 
	import flash.media.Camera; 
	import flash.media.Microphone; 
	import flash.media.Video; 
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.events.MouseEvent;
	import flash.utils.ByteArray;
	import flash.net.SharedObject; 
  	import flash.events.SyncEvent; 
  	import flash.events.Event;
	import flash.display.Loader;

	public class VideoCapture extends Sprite
	{
		// Shared Object for communication
   		private var sharedObject:SharedObject;

 		// Overall NetConnection for communicating with FMS
        private var nc:NetConnection; 
        
        // RTMP URL, same as directory on FMS
        private var rtmpURL:String = "rtmp://xxxx/webcam";
         
        // NetStreams for each stream 
        private var netStreamOut:NetStream; 
        private var netStreamIn1:NetStream; 
        
        // Camera
        private var camera:Camera; 
        // Microphone
        private var microphone:Microphone; 
                
        // My Video
        private var videoOut:Video; 
        
        // Video Components
        private var videoIn1:Video; 
        
        // Stream Names
        private var outStream:String; 
        private var inStream1:String; 

        public function VideoCapture() 
        { 
        	// Construct NetConnection and connect to server
             nc = new NetConnection(); 
             nc.connect(rtmpURL); 
             
             // Add a listener for connection
             nc.addEventListener (NetStatusEvent.NET_STATUS,connectionHandler); 
        } 
        
        
        // Listener for connection
        private function connectionHandler (e:NetStatusEvent):void 
        { 
        	// If we are connected
             if (e.info.code == "NetConnection.Connect.Success") 
             {

				// Set up the shared object
				// We'll call it SimpleSO, pass in the app url and not make it persistent
				sharedObject = SharedObject.getRemote("SimpleSO",nc.uri,false);
				
				// Tell the shared object to call methods in this class if requested
				sharedObject.client = this;				
				
				// Connect the shared object to our netConnection
				sharedObject.connect(nc);
				
				// All of the video streaming
				doEverything();
				
				// Register mouseclicks, how we'll determine when to send a frame
				stage.addEventListener(MouseEvent.MOUSE_DOWN, saveFrame);
             } 
        }
        
        // Gets the results from the server
        private function doEverything():void 
        { 
        	// Name of streams
			outStream="one"; 
			inStream1="one"; 
        
			// Setup the camera        
			camera = Camera.getCamera(); 

			// setup the microphone             
			microphone = Microphone.getMicrophone(); 

			// Video components
			videoOut = new Video(); 
    		videoIn1 = new Video();

			// Set positions 
			videoOut.x = 0;
			videoOut.y = 0;
			videoIn1.x = 320;
			videoIn1.y = 0;

			// Add them to the screen
			addChild(videoOut);
			addChild(videoIn1);

            // Publish our stream
            netStreamOut = new NetStream(nc); 
            netStreamOut.attachAudio(microphone); 
            netStreamOut.attachCamera(camera); 
            netStreamOut.publish(outStream, "live"); 
             
            // Attach camera to our video 
            videoOut.attachCamera(camera); 
             
            //Play incoming streamed video 
            netStreamIn1 = new NetStream(nc); 

			videoIn1.attachNetStream(netStreamIn1);

			netStreamIn1.play(inStream1);
        }
        
        private function saveFrame(e:MouseEvent):void
        {        
        	// Save the frame to the bitmapdata object
        	var bmpd:BitmapData = new BitmapData(320,240);
        	bmpd.draw(videoIn1);
                	
        	// First encode as JPEG so it is smaller
			var jpgEncoder:JPGEncoder = new JPGEncoder(100);
			var jpgByteArray:ByteArray = jpgEncoder.encode(bmpd);
			
			// Send it via the shared object
			sharedObject.send("newBitmap",jpgByteArray);
        }
        
		
		public function newBitmap(jpgByteArray:ByteArray):void
		{
			var loader:Loader = new Loader();
			loader.contentLoaderInfo.addEventListener(Event.COMPLETE, gotBitmapData)
			loader.loadBytes(jpgByteArray);
		}
		
		private function gotBitmapData(e:Event):void
		{
			var decodedBitmapData:BitmapData = Bitmap(e.target.content).bitmapData

        	// Create the bitmap image
        	var bmp:Bitmap = new Bitmap(decodedBitmapData);
        	
        	// Add it to the stage
        	bmp.x = 0;
        	bmp.y = 240;
        	addChild(bmp);			
		}
	} 
}

ITP Show in full swing

ITP’s end of semester show is all the rage at the moment. Been floating around playing with projects and taking some snapshots.

More soon but here is a teaser courtesy of Meredith Hasson’s Video Mosiac

And another courtesy of Nobuyuki Nakaguchi’s Breath Note

(I love the digital take home)

Online Video Player Survey

For a project that I am working on, we are doing a quick survey to see what features and functionality people are interested in with regards to online video players. This doesn’t pretend to be an exhaustive survey of the various players out there, rather it is something that we will be using to gauge what people like and dislike in the range of features.

If you have a moment, check it out:
Online Video Player Survey