With the Oculus store continuing to get some interesting titles while I wait with baited breath over getting my very own Oculus Touch Controllers when they get released, its easy to forget about 360 Video. Some say 360 Video is not VR at all like Vive engineer Alan Yates:
The problem Mr. Yates seems to have is that 360 Video chops off some key VR immersion factors like being able to move your body (not just your head) as well as being able to interact with the scene and have the scene interact with you.
In addition to lacking some key things that make VR the immersive thing it is when you think about content like games, it can also box in the content creators to doing things that are less than ideal in VR. A big one here is knowing that when in VR, an experience can induce a little motion sickness by moving the user against their will. Another is dropping a user into a space leaving them disoriented and confused while they figure out where they are. 360 Video continues to suffer from this as traditional video creators make the choice to pan their cameras, or don't pay enough attention to their viewer when cutting between scenes.
All that said, whether it is or isn't VR, it's certainly an emerging art form. It's enough like traditional video to be approachable for longtime video creators, but also deceptive in that these same creators need to question fundamental techniques or risk creating bad or even vomit-inducing experiences.
It had been a while since I last spent a couple hours enjoying 360 Video, so yesterday afternoon I decided to go back in. Things have improved, and there are some fascinating shorts being produced as well as not so fascinating shorts with gimmicks that may or may not work. Let's talk about them.
Mr. Robot - Written and Directed by Sam Esmail #
Runtime: 13:03
Filesize: 586MB
Kicking things off with Mr. Robot is a bit unfair. Not only do I love the TV show, but of the shorts I've seen so far I like this one the best. It uses a number of gimmicks that don't feel gimmicky, and breaks some other rules in a way that feels OK.
Also interesting, is that many of the best videos that I've seen and I want to talk about are directed or co-directed by Chris Milk who runs a VR Storytelling company called Within (and then posted on the Within Oculus channel). Despite Mr. Milk making some compelling shorts, Mr. Robot shines greater than any of them for me AND it is directed by original Mr. Robot creator Sam Esmail.
BTW...only read the rest of this one if you can handle Season 1 spoilers.
Elliot, the main character, turns out to have some major split personality disorder. Through season 1, Elliot's routinely expresses his thoughts to you, the viewer. It would be a bit gimmicky breaking the fourth wall like this all the time, except when you consider the possibility that the viewer is another personality or just a remnant of Elliot's troubled mind.
Elliot acknowledging the user
The 360 short uses this setup to it's advantage, and as you might expect it just so happens to be perfectly suited for VR. You enter the experience next to Elliot on his couch while listening to his internal thoughts (expressed as a voice over) lamenting about some past memories. In true Mr. Robot fashion, Elliot looks at you to acknowledge your presence occasionally. Turns out this works great for VR, too. The user having a presence (even if just an inactionable one) does great for immersion.
An interesting thing that happens early on is that the director wants to focus on the laptop in the kitchen. It's a bit weird, in that it feels like a throwaway story point that never really matters. That said, with traditional video, a director might edit the shot and cut over to the laptop. However, with 360 video we can't have hard edits like this that disorient the viewer, so instead the lights flicker in the kitchen around the laptop and the user's attention is drawn there.
Elliot also happens to be smoking a joint which presents some interesting opportunity for gimmick. Elliot takes a big puff and exhales at the camera which offers an opportunity to transition to a past memory. While this isn't necessarily a 360 video gimmick, what follows is him sitting in the exact same spot in his past memory. In fact the scene looks no different, which is an important point not to disorient the user's experience. The use of whiting out the scene with smoke serves to transition the story but not necessarily the set.
The marijuana use also provides a convenient way for the camera/viewer to get "high". As the marijuana starts taking effect, the camera starts floating to the ceiling offering a wider view of the shot and allowing Elliot's past love interest to enter. He even calls out "Oh wow, I got really high...look at us up here". Very important to reiterate here that Esmail coupled the camera transition with talking to the user about it in while SIMULTANEOUSLY pretending its part of the story.
Camera floats up as Elliot gets high
To further layer on this storytelling feat, the camera view slightly shifts the user to look at the door in preparation for his former love interest Shayla to walk in.
As Shayla knocks on the door, something a bit awkward happens which is that with each knock, the camera cuts slightly in every direction a few times. It feels like a mistake, but perhaps it was an opportunity to cover up a production mistake where the shots weren't aligned.
Shayla enters and wanders a bit around the room talking to Elliot. As she enters the bedroom, a light turns on and illuminates the room for a moment. To me it was a bit awkward and I couldn't find any purpose to it, but its over quickly.
As she wanders around the camera pans a bit, which is breaking a bit of a VR "rule" since you have no control over it - but it's done gently and after the initial camera floatation and other movements so doesn't feel wrong in the least. Here the full 360 comes into effect as Elliot stays on one side of you and Shayla walks behind you. You, the viewer are in the middle of this conversation and it feels like being in the middle of a group, turning your head each way to see each person speak.
Shayla and Elliot talking
After the two leave there are some establishing shots of the beach, the shut down amusement park that would be later used as a base of operation. In these shots there is some gentle movement. Again, it seems to break a bit of the VR rule of not forcing the user to move in a way they aren't actually moving their body - but here it feels right and I think making it a short establishing shot that's not integral to on-screen dialog is the right way to go.
More of the story happens when the video cuts to inside a ferris wheel car. As the video goes on, Esmail seems to be limiting the storytelling to slow paced enclosed areas with the dialog being a bit slow paced as well - more like what you'd find in real life, not fast moving heated dialog with fast cuts. Again, in the Ferris Wheel scene, you must turn to each character as they talk, much like you would do as the third and silent wheel in a real conversation sitting behind two people.
Shayla and Elliot in the Ferris Wheel
What's interesting here, is that I did previously watch this 360 video on a monitor using my mouse to pan around. I thought it was a bit boring, and I didn't make it past the opening title, judging it as another property to jump on the 360 bandwagon. But, here in VR, what didn't work on a screen is a great storytelling experience.
In the next scene, Elliot and Shayla are strolling along the boardwalk. Important to note here is that the camera is following them again, moving the user. Esmail didn't put any important dialog in this scene, only using the tone and mood to convey a story point (that Elliot and Shayla have made up after some cold-shouldering and are having a happy memorable time). I find this interesting to contrast with the slow pacing and slow conversations that are placed in static scenes. To get in Esmail's head a bit, I might be inclined he believes that the camera shouldn't be moving at all when you need the viewer to focus on a important bit of story. This scene itself transitions to a interesting colorful montage.
Happy montage
For sure, Esmail did lots of interesting things here. I'm sure I can rewatch again and again and find more. I do want to move on to other videos as interesting as Mr. Robot is. That said, I DO want to end Mr. Robot with one scene that really stood out for me, and that's when Shayla and Elliot are relaxing in bed. I am interested in 360 Video to offer perspectives that aren't seen in real life, and this bed scene touches upon that desire but doesn't go overboard. Check out the following shot with the camera positioned above the bed making them look upright. This is often used in TV/film and looks completely normal. In 360, however, it takes some getting used to. There's some dissonance while you figure out your orientation - but once you do, there's a bit of an aha moment that really adds to the experience. Other than the camera orientation, this scene is more of the slow, enclosed, conversational scenes that make the rest of the piece work well.
Shayla and Elliot in bed
Saturday Night Live: Jerry Seinfeld hosts a Q&A with Surprise Guests - Directed by Chris Milk #
Runtime: 8:00
Filesize: 787MB
To be clear, I really did like this video, which is surprising because I'm one of those people that think SNL has dropped way off in quality since
Since there's not much to say, its the perfect opportunity to note opening credits for most 360 Video I've seen. It's really no more than noteworthy, but opening credits and closing credits all seem to have standardized around displaying what they need to display in a narrow field of view, much like what you'd see on screen/TV and then replicating it 3 or 4 times over around your head so you can see the text and intro graphics regardless of where you look.
Opening credits and corner of repeated title screen
With that out of the way, we can talk the actual video. In this experience, the 360 camera looks like it's positioned just above the traditional TV camera. You are facing the host, Jerry Seinfeld, and can turn to look at the crew and the audience...the entire room.
Seinfeld Hosting
If you've never been an audience member for SNL before (I haven't), it's interesting to experience what it's like behind the camera. You can see camera operators, a photographer, the boom mic crane, what the set looks like, etc. Its fairly interesting.
Unfortunately, the action takes place fairly quickly right away and you have to start paying attention. Contrast this with other VR stories and 360 video. Typically you might want to give the user some time to acclimate to the scene before starting the action. Here, being in the SNL audience is interesting, but Jerry Seinfeld is on stage in just 15 seconds and starts delivering his monologue so I was a bit torn what I want to pay attention to.
If it was JUST watching a monologue intended for TV, this would be a disappointing experience. However, it turns into a sketch. Jerry launches into a Q&A with the audience who just happen to all be celebrities and/or past hosts.
Yes, its funny. And YES it makes use of the 360 experience. The viewer is immersed into the audience here, because you watch Seinfeld call on somebody in the audience, and you turn to see if you can find them. In this way, the video sort of acknowledges that you're there by making you do what you would have to do in real life as an audience member.
Here's where things break down, though, and its purely technical. Check out this shot of when James Franco asks his question:
James Franco asks a question
Can you even tell that's James Franco? More importantly, do you think it would be easy to identify his facial expressions and body language? Recognition is so important to this bit involving celebrity. And facial expressions and body language are key to comedy and acting. You might think this is an anomaly because he's a bit far away. After all, John Goodman is another feature and he's fairly recognizable but also fairly close (hint: he's just past the lady at the bottom center). Its a fair point if you're just looking at this image, but in the experience Franco looks fairly close...just blurry and not crisp enough in the encoding. As a viewer you feel like you SHOULD be able to see him better and it's imperative to the experience, but the detail of the capture and/or the encoding prevents this.
Oddly enough, Mr. Robot didn't suffer from this despite being longer and having less overall file size. This point is exactly why I'm prefacing each writeup with the duration and file size. This SNL video is closer to what you might expect from live 360 shooting without the benefit of planning and script to overcome these type of issues.
Last disappointing bit is that while it's interesting to see camera operators, the boom mic, etc, you can ALSO see the folks that hold the cue cards. It really detracts from the spontaneous and hilarious premise of this off the cuff Q&A to have cue cards right in front of you.
Center: Assistant holding a cue card for Larry David's question
All in all, this is a fairly successful 360 video. I quite enjoyed it, but where it falls down, it suffers because 360 isn't really the medium they intended to target with this bit.
Vice News VR: Millions March in NYC - Directed by Chris Milk & Spike Jonze #
Runtime: 8:08
Filesize: 1.07GB
Vice News, from my understanding, is a bit more gritty and risky/risque than traditional news media. When I come across Vice stories and videos I seem to recall reporters inserting themselves into dangerous scenes and doing raw reporting. Even I've put Vice in the wrong nutshell, that seems exactly what they are doing in this 360 video (though to be fair, a protest esp in NYC is probably not going to be dangerous at all). This protest in particular is to highlight the unfair treatment of black men and women at the hands of the US justice system and police.
One interesting thing done right off the bat is in the opening title. I criticized SNL's 360 video for not giving enough time for the viewer to get acclimated to the scene. Here it's just right, and they've incorporated the title of the piece in an interesting way. It's wrapped around your head with most of the title outside your field of view at all times. So, to read it, you must look all the way to the left and pan your head from left to right. Meanwhile, a crowd (protesters) appear below you.
Vice News Title Screen
Initially I panned this as a bad choice. But, after thinking about it, having to work across 270 degrees to read the text doubles as a mechanism to also take in the scene around you. Given that an Oculus user already had to click on the video thumbnail and download, having the title be legible again in the video is not as necessary as one might think. So, even if the user fails to read and struggles to take all the text in in one go, its still OK.
After the opening scene, we cut to being eye level right next to a demonstrator. This group of men chants "I can't breathe with y'all on my neck", of course a reference to the killing of Eric Garner.
Demonstrators chanting "Can't breathe"
What was interesting for me is my reaction to being right up next to the demonstrator. In real life, I tend to stay far away from demonstrations like this, whether it be Black Lives Matter, a subway musician, or someone in a city park on a microphone calling for sinners to repent. Reflecting, I think it comes down to one thing for me: how to react and what kind of body language am I using in response. For example, someone standing on a soap box saying the world will end tomorrow is super interesting. I don't take them seriously, of course, but I would love to hear their whole speech for the entertainment value. On the opposite end of the spectrum - a demonstrator like this, representing a real cause, I might like to hear what they are saying, but especially with myself being white and someone who historically COULD look down on them, I may be a bit self conscious of what type of message I'm sending to them with my reactions (or lack thereof) as I stand there and observe them.
I talked before about how 360 videos do well to acknowledge the viewer as part of the scene. Mr. Robot does this exceptionally well, and SNL with Seinfeld did this to a small extent. In a scene like this, NOT acknowledging the viewer seems to work exceptionally well. I can observe and actually listen to the message without worrying about my reactions or being self conscious.
In addition to watching demonstrators, I've never been part of a march. So it was interesting to be swept down the street passing chanting, banners, stalled traffic etc. As this is a camera, regardless of whether it's 360, if it's being carried somewhere, there needs to be an operator. While the cameraman usually stays perfectly behind the viewer through most of the video, he's bit close for comfort in this scene:
Cameraman, a bit too close for comfort
Like I said, its a bit disconcerting, but its 360 footage being capture in an uncontrolled environment. He can hardly be blamed!
In the next scene, we follow a reporter down the street to a "Die In". A few dozen people are lying on the ground to demonstrate dying in the streets. Unfortunately, the technology and more specifically the capture resolution/encoding failed here much like it did in the Saturday Night Live video. For starters it was night time, so the visibility wasn't great, and well...can you tell what is actually happening in this scene?
A "Die In" demonstration
This image, as in VR is extremely hard to comprehend. Its actually worse in VR because you feel like you're there and because of that you think you SHOULD be able to pick out the shapes on the ground as people and bodies. I was actually a little surprised when they started getting up because some of those people were white. I convinced myself that part of the reason that I couldn't make heads or tails of the scene was that a black man or woman's face at night lying on the ground would be surely be hard to see. But no in fact, there were many white men and women as well.
I'll end this video's analyzation with an interesting reaction I had to one of the protestors. In the same "Die In", as people were starting to get up, one lady got up half way, raising her hands and ended up on her knees. The reporter we were following crouched down next to her to interview and get her thoughts.
Reporter crouching to interview a demonstrator
What was interesting for me was my posture as this happened. Previously, I was sitting upright in my office chair as I watched this video. However, when the reporter crouched down and the camera followed, my entire posture in my chair went lower into a seated squat. I took note that it was an interesting thing that with enough direction from on screen cues, my body would follow!
Catatonic - Directed by Guy Shelmerdine #
Runtime: 8:09
Filesize: 481MB
Catatonic was a fun little horror experience akin to what you'd get when you and your friends drive for a couple hours on Halloween to a small rural town where someone has redone their barn or farm to be a spooky experience complete with actors jumping out at you to scare you.
This 360 video takes place in a run down insane asylum. Despite thinking it worked pretty well, it did what contemporary VR creators dictate you should not do: put the camera on wheels and roll around. I eluded to this before when some of the videos above had this to a lesser effect, and it harkens back to early VR tests when lots of people experimented with putting you on some kind of track like a rollercoaster. The movement depicted in your vision contrasted with the lack of movement felt in your body was named as the prime reason for feeling motion sick. So, of course, content creators name this as a prime thing not to do.
All that said, I personally felt fine despite being on a moving track the entire time. In the story, you are a patient being wheeled through the asylum in a wheel chair. In addition to being slow, you can also look down and see your body and chair. Perhaps this addition of a "reference object", or something that persists in the same place in your field of view, cancels out or minimizes the motion sickness.
In a wheelchair (reference object to reduce motion sickness?)[/caption]
Remember I talked about those Spooky Barns? Well, some people get scared by the things jumping out at you. Not me (or my wife for that matter), we see things coming maybe get a little surprised, but really just end up giggling at the attempt. Same here. The first thing you encounter is this zombie looking girl that kinda snarls and lunges at you as you roll past. I had the same reaction. Ironically, I was much more concerned as I was wheeled into the room that my knees would smash into the doorway (no seriously, it made me a bit uncomfortable).
Scary girl
All in all, it was more of the same. Interesting and fun no doubt, but not TOO much more noteworthy to speak of. I was wheeled past disturbing patients. Tricks of light and time dilation made things creepier as well.
One thing that really made me take notice of after I experienced it was the use of time and making me impatient to look around. There is a quite normal looking man that wheels you around for the first half of the experience. He even tries to soothe you in the beginning. But, he's behind you. It's an effort to look backward and up to take note that there's someone there. I think I only did it once out of curiosity.
However, an interesting thing happened. After a particularly fast paced period of time when lighting suddenly changed and time sped up for a second and things got creepy, there was a few seconds of complete inaction. I was left sitting in the chair standing still and nothing was happening. The scene had the effect on me to make me impatient and look behind me to figure out why I wasn't moving. It turned out the nice man was gone, and a scary hooded figure lurched out a door and took over. If I wasn't given time to get impatient (possibly after such an actionable few seconds), I would not have looked backwards (again it's awkward to do so) to see what I was supposed to see.
From there, the cheesy horror and effects picked up!
In real life I've been getting bloodwork lately, and I JUST CAN'T look at the needle going into my arm. It's too much...freaks me out. However, when presented with the following:
...I can't look away! I know it's not real, so I guess I feel empowered to watch it and laugh off my complete irrational freak out on needles.
And from then on its more good, cheesy horror with some personal bubble invasion thrown in to try to make you uncomfortable.
Invading personal space
So that's Catatonic! I figure if those cheap horror movies and Halloween barns scare you, this might as well. For me, it was good B-Movie fun.
New Wave - Directed by Samir Mallal & Aron Hjartarson #
Runtime: 2:17
Filesize: 159MB
This is a quick one! It has a bit of a novel approach, though. I'm not sure how well it works to be honest with you. The video opens on a beach. In front of you is a washed up boat. Really just a nice relaxing scene, and it holds there for around 40 seconds for the viewer to get acclimated. Again this seems fairly standard practice for VR Storytelling.
Prior to the narrative starting, a lady walks her dog directly in front of your view. My first couple of times through, the dog walking seemed a bit meaningless and odd. I ignored it waiting for something ELSE to start up. It turns out though, on my 3rd viewing, I noticed its a guiding action. It was a bit of action meant to make you follow your head behind you where the actual narrative started with the two main characters.
Walking the dog. A guiding action to the main story
So obviously this bit of direction missed the mark for me. Luckily, I hear a voice over narrative, and I know to look around for what was going on.
The interesting bit about this experience is the spatial audio. The setup is that this couple is fighting and go off to different areas of the beach. You can see each by turning your head, but also when you turn your head you can hear each of their thoughts...a narrative of their anger towards the other from their perspective.
Split View
Unfortunately, I didn't think this worked so well, because it took a long time in the short span of this video to figure out that there was different audio depending on where I looked. When I figured it out, I got a bit frustrated because I couldn't listen to both dialog at once and felt like I was missing things.
All that said, it was an interesting device to tell the story!
LoVR - Created by Aaron Bradbury #
Runtime: 5:20
Filesize: 273MB
LoVR is an interesting concept. Its all computer generated data visualization that you fly through, and it's about love. Aaron's description verbatim is this:
A story of love, told through neural activity. Chemicals are released, neurons are activated and a form of poetry is revealed within the data.
You know, to be perfectly honest, I'm not quite sure this needs to be in VR. I dig the concept of measuring the brain's neural activity and pinpointing the moment that falling in love happens. At that moment the music picks up and the dataviz starts getting extreme.
I want to guess that this experience was done with VR in mind, but the creator wanted to expand reach to flat screens as well so made an experience that could encompass both. Flying through the visuals is a nifty experience, but at the same time, not much of your periphery or even behind you matters.
All that said, its a nifty concept and video!
Baseline reading, not in love yet
Flying through, noticing beauty and starting to sweat a bit - music is picking up
Looking back on the chaos at the moment of love
Lowes Home Improvement #
Runtime:4:56
Filesize: ???
I'll end this post with a weird one. Despite having some negative comments on various aspects on all the 360 videos I talked about, the criticism is just to point out interesting decisions and aspects. Overall, the videos I watched were pretty damn great. 2016 is just the tip of the iceberg as well. 360 video will continue to evolve as an art form and I think we're mostly in the experimental stage of things right now. All of the above videos were from Within, and its certainly no mistake that a company founded on great VR Storytelling would produce and highlight great 360 video.
What I'm about to mention next isn't anything like that, but it has a unique take on instructional videos!
I've been to Lowes Home Improvement stores before for various projects, and they really do try to help guide you through your projects. Their website is no different. Having owned a home, I've gone through some of their instructional videos or tutorials a few times to make or fix something. It does help, for sure.
However, when your hands are full and you're trying to fix something while at the same time trying to scrub a video back or paginate the instructions back because you missed something...well it's a pain!
This video attempts to address that problem. I question the effectiveness as I wonder how unweildly wearing a headset (even if wireless like the GearVR) would be while trying to do home repair. All the same, its a bright idea!
This instructional video is to make a quick DIY cement table with wooden legs. Instead of introducing the steps over time, the steps are done in space. So step #1 is right in front of you. As you turn your head in 360 degrees you can see the rest of the steps. This makes it easy to go back and forth between steps you might not understand...just look the other way! Each step video is on a continuous loop so the action is incredibly easy to revisit.
Making a table
Finished!
And that's just a few.... #
This was a bigger blog post than usual, but it forced me to think through some very interesting pieces and evaluate what's good, what's bad, and just where we are technically and experience wise in 2016 for 360 video. I picked the few that I though were most interesting - so everything here I enjoyed and send my kudos to the creators. There are, of course, even more that are as interesting - but lots that fall flat as well. The most important thing to note is the fact that everyone is experimenting with what works and we are at the beginning of a new way of thinking about video. It will be quite interesting to see the caliber of stuff that 2017 brings!