Video: Interlacing Headaches

Well, this morning I had my wisdom teeth out, and my jaw hurts. So far, I haven't resorted to the oxy I was prescribed, because the ibuprofin seems to be doing the job. I'm really hungry though.

As a result, I've spent some time today doing some recreational video encoding. As previous posts may have lead you to believe, I'm fascinated by figuring out how to utilize my fair use rights to convert video from one form to another. Lately, I've received an unlikely, and perhaps unwitting, ally in the fight from Sony, who in a strange turn, have released a spiffy portable device capable of playing user-created kick-ass video.

One outcome of this is that there's a lot of interest in the blogosphere in how to convert video from DVDs into more portable forms. As I've mentioned before, this is possible, but there are practical and legal impediments. With a will, I set myself to the task of converting our new Incredibles dvd to a portable form. (Disney is notoriously stingy when it comes to video releases, so the prospects for an offical portable release seem dim.)

What I'd like to say is that if you're on a Mac, there's a great tool for starting this process called Handbrake. The old version (0.6.2) was great, the new beta (0.7.0-beta 2) looks even better. It comes very close to the ideal of inserting DVD, pressing go, and getting a useful video file that you can play on your computer, burn to CD, stream over your intranet, or put on a portable device. If you have to convert some DVDs to computer form, it's a great tool.

People who aren't interested in annoying technical nitty-gritty can go download it and stop reading here.

For the rest of you, I was perusing the Handbrake forums and came across this interesting tidbit.

I've noticed that while the individual deinterlaced frames look good, when you play them, it'll skip every two seconds or so (really noticeable in long pans). It looks really out of place. I've tried it in both XviD and FFmpeg (both at 1280 kbs) and still the same thing. Just wondering if there's any way to fix this. If you don't know what I mean I can upload a sample. I just really want to know what's going on here. Thanks.

To which I offered the following response, which I think gives some interesting insight in to just what kind of a hassle it is to deal with digital video from analogue sources behind the scenes.

The reason that this is happening is possibly because the original source material (pre-interlace) was at 24fps. When someone converts 24fps film into 29.97fps interlaced video (often referred to as 60 fields henceforth) they have a few options, but the one that is typically used is to spread the 24 fps video frames out over alternate groups of 3 or 2 frames by using the interlacing.

So, what often happens is that:

Source frame 1 is field 1,2,3
Source frame 2 is field 4,5
Source frame 3 is field 6,7,8
Source frame 4 is field 9,10
Source frame 5 is field 11,12,13
Source frame 6 is field 14,15

6 frames at 24 fps = 1/4 second.
15 fields at 60field/sec = 1/4 second.

This is pretty groovy, and looks pretty smooth, but (wait for it) here comes your problem. deinterlacing is pretty tricky stuff. I don't know anything about the guts of handbrake, but however it's converting 60 field/second video into 30 frame/second video, the result is going to look like this: (assuming the odd frames are dominant).

output frame 1 comes from field 1 and thus source frame 1
output frame 2 comes from field 3 and thus source frame 1
output frame 3 comes from field 5 and thus source frame 2
output frame 4 comes from field 7 and thus source frame 3
output frame 5 comes from field 9 and thus source frame 4
output frame 6 comes from field 11 and thus source frame 5
output frame 7 comes from field 13 and thus source frame 5
output frame 8 comes from field 15 and thus source frame 6.

Note that this introduces a duplicated frame every 5 frames. This is not too troubling normally, but it tends to make pans and smooth movement look a little jerky. It _might_ be possible to use a smart deinterlacing mechanism (called a 3:2 pulldown) to turn the video back into its original 24fps frames, but in my experience this is an approach fraught with perils, and to be avoided if possible.

Finally, because 29.97 frames per second is the reality instead of the cleaner 30fps, we'll have an extra hiccup every once in a unless the source clip was at 23.985 instead of 24fps.

Short answer: Try to get a copy of your source material that isn't interlaced in the first place, interlacing is the bane of rational, easy-to-understand, computer video manipulation.

Neat, huh?

Family: Ivy 1, Santa 0

So as I was driving home from playgroup tonight, with Ivy and Rose sitting in the back, Ivy suddenly begins an interesting conversation. Here is a partial re-enactment, for the benefit of history.

Ivy: Daddy, I have a friend at school who says that her Mom and Dad are Santa Claus for her.
Glenn: Ah.
Ivy: She's wrong, right?
Glenn: What do you think?
Ivy: "I think that she's wrong, and that Santa Claus comes to her house."
Glenn "Why do you think that?"
Ivy: Because.
Glenn: Ah.

A few minutes pass. At this point, I have two worries. The first of which is that Rose will rouse from sleep, overhear this discussion and get upset. The second is that Ivy will press the point further and become upset if I tell her the truth.

Ivy: Daddy, would you tell me the truth?
Glenn: Yes, but I want to ask you some questions. Why is this important to you?
Ivy: I just want to know the truth.
Glenn: Is it because you're curious? Or because you're afraid you won't like the truth?
Ivy: Curious, I think.
Glenn: Do you remember when you asked me about Santa's reindeer, and I said that they could fly because they were magic?
Ivy: yes.
Glenn: I also told you once that magic was "just pretend", and when I said that about the reindeer, you said 'But you said magic was pretend.' At the time, I asked you what you thought about that, and you said you didn't know. What do you think now?
Ivy: (with dawning smugness) I know! I know! Mommies and Daddies dress up in Santa clothes.

Interestingly, at this point, Ivy, rather than bursting into tears, became fascinated by the logistics of the illusion. Most of this centered around Mall santas and the fact that they looked different, which she confessed to having observed. But I thought the following was interesting:

Ivy: Do the mall santas carry the presents to the Mommies and Daddies?
Glenn: (laugh) No, the parents buy the gifts.
Ivy: But how can they go to the store at night?
Glenn: No, we buy them beforehand.
Ivy: Then we'd see them sitting on the counter!
Glenn: We hide them.
Ivy: Oh!

Finally, I asked her if the discovery made her sad, to which she said no. In fact, she was very excited to tell mommy that she was "in" on the big secret. I told her not to tell her sister, but I imagine that's a doomed effort and that Ivy will let the truth slip sometime as Christmas approaches. We'll see.

I share with you this final exchange we had while talking about my elaborate marble trail.

Ivy: (suddenly) You ate the cookies and milk!
Glenn: Yes, they were very tasty cookies.
Ivy: Thanks, daddy!

Politics: Fiddling while Rome Burns

I can not fucking believe that while our economy lies in near-ruins, our debt is skyrocketing out of control, and our soldiers are overseas at war, the most important elected officials in the United States are wasting their time destroying 200 years of jurisprudence to overrule the husband of someone who's been in a persistant vegetative state for 15 years.

Sure, the helpless have rights, and the courts and laws need to uphold them. But you can't convince me that this hasn't escalated beyond the rational. In fact, the courts have already had their say. But, here we have it, instead of going all-out to save the livelyhoods of millions of American citizens, or going all-out to save the thousands of soldiers who will die in the desert, or going all-out to do something good for the billions of people in the world who are affected by the United State's governance, we're going to go all-out to save the life of somone who hasn't really even been a person for more than a decade.

I don't know what the actual motivation is behind this stupid political grandstanding, but I'm disgusted.

Gaming: Character Sheet Display Engine

I've finished the preliminary work on my Flash engine for displaying character information embedded in a wiki page. I don't have much up yet by way of example content, but things are looking pretty functional at the moment. You can check it out on our gaming wiki. Further improvements may come, but for now, I'm very happy with it.

Web Comics: Gabe on Rick Dees

As many know, (possibly from the numerous web comic links in my sidebar), I'm a big fan of web comics. They're quite good, often touch on dangerous topics, and are predominately free. In general, I think of them as alternative as well, but today I had a little surprise.

I'm listening to Rick Dees Weekly Top Forty this morning, and all the sudden, Rick Dees starts talking about the Sony PSP. No surprise there, after all, it's an upcoming gadget with a lot riding on it, and Sony is doubtlessly dishing out the millions to anyone that's willing to shill. Suddenly Rick Dees says, "...and I have Gabe from Penny Arcade here in the studio."

For those who aren't in the know, Gabe is the alter-ego of one of the two creators of one of the most successful web comics, Penny Arcade. It's a pretty good comic, with the proviso that you often need to either read the rant or know about happenings in the world of computer games. I read it regularly, along with lots of other folks. But I wouldn't have thought that elevated its creators to the kind of celebrity status I typically associate with guests on national top 40 radio shows.

To be honest, since the strip can be very off-color, it must be even more popular than I thought for Rick Dees to be willing to directly link to it, seeing as it'll almost certainly generate some complaints from parents. The only reason I could see that they'd be willing to deal with that is a big paycheck from Sony. And why would Sony pay to have Gabe talk about the PSP unless it was a big draw?

I don't know, maybe web comics are moving up in the world.

So, just in case I can ride that particular wave, here are some other excellent web comics for you to check out.

Life on Forbez: This sci-fi comic, set in a school on an alien world, is incredibly well drawn black and white. While it is often more dramatic than funny, it's worth reading through the archives to see the storyline and characters develop. I believe that the creator is about to try and achieve three postings a week, which is close to being the web-comics standard. (Possibly because of the influence of Penny Arcade, interestingly enough.)

Questionable Content: This interpersonal romantic tension comic has soap-opera tendencies, but I'm liking it anyway. Despite the name, I doesn't really have any questionable happenings. It also makes a lot of jokes about the indie music scene and the young hipsters that follow its trends. It updates every weekday, and is well worth a look see.

Fallen: I think the full color art here is simply amazing, and setting is complicated enough to be truly intruiging. Unfortunately, once you've plowed through the archives, you may be left hanging for a while, because the creator lost her scriptbook and some finished pages, and the comic hasn't been updated lately. Here's hoping she brings more soon.

The Wotch: This Buffy-esque comic about a young girl with special powers who moves to town is a lot of fun, if a little silly.

Alice!: This sweet and fairly traditional comic strip about an over-imaginative girl has been around for quite a while, but has been defunct until just recently, when it returned with a vengeance. The current update schedule seems to be every weekday.

Glenn: Status

Well, there are a number of things going on right now.

• Most of us are feeling better, but Ivy is sick.
• Liana and I laid a lot of groundwork for the Avarrè campaign setting, more is on the way. There are crude maps now.
• I'm working on a flash-based character sheet display program for use as a wiki extension. It will read XML data that will contain one or more characters at one or more power levels in one or more rulesets. You can see the current state of development here. If it's not exciting, check back in a day or so.
• I have a long entry for Spike's Journal about three quarters written, but It keeps breaking down into a rambling blow-by-blow which is not at all the feel I'm going for with the journal. I think I need to pick a theme for the post and leave out some detail.
• I set up a gmail account today, and decided to use it as the contact email for this blog. You should see the email link on the sidebar. Posting the email address to the web at large ought to really put its spam-filtering to the test. Conclusion so far: It's fairly impressive for a web-based email system.
• In other Google tidbits, the customization feature for Google News means I can customize my news page to not show sports, and the super-cool Google Maps now works with Safari.
• I'm beginning to put together the film list for my film party at OddCon, but I've decided not to put the list on the web. If you're curious, email me.
• There's a similar list for my film party at Confusion, which is the convention-in-miniature my friends and I have in Door County. I designed the logo this year.
• I've seen the newest Dr. Who. It's great, and still very doctor-who-ish. It's also really strange to see the internet play a role in a Dr. Who episode, since the last episode was in 1989. I hope that someone will show it state-side for those of you who can't or don't want to download TV episodes.
• In other film news, Clotho (my Business) did a lot of the heaving lifting for the Wisconsin Film Fest website. Though I personally was working on other projects, and won't want to attend, since I'm getting my wisdom teeth out on March 31st.

Gaming: Bad Impressions

It has recently come to light that the Israeli Army has an actual policy that soldiers who admit to playing D&D will only be granted a low security clearance. I was curious about details, so I read this article.

Man, talk about a bummer. The list of things that those responsible believe and do makes a pretty depressing litany.

• Players are "detached from reality and suscepitble to influence".
• Soldiers who admit to playing are "sent to a professional for an evaluation, usually a psychologist".
• Finally, such people may "have a weak personality".

Currently, with the success of the video game industry and movies like the Lord of the Rings trilogy, geek culture is ascendant in the States and abroad. However, I still remember that when I was in high school, the various pressure applied (mostly by religious groups) actually lead the parents of one of my friends to forbid him to play. He never told me in so many words, but I believe he wasn't even supposed associate with me. Stuff like this IDF policy, even happening in Israel, makes me worry, just because there are still plenty of people out there that hate my favorite pastime.

So, I was looking for some differing perspectives to help cheer me up after this. I found this darkly sarcastic take which I found pretty humorous. I also found this commentary, however, which I thought was more uplifting.

To put it simply, Dungeons and Dragons reinvented the use of the imagination as a kid's best toy. The cliche of parents waxing nostalgic for their wooden toys and things "they had to make themselves" has now become my own. Looking around at my toddler's room full of trucks, trains, and Transformers, I want to cry out, "I created worlds with nothing more than a twenty-sided die!"

Update: You could also check out this fine Dork Tower strip on the subject.

Wiki: Adding Extensions to MediaWiki

Since I often use Flash to illustrate topics and create animations, and may use it for maps, I wanted to have the capability to use flash in my new wiki. (see previous post). Alas, by default, my installation of mediaWiki doesn't seem to include this capability.

However, using just a bit of php, one can write an extension to mediaWiki that allows for custom tags that can do just about anything, so, with some programming, I've added this capability. Now, if you include

<gflash>480 340 http://www.somewhere.com/someflash.swf</gflash>

in your wiki markup, it should embed the flash at the url inline, 480 wide by 340 tall, in your wiki page. Here's a working example on the GlennGameWiki. For people that want the code of the extension, I'm including it as a comment, below.

Update: Since being indexed by Google, this post appears to have become something of a resource for people looking into how to do this. For instance, a large business linked to this article from the main page of their internal wiki for a while. Since my business Clotho Advanced Media, Inc. does internet development work of this sort, I'd certainly invite anyone who is having trouble with this, or needs custom php extensions to mediawiki, to contact my office (608-294-7900, ask for Glenn) or email me. I might be able to help you out, and Clotho can always stand to develop new business-to-business relationships.

Update 2: (5/26/05). My wiki server isn't up to much bandwidth, so I don't allow the uploading of images. Perforce, this means that the flash content referred to by my extension is hosted elsewhere. However, if you're interested in making an extension to allow the inclusion of uploaded content (such as swfs), I refer you to this page. The extension there also has a more sophisticated parameter passing methodology, which may be of interest as well.

Gaming: New Gaming Wiki

I've been busy setting up a personal mediaWiki server to use for gaming purposes. Right now, it has the beginnings of world information for Avarrè, which is the campaign setting I'm using for playtesting Glennworld. I'm also going to use it to post a copy of the Glennworld rules in-progress, so that people can comment on them or collaborate. (Though I haven't done this yet.) Finally, I may use it to post shared information about other games that are currently in progress.

I have some hopes that if I can get a few people interested in helping me write the Avarrè stuff, it can fairly quickly develop into a more full-fledged RPG gameworld. The first things I need to take care of are maps, and then I believe I'll be moving on to simple descriptions of each of the kingdoms around Avarrè, most of which don't even have names yet. What does exist so far are history, laws, gods, organizations, and the basic calendar.

Anyway, you are all cordially invited to drop by, peruse what's there, and comment or contribute if you see fit. Right now it's running at the fairly unimpressive address of http://66.222.39.45/wiki/.

(This is my first attempt at running any kind of server through my home NAT router, so please let me know if it isn't working, and I'll try to fix it.)