Geek

YouTube Downloader

I watch a lot of YouTube videos. It has almost completely replaced anything that comes from traditional television in my viewing habits. The YouTube interface isn’t terrible, but I really like the idea of a DVR style interface where new shows automatically appear and then I can delete them when they are done. I want to be able to stream them to whatever device I’m on and remember my playback position in case I stop mid way. In true geek fashion, I have a technical solution (which I’ve partly described before.)

One of my computers runs a custom program that checks all of my YouTube subscriptions for new videos and then downloads them to my local computer. From there I can stream them to any of my devices via Plex. I thought about trying to package up that custom program because it would be useful for people like my parents who get free bandwidth between 12 and 5am but not during the day. This would allow them to schedule YouTube videos to be downloaded. But unfortunately my homegrown solution has gotten pretty complex over the years and it’s not super simple to package it up for re-use. Bad design I guess.

There are various programs and websites that will help you download local copies of YouTube videos, but the key here is getting something that can be scheduled to specifically solve my parents internet constraints. (Basically this is just an email to my Dad that I thought other people might find useful too.)

The good news is that the core of my solution isn’t too difficult to reuse. It’s a program called youtube-dl. It was originally just a python script but it looks like they have packaged it all up into a Windows executable now. You can get all the info along with their source code and the compiled executables on their GitHub site. (Look for the part that says “Windows users can download an .exe file…”)

Once you have that executable, you need to figure out what command line parameters to pass to it. I’m still using the straight Python version, but I’m guessing your command line would look something like this:

youtube-dl.exe http://www.youtube.com/watch?v=KDL2LcKnISU -f 22 -o “C:\kayaking.mp4”

Once you get that figured out, then you could schedule it via the Window Task Scheduler. That would be fine for one time video downloads. If you want to do this more often, then you’ll need to build up some automation around it to make it easier to use. I hooked into the YouTube APIs to get videos from my subscriptions and compare them with a list of videos that I’ve already downloaded, etc. I can provide more info if you want to go that far.

This all sounds crazy but it works really well. My setup has downloaded over 4000 YouTube videos over the years and most of the time I just plop down in front of the TV and watch whatever new videos have been placed there for me. I don’t have to think about the magic happening behind the scenes.

benmartens October 3, 2016 Geek

Chrome

The first web browser I ever used was Netscape Navigator. Sometime around the release of Internet Explorer 4 (1997), I switched over to IE and I’ve used that almost exclusively since then. Coincidentally, I ended up working with one of the guys responsible for the Trident engine which powered IE4 and is credited for giving it such a big market share over Netscape.

I’ve been on board with IE for almost 20 years, but now I’m switching away. It was great for a long time but it has atrophied in recent years. The focus seems to have shifted to the new Edge browser and while that is very fast and standards compliant, it also lacks a plugin model so I can’t use things like Ad Block Plus or LastPass. That’s a non-starter for me.

So goodbye Microsoft browsers and hello Google Chrome. It’s hard to find accurate statistics, but pretty much everyone agrees that a majority of users use Chrome with the remaining users scattered among IE, Edge, Firefox, Safari, and others. Chrome has a huge array of extensions available for it. My favorite are Ad Block Plus, Delicious, and LastPass. Of course if you include too many extensions, you’ll bog the browser down, but generally it’s very snappy and works with almost all websites. (There are still a couple old company intranet sites that require to me to use IE.) You can sign in to the browser and let it sync your favorites and extensions across your various computers.

Switching to a new program after two decades takes a little while to get used to, but I’ve been using Chrome for about a month and I don’t think I’ll be going back for quite a while unless Edge can really bring the fight to Chrome.

benmartens September 5, 2016 Geek

Power Over Ethernet

I dare you to try and buy just one security camera. That’s how it started for me. Now I have six. I might need an intervention.

The last one I put up was an “extra” that we had lying around after we upgraded one of the cameras to a newer 1080p. This old one is only 640×480 but hey, why not use it? I decided to stick it on the back corner of the house, but that meant a long cable run from my network closet and there’s also no power nearby. Power over Ethernet (PoE) to the rescue!

In your standard ethernet cable, half of the wires are unused. The PoE specification defines a common way of transmitting power over those extra wires. This means that instead of running a network cable to the camera along with installing a new outlet nearby, I just had to run a network cable.

You also need something to combine the network signal and the power on one end and deconstruct it on the other end. For the injection side, you can either get a standalone plug/ethernet jack combo thing or you can purchase a switch with PoE built in. I chose the latter since I’ll be using this more in the future. My WiFi access points are already PoE so I can get rid of the two injector plugs and switch them over to this unit. And while it’s possible to buy PoE versions of these cameras, I don’t have those yet so I had to buy a splitter for the camera end.

The hardest part was running the wire 40 or 50 feet under my house and even that wasn’t hard, just dirty and cramped. Elijah loves it when I go down there and always asks if he can poke his head in there too.

The end result is a really nice view of either the cul-de-sac or our back yard depending on which way the camera is rotated. It’s not going to do much for security, but it’s fun to see Elijah playing during the day while I’m at work or just get a little extra assurance that everything is ok while we are away.

P.S. If you are interested in setting up something like this at your house, I have a post detailing my basic setup. You can get more posts by searching my site for “foscam“.

benmartens August 9, 2016 Geek, Home Improvement

Facebook Data Retention

Many of us end up dumping a lot of information into Facebook. You might want to download that to have a local copy/backup. It turns out this is pretty easy to do. The Facebook help docs have these instructions:

Click at the top right of any Facebook page and select Settings
Click Download a copy of your Facebook data below your General Account Settings
Click Start My Archive

It takes a while to generate your archive. You’ll receive an email when it’s done. Click the link in the email and save that file to your computer.

Now that you’ve got it all backed up, are you sure you want to leave it sitting on the Facebook site? I don’t. Deleting your data from Facebook is a lot harder than backing it up. They don’t provide an easy way to selectively delete data. Thankfully there are some tools to help you out. I like the Facebook Post Manager extension for Chrome. It’s not super fancy or easy to use, but it did the job for me. I wiped out most of my posts, shares and some of the photos from before 2016.

It takes a little while, but I think it’s worthwhile to not have that all sitting around. I’ve been doing the same thing with my email with MailStore for the past couple years. Did you know that until some time in the last year or so, no warrant was needed to read email that you left on the email server for more than six months?

In the big scheme of things, this probably makes no difference. There’s so much data about us out there already, that deleting my old Facebook posts doesn’t matter, but oh well, it makes me feel a little better. Facebook probably still keeps them around on their server but now random internet people can’t view them all.

benmartens August 3, 2016 Geek

The LEGO Movie

I finally got around to watching The LEGO Movie. It’s… awesome! Everything is awesome! The cast reads like a list of my favorite actors: Will Arnett, Alison Brie, Charlie Day, Will Ferrell, Will Forte, Morgan Freeman, and Nick Offerman and Chris Pratt. The kid in me loved the story and the geek in me loved the awesome computer graphics.

Long time followers of my nerd adventures may remember that I’ve made a few movies of my own. I finally uploaded them to YouTube for your amusement.

Up first is a stop motion video made with cousins Tim, Mark and Ryan. It was probably during Christmas and I’d guess it was around 1987 or 88. We did it in Tim’s basement with his VHS camcorder. There was a feature that would record a few frames with the press of a button. When we had the idea, I remember talking about who was going to build each piece. Before Mark went back to his house to build everything, we typed in a bunch of reminders on his awesome Casio calculator watch. The video took forever and we had a “script” change at the end. As a Lego man was moving across the crosswalk, he fell down perfectly in the middle of one of the recording clips. We went with it (what else can you do?) and turned it into a medical emergency. It’s a riveting story. I sped up the video below so that you can get your boredom in a shorter amount of time.

I thought this project was incredibly cool so a couple years later, I decided to try it again on my own. I borrowed our neighbor’s VHS-C camcorder (so “tiny”!) and set up a table in our basement. The city I built was much simpler than the first one and the story was non-existant.

And last, but certainly not least, I fell down a rabbit hole in college. My Junior and Senior year and then part of the year after graduation were spent doing a Lego movie on the computer. This was about 2000-2002 so there were none of the fancy Lego CAD programs available. Instead, I painstakingly built up a library of Lego pieces in a program called POVRay. It was a free tool that did raytracing but the modeling was all done in text. So to build a piece, I’d start with a rectangle, add some height and draw the nubs on top. I even had versions of the pieces with the word LEGO writen on top of each nub. Something like a Lego person is a huge amount of code. I had posted it to the web a while back and found it on archive.org. This link shows you what it takes to draw a single Lego character. Before I got too deep into the project, I built a short little sample video based on the old Budweiser Wazabi commercial.

The pieces were incredibly detailed and it literally took years to build it all. I also wrote a bunch of crazy math functions to animate the camera movement through the virtual world and also to animate things like walking characters. Rememember that since it is all text based input, “walking” means manually adjusting the rotation of various geometric shapes while translating the whole object forward. It was crazy complicated, but obviously I really enjoyed it for whatever reason. It was kind of relaxing to sit down for a while every night and “build” more pieces from scratch and at a time when I had very little money, it was fun to have limitless amounts of Legos to play with.

After spending eons on the graphics, I put about 30 seconds of thought into the story and here’s the result:

It’s not too far-fetched to say that this project got me my job at Microsoft. It took about 11 minutes to render a single frame of the movie (on my P2 350Mhz computer) and there were 2538 frames. That comes out to about 20 days of non-stop computing power. But oh yeah, don’t forget that I screw up a lot so I had to redo a bunch of it. To get it all done more quickly and to enter a programming contest at Purdue, I wrote a distributed rendering program. POVRay could be controlled via the command line so you would install POVRay and my client app on your computer. A central server would hand out individual frames for all the clients to render and then the client would send back the final image. (Archive.org has a copy of the design document for this project.) It actually worked and saved me a bunch of time. Not only that, I won the programming contest which netted me an original XBox and an HP Jornada PocketPC. But better yet, the guy who ran that comptetition went on to work at Microsoft. When I was looking for a job, JimM helped me reconnect with him and I ended up working in his team for the next six years!

So yes, this Lego project was ridiculously complex and within a few months of me releasing that video, there were bigger and better Lego rendering tools available. But I do think that finding fun projects like this area great way to stretch your abilities and feed your passion. This single project helped me in a lot of interviews when I was exiting college and it taught me a lot about programming.

benmartens July 15, 2016 Geek, Maker

Yes We Can

Two of Elijah’s favorite TV shows are Thomas the Train and Bob the Builder. As we were watching Bob the Builder one night, I had an idea for a video mashup…

benmartens July 13, 2016 Geek

Beer Stats

Dad, Luke and I all use the Untappd app pretty regularly. Think of it like Facebook for beer. You “checkin” when you have a beer and you give it a rating. You can see what your friends are drinking, get recommendations, etc. I love it because it’s an easy way to keep track of all the different beers that I’ve tried and how much I like each one. It’s a great encouragement to keep trying new things.

I recently tied into their API and wrote a quick app that lets me download the data locally for analysis. There are lots of questions that I want to answer, but here are a couple fun charts to get started.

The first is a chart showing how many checkins we’ve each had by IBUs (International Bitterness Unit). It doesn’t show how much we LIKE them, but it does show which ones we generally try.I’d love to expand this to show how this has changed over time.

Here is one showing the cumulative number of checkins we’ve had over time. This chart is a little sloppy because each user series should have it’s own color, but there are only three of us so it’s not too hard to follow. If you look at the end of the chart, Luke is the top line, I’m the middle and Dad is the bottom. You might think that after a while, you run out of new beers, but if that’s true, we haven’t hit that point yet. We’re all on a pretty steady upward trajectory.

I took a look at which beers have the highest combined score from the three of us. The beers with wider distribution are more likely to win because to get to the top it helps if we’ve all tried it. Since I’m out on the west cost and they are in the midwest, most of the time our beers don’t overlap unless they are bigger breweries. Here are the top six in order.

Deschutes Brewery Red Chair NWPA
Bell’s Brewery Two Hearted Ale
New Belgium Fat Tire
Great Lakes Elliot Ness
Great Lakes Burning River Pale Ale
Deschutes Brewery Mirror Pond Pale Ale

Looking at the bottom of that list is a little silly because there are only 40 beers that all three of us have tried. I expanded it out to beers that at least two of us have had. There are 129 in that list and here are the bottom three. I bet you won’t be surprised.

Coors Light
Miller High Life
Bud Light

I’m looking forward to playing around a lot more with this data. If you want access too, just let me know.

benmartens May 2, 2016 Geek

Coding At Home

When people ask me how to get a job where I work, I tell them to be curious and experiment at home. I’ve learned so many valuable things from tinkering with code at home and it has made a big, positive impact in my career.

In the past it has been a lot of website stuff, but since my move to WordPress, that has pretty much disappeared. Lately it has been stuff that a computer science geek would call “ETL” (Extract Transform Load) programs. I enjoy taking my data out of one system and storing it in a common place that I control. For example, my Ecobee thermostat logs a ton of data every day but it lives in the EcoBee server. I use their programming interface to download the data and store it in my own database. I do the same thing with my irrigation controller, fantasy football and a few other gadgets and services. There’s not a lot you can do with the initial data right at the beginning, but as time goes on, you build up a pretty large data set and you can start to see some interesting trends. As I move into more of a data science position it’s great to have a big dataset of data that’s interesting to me personally.

Learning is a lot more fun when you’re doing it to achieve a goal that interests you. So why not find interesting things to learn that can benefit you in your job?

benmartens April 29, 2016 Geek

Reminders

“Cortana, remind me on Saturday to ride the monorail.” I apparently use reminders on my phone so often that phrases like that come out of my three year old’s mouth. Whether you have iOS, Android, Windows Phone, or even Windows 10, you should get to know the reminder capabilities. On my phone I just hold down the search button, wait for Cortana to start listening and then tell her what I want to be reminded of. That reminder can be triggered at a specific time, a place (via GPS), or when I’m calling or texting someone. It took a little while to get used to using it, but now I’d have a hard time living without it. Our brains are terrible at remembering todo lists. Offload it to the computer in your pocket!

benmartens April 21, 2016 Geek

Azure IP Blocking

A few days ago I noticed that my website was getting a little sluggish. It wasn’t much but it wasn’t as snappy as usual. I checked out the website dashboard on Azure and noticed that the number of requests to my server were steadily rising. A quick scan of the logs revealed that a single IP was flooding my server with requests. It was trying to hit the URL where I used to serve up CascadeSkier data so I doubt it was an intentional DOS attack, but the effect was ramping up to be similar.

I wrote previously about blocking traffic to Azure websites, but new features are always getting added so I looked around again. Indeed they have added the ability to block specific IPs. Check out the final section of ScottGu’s blog post for more info.

It was a one line configuration file change and the results were immediately apparent. Even if you don’t understand any of the geek gibberish that I just wrote, I bet you can figure out when I implemented the change:

benmartens April 14, 2016 Geek, Site News

Studio711.com – Ben Martens

Just trying to get through life without looking stupid, so far it's not going very well.