Well, the video bug has bitten me again. It's actually been percolating for a little while now and was born from our bike rides. See, we ride through a bunch of really cool areas and I've always wanted to capture the experience somehow. The solution, of course, is to have some sort of camera that records the ride.
A still camera isn't really practical and not what I want. I don't want to pull over and take a picture. I want to record video that I can then edit and make into a kind of travelogue. There are basically two options - mount the camera on the bike, or mount it on me. Mounting on the bike limits the visibility of the camera as it's fixed in position and can't see the various views I'm seeing in different directions. It also would get a lot of road vibration and probably wouldn't look very good.
So, the solution is to get a helmet cam. Yeah, like the kind the skydivers wear when they jump out of planes. There are two main contenders in the small, relatively inexpensive helmet wars - Contour and GoPro. Initially, I liked the Countour since it was small and relatively unobtrusive (as much as a camera mounted on your helmet can be unobtrusive). Recently though, I'd been hearing a lot about the GoPro so I looked into it further. Here's their promo video
Not that I want to jump off a mountain on my skis or anything but that's some pretty cool video.
Anyway, I did a little more research and decided to go with the GoPro camera. I got a helmet mount and a few other so I could attach it to the bike as well for some other shots.
For a test, I attached it to the top of the car and drove around. Kind of boring footage but I wanted to make sure I aimed it properly and could bring the video into my editor and cut and trim. That all seemed to go okay so, if it ever stops raining, I'll try to get some bike footage.
Should be fun.
Saturday, April 23, 2011
Monday, April 04, 2011
Big Data
It's the new buzzword. Big Data refers to the vast quantities of information that is being generated and collected by various companies, web sites, governments, whatever. More specifically, it refers to what those entities might want to *do* with all that data.
Working at EMC, we've been dealing with large amounts of data for a long time. Our products (in general) are gigantic disk drives that store and protect "Mission Critical" data for companies large and small. Our Marketing people are quick to point out mind boggling statistics like - the amount of information produced last year is larger than all the of information produced in all the previous years combined - things like that.
Information production is only getting faster and faster too. It's one thing to have to store all that information but increasingly, companies want to be able to *mine* that information. As the amount of information grows, data analysts can apply statistical methods to look for patterns in the data and 1) determine behaviors and 2) predict activity.
So, analysts can look at sales figures and see that people are buying more of one product than another and adjust inventory levels or do other things to make sure their business is positioned correctly. They can also look at the data, combine it with other data and create models that let them predict what people are going to do when this or that changes.
Now, analysts have been doing this kind of thing for a long time - it's not really new. What is new is the amount of data being processed and the need to process it very, very quickly - real-time analytics.
Real-time analytics means looking at the data as it comes in and analyzing it right then. In the past, the analysis had to be performed on small subsets of the data in the "Data Warehouse". The analysis systems were not big enough or powerful enough to plow through all the data, they had to take a sample and hope that it had enough information to provide meaningful insight. Plus, it took them hours and hours to run those models and get an answer
Obviously, if you don't get a big enough sample, you could arrive at inaccurate conclusions. For example, you can take a look at the stock market values over any period of time. Depending on which week you happen to pick, you might conclude that the market is going up, down, or staying the same but that might not represent the larger trend. To get a "better" picture, you really need to look at more data - data that represents a longer period of time. In general, some patterns don't emerge until you get a sufficiently broad look at the data. Thus the dilemma. You need large samples of data to analyze and the bigger the sample, the more time it takes to analyze. But, to beat your competition, you need to get the results in seconds, not hours or days of the traditional systems. You want to be able to look at the cash register as the clerk is scanning items, find what else that customer has bought from you, and offer them an accessory that would beautifully match the dress they just bought today and the shoes they bought last month. Winning!
So the term Big Data not only means the *amount* of data out there but how to process and use that data to gain an edge in business. At work, we've been getting into this more and more. We're no longer interested in just storing the data for our customers, we need to help them mine it and get value out of it.
There are lots of interesting applications. We are currently working with Utility companies to help them figure out how to manage Smart Meter Data. It used to be that the power company would come by every month and read your power meter to figure out how much electricity you used so they could send you a bill. They got a little more advanced by installing meters that they could read from a truck as it drove by your house - no need to get out, find the meter, and write it down.
Enter the age of Smart Meters. These meters will now look at your power consumption and send it back to the Utility company every 15 minutes. So, instead of getting 12 readings a year from each customer, they are now getting 34,000 readings per year from each user. That's 3,000 times more information than they had been getting previously.
Not only are they looking for ways to manage this info, they are looking for ways to *use* it - beyond just sending you your bill. For example, it's really, really, expensive and hard to build a new power plant but there is a never ending demand for power. And it's the peak power demand that is killing the power companies. When everyone comes home at night and cooks dinner or washes clothes on Saturday, they have to have enough capacity to handle the peaks but that capacity goes unused in the valleys. If they could lower the peaks, they wouldn't have to build more plants.
Enter the Smart Meter. The power company can offer you an incentive and say, if you reduce your consumption from 5:00 pm to 11:00 pm - our peak demand time - we'll offer you a rebate (or some other incentive) on your bill. With the Smart Meter, they can tell not only how much power you use but when you use it, and try to adjust your behavior. In some scenarios, they can even tell what kinds of things are using your power and be able to send you a letter that says "We see you have a 1995, Kenmore model C-RAP dishwasher. Newer models use much less energy so we'll give you a rebate if you replace it."
They can also look at the grid of meters and get a better picture of their delivery system. With real-time analytics of the information coming in, they can detect say, voltage variations in a particular neighborhood. They can see that one particular transformer is common to all the affected meters and, using their statistical models, predict that it will fail in 2 weeks. They can then roll a truck to replace it *before* it blows, avoiding down time, angry customers, and unfavorable news reports..
Cool.
Working at EMC, we've been dealing with large amounts of data for a long time. Our products (in general) are gigantic disk drives that store and protect "Mission Critical" data for companies large and small. Our Marketing people are quick to point out mind boggling statistics like - the amount of information produced last year is larger than all the of information produced in all the previous years combined - things like that.
Information production is only getting faster and faster too. It's one thing to have to store all that information but increasingly, companies want to be able to *mine* that information. As the amount of information grows, data analysts can apply statistical methods to look for patterns in the data and 1) determine behaviors and 2) predict activity.
So, analysts can look at sales figures and see that people are buying more of one product than another and adjust inventory levels or do other things to make sure their business is positioned correctly. They can also look at the data, combine it with other data and create models that let them predict what people are going to do when this or that changes.
Now, analysts have been doing this kind of thing for a long time - it's not really new. What is new is the amount of data being processed and the need to process it very, very quickly - real-time analytics.
Real-time analytics means looking at the data as it comes in and analyzing it right then. In the past, the analysis had to be performed on small subsets of the data in the "Data Warehouse". The analysis systems were not big enough or powerful enough to plow through all the data, they had to take a sample and hope that it had enough information to provide meaningful insight. Plus, it took them hours and hours to run those models and get an answer
Obviously, if you don't get a big enough sample, you could arrive at inaccurate conclusions. For example, you can take a look at the stock market values over any period of time. Depending on which week you happen to pick, you might conclude that the market is going up, down, or staying the same but that might not represent the larger trend. To get a "better" picture, you really need to look at more data - data that represents a longer period of time. In general, some patterns don't emerge until you get a sufficiently broad look at the data. Thus the dilemma. You need large samples of data to analyze and the bigger the sample, the more time it takes to analyze. But, to beat your competition, you need to get the results in seconds, not hours or days of the traditional systems. You want to be able to look at the cash register as the clerk is scanning items, find what else that customer has bought from you, and offer them an accessory that would beautifully match the dress they just bought today and the shoes they bought last month. Winning!
So the term Big Data not only means the *amount* of data out there but how to process and use that data to gain an edge in business. At work, we've been getting into this more and more. We're no longer interested in just storing the data for our customers, we need to help them mine it and get value out of it.
There are lots of interesting applications. We are currently working with Utility companies to help them figure out how to manage Smart Meter Data. It used to be that the power company would come by every month and read your power meter to figure out how much electricity you used so they could send you a bill. They got a little more advanced by installing meters that they could read from a truck as it drove by your house - no need to get out, find the meter, and write it down.
Enter the age of Smart Meters. These meters will now look at your power consumption and send it back to the Utility company every 15 minutes. So, instead of getting 12 readings a year from each customer, they are now getting 34,000 readings per year from each user. That's 3,000 times more information than they had been getting previously.
Not only are they looking for ways to manage this info, they are looking for ways to *use* it - beyond just sending you your bill. For example, it's really, really, expensive and hard to build a new power plant but there is a never ending demand for power. And it's the peak power demand that is killing the power companies. When everyone comes home at night and cooks dinner or washes clothes on Saturday, they have to have enough capacity to handle the peaks but that capacity goes unused in the valleys. If they could lower the peaks, they wouldn't have to build more plants.
Enter the Smart Meter. The power company can offer you an incentive and say, if you reduce your consumption from 5:00 pm to 11:00 pm - our peak demand time - we'll offer you a rebate (or some other incentive) on your bill. With the Smart Meter, they can tell not only how much power you use but when you use it, and try to adjust your behavior. In some scenarios, they can even tell what kinds of things are using your power and be able to send you a letter that says "We see you have a 1995, Kenmore model C-RAP dishwasher. Newer models use much less energy so we'll give you a rebate if you replace it."
They can also look at the grid of meters and get a better picture of their delivery system. With real-time analytics of the information coming in, they can detect say, voltage variations in a particular neighborhood. They can see that one particular transformer is common to all the affected meters and, using their statistical models, predict that it will fail in 2 weeks. They can then roll a truck to replace it *before* it blows, avoiding down time, angry customers, and unfavorable news reports..
Cool.
Saturday, April 02, 2011
Okay computers, this is getting ridiculous
Wow, it's been a long time since I've posted. Have to get better at this.
I'm a little late on this latest bit of technology but it's still amazing. To set the stage, I've started doing Sudoku puzzles. I held off for several years while Dad, Carly, and Deb all exclaimed how fun it was. In fact, I was a little worried that Dad was on a personal mission to convert everyone to be a Sudoku Jedi Master.
I'm not really into puzzles or brain teasers much so I ignored them as best I could. What finally converted me was I downloaded a Sudoku App for the iPad and tried it. It took a few runs but now I'm hooked. Deb and I now play a game or two on the iPad before turning out the lights every night.
The App itself is pretty amazing. It has different levels and actually has teaching modes to help you learn different techniques to solve the tougher puzzles. While in the puzzle, if you get stuck, you can ask it for hints. It will start by saying something like "Ponder the digit 8". This will usually be enough to help you move on since you can focus on all the 8's and see a move you hadn't noticed before. If that doesn't help though, you can ask for more and it will give more and more specific hints, to the point where it will just fill in the square for you. I have the sound off but I think it makes a little noise like "Jeeeezus!" when it has to hold your hand that much.
So that's cool but not really the reason for this post. I'd heard about Google Goggles which is an app for the phone that let's you do Google searches but in a couple of interesting ways. You can type in your search words like normal of course but you can also just speak your search. You press the microphone icon, say "Nuclear Power", and it will pull up search results just as if you'd typed it in. Amazing, but oh so five minutes ago.
No, to really be cool, you just take a picture of what you want information about. So, take a picture of say, a book or DVD, and Google will bring up information about that item. In the bookstore, take a picture of the barcode, Google will identify it and tell you what the price is on Amazon so you can order it right then.
(Which really sucks for the bookstore of course. Reminds me of a funny picture I just saw. It was the front window of a Borders Bookstore with signs listing all the liquidation sales. There was also a sign that said "No bathrooms. Check Amazon" Ouch!)
Which gets us back to Sudoku. We do the puzzle in the paper on Saturday and Sunday so today I tried a trick I'd heard about. In Google Goggles, I pointed the camera at the puzzle in the paper and took the picture. Google identified it as a Sudoku puzzle!
Then it asked me if I wanted it to solve it.
Two seconds later, it was done.
Hello Sky-Net...
I'm a little late on this latest bit of technology but it's still amazing. To set the stage, I've started doing Sudoku puzzles. I held off for several years while Dad, Carly, and Deb all exclaimed how fun it was. In fact, I was a little worried that Dad was on a personal mission to convert everyone to be a Sudoku Jedi Master.
I'm not really into puzzles or brain teasers much so I ignored them as best I could. What finally converted me was I downloaded a Sudoku App for the iPad and tried it. It took a few runs but now I'm hooked. Deb and I now play a game or two on the iPad before turning out the lights every night.
The App itself is pretty amazing. It has different levels and actually has teaching modes to help you learn different techniques to solve the tougher puzzles. While in the puzzle, if you get stuck, you can ask it for hints. It will start by saying something like "Ponder the digit 8". This will usually be enough to help you move on since you can focus on all the 8's and see a move you hadn't noticed before. If that doesn't help though, you can ask for more and it will give more and more specific hints, to the point where it will just fill in the square for you. I have the sound off but I think it makes a little noise like "Jeeeezus!" when it has to hold your hand that much.
So that's cool but not really the reason for this post. I'd heard about Google Goggles which is an app for the phone that let's you do Google searches but in a couple of interesting ways. You can type in your search words like normal of course but you can also just speak your search. You press the microphone icon, say "Nuclear Power", and it will pull up search results just as if you'd typed it in. Amazing, but oh so five minutes ago.
No, to really be cool, you just take a picture of what you want information about. So, take a picture of say, a book or DVD, and Google will bring up information about that item. In the bookstore, take a picture of the barcode, Google will identify it and tell you what the price is on Amazon so you can order it right then.
(Which really sucks for the bookstore of course. Reminds me of a funny picture I just saw. It was the front window of a Borders Bookstore with signs listing all the liquidation sales. There was also a sign that said "No bathrooms. Check Amazon" Ouch!)
Which gets us back to Sudoku. We do the puzzle in the paper on Saturday and Sunday so today I tried a trick I'd heard about. In Google Goggles, I pointed the camera at the puzzle in the paper and took the picture. Google identified it as a Sudoku puzzle!
Then it asked me if I wanted it to solve it.
Two seconds later, it was done.
Hello Sky-Net...
Subscribe to:
Posts (Atom)