NPR has a two-part report that looks into the business of gathering data on consumers and how it is used, as well as the search for analysts to make sense of that vast stores of “big data.”
What do Facebook, Groupon and biotech firm Human Genome Sciences have in common? They all rely on massive amounts of data to design their products. Terabytes and even zettabytes of information about consumers or about genetic sequences can be harnessed and crunched.
The practice is called big data, and as the term suggests, it is huge in both scope and power. Analyzing big data enables anything from predicting prices to catching criminals, and has the potential to impact many industries.
One way to understand how big data works is to think about your daily life. You write an email, call your boss, pass a security camera, maybe buy a plane ticket online. Taken alone, this is disjointed, boring information. To Elizabeth Charnock, it makes up your digital character. […]
Charnock founded Cataphora, a company that can process huge amounts of this sort of data about employees to determine patterns. She says those patterns can predict everything from a person’s mood to their skill as a manager to a person’s inclination to commit fraud. […]
Big data can, and occasionally does, go wrong. Comic examples of that include mismatched recommendations, like “My TiVo thinks I’m gay.”But think about a company divulging your Web-surfing history with your name attached, and you begin to get a sense of how big data opens the door to new possibilities of security or privacy breaches.
Businesses keep vast troves of data about things like online shopping behavior, or millions of changes in weather patterns, or trillions of financial transactions — information that goes by the generic name of big data.
Now, more companies are trying to make sense of what the data can tell them about how to do business better. That, in turn, is fueling demand for people who can make sense of the information […]
DJ Patil, with venture capital firm Greylock Partners, is on a perpetual manhunt, looking for a rare breed: someone with a brain for math, finesse with computers, the eyes of an artist and more.
“There’s one common element across all these people that stands out above everything, and that’s curiosity,” Patil says. “It’s an intense curiosity to understand what’s behind the data.”
He compares raw data to clay: shapeless until molded by a gifted mathematician. A good mathematician can write algorithms that can churn through billions or trillions of data points and show where patterns emerge.
For example, patterns indicated early on that mothers were heavy users of social networks, which in turn led to the creation of social circles. Patil says a good mathematician can figure out what matters and what doesn’t in a huge trove of data.