What's After Big Data? 87
gthuang88 writes: As the marketing hype around "big data" subsides, a recent wave of startups is solving a new class of data-related problems and showing where the field is headed. Niche analytics companies like RStudio, Vast, and FarmLink are trying to provide insights for specific industries such as finance, real estate, and agriculture. Data-wrangling software from startups like Tamr and Trifacta is targeting enterprises looking to find and prep corporate data. And heavily funded startups such as Actifio and DataGravity are trying to make data-storage systems smarter. Together, these efforts highlight where emerging data technologies might actually be used in the business world.
Re:Big Mac (Score:2, Insightful)
Rare. I prefer to have Big Macs as rarely as possible.
Hopefully a return to real science (Score:5, Insightful)
The problem with 'Big Data' is everyone is trying to use it as a substitute for actual hypothesizing and experimentation.
I am not suggesting it isn't useful, it is, and it can be a huge help in identifying non-intuitive relationships that may exist. Its not being marketed that way though! Everyone is trying to sell it as the solution to all their unresolved problems and knowledge gaps.
At the end of the day all it can ever show is correlation, never causation. All the fancy AIs we add on top are really just correlation engines as well. One day real-soon-now WATSON or something like it will diagnose your cancer. It won't 'discover' the cure though, it will just apply the 'KNOWN' treatment that statistically correlates with the best outcome, hopefully excluding some which correlate with especially un pleasant side effects.
Same is true with the financial markets. Big Data alone will never discover a unified theory that explains market behavior. It will probably make a handful of people stupid amounts of money based again or event correlation and speed. As long as those are the drivers though we will remain forever at risk of sudden meltdowns.