We have entered the era of big data. We can now access petabytes of transactions, clickstreams, media, archives, research, as well as data from social networks, mobile phones, and sensors. This data is growing exponentially — as is the technology to extract insights, discoveries, and meaning from it.
Big data will transform every sector of the economy. Cloud storage and advanced software now offer professionals access to real-time data. Data-driven decision-making has already increased productivity in financial services and social media. Health care, public administration, retail, and manufacturing are poised to achieve similar growth as they harness the technologies underlying this data revolution.
Data science has been dominated by computer scientists, mathematicians, and statisticians working in finance and information industries. But the tools for working with big data are improving fast, and a much wider range of sectors, including health care, manufacturing, and government, now recognize the value of data-driven decision making.
We need a broader conception of the field — one that highlights not only technical skills for data collection, aggregation, and analysis but also the ability to frame a relevant problem, to derive insight from the data, and to visualize and communicate the findings and why they matter.
Are you ready? Big data demands individuals with the technical skill to aggregate huge amounts of data, the analytical skill to understand it, and the design and storytelling skill to synthesize it. Silicon Valley businesses are already struggling to find qualified data scientists. This talent shortage will intensify as mainstream businesses seek managers fluent in the language and tools of big data to extract value from their un-mined stores of data.
What skills do you and your organization need to thrive in a data-intensive economy? What skills will all professionals need to succeed in the era of big data? What balance of technical, social science, and design skills will we expect of a new generation of data scientists? What tools must educators master to train professionals to extract insights, products, and services from enormous databases? How do we protect privacy and security when almost every move we make, mouse we click, and dollar we spend is captured as data?
Join the conversation. The DataEDGE conference at UC Berkeley will bring together senior industry and academic leaders for a conversation about the challenges and opportunities created by the rise of big data.