Nowadays, as IT technology grows at an accelerating pace, the amount of data is increasing accordingly. Data processing using AI is essential when analyzing big data. This article presents a common understanding of big data and introduces examples of how big data can be utilized using AI.
Big data overview
Big data refers to huge groups of data that are difficult to record, store, and analyze using conventional database management systems. In today’s information society, it is possible to obtain all kinds of data. Analyzing this vast amount of data is expected to improve productivity in daily life and business.
Big data definition
There is no unified definition of big data, but in recent business situations, anything that satisfies the “three Vs” below is sometimes considered big data.
- Volume (capacity) Amount of data ranging from several terabytes to several petabytes
- Variety: Various data such as text and audio
- Velocity: Data responds in real time in a changing environment
Value is sometimes added and called the “four Vs.”
Benefits of using big data
The use of big data is expected to be a way to effectively utilize data and create new value. There are two major benefits to using big data. One is the ability to analyze the past and present and predict the future. Future predictions are also related to improving productivity and profitability. Second, we can add new value to existing services. For example, web advertising such as Google and Yahoo provides added value by using analytics tools to visualize cost-effectiveness, which was a black box in conventional advertising.
Introduction of AI usage examples
Data utilization through AI is being used in many industries and business settings. Here we will introduce three examples.
Utilization in human resources
Recruiting human resources is essential for companies. When hiring, we conduct document screening and aptitude tests, but it is currently difficult to judge which human resources are truly suitable for your company based on that alone. This is where AI data analysis comes into play. We collect and analyze the data necessary for hiring, such as the educational background, special skills, and contribution to work of people we have hired so far, and determine what characteristics a person has and can be successful in what department. It will be possible. This allows you to quantitatively visualize the trends of the best human resources for your company, which cannot be determined just by looking at documents or interviews.
Automatic driving function
AI-based data analysis is also used in self-driving technology, which is a hot topic these days. AI collects data from driving situations such as sunny days, rainy days, daytime and night, and uses deep learning to learn how to drive in various situations. By constantly learning data, AI increases the probability of being able to drive safely without an accident in any situation.
Application to medical practice
Data analysis using AI is also effective in the medical field. In NTT Data’s demonstration experiment, information obtained from various medical devices in the ICU (intensive care unit) is aggregated into one platform, and based on various data including aggregated vital signs, patients develop complications. We are developing a model to predict risk.
Challenges of using big data
Although the amount of data in circulation is increasing day by day, there are also disadvantages that arise from the increase in the amount of data in circulation. We will introduce what precautions need to be taken in business and what risks there are when using big data.
Quality is not uniform
Even though big data is not centrally managed, its quality is not uniform. Big data also contains typographical errors and meaningless information. In order to improve the accuracy of data analysis and utilization, it is necessary to organize it into an analyzable format and ensure uniform quality.
It is necessary to prepare an environment for handling big data.
As mentioned above, it is necessary to equalize the quality of data as a prerequisite for data analysis, so it is necessary to prepare a mechanism to achieve uniformity. In addition, storage to store large amounts of data and computers to perform calculations must be provided. Additionally, data such as customer information is often confidential and requires a high level of security.
Lack of data scientists
The demand for hiring data scientists is increasing day by day, and companies are also struggling with recruitment activities. In addition to the lack of IT human resources in Japan, human resource development requires significant cost and time.
summary
There are many examples of utilizing big data using AI. Although there are many benefits, it is necessary to prepare an environment and secure specialized human resources in order to handle big data.
