Big data is a huge data group that includes data of various types and formats . It consists of three elements: “volume,” “variety,” and “velocity of input/output and processing .”
Big data has made it possible to accumulate unstructured data (video, voice, text, etc.) and real-time data that were previously difficult to utilize.
Big data is also attracting attention as one of the important factors in promoting DX , and is closely related to other advanced technologies such as IoT and AI .
What is big data?
We will explain the meaning and definition of big data, and the benefits of using it.
What is big data? Three “V” points
Types of data included in big data
Big data is a huge data group composed of various types and formats of data, including unstructured data .
Unlike conventional structured data (data in table format consisting of “columns” and “rows” like Excel), data obtained from text, audio, video, images, SNS, web pages, etc.
Gartner, a major American IT research company, defines the characteristics of big data as “three Vs” .
・Speed of input/output and processing (Velocity)
・Diversity of data types and information sources (Variety)
Advances in technology have made it possible not only to handle huge amounts of diverse data at once, but also to instantly analyze real-time data that was previously difficult to store and utilize. , the use of big data has become widespread.
Definition and scope of big data
What kind of data does big data mainly refer to?
According to the Ministry of Internal Affairs and Communications’ White Paper on Information and Communications (2017 edition ), big data is defined as consisting of the following four types of data.
1) Government: “Open data” provided by national and local governments
2) Companies: Digitalized and structured data of tacit knowledge (know-how) (referred to as “digitalization of knowledge”)
3) Companies: M2M (Machine to Machine) streaming data (referred to as “M2M data”)
4) Individual: “Personal data” related to personal attributes
Advantages of using big data
As the use of big data spreads, it has become possible to handle data that could not be collected in the past, and at the same time, it is also possible to combine various types of data . As a result, useful data from unprecedented new perspectives is created, and new systems and businesses are being created one after another.
Background of big data attracting attention in the business scene
In Japan, the Ministry of Internal Affairs and Communications’ ” White Paper on Information and Communications (2012 edition)” emphasized the importance of big data for ” creating new added value as a knowledge and information infrastructure .” rice field.
In addition, the White Paper on Information and Communications (2017 edition ) states that “the first year of big data utilization has arrived,” and that big data will be efficiently collected and collected with the development of laws related to data utilization and the spread of technologies such as IoT and AI. It is written that an environment that can be shared is being realized.
In recent years, the “ DX Report ” has also taken up the use of data as an element to realize the DX scenario, saying that it “makes it possible to respond to speedy changes in direction and global expansion . ”
What you can do with big data
In what situations is big data used?
Here, we will divide the role of big data into two broad categories: “decision-making based on data” and “prediction.”
data-driven decision making
By utilizing big data, it is possible to draw out the knowledge necessary to solve problems from a vast amount of diverse information and make business decisions .
Collecting, accumulating, and visualizing big data in this way so that it can be used for business decision-making is called “data-driven .”
Since it is possible to formulate policies based on data, which is an objective basis, it is easy to gain understanding from those around you. In addition, since the effectiveness verification can be performed based on the data, there is also the advantage that it is easy to turn the PDCA cycle in detail. It is used in various fields from marketing to product development.
predict
By using big data, it is possible to analyze trends from a huge amount of past performance data and make highly accurate forecasts . For example, by predicting the demand for products and services, it is possible to properly manage inventory and production volumes, reducing unnecessary costs and improving operational efficiency.
In addition to being used to predict product demand and improve operational efficiency, it is also used in various fields such as accident and crime prediction and health management.
Relationship between IoT and AI
Big data is also attracting attention as one of the important factors driving DX . It is also closely related to technologies such as IoT and AI , which are also the technologies that support DX .
We will explain in detail the relationship and role of each technology.
IoT × big data
IoT stands for “Internet of Things”. The basic role of IoT is to detect the state and movement of things and acquire data using sensors, cameras, and wireless communication installed in things. The obtained information is transmitted to people and things via the Internet.
In recent years, many products equipped with IoT technology, such as smart speakers, smart homes, and self-driving cars, have been used in people’s lives.
IoT technology makes it possible to acquire various data related to people’s lives in real time, which could not be collected in the past . This information is collected and accumulated as big data and utilized in various areas such as services, products, and marketing.
AI x Big Data
Big data acquired by IoT devices is analyzed using AI. Based on the results, a new AI model is born and used for services and products.
If a new AI model is installed in an IoT device, it will have more functionality than before.
By repeating the cycle of IoT → big data → AI over and over again, it is expected that better data and AI models will continue to be created.
How to introduce and utilize big data
In order to actually utilize big data in business, the following data-driven implementation process is required.
Data collection and accumulation
In order to start using data, it is essential to have a platform for collecting and accumulating data . It is common to collect data from corporate business systems, mission-critical systems, web servers, IoT devices, external services, tools of other companies, etc.
Data visualization
Before analyzing the collected data, the process of organizing the information and visualizing it in an easy-to-understand manner in order to objectively grasp what kind of content is contained in the huge amount of data.
Data analysis/analysis
Based on the processed data, analysis is performed according to the problem to be solved. The point here is to derive not only quantitative data such as ranks and maximum and minimum values, but also qualitative data such as changes and trends.
After going through the process up to this point, it is finally possible to utilize the knowledge obtained from the data in measures.
Examples of big data utilization (by industry)
Big data is used in various fields. Here is an example.
[Retail] Demand forecasting and inventory management
In the retail industry, grasping changes in demand due to seasonal factors and other factors, as well as associated production and inventory management, are often issues.
By utilizing big data, it is possible to forecast demand based on data such as past sales results, user behavior at online shops, and weather . By utilizing demand forecasting, it is possible to reduce costs by resolving inventory shortages and excesses, and to make personalized recommendations. In addition, in the case of mass-produced items, it becomes easier to plan the production volume, which leads to improved work efficiency.
Hitachi, Ltd. uses AI to solve problems such as inventory management and disposal loss and resale loss in ordering operations. We calculate highly accurate demand forecast values and order quantities, and support the efficiency of ordering operations and inventory optimization.
[Medical] Diagnostic Assistant
When making a decision, searching for information that can be used as a basis for judgment from a huge amount of data each time is a time-consuming task. However, if we build a system in which AI analyzes big data, it will be possible to quickly extract accurate information from a vast amount of information and make decisions .
In recent years, attempts to combine big data and AI to extract appropriate information from a huge amount of information such as cancer treatment guidelines, abstracts of medical literature, and public library data, and to support doctors’ diagnosis in the medical field have become a hot topic. became.
[Agriculture] Weather and harvest predictions
Attempts to use advanced technology in agriculture are attracting attention as “smart agriculture.”
Taking advantage of big data’s strength of being able to integrate real-time data in various formats such as images, sounds, and information from satellites, it is now possible to predict weather and yields and monitor product quality. .
Shonowa Co., Ltd. is a company that aims to solve various problems with smart agriculture using IoT and AI. The smart rice field service “paditch” is a service that allows you to easily check changes in rice fields and water temperature remotely using a PC, smartphone, or Galapagos mobile phone, and to open and close water gates and valves. This helps improve work efficiency and prevent farm work accidents.
Summary: Data utilization determines business success
Big data is a huge data group composed of various types and formats of data, including unstructured data.
By utilizing big data, it is possible to draw out the information necessary for decision-making and make highly accurate predictions. It is used to solve various problems. It can be said that the utilization of data is now becoming a major factor that determines the success of a business.
