Huge amounts of data are generated by smart meters, power distribution automated devices, digital protection devices, and other intelligent devices in the smart grid, thus forming the electric power big data. It is very important to compress the data to relieve transmission pressure on communication lines and reduce the storage overhead of data centers, as well as to enhance the efficiency of data mining.
There are many machine learning and data analysis methods used in different areas of smart metering.
The load profiles of different consumers on different days are diverse. They are used to find the basic electricity consumption patterns of each consumer or a group of consumers. Having a better understanding of the volatility and uncertainty of the massive load profiles is very important for further load analysis. The results can be further used to train training a model such as a forecasting model or clustering model.
Load forecasts have been widely used by the electric power industry. Power distribution companies rely on short- and long-term forecasts at the feeder level to support operations and planning processes, while retail electricity providers make pricing, procurement and hedging decisions largely based on the forecasted load of their customers. How smart meter data contribute to the implementation of load management is summarized from three aspects in this section: the first one is to have a better understanding of sociodemographic information of consumers to provide better and personalized service. The second one is to target the potential consumers for demand response program marketing. The third one is the issue related to demand response program implementation including price design for price-based demand response and baseline estimation for incentive-based demand response [9].
There are several academic research and applications on the diverse implementation of machine learning methods.
For example, a case study in the UK is presented with assuming 27 million domestic electricity consumers will generate data at a rate of approximately 13,000 records per second Its implemented platform is named Smart Meter Analytics Scaled by Hadoop (SMASH). It has demonstrated performing data storing, querying, analysis and visualization tasks on large data sets for smart meters.
A case study of Kunshan City in China is presented, using the daily electricity consumption data of 1312 low-voltage users within a month. The analysis is based on the fuzzy c-means (FCM) clustering method and a fuzzy cluster validity index (PBMF) to discover the electricity consumption patterns of residential users in China.