Analysis of New RAID Technology in Security Storage

In recent years, with the continuous development of information technology and the popularization and application of security products and systems, data security issues in the security industry have received increasing attention. With the rapid development of high-definition surveillance, large-scale projects such as safe cities and smart cities have been rolled out, and the amount of data has been continuously increasing. As a result, the storage problems and data security problems brought about by big data have gradually become video network monitoring. New topics in the field.

The storage of security data is accompanied by the development of security technologies and storage technologies. Experienced in the simulation front-end + DVR storage, analog front-end + DVS / DVR + network storage, network front-end + NVR, network front-end + network storage and other stages. With the increase of network bandwidth, high-definition applications are becoming more and more common, and higher requirements have been placed on security storage systems.

First, new requirements for security data storage

Today's security development features can be summarized in the following three points:

1) HD

l High Line Simulation: 600/650/700 Line Analog Camera + High Resolution DVR

l Digital HD: SDI/HDSDI, HDCCTV

l HD Network: 1.3 million/2.0 million/5 million IPC+HDVR+NVR, 4K applications

2) Intelligent

l Engaged in post-responsibility and passively applied the transition to advance prevention and control and active defense to further liberate manpower and increase efficiency.

l From the technical level of intelligent analysis to the practical application stage, such as perimeter prevention, personnel tracking, etc., has been used in the public security, transportation, finance, and justice industries.

3) Fusion application

From the single system independent use to the multi-system platform integration use stage, the main performance

l Large-capacity video information collection and sharing

l Security system internal integration, butt upper business management system

l Multi-system integration, such as emergency linkage, emergency dispatch, and public security firefighting

l The integration of security and IT technologies

According to the new requirements of current security systems, the requirements for security storage systems can be summarized as follows.

Second, security storage technology: data security from the bottom

From the above analysis, it can be seen that the security system has entered the actual combat stage, and the actual operations of intelligent analysis, active defense, and fusion application must be established on a reliable monitoring data storage system. Here are two kinds of RAID technology, which can ensure the safe storage of data from the bottom.

1, VRAID (VedioRAID) technology

RAID technology is already familiar and belongs to the basic technology of disk array. After continuous development, there are now eight basic RAID levels from RAID 0 to RAID 7 and two-dimensional RAID like RAID 10 and RAID 50. Different RAID levels can provide users with the appropriate RAID performance and data security. However, any kind of RAID technology has a fault tolerance limit, that is, after the number of bad disks reaches a certain number, the entire RAID group fails and all of the data is lost. For example, in a traditional RAID 5, when a hard disk is damaged, RAID can ensure that the data is not lost through the redundancy mechanism, but when the hard disk is damaged by more than 2 blocks, it is unfortunate that RAID 5 can no longer restore the data of the hard disk for you. In general, it must be all data with you. Said bye bye!

The VRAID technology introduced here is a new RAID technology for security applications (Hikvision's new technology for video applications). VRAID technology can effectively solve the problem of complete loss of data in the entire RAID group after multiple (2 or more) hard disks in the traditional RAID are damaged. That is, in a VRAID, more than two disks are bad (ideally, only one available disk can be used). The data in the remaining disk is still readable, and new data can be written continuously.

Technical Principle: The implementation of VRAID is based on the streaming media data management structure and RAID optimization technology that is different from the traditional file system at the bottom of the storage device. When multiple disks in a VRAID are damaged, under the streaming media data management structure, the video data in each stripe of the good disk can still be played independently; when it is played to a bad disk stripe, it is skipped automatically and the next one is played naturally. The data on the strip ensures that the video screen plays smoothly. At the same time when new data is transmitted, it can still be written.

2, network RAID technology

Traditional RAID is limited by the number of hard disks and cannot meet the demand for unified and flexible deployment of resources in the era of drastic increase in data capacity. The trend of smart and efficient storage requires that the underlying data distribution architecture must be highly flexible and expandable. With the increase in hard disk capacity, managing data in hard disk units becomes more and more difficult. The general disk array can only do RAID within the same physical node, and a hard disk can only be located in a RAID group. When the controller is down or the disk is damaged, all the data is lost. This is traditional. Disk array RAID has a large disadvantage in terms of availability.

Network RAID is a scale out technology. Through network communication, RAID can be established between different physical nodes. And the basic unit that forms RAID is no longer the entire hard disk, but each hard disk Chunk block, and one hard disk can be in multiple RAID groups. When a physical node goes down, data of the entire system can be recovered through other physical nodes. When a single hard disk fails, multiple disks can participate in the reconstruction. The reconstruction process can be completed within minutes.

Technical principle:

1) Divide all hard disks of each physical node into Chunk blocks

2) Create a RAID based on the hard disk Chunk blocks of different physical nodes

The entire RAID is built based on Chunks, disks from different physical nodes. The entire RAID group can be constructed by all physical nodes (one or more hard disks).

The first set of RAID: 3+1 - 3 valid data, 1 check data

The second set of RAID: 3 + 2 - 3 valid data, 2 check data

3) Single hard disk failure and processing

A hard disk can be located in multiple sets of RAID. Therefore, when a single hard disk is damaged, multiple sets of RAID are jointly reconfigured. In the extreme case, all hard disks participate in the reconstruction calculation at the same time, and the speed is much higher than that of ordinary RAID. Even seconds.

4) The entire physical node failure and processing

Assume that the third device fails. In normal RAID, the data of the entire device is lost.

In the network RAID, the impact on each RAID is as shown below. Only part of Chunk data in each RAID group is lost. The remaining data blocks in the RAID group can be completely restored.

3+1 RAID 3+2 RAID

Although Network RAID is called RAID, it is actually a distributed storage technology. It can solve the problems of long reconstruction time under normal RAID, large management burden when multiple storage nodes are stacked, and can not meet hotspot requirements, data loss when nodes fail, and eliminates greater system availability risks.

In network RAID, the entire disk is no longer used as the management unit and check unit, disk segmentation (Chunk) management is divided into zeros, providing fine management, high-level fault tolerance, and high-performance data recovery, providing high data storage. reliability.

In the network RAID, the general disk error can be reconstructed in seconds; when the disk fails, a large number of disks participate in the reconstruction, and the reconstruction process is accelerated several times; the storage resources are distributed across nodes, the management is simpler, and more importantly, the entire When a node fails, data can be guaranteed to not be lost. The highly fault-tolerant design can be easily handled and implemented, whether it is a bad disk or a node replacement and expansion, thereby enhancing the continuous availability of the overall storage solution.

Third, security data storage future

The development of security storage must be the coordinated development of the underlying technology, physical equipment, and system architecture. Reliance on any point alone cannot meet the demand.

For common large, medium, and small-sized surveillance systems, video stream write-through storage architecture without storage servers and security-oriented VRAID technology can both simplify system deployment and fully protect user data security. At the same time, with the implementation of the GB/T28181 standard, video streaming direct-write storage solutions using standard streaming media protocols will also be increasingly favored by the market.

For ultra-large-scale monitoring systems (the 10,000-way video front-end), it is required to achieve the PB-level storage capacity. The single-device stacking and common file systems cannot provide secure and efficient application guarantees, and must be virtualized, clustered, and discretely stored. s solution. The video cloud storage technology for security applications adopts the concept of cloud computing. The underlying network RAID technology can provide users with convenient cloud storage services that are virtualized, clustered, and integrated.

Security cloud storage system features:

1) Virtualization technology fully integrates storage capacity, allocates on demand, and has the flexibility to adjust the size.

2) The front-end writes cloud storage directly through the video streaming protocol, saving user investment.

3) The video stream is balanced by a discrete algorithm and provides data concurrency services in a cluster response manner.

4) The bottom layer uses network RAID technology to protect user data at multiple levels.

5) The system-wide cluster working mode ensures real-time load balancing among devices and improves equipment utilization.

6) Globally integrated indexing technology to increase the query rate and achieve second-level response.

Security cloud storage solves the current security industry has begun to face the management and application of mass data storage, from 2012 to 2013, Hikvision has a number of safe city projects have been successfully applied, and through the actual use of the user has been very successful Video business application experience. In the next few years, in the construction of large and medium-sized security projects, security cloud storage will occupy the main position of video surveillance storage solutions. It is believed that security cloud storage will inevitably develop into a mainstream solution for the video surveillance storage market and will be the preferred choice. Program.