Build security video cloud storage, say goodbye to the troubles of traditional security video business!

security video store business cloud characteristics and challenges

Due to the lack of metadata index of traditional video data, video information cannot be located quickly and effectively when the precise event occurrence time cannot be determined, and a lot of manpower is required to retrieve the stock video information. As a result, huge labor costs are introduced, and more time is required to complete the video retrieval.

In addition, traditional security video has the following characteristics:

• Use RTMP/RTSP protocol to realize video streaming.
• Massive, reliable video storage is required.
• Look back only when there is an event and need to understand the reality.
• The review framework needs to ensure that it is available 7*24, and can be quickly accessed when needed to realize the review function.

The above features mean:

• Additional software/hardware resources need to be deployed in the cloud or on-premises to implement video stream reception, which introduces additional costs.
• Traditional local video storage is limited to a single copy, and there is a risk of video loss due to a single point of failure. The fixed local storage capacity makes it necessary to consider the storage facility capacity planning and deployment plan when increasing the number of cameras, which makes the project cycle longer and the project planning more complicated.
• The access frequency of the review frame is not high, but 7*24 availability means that even during the time period when there is no review demand, the corresponding system operating costs still need to be paid.

In view of the above difficulties and characteristics, that can be implanted into the IPC for the cost optimization scenario of IPC video cloud storage, and used the cloud hosting service provided by Amazon Cloud Technology to build video retrieval and review. The platform can reduce the basic operating cost of the review platform, and realize that the review cost is related to the cumulative review time. When the review platform is not used, there is almost no need to pay the operating cost of the platform, and when someone accesses it, it can quickly provide services.

📢 To learn more about the latest technology releases and practical innovations of Amazon Cloud Technology, please pay attention to the 2021 Amazon Cloud Technology China Summit held in Shanghai, Beijing and Shenzhen! Click on the image to sign up

Solution Architecture and Main Function Modules

System Architecture Diagram:

Main functional modules:

• IPC C SDK

The SDK embedded in the IPC is used to implement video segmentation and encapsulation, and store the video segments that can be viewed directly in the Amazon S3 bucket. And through IAM permission management, the management of IPC upload permissions is realized, so as to ensure that the upload permissions of each IPC video are independent of each other.

Use the SDK to directly upload videos to Amazon S3 buckets, so as to get rid of the resources required for local or cloud video streaming, and realize that the cloud storage cost is related to the accumulated video storage time.

• Amazon S3 video bucket

It is used to store IPC video fragment information, supports the use of Amazon S3 object tags and lifecycle management functions, and realizes video fragment storage cycle management and video storage cost optimization.

At the same time, using Amazon S3 to provide the performance of 3500 PUT and 5500 GET requests per prefix, using prefix naming rules, to achieve automatic expansion of cloud storage space performance, so as to use a unified framework to support from a few to millions of IPC devices .

• Frame the video and call the AI interface and DynamoDB metadata storage

In view of the difficulty in retrieval of traditional IPC video storage, AI services on the cloud are introduced to realize intelligent video analysis, and according to the video content, corresponding metadata information is generated and stored in DynamoDB. Users can choose to use the information in DynamoDB to retrieve video content, so as to quickly locate the required video clips and corresponding time, thereby effectively improving the efficiency of video retrieval and shortening the time required for video retrieval.

• AI service

Amazon Cloud Technology provides rich AI services for video analysis and retrieval. The main AI services currently supported can refer to the following solutions:

https://www.amazonaws.cn/solutions/ipc-ai-saas-solution/

Due to the large content of AI services and limited space, the solution will be introduced in a separate article. Stay tuned.

• Serverless Architecture Builds a Playback Platform

In order to effectively reduce the operating cost of the platform, especially the cost when there is no review request, this solution adopts a cloud-native architecture. It has extremely low cost when no user is used, supports fast startup in seconds when a request is received, automatic expansion, and the unified system architecture supports simultaneous access by several to millions of users.

IPC-C-SDK

Users can download the SDK code from Github and compile it into their own IPC:

https://github.com/aws-samples/ipc-h264-hls-c-sdk/

The internal working logic of IPC-C-SDK is as follows:

IPC-C-SDK obtains video and audio streams from IPC's internal video and audio encoders, and adds TS, PAT, PMT, and PES headers to form TS-format audio and video streams, and stores them in the IPC SoC memory buffer.

When the audio and video streams in the memory buffer reach a certain threshold, the IPC-C-SDK will automatically push the video content to the Amazon S3 bucket in the form of objects through the built-in Amazon S3 client.

Thanks to the IPC-C-SDK, video segments can be directly stored in Amazon S3 without the need for other audio and video stream processing equipment or resources. This effectively reduces costs to the point where only Amazon S3 storage and API calls are required. Cost optimization is achieved.

IPC-C-SDK uses Amazon Cloud Technology IAM for permission management. Support AK+SK and AK+SK+Token authentication modes. In the actual production environment, it is recommended to use Amazon IoT for device management, and cooperate with the IoT Credential Provider service to realize the replacement of IoT device certificates for temporary IAM identity information. This solution provides the best data security on the cloud, and supports the management of device permissions according to the device certificate ID, thereby ensuring the security of IPC cloud storage videos. Its reference architecture is as follows:

Users can refer to the following documents:
https://docs.aws.amazon.com/iot/latest/developerguide/authorizing-direct-aws.html

Amazon S3 Video Bucket

Amazon S3 video buckets offer up to 99.999999999% data durability. This means that if you save to Amazon S3 in 3-second shards, only one 3-second video shard is lost every 10,000 years. Thereby ensuring the security of the stored video.

In addition, when storing videos in the cloud, it is recommended to use the following formats for storage:

S3://{bucket name}/{device certificate ID}

/{year}/{month}/{day}/{hour}/{minute}/{second}.ts

This format can provide the following advantages:

• Secure data storage and access

Using the device certificate ID as the first-level prefix, the permissions of each IPC device are independent and isolated from each other. That is, each camera can only access the video segments under its own certificate ID prefix. Through IAM role configuration, it is possible to limit the IPC device to only upload operations, but not to download video segments, thus ensuring the security of video cloud storage.

• Automatic scaling of performance

Since each IPC belongs to a different prefix, Amazon S3 provides a performance of 3500 PUT and 5500 GET requests per second for each Amazon S3 prefix, so when the number of IPC cameras increases, Amazon S3 will automatically optimize and provide corresponding performance processing power. This enables the ability to support from a few to millions of IPC devices using a unified architecture.

video frame is extracted and the AI interface is called to store the result in

When video segments are uploaded to an Amazon S3 bucket, users can choose to use AI technology to analyze the video content. For example: detect faces and corresponding positions in videos, and save the results to DynamoDB for subsequent queries.

After video data is uploaded to Amazon S3, Amazon S3 event notifications are triggered. Through the cloud Lambda function, video key frames can be extracted, converted into JPEG format, and sent to the AI interface for analysis.

The results of the AI analysis will be returned to the Lambda function in JSON format, and the Lambda function will store the analysis result information in the DynamoDB database for retrieval by the review platform.

Architecture Building Video Review Platform

Thanks to the video slices saved in TS format, cloud-storage video slices can be connected to a whole video for review in any time period. Therefore, there is no need for secondary transcoding operations in the cloud. Thereby reducing system cost.

The serverless video review platform uses the open source Video.js framework and provides two review modes:

• Near real-time review (6-20 seconds delay, depending on network and client configuration)

In the near real-time review mode, the client requests the m3u8 playlist through the API Gateway HTTP API, the Lambda function checks the shards stored in the Amazon S3 bucket, and returns the last 3 video shards uploaded to the client within 2 minutes end. By scrolling and updating m3u8 to return the list of shards, the video content can be viewed in near real time.

• Check back at a specified time period

The client requests the m3u8 playlist through the API Gateway HTTP API, and provides the start and end time of the playback period. The Lambda function checks the shards stored in the Amazon S3 bucket, and adds the video shards in the specified time period to the m3u8 playlist in order and returns it to the client, so as to realize the video playback function of the specified time period.

Specified time period playback support retrieves the video metadata stored in DynamoDB, obtains the desired time period range for playback, and returns the result to the client for playback.

You can use the link below to deploy the Lookback Framework in one click in the Amazon cloud environment. The review application in the frame comes with a sample video for your reference.

https://console.aws.amazon.com/cloudformation/home?#/stacks/create/template?stackName=IPC-H264-HLS&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/ipc-c-sdk/v1.0.0/ipc-h264-hls-c-sdk.template

📢 July 21-July 22, Shanghai World Expo Center, 2021 Amazon Cloud Technology China Summit - Shanghai Station strong landing! If you haven't registered yet, please click the card above to register quickly! Builders who have successfully registered for the conference should also remember to attend the conference on time~

Author of this article

Li

Amazon Cloud Technology Solutions Architect

Xu

Amazon Cloud Technology Innovation Solutions Architect

Chen

Amazon Cloud Tech UI/UX Developer

Li Zhiqin

Amazon Cloud Technologies IoT Prototype Architect