Batch data processing method and device, computer equipment and storage medium

A data processing and batch data technology, applied in the field of big data, can solve problems such as unreasonable system resources, achieve the effects of improving determination efficiency, reducing the amount of single-processing data, and speeding up

Pending Publication Date: 2019-10-01
PING AN TECH (SHENZHEN) CO LTD
View PDF7 Cites 34 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Benefits of technology

This patented technology allows for efficient use of computing systems by selecting specific jobs during certain periods while minimizing interruptions or delays caused by other processes waiting therefor. It also uses an algorithm called Segmenting Process (SP) to determine if any job needs more work before it goes through its entire process. If this requires extra space then allocating some spare processor capacity instead of doing everything else. By optimally assigning these resources, the overall effect will improve server performance without sacrificing speedup over time.

Problems solved by technology

The technical problem addressed by this patented method relates to efficiently handling huge amounts of data while performing certain types of operations on them within one period or unitary cycle. This can lead to slowdown times caused by overlapping systems' workload, delayed responses from user requests, etc., leading to poor performance and potential issues such as data backup delays associated with these processes.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Batch data processing method and device, computer equipment and storage medium
  • Batch data processing method and device, computer equipment and storage medium
  • Batch data processing method and device, computer equipment and storage medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0031] The batch data processing method provided by the embodiment of the present invention, the batch data processing method can be applied as figure 1 shown in the application environment. Specifically, the batch data processing method is applied in a batch data processing system, and the batch data processing system includes such as figure 1 The client and server shown in the figure, the client and the server communicate through the network to accurat

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a batch data processing method and device, computer equipment and a storage medium. The batch data processing method comprises the steps: selecting a target batch task from a non-real-time task queue, and creating a data processing queue based on the target batch task; determining target idle information from the target idle time queue, obtaining the number of to-be-processed data corresponding to the to-be-processed data, obtaining the target number based on the number of the to-be-processed data, the target idle duration and the estimated thread number, selecting theto-be-processed data corresponding to the target number from the data processing queue, and determining the to-be-processed data as segmentation processing data; when the original load of the system is smaller than a busy load threshold value, obtaining a target processing thread, adopting the target processing thread to perform data processing on the segmentation processing data in the target idle duration, and obtaining a data processing result; and based on the data processing result, updating the task state of each split processing data in the data processing queue. According to the batchdata processing method, system resources can be reasonably distributed, and the batch data processing efficiency is ensured.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Owner PING AN TECH (SHENZHEN) CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products