Processing of streaming data with keyed aggregation

a technology of streaming data and keyed aggregation, applied in the field of processing streaming data, can solve the problems of excessive amount of data processed, excessive amount of resources consumed, and overwhelming amount of data output by these systems

Inactive Publication Date: 2011-02-17
IBM CORP
View PDF17 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Benefits of technology

This patent describes a way to process streaming data quickly without having to manually check if any changes have happened during the analysis. It uses a special software that can determine when certain events occur and replace them with older ones based on their importance. These modifications help improve the accuracy of the analysis.

Problems solved by technology

The technical problem addressed in this patent is how to improve processing of streaming data while maintaining their accuracy.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Processing of streaming data with keyed aggregation
  • Processing of streaming data with keyed aggregation
  • Processing of streaming data with keyed aggregation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019]In accordance with an aspect of the present invention, a capability is provided for improving processing of streaming data by employing a keyed aggregation operator that uses replacement of older information with more recent information. A capability is provided for aggregating the values from a stream of data events, in which, for example, there is a flow of events (i.e., records) where some events are for the same entity as previous events and they replace the older events (i.e., records related to an entity received earlier), and these replaced events no longer factor in the aggregation calculations.

[0020]As one example, in order to incrementally calculate aggregations even in light of events for specific entities being replaced by newer events for those entities, the most recent event for each unique replacement key value(s) is maintained, running aggregations are kept, and the aggregations are output. When a new event arrives, in one example, if a previous event exists for t

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

Keyed aggregation is used in the processing of streaming data to streamline processing to provide higher throughput and decreased use of resources. The most recent event for each unique replacement key value(s) is maintained. In response to an incoming event having a same key as a previous event, the effect on an aggregation of the previous event is removed. The aggregation is then updated with one or more values from the arriving event and the updated aggregation is output.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Owner IBM CORP
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products