Scribd Logo
Upload

Welcome to Scribd!

  • 47 views

    Introduction To Stream Concepts - Stream Data Model and Architecture

    Uploaded by

    Anyam Mohan
    This document introduces stream data concepts. It discusses how data stream management systems are needed to analyze real-time streaming data from various sources like financial, network, security, and web applications. It describes key aspects of the streaming data model including that data arrives continuously and cannot be stored entirely, requiring approximate summaries. It also outlines characteristics stream queries must have to process unbounded data and challenges of mining streaming data at high volume and velocity with concept drift.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Introduction To Stream Concepts - Stream Data Model and Architecture

    Uploaded by

    Anyam Mohan
    47 views8 pages
    This document introduces stream data concepts. It discusses how data stream management systems are needed to analyze real-time streaming data from various sources like financial, network, security, and web applications. It describes key aspects of the streaming data model including that data arrives continuously and cannot be stored entirely, requiring approximate summaries. It also outlines characteristics stream queries must have to process unbounded data and challenges of mining streaming data at high volume and velocity with concept drift.

    Original Title

    1567964522

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document introduces stream data concepts. It discusses how data stream management systems are needed to analyze real-time streaming data from various sources like financial, network, security, and web applications. It describes key aspects of the streaming data model including that data arrives continuously and cannot be stored entirely, requiring approximate summaries. It also outlines characteristics stream queries must have to process unbounded data and challenges of mining streaming data at high volume and velocity with concept drift.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    47 views8 pages

    Introduction To Stream Concepts - Stream Data Model and Architecture

    Uploaded by

    Anyam Mohan
    This document introduces stream data concepts. It discusses how data stream management systems are needed to analyze real-time streaming data from various sources like financial, network, security, and web applications. It describes key aspects of the streaming data model including that data arrives continuously and cannot be stored entirely, requiring approximate summaries. It also outlines characteristics stream queries must have to process unbounded data and challenges of mining streaming data at high volume and velocity with concept drift.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 8

    Introduction to Stream Concepts –

    Stream Data model and architecture

    9/8/2019 1
    • Data stream real time analytics are needed to manage the data
    currently generated, at an ever increasing rate from these
    applications.
    • Examples:
    • Financial
    • Network monitoring
    • Security
    • Telecommunications data management
    • Web applications
    • Manufacturing
    • Sensor networks
    • Email
    • blogging

    9/8/2019 2
    Data Stream
    Management System
    • Traditional databases store and retrieve data
    in static manner
    • To manage streaming data – DSMS – with
    continuous query languages and query
    evaluation.

    9/8/2019 3
    Data Stream Model
    • A data stream is a real time, continuous and
    ordered sequence of items.
    • Not possible to control the order in which the
    items arrive, nor it is feasible to locally store a
    stream in its entirety in any memory device

    9/8/2019 4
    Characteristics of data
    stream model
    • Data model and query processor must allow both order-based and time-
    based operations
    • Inability to store a complete stream indicates that some approximate
    summary structures must be used.
    • Streaming query plans must not use any operators that require the entire
    input before any results are produced.
    • Any query that requires backtracking over a data stream is infeasible. This
    is due to storage and performance constraints imposed by a data stream
    • Applications that monitor streams in real-time must react quickly to
    unusual data values.
    • Scalability requirements dictate that parallel and shared execution of
    many continuous queries must be possible.

    9/8/2019 5
    Data stored into 3 partitions
    • Temporary working storage
    • Summary storage
    • Static storage for meta-data

    9/8/2019 6
    Data Stream Mining
    • Data stream mining is the process of extracting useful
    knowledge from continuous, rapid data streams.
    • Three challenges during mining of big data streams
    • Volume
    • Velocity
    • Volatility
    • Volume and velocity require high volume of data to be
    processed in limited time
    • Concept Drift is a phenomenon that occurs because of feature
    changes or changes in behaviour of the data itself.

    9/8/2019 7
    Examples of Data
    Stream Applications
    • Sensor networks
    • Network Traffic analysis
    • Financial Applications
    • Transaction Log Analysis

    9/8/2019 8

    You might also like