DirectShow RTP support for network multimedia applications
Lucky2all@yahoo.com
Intel Architecture Laboratory
Foreword
Interactive collaboration applications, or a distributed game of many independent multimedia programs, running synchronously and / or playing multiple audio and video streams while running. As a single stream changes and stream / applications are started or finally the demand for resources, the total amount of resources available will change. Network Multimedia Application (NETMM) must be prepared to adapt to these changes, using them to provide users with different levels of services that can accept. This paper focuses on the problems that add network and host adaptability to component-based DirectShow RTP.
DirectShow is Microsoft's set of system architectures for video data acquisition and display. DirectShow RTP is a framework that expands the DirectShow system architecture, which adds support for the use of RTP protocols through network transmission multimedia application data. The DirectShow RTP framework is designed to support multimedia flow tasks for high-scalability. DirectShow RTP is sent as part of the operating system as part of the operating system as part of the WINDOWS NT5.0. We have expanded this architecture that adds support for convection applications, which can be dynamically compensated for local hosts and computer networks to change the available resources due to distribution and receiving multimedia data.
Our extensions include information that can be adapted to make options, making decisions based on this information, and methods that can be implemented in the infrastructure. This article is useful for application development and architectural design for developing these applications.
1. Introduction
NETMM is a class that is the strongest in the executive of the client computer and a single user workstation. These applications have high requirements for host processing capabilities and network bandwidth. These applications also often require the underlying operating system and the computer network to approach the necessary resources that can perform in real time. The delay in any of the above resource access will lead to a significant decline in the quality of the user. With the changes in the demand for resources, the live and network availability resources will change significantly with the start and stop of the single NETMM program. Because the available resources and demand resources will change significantly during operation, NETMM must be able to adapt to these changes at any time.
In this paper, we mainly target two types of adaptations ------ network adaptation and host adaptation. Network adaptation refers to the ability to make full utilization of network resources through a variety of ways with bandwidth, network jitter, data loss, etc. Host adaptations can be defined as applications based on local hosts, including CPU utilization, available memory, to change their behavior. The examples of the examples listed below are useful to network and host adaptation:
Multi-stream competition for resources. Suppose an application has an audio stream, a high bit rate video stream, and a burst slide stream, where the audio stream is the highest priority to the user. If the quality of the audio stream is affected, in order to satisfy the user's priority, the program can adapt the slide show and video stream to reduce system resources. Applications must also detect competition for network resources, and adjust it accordingly to send and receive behavior of each stream.
Allow users who have different networks and processor resources to experience. All nodes will not receive all streams at a video conference or interaction meeting of heterogeneous users of different bandwidth and processing capabilities. Under this condition, all conference staff can participate in this classification code. This adaptation form is compared to a display that does not use the hierarchical coding stream, allowing heterogeneous participants with very little resource requirements to achieve great satisfaction.
Compensate changes in application roles of different importance. In a single user environment, an important degree of a program relative to other programs will change over time. For example, when a user is switched from a view news broadcast to a compiled or design task, an operating system like Windows 95 generally scheders a multimedia program to the background running at a lower priority. When the user switches to the news broadcast, go to the incident of interest (such as the latest baseball message), the operating system will increase the execution priority of this program accordingly. NETMM should also respond to the background or reception. In this way, the NETMM program should reduce the network and CPU usage after switching to the background, and return to the front desk. The above additional response and priority scheduling strategy that exceeds the operating system are assigned to those with an urgent need. Researchers have prepared several ways to adapt to NETMM programs. These methods include different forms of source stream control, adopting the accepted drive of hierarchical video, and adaptation of host resources. Although the protocols and mechanisms like RSVP and ATM provide QoS protection have been able to meet NETMM programs, researchers have been adapted under the conditions of resource constraints and resource changes.
Assuming that the impact of adaptation to application quality is positive, we decided to study how the application developers can not worry about the complexity of adaptation, add advice to applications. In this paper, the NETMM programs are studied in detail how the multimedia stream component architecture is adapted to network and host conditions. The paper discusses our results in this area, including new research on resource restrictions, creating middleware for NETMM supporting network host adaptation, and some application developers or architect designers using adaptive A beneficial lesson.
We use Microsoft's DirectShow as the foundation of our architecture. DirectShow provides a modular and scalable system that implements multimedia applications. In the DirectShow architecture, we add a framework for implementing the RTP protocol, we call DirectShow RTP, used to create a NETMM application,
The remainder of this thesis is organized as follows. Part 2 tells the DirectShow architecture and DirectShow RTP. Part 3 discussed this, analyzing some methods of adding adaptive functions to components based on DirectShow RTP. Part 4 gives the adaptation implementation based on the data source, Part 5, introduces the adaptation implementation of the reception drive of the layer multicast. Section 6 provides an overall empirical analysis of the implementation of components-based flow frame networks and host adaptation implementation. Section 7 discusses the work fields that may further improve the next step of adaptation in NETMM. Finally, this article is ended in accordance with the appropriate discussion of adaptation in the component architecture.
2. DirectShow architecture
2.1 Microsoft DirectShow
The DirectShow architecture is used to provide basic multimedia stream processing functions for many applications. It is used in all-round multimedia processing, including collection, codec, playback, and audio and video data storage. DirectShow uses four main abstractions to operate multimedia data. These abstract terms are filters, pin, media samples, and media types (Media Types).
MPEG File Mpeg Decoder Video Render
Data Flow
DirectShow Filters is used to encapsulate one or more tasks related to multimedia flow. The DirectShow Filters example contains video acquisition filter, which is used to control video camera devices, output raw RGB video frames; H.261 codec FILTER, is used to compress the original RGB video data to H.261 frames and decompression. There are similar filters, audio acquisitions, Filter, and G.711 codec for audio stream. Filter is also used back to play video on local devices. To allow program integrated features to process audio or video, DirectShow uses Filter Graphs. Filter Graph is a single serial connection, processing multimedia data buffer Filters. Figure 1 shows an example filter graph that contains a file read filter, an MPEG decoding filter, and a video rendering filter for playback of the video. Filters Connect to the Filter Graph through the Pins. Pins has two main functions in DirectShow. The first is to negotiate MEDIA TYPE, and the allocation memory when the FILTER is interconnected at Filters. Media Type negotiation is a way for Media Type management two identified Filters data interactions. The memory allocation negotiation is used to specify where to save where the multimedia buffer (also known as MedIA Sample) is allocated, allocating how large memory (byte alignment, using memory from the memory map device, etc.).
The second task of Pins in DirectShow is to hide the way of exchange data between Filters. Once a link has been successfully established, Filters just simply receives and distributes Media Samples to its Pins, and Pins performs actual operations, that is, Samples is distributed to the Filter Graph in the next filter.
Media Samples is an abstraction of the storage multimedia data buffer in DirectShow. In addition to the multimedia data that they contain, Media Samples also contains start and termination timestamps for determining the Samples Life. These values can be used by the renderer to determine when playing and detecting performance issues.
DirectShow Media Types Specifies the format of the data between the Filters that contains the data containing the Media Sample. Media Types contains several parts; most of which are Major and Minor types. A primary type is generally used in the semantic guidance of the highest level. Majortype_video and Majortype_Audio are two kinds of main types. Types are generally used to determine different formats, such as minorType_audio_g711a and minortype_audio_g723. If the Pins between the two filters can find the same Media Type, then they can establish a connection. DirectSHOW allows you to define new Filters, Pins, and Media Types. Take advantage of DirectShow's inner scalability, we define two new Media Types and several filters to implement multimedia data streams through the network under the DirectShow architecture using RTP protocols. This new NetMM frame is called DirectShow RTP.
2.2 DirectShow RTP
DirectShow RTP defines a set of Fitlers and Media Types that supports multimedia streams using RTP protocol networks.
New defined filters are RTP Source Filter, RTP Render Filter, RTP Demux Filter, RTP Receive Payload Handler (RPH) Filter, and RTP Send Payload (SPH) Filter. It is possible to construct a data platform application that can pass the RTP protocol network traffic video stream using these five Filters (simultaneous use of standard CODEC and acquisition / rendering filters). RTP Source Filter is used to receive RTP and RTCP packages from a separate RTP session. These packages are encapsulated in Media Sample to Filter Graph. This Filter is used to connect to broadcast Media Type, including primary types.
RTP_MAJORTYPE_MULTIPLE_STREAM and the second type RTP_MINORTYPE_PAYLOAD_ANY, which are part of the DirectShow RTP framework. Such a Media Types combination indicates that this filter can generate a stream containing one or more RTP streams, which may be a single load type or multi-load type. This Filter provides an interface specified to other host RTCP receivers and specifying network addresses and port interfaces to receive RTP sessions.
The RTP Source Filter is RTP Render Filter. This Filter receives the Media Type primary type to RTP_majortype_single_stream, and any type of external connection. In order to comply with the AVP to the sender's regulations on the RTP stream on load type and SSRC, the selection of more stringent primary types is necessary. This Filter provides an interface similar to RTP Source Filter.
RTP Demux Filter is used to separate RTP packages from RTP Source Filter. Multi-channel separation follows the load type of SSRC and each package. This Filter receives the main type RTP_majortype_multiple_stream and the PIN of the Type of RTP_MINORTYPE_PAYLOAD_ANY. This Filter registers one or more outputs of PIN,
The main type is RTP_MAJORTYPE_SINGLE_STREAM, which is associated with a single distribution stream load on the PIN that is discussed. This Filter provides interfaces for how to control how multi-channel separation and how to assign to a specific output PIN.
RTP RPH Filter is used to transform the RTP package from a single fixed load type source for their corresponding unpacking form. Therefore, one version of this Filter is used to obtain RTP H.261 packets and generate H.261 compressed video frames. This version of Filter has been designed to support H.261, H.263, INDEO, G.711, G.723, and G.729 and common multi-audio and video load types. These common audio and video RPH filters compliant with the RTP AVP specified to package Media Sample. It provides a specified number of buffer interfaces to resolve the waiting time before data loss (or forwarding), especially when the frame is rebuilt, and the buffer weight is required. RTP SPH Fitler is similar to RTP RPH Filter, and its task is to decompose the Decomposition of the audio and video compression FILTER to the RTP package. It provides the interface, specifying the maximum generated package size (for the MTU (maximum transmission unit)) and PT values for different networks (allowing the dynamic RTP PT value).
Video Capture Codec SPH RTP Render
Data Flow
Figure 2 Sending network data using DirectShow RTP
RTP Source RTP Demux Rph Decoder Video Render
Data Flow Figure 3 Using DIECTSHOW RTP Receive Network Data
Figures 2 and 3 show how Filters defined in DirectShow RTP is used. Figure 2 is a Filter GRAPH that collects local multimedia data and transmitted through the network using the RTP protocol. It contains a video acquisition filter of an output original video frame, following the encoded filter of a compressed frame. Once compressed, these frames are sent to RTP SPH Filter, fragmentation packages, generate RTP packets, and transmit them to RTP Render Filter, transfer them over the network. Figure 3 shows a filter graph to receive the video stream RTP package, play video. This Graph is composed of RTP Demux Filter, which is used to receive a package, a RTP Demux filter that is classified according to the source and load type, and a RTP RPH Filter that converts the RTP package to a compressed video frame. These filters are then used to decode the decoded filter that displays the uncompressed frame.
The most important thing in the DirectShow RTP framework is the definition of Media Type used to display the RTP stream and the five filters used to receive, process, and send the RTP package. The definition of Media Type provides a useful way to describe the RTP stream in the DirectShow system structure, allowing future definitions of files that can add new processing RTP flow methods. The implementation of these Filter provides a set of components that can be easily created based on this NETMM application and provide a system structure that can be used in further research in the future NETMM.
