3.4. Data stream in Filter Graph This section mainly describes how media data flows in Filter Graph. If you just have to write a DirectShow application, you don't need to know these details, of course, you know that these details are still helpful for writing DirectShow applications. But if you want to write DirectShow Filter, you must master this part of knowledge. 3.4.1. DirectShow data flow exceeds how this part is roughly describing how the data flow in DirectShow works. The data is first saved in the buffer, in the buffer, they are just one byte array. Each buffer is included in a COM object called Media Sample, and Media Sample Provides an ImediaSample interface. Media Sample is created by another COM object called Distributor (Allocator), and Allocator provides an IMallocator interface. Each PIN connection specifies an allocator, of course, two or more PIN connections can also share several allocator.
Every Allocator creates a Media Sample pool and assigns buffers for each Sample. Once a filter requires a buffer to fill the data, it calls the Imemallocator :: GetBuffer method to request an Sample. As long as Allocator has a SAMPLE, it is not used by any filter, and the getBuffer method returns a Sample pointer. If all the Sample of Allocator has been used, this method is blocked there until there is a SAMPLE to become available. After getBuffer returns a SAMPLE, the Filter writes the data into the Sample buffer and sets the appropriate tag on the Sample, and then submit it to the next filter. When a renderer filter receives a SAMPLE, renderer filter checks the timestamp and saves Sample first until the Filter Graph's reference clock indicates that the data of this Sample can be rented. When Filter render, it releases Sample, at this time, SAMPLE does not immediately go back to Allocator's Sample pool, unless the reference count on this SAMPLE has changed to 0, indicating that all filters have released this Sample. .
The upstream Filter may run before renderer, which means that the upstream Filter fill buffer can be destroyed by renderer. However, despite this, Samples is not necessary to be earlier earlier, because rendere will keep them until the appropriate timing goes to render, and upstream Filter will not accidentally cover the buffers of these Samples, because the getSample method will only Returns those SAMPLE that is not used. The number of SAMPLEs that upstream Filter can use in advance depends on the number of samples in the Allocator assignment pool. The front chart only shows an allocator, but in general, there will be multiple allocator in each stream. Therefore, when rendere releases an SAMPLE, it produces a cascading effect. As shown in the figure below, a Decoder saves a video compressed frame, which is waiting for Renderer to release a Sample, and Parser Filter is also released a Sample in Decoder.
When Rendere releases a SAMPLE, Decoder completes the unfinished getBuffer call. The Decoder can then decode the compressed video frame and release it saved Sample so that Parser completes its getBuffer call. 3.4.2. Transports (Transports) In order for media data to flow in Filter Graph, DirectShow Filter must support one of multiple protocols, which are called Transports. When two Filter are connected, they must support the same transport protocol, otherwise they will not exchange data. Typically, a transport protocol requires a certain PIN supporting a specific interface, when two Filter connections, another pin calls this interface of this PIN. Most DirectShow Filter saves media data in the main memory and submits data to another through the PIN connection, which is called local memory transfer protocol (Local Memory Transport). Although this type of transport protocol is most common in DirectShow, it is not all filter to use it. For example, some Filter passes data by hardware pathways, using PIN is just to deliver control information, such as the IOverlay interface. DirectShow defines two mechanisms, push mode and pull mode for a local memory transfer protocol. In push mode, Source Filter generates data and submits it to the downstream file, passively receives data passively and processes them, and then passes the data to its downstream file. In the pull mode, Source Filter is connected to a Parser Filter, the Parser Filter requests data to the Source Filter, and the source filter responds to the request and passes the data. Push mode uses the IMEMINPUTPIN interface and pull the pattern uses the IASYNCREADER interface. Push the mode than a wider range. 3.4.3. Media Sample (SAMPLE) and Distributor (Allocator) When a PIN passes media data to another, it does not directly pass a memory buffer pointer, but a pointer to a COM object, this COM The object manages memory buffers, called media samples, exposes the IMEDIASAMPLE interface. Receiver PIN accesss memory buffer by calling an IMEDIASAMPLE interface, such as iMediaSample:: getPointer, iMediaSample :: getSize and iMediaSample :: getActualDatalength. SAMPLE always transmits down from the output PIN to the input PIN. In the push mode, the output PIN passes an Sample by adjusting the IMEMINPUTPIN :: Receive method on the input PIN. Enter the PIN or process the data in synchronously inside the Receive method, or another work thread perform asynchronous processing. If you enter the PIN requires a wait for resources, allow it to block in Receive. Another COM object used to manage media samples, referred to as a dispenser, which exposes the IMallocator interface. Once a FILTER requires an idle media sample, it calls the Imemallocator :: GetBuffer method to get the Sample pointer. Each PIN connection shares an Allocator, when two PIN connections, they negotiate to decide which filter to provide allocator. PIN can set the properties of allocator, such as the number of buffers and the size of each buffer.
The following figure shows the relationship between Allocator, Media Sample, and Filter:
Media Sample Reference Count One Allocator creates a Sample pool with limited Sample. At some point, some SAMPLE is being used, some can be used by the GetBuffer method. Allocator uses the reference count to track the Sample reference count returned by the getBuffer method. If the reference count becomes 0, SAMPLE can return to the Sample pool of Allocator, so that it can be used by the getBuffer method again. During the reference count is greater than 0, SAMPLE cannot be used by GetBuffer. If each of the SAMPLEs belonging to allocator is being used, the getBuffer method is blocked until some SAMPLE can be used. For example, assume that an input PIN receives a SAMPLE. If it handles it in the Receive method, the Sample's reference count does not increase. When Receive returns, the output PIN releases this sample, the reference count belongs to 0, and SAMPLE returns to the Sample pool. Alternatively, if you processed SAMPLE, it will add the SAMPLE reference count before the Receive method returns 1, and the reference count becomes 2. When the output PIN releases this SAMPLE, the reference count becomes 1. SAMPLE cannot return to the Sample pool until the work thread of the asynchronous processing completes the work, calling Release release this Sample, the reference count is 0, it can return to Sample pool. When a PIN receives a SAMPLE, it can copy data to another Sample or modify the original SAMPLE and pass it to the next filter. A SAMPLE may be passed throughout the Graph length, and each filter calls AddRef and Release. Thus, the output PIN must not reuse the same sample after calling Receive, because the downstream file may be using this Sample. Output PIN can only call GetBuffer to get new Sample. This mechanism reduces the total memory allocation process because filter can reuse the same buffer. It also prevents the data from being accidentally covered before being processed. When the amount of data is large after the Filter processing data, the amount of decoded data is decoded, and a FILTER can assign different allocator to input PIN and output PIN. If the output data is not larger than the input data, the filter can process data without copying it to a new Sample, in which case two or more PIN connections share an Allocator. Submit (Commit) and Decommit Distributor When a Filter creates an allocator, allocator does not assign memory buffering, if the getBuffer method will fail. When the stream begins to flow, the output PIN calls Imemallocator :: Commit to submit the AlLocator to allocate memory. At this point, PIN can call GetBuffer. When the flow stops, the PIN calls IMEMALLOCATOR :: DeCommit to reverse the ALLOCATOR, and later getBuffer calls will fail before the Allocator is submitted again, and similarly, if there is blocking, it is waiting for the getBuffer call, will return failed information immediately. . Decommit methods release memory depends on implementations, such as CMEMallocator classes until the memory is released.
3.4.4. Filter State Filter has three possible status: Stopped, ready (PAUSED) and Running. The purpose of the ready state is to make Graph to be prepared in advance to respond immediately when the Run command is issued. Filter Graph Manager controls all state conversions. When an application calls iMediaControl :: Run, iMediaControl :: Pause, iMediaControl :: Stop, Filter Graph Manager, Filter Graph Manager, invasive IMEDIAFILTER methods on all Filter. When the stop state and the running state are always ready, if the application is called with RUN on a GRAPH in a stopped state, the Filter Graph Manager will turn it to the PAUSE state before running it. For most Filter, the operating status and ready state are equivalent. Look at this graph: source> Transform> Renderre assumes that this source filter is not a real-time acquisition source. When Source Filter is ready, it creates a thread to generate new data as quickly as possible and write to Media Sample. Threads Tune the IMEMINPUTPIN method to the downstream file by calling the IMEMINPUTPIN method on the input PIN of Transform Filter. Transform Filter receives data in the Source Filter thread, which may also use a work thread to deliver Sample to Renderer, but in general, it delivers them in the same thread. Such as renderer processing, it waits for receiving Sample, when it receives one, it or blocks or saves the Sample, if this is a video render, it displays Sample as a static picture, only when necessary Refresh it. At this time, the stream is ready to be fully removed. If Graph still processes ready state, Sample will accumulate after each Sample until each Filter is blocked under Receive or GetBuffer. No data will be lost. Once the Source thread is blocked, it is just simply recover from the blocking point. Source Filter and Transform Filter ignore from ready state - they are only as fast as possible to process data. But when rendere runs, it will start render sample. First, it render Samples saved in ready state, then each receives a new Sample, which calculates the presentation time, rendere saves each Sample until their rendering time reender. When waiting for a suitable presentation time, it or blocks on the Receive method, or receives data in a working thread and put it into the queue. Renderer's previous Filter does not care about these issues. Real-time source, such as acquisition equipment, is an exception in the usual situation. In the real-time source, it is not suitable for preparing data in advance. The application may put Graph in ready state, and then wait for a long time to run it again. GRAPH should not render Sample during render, so a real-time source does not generate new Sample when ready state. To notify this situation to the Filter Graph Manager, Source Filter's iMediafilter :: getState method Returns VFW_S_CANT_CUE.
This return value indicates that the Filter has been switched to the ready state even if rendere has not received any data. When a FILTER is stopped, it no longer receives any data passed to it. Source Filter Close their streaming threads, other Filter closes all their work threads you created. PIN reverse submit (Decommit) their allocator. Status Conversion Filter Graph Manager Completes all state conversions by the order from downstream file to upstream file, starting from rendere until Source Filter, this order is necessary, prevent data loss or graph lock lock. The most important state conversion is the conversion between ready state and stop state: * Stop state to ready state: When each Filter is ready to be read, it is ready to receive SAMPLE from the previous Filter. Source Filter is the last Filter that is placed in a ready state, which creates a data stream thread and starts to pass the SAMPLE. Because all downstream filters are ready, no Filter will refuse to receive SAMPLE. When all renderer in Graph receives a SAMPLE, the Filter Graph Manager completely completes the status conversion work (except for real-time sources). * Ready Status to Stop Status: When a FILTER is stopped, it releases all the Samples you saved, will release all upstream filter to call GetBuffer. If Filter is waiting for data in the Receive method, it stops waiting and returning from the Receive to relieve the blocking. Thus, when the Filter Graph Manager converts upstream filter to a stop state, it has no longer blocking the getBuffer and Receive, so that the stop command can be responded. Upstream FILTER may pass some outdated Sample before getting the stop command, but downstream file no longer receives them, because the downstream file is already in a stopped state. 3.4.5. Pull mode In the ImeminputPin interface, upstream file determines which data is to be sent, then push the data to the downstream file. However, in some cases, the pull mode will be more appropriate. In the pull mode, only the data is passed when the data is requested from the upstream file, and the data is initiated by the downstream Filter. This type of connection uses an IasyncReader interface. Typical pull mode applications are file playback. For example, in an AVI playback Graph, the Async File Source Filter completes a general file read operation and passes the data as a byte stream, and there is no format information. Avi Splitter Filter reads AVI heads and decomposes data streams into video and audio Sample. Avi Splitter is more than the Async File Source Filter to make what data they need, so you need to replace the IMEMINPUTPIN interface with the IASYNCReader interface. To request data from the output PIN, enter a PIN call one of the methods: * iasyncReader :: request * ivreader :: syncread * ivreadaligneder :: SyncReadALIGNED The first method is asynchronous, support multi-read operation. The rest is synchronized. In theory, all Filter can support IASYNCREADER, but in fact, it is only used on the Source Filter connected to a Parser Filter.
