DirectShow audio and video synchronization experiment report (2)

xiaoxiao2021-03-06  54

Single video stream

Filter Graph As shown in Figure 2:

Figure 2 Filter graph of single video stream

Note: Close to the last level of the previous level of Video Rendere, the VIDEOINFOHEADER structure provided by the GetMediaType function requires full, not only including the width, height, the number of pixel positions, but also the size of the BitmapInfoHeader structure (Bisize Specify as sizeof (bitmapInfoHeader), flat number (BIPLANES, specified as 1). If you need a palette, the BitmapInfoHeader data structure is also taken back to the palette data; if it is a 16-bit RGB format, the BitMapInfoHeader data structure is behind the RGB component extracted mask. The code reference is as follows:

VideoInfo mvideoinfo;

ZeromeMory (& MVideoInfo);

MVideoInfo.avgtimePerframe = 333667;

Mvideoinfo.bmiheader.bisize = sizeof (BitmapInfoHead);

MvideoInfo.Bmiheader.biWidth = 352;

MvideoInfo.Bmiheader.biheight = 240;

MVideoInfo.Bmiheader.biBitcount = 16;

MvideoInfo.Bmiheader.Biplanes = 1;

MvideoInfo.Bmiheader.Bicompression = BI_BITFIELDS;

MvideoInfo.Bmiheader.bisizeImage = mvideoinfo.bmiheader.biwidth *

MvideoInfo.Bmiheader.biheight * mvideoinfo.bmiheader.biBitcount / 8;

For (int i = 0; i <3; i ) // important for 16 bit rgb!

{

MvideoInfo.dwbitmasks [i] = bits565 [i];

}

1. Use Filter Graph Manager default reference clock

Note: If there is no Filter in the Filter Graph implements the IReferenceClock interface, the Filter Graph uses the system clock by default as a reference clock.

1.1 Video Sample does not call the timestamp, send out

Phenomenon: Video is played at the fastest speed. Video Renderer does not send a Quality Control message. The flow time is linear.

Msiavsrc.ax (TID 920) 2307: Stream Time: 120000

Msiavsrc.ax (TID 920) 2309: Stream Time: 140000

Msiavsrc.ax (TID 920) 2311: Stream Time: 160000

Msiavsrc.ax (TID 920) 2314: Street Time: 190000

Msiavsrc.ax (TID 920) 2316: Stream Time: 210000

Msiavsrc.ax (TID 920) 2318: Stream Time: 230000

Msiavsrc.ax (TID 920) 2320: Street Time: 250000

......

Msiavsrc.ax (TID 920) 3569: Stream Time: 12740000 (last Sample) 1.2 Video Sample Do not call the timestamp, the gap is sent (analog network blocker)

Phenomenon: Video Plays a meal. Video Renderer does not send a Quality Control message. The blocking of video data streams does not affect the flow time. The flow time is linearly increased, and the time in the gap is reflected in the flow time obtained twice before and after.

Msiavsrc.ax (TID 970) 2385: Street Time: 680000

Msiavsrc.ax (TID 970) 2389: Stream Time: 720000

MsiaVsrc.ax (TID 970) 4390: Sleep (2000) ... (Delivery of video playback)

MsiaVSRC.AX (TID 970) 4391: Stream Time: 20740000 (flow time reflects 2S of Sleep)

Msiavsrc.ax (TID 970) 4393: Street Time: 20760000

Msiavsrc.ax (TID 970) 4395: Street Time: 20780000

......

Msiavsrc.ax (TID 970) 39826: Stream Time: 375090000 (Last Sample)

1.3 Video Sample hits (continuous) timestamps, send out

Phenomenon: Video continuously play. Video Renderer Sends a Quality Control message for feedback control. The flow time is increased, and the flow time difference between the two acquisitions is roughly a frame display time.

Msiavsrc.ax (TID 970) 4105: Street Time: 3600000

Msiavsrc.ax (TID 970) 4106: Quality Control (Famine) Received.

Msiavsrc.ax (TID 970) 4139: Stream Time: 3940000

Msiavsrc.ax (TID 970) 4140: Quality Control (Famine) Received.

Msiavsrc.ax (TID 970) 4171: Street Time: 4260000

MsiaVSRC.AX (TID 970) 4172: Quality Control (Famine) received.

Msiavsrc.ax (TID 970) 4205: Street Time: 4600000

MsiaVSRC.AX (TID 970) 4206: Quality Control (Famine) received.

Msiavsrc.ax (TID 970) 4238: Stream Time: 4930000

MsiaVSRC.AX (TID 970) 4239: Quality Control (Famine) received.

......

Msiavsrc.ax (TID 970) 21922: Street Time: 181770000 (last Sample)

1.4 Video Sample Timing Time Poke (Time Touch, Time Stamp from 0), send out

Phenomenon: Playing is more continuous. Video Renderer Sends a Quality Control message for feedback control. There is a "fast lens" after the timestamp is reset, and the time difference transmitted by the adjacent SAMPLE is smaller (the display time should be a frame image), indicating that the video rendere is returned immediately after receiving SAMPLE, with the fastest speed processing SAMPLE Make the Sample Timestamp to catch the flow time again. Msiavsrc.ax (TID 964) 2133: Street Time: 12280000

Msiavsrc.ax (TID 964) 2165: Stream Time: 12600000

Msiavsrc.ax (TID 964) 2166: RESET TIME STAMP ...

Msiavsrc.ax (TID 964) 2199: Stream Time: 12940000 (Slow time increment)

Msiavsrc.ax (TID 964) 2201: Street Time: 12960000

Msiavsrc.ax (TID 964) 2203: Stream Time: 12980000

Msiavsrc.ax (TID 964) 2205: Stream Time: 13000000

Msiavsrc.ax (TID 964) 2208: Stream Time: 13030000

Msiavsrc.ax (TID 964) 2210: Stream Time: 13050000

Msiavsrc.ax (TID 964) 2212: Street Time: 13070000

Msiavsrc.ax (TID 964) 2214: Stream Time: 13090000

......

MsiaVSRC.AX (TID 964) 17748: Stream Time: 168430000 (last Sample)

1.5 Video Sample Timestamp (Timestam Time A delta), send out

Phenomenon: Playing continuous after waiting for Delta. Video Renderer Sends a Quality Control message for feedback control.

Msiavsrc.ax (TID 848) 821: Street Time: 4167195720000 (random value, this SAMPLE is sent out by Video Renderer)

Msiavsrc.ax (TID 848) 5828: Stream Time: 49940000 (Delta is reflected here)

Msiavsrc.ax (TID 848) 5860: Street Time: 50250000

Msiavsrc.ax (TID 848) 5892: Street Time: 50580000

Msiavsrc.ax (TID 848) 5927: Street Time: 50930000

......

Msiavsrc.ax (TID 848) 22677: Street Time: 218430000 (last Sample)

1.6 Video Sample Timestamp (Timestamp randomly), send out

Phenomenon: It has a quick shot. Video Renderer Sends a Quality Control message for feedback control. During the entire playback process, if the new SAMPLE timestamp is more than flow time, Sample will be blocked by Video Renderer until the flow time reaches the Sample time; if the new Sample time stamp is stagnant, Sample arrives in Video Renderer will be played immediately, so a quick lens is present until the timestream of the Samples cars back the flow time to return to normal speed. In summary, the video stream does not affect the normal operation of the reference clock. Msiavsrc.ax (TID 610) 1774: Street Time: 5590000

Msiavsrc.ax (TID 610) 1807: Street Time: 5930000

Msiavsrc.ax (TID 610) 1808: Add 5 SECOND DELTA ...

Msiavsrc.ax (TID 610) 1841: Stream Time: 6270000 (This Sample is sent out after blocking)

MSIAVSRC.AX (TID 610) 6873: Stream Time: 56590000 (Spending time reaches the timestamp on SAMPLE, continue playing)

Msiavsrc.ax (TID 610) 6906: Street Time: 56920000

Msiavsrc.ax (TID 610) 6940: Stream Time: 57260000

......

Msiavsrc.ax (TID 610) 7440: Stream Time: 62260000

Msiavsrc.ax (TID 610) 7473: Street Time: 62590000

Msiavsrc.ax (TID 610) 7474: Add -3 second Delta ...

Msiavsrc.ax (TID 610) 7508: Stream Time: 62940000 (SAMPLE sent to Video Rendere) with a very fast speed

Msiavsrc.ax (TID 610) 7509: Street Time: 62950000

Msiavsrc.ax (TID 610) 7511: Stream Time: 62970000

Msiavsrc.ax (TID 610) 7512: Stream Time: 62980000

Msiavsrc.ax (TID 610) 7513: Stream Time: 62990000

......

Msiavsrc.ax (TID 610) 7563: Street Time: 63490000

Msiavsrc.ax (TID 610) 7564: Street Time: 63500000

Msiavsrc.ax (TID 610) 7566: Do Not Add Delta ...

Msiavsrc.ax (TID 610) 7566: Stream Time: 63520000

Msiavsrc.ax (TID 610) 7567: Stream Time: 63530000

Msiavsrc.ax (TID 610) 7568: Stream Time: 63540000

Msiavsrc.ax (TID 610) 7570: Street Time: 63560000

......

Msiavsrc.ax (TID 610) 7719: Stream Time: 65050000

Msiavsrc.ax (TID 610) 7721: Street Time: 65070000

Msiavsrc.ax (TID 610) 7722: Street Time: 65080000

Msiavsrc.ax (TID 610) 7746: Stream Time: 65320000 (Sample timestamp finally catching up the flow time)

Msiavsrc.ax (TID 610) 7779: Stream Time: 65650000

......

Msiavsrc.ax (TID 610) 19390: Street Time: 181760000 (last Sample)

1.7 Video Sample hits (continuous) timestamps, gaps send (analog network obstruction)

Phenomenon: Video Plays a meal. Video Renderer Sends a Quality Control message for feedback control. When there is no data, the flow time is still walking; so it is "late" when the next Sample arrives at Video Renderer, so a fast lens will appear.

Msiavsrc.ax (TID 678) 1667: Street Time: 8940000

Msiavsrc.ax (TID 678) 1700: Street Time: 9270000

MsiavSrc.ax (TID 678) 3701: Sleep (2000) ... (at this time, blocking the image on the previous Sample)

Msiavsrc.ax (TID 678) 3702: Stream Time: 29290000 (fast lens)

Msiavsrc.ax (TID 678) 3704: Stream Time: 29310000

Msiavsrc.ax (TID 678) 3706: Stream Time: 29330000

......

Msiavsrc.ax (TID 678) 3762: Stream Time: 29890000

Msiavsrc.ax (TID 678) 3766: Stream Time: 29930000

MsiavSrc.ax (TID 678) 5767: Sleep (2000) ... (at this time, blocking the image on the previous Sample)

Msiavsrc.ax (TID 678) 5768: Stream Time: 49950000 (fast lens)

Msiavsrc.ax (TID 678) 5770: Street Time: 49970000

Msiavsrc.ax (TID 678) 5772: Street Time: 49990000

......

Msiavsrc.ax (TID 678) 38788: Stream Time: 380150000 (last Sample)

2. Filter graph does not use a reference clock

Phenomenon: No matter whether you don't call the timestamp, no matter whether the timestamp is played correctly, the video is played at the fastest speed. And Video Rendere does not send a Quality Control message. If there is a gap-based pause during the Video Sample, the video will also have a gap.

3. Residual (in unconnected state) in Filter Graph) A Audio Renderer

Note: By default, Audio Rendrer is selected for the reference clock for Filter Graph.

Filter Graph is shown in Figure 3:

Figure 3 Single video stream of residual Audio Rendere Filter Graph

The test results are similar to the case where the system clock is used as the reference clock of Filter Graph. summary:

v At any time, the timestamp on Video Sample does not affect the flow time of the filter graph.

v If there is no timestamp on Video Sample, Video Rendrer handles Sample data at the fastest speed; if there is a timestamp, schedule Sample content (video images) is scheduled based on the timestamp and the current flow time. If the timestamp of SAMPLE is superflow, Video Rendrer blocks the Sample, until the flow time reaches the Sample timestamp and starts playback; if the timestamp on the SAMPLE is hysterer, Video Rendere is immediately displayed immediately after the Sample content is displayed. Handling Sample at the fastest speed to chase the timestream timestall as soon as possible.

v Video stream playback process has a Quality Control mechanism; Quality Control message sender is Video Renderer, feedback to the data sending thread to speed up or slow down the data transmission speed, try to improve service quality.

v

If you can get more important conclusions from the above data, please let me tell me Luqiming@263.net.

转载请注明原文地址:https://www.9cbs.com/read-117465.html

New Post(0)