When the media content is played, the most headache is that the audio and video are not synchronized. From a technical example, the best solution to solve audio and video synchronization is the timestamp: first select a reference clock (the time required to refer to the reference clock is linearly incremented); give each data on the reference clock when generating data flow Blocks have a timestamp (generally including the start time and end time); when playback, read the timestamp on the data block, and refer to the current reference clock to schedule play (if the start time of the data block is greater than the current reference clock The on-time, no rush to play the data block until the reference clock reaches the start time of the data block; if the start time of the data block is less than the time on the current reference clock, "" as soon as possible "playing this data or index "Discard" to make the playback progress to the reference clock).
Figure 2.8 Timestamp scheme for solving audio and video synchronization problems
It can be seen that avoiding audio and video disagreement phenomena - One is to play the correct timestamp when generating a data stream. If there is a problem with the timestamp on the data block itself, then how to adjust when playing is playing. As shown in Figure 2.8, the video stream content is started from 0s. If someone starts to speak, it is required to match the audio stream, then the start time of the audio stream should be 10s, if timestamp starts from 0S or other time, this The mixed audio and video stream itself has problems in time synchronization. When you call the timestamp, the video stream and audio stream are the time reference clock, and the data stream does not have a reference relationship; that is, the video stream and audio stream is through a neutral third party (that is, the reference clock ) To achieve synchronization. The second critical place is that the control is based on the control of the data stream based on the playback, that is, the data block is early or late to take different processing methods. In Figure 2.8, the reference clock time is played in the video stream content within 0-10s, even if the audio stream data block is received, it must wait until the reference clock has reached 10s, otherwise it will cause sound. Video does not synchronize problems.
Based on the timestamp playback, only the morning-to-late or late data block is waiting or rapidly handled, sometimes it is not enough. If you want to be more active and effectively adjusting play performance, you need to introduce a feedback mechanism, which is to feed back the current data stream speed or too slow, so that the source is slowed down or accelerated. . Readers who are familiar with DirectShow must know that Quality Control is such a feedback mechanism in DirectShow. DirectShow Synchronize for audio and video is quite well. But WMF SDK is only responsible for reading and decoding ASF data when playing, and does not responsible for the final presentation of tone video content, so it lacks such a feedback mechanism.
In order to better understand the timestamp-based audio and video synchronization scheme, the example in life will be held. Suppose you and your friends have gone about 18:00 today in Shanghai, then eat together, then play the game. In fact, this 18:00 is a point in time with your friends. As a result, you have arrived in Shanghai at 17:50, then you have to wait for your friends. After 10 minutes, your friends have not yet arrived. At this time, he said that there is something to delay, and it will be late. You have no way, because you have already booked a location next to the restaurant. If you don't rush it, the book will be canceled, so you tell your friends to meet directly to the restaurant, I want him to get up. So at some point in time in the restaurant becomes you and your friend's synchronization point. Although the specific time is not necessarily (to see the speed of your friends), the direction of this efforts is right, you and your friends will definitely see the face at the restaurant. The results of it? Your friend finally came over at 18:30, and you finally "synchronize". After dinner at 19:30, you have something to handle, so I have been about 20:00 with your friend 20:00 in the nearby game hall. You don't have sync, but you will still sync again at some point in the game hall. What is the truth? In fact, synchronization is a dynamic process, a process of someone waiting, some people catch up. Synchronization is just temporary, and it is normal. People always oscillate fluctuations in the horizontal line of synchronous, but will not deviate from this baseline too far.
