Multimedia stream Internet Transport Demo Laostea Manufactured For ZXX Company Interview: This interview because the recruiting party does not look at the fine traditional habit of the applicant Demo program, and is only the procedure under UNIX and does not in local research and development. Do Net Conf System, but play Database. (I hate Database, you have to make a data report, key constraint, workflow, and compare with OpenGL to Demo, Database is boring to the pole ...) (Wow , Old, I service u, go to the interview, even the other party's work platform and work content are not clear ...) The final result is that it is ignored me, I ignore it (continue to hibernochan in the organ ... zzzzzzz) In fact, the essence of these things (whether socket, RTP protocol, thread synchronous design, compressed transmission design) is no platform characteristics or write these days of learning experience, give reference (for example, e-class, Net) Conference, related content graduation, guy) (but there may be bugs, Matt Pietrek, Jeffery Richter's book also has bug, let alone I am old, please go to you, don't take a brick) If you learn, work Help, then it is best, my old man is the most conservative thoughts of small farmers. :) (Quote To write the place, you can improve the IMAGICSTUDIO's Counter value, do not learn 9CBS) development goals: real-time audio and video transmission under low bandwidth conditions. The software implementation of the RTP protocol. Several implementation methods: First, use Netmeeting SDK directly to make something similar to NetMeeting, based on a multimedia communication protocol stack (such as Open Source H.323), which has implemented the IP layer. Directly based on the IP protocol to use the third method, but only a Demo, time and energy limit, it is impossible to implement the H.323 protocol, so only part of the H.323 protocol, of course, The general characteristics of H.323 are not available. Technical difficulties: low bandwidth multimedia data stream synchronization, continuous video audio stream estimation is the biggest difficulty. Because condition restrictions, data compression can only use soft CODEC. The corresponding hardware will definitely be implemented in practical applications. Parameter selection in multi-threaded sampling coding decoding, such as the size and number of sound buffers, required experiments. In terms of data communication, Peer-to -peer generally adopts the connection-oriented communication, that is, stream socket, has a reliable transmission of reliable, orderly, which can better ensure the quality of communication, but due to TCP's three-way handshake characteristics, its speed Slow, and does not support multiple characteristics, and after tracking analysis NetMeeting, NetMeeting only uses TCP protocols when connecting handshake, transmitting real-time view and audio does not adopt a TCP protocol, but a faster UDP protocol. Therefore, this main study of UDP applications. Broadcast or multi-cast mode belongs to UDP socket, does not guarantee reliable data, or repetitive, so programming is complicated than that of TCP stream socket. (Note: Although the ATM network has no problem, there is data blocked and burst of phenomenon), so the impact of eliminating delay and packet loss on communication is universally common. Imitation Netmeeting, TCP / IP connection can be used for handshake and establishing connections for both ends, actual transmission UDP.
You cannot use TCP / IP introduced by some articles to implement multimedia streaming, and the practical system is used to transmit multimedia stream information on the Internet. There are two ways of working: point-to-point and multicast mode, select how work is selected. Point-to-point: Wait a TCP / IP connection, once there is a connection request, ask the user if the connection is received, allowing the UDP port to be two-way transmit and receive data. Multicast: Directly, the receiver is received. Non-compression: video data per second (QCIf is an example): 176 * 144 * 24/8 = 76032 (bytes) 16bit Mono audio data transmitted per second: Samplefrequency * SampleBits / 8 = 16K BYTES compression consideration : Audio: Consider the following compressed CODEC G.711, G.729, G.723.1, Non-H.323 content: Truespeech, GSM and other video CODEC: H.261, H.263 Non-h.323 content: Wavelets, Motion -JPEG, JPEG Intel implementation, selection of compression ratios, compression speed, and several indicators of several indicators. Finally, the M-JPEG CODEC, the ratio 1:24, the free inter-frame-free video compression, eliminating the flow of the flower screen in H.263 GSM ratio 1:10 audio mode: 16 bit, 8 kHz results rate is 1.6 k Bytes / sec, the effect can accept data parameters: Video Internet: QCIF 176 * 144, 5 fps Audio: 8kHz, 16bit, Mono Control Protocol: RTP / RCTP understands its main ideas (how to add sequential sequence for dictation) and programmed Implement Network API: Windows Socket 2, point-to-point or multicast mode (Multicast), how to consider asynchronous or synchronous mode. Multi-threaded blocking (synchronous) mode has better flexibility and efficiency, making it possible to play the computing power of the CPU. Safety: This example does not consider the design of the communication protocol: Try to write the following procedure, the sender continuously transmits the packet of the sequence number. The receiving end receives the sequence number of the package per 500 packages, you can see the following package serial number, 511, 1029, 1618 .... This shows that the UDP can provide multiple and basic checksum services, but it It is in the order, unreliable. More control information is needed to be reliable. In the H.323 protocol, the UDP package is controlled by the Real Time Transport Protocol (RTP). RTP provides traffic measurement, packet sequence number, timestamp, and transmission management of unreliable UDP.
Refer to the principle of the RTP protocol in H.323, after reading RFC 1889, we know é ", the RTP head has the following format: (Win32 SDK RTP.H): typedef struct _rtp_header {// --- NetWork Byte Order Begin - - // Word Numcsrc: 4; // Description CSRC number Word FEXTHEADER: 1; // If is 1, indicating the head data word fpadding: 1; // fill if the fixed head ends: 1; // Fill, if 1, Indicates that PAYLOAD is filled with a filling section Word Version: 2; // RTP version Word PayLoadType: 7; // payload type, such as video, audio, image, data, etc. Word FMarker: 1; // indicates a certain boundary condition / / ---- NetWork Byte Order End ---- // Word sequencenum; // serial number, indicating the order of each package, used to re-combined into a complete information dword timestamp; // timestamp, used to represent the video Frame or voice flow time flag DWORD SSRC; // Sign Information Source ID, is a random number, the same session source has the same SSRC} RTP_HEADER, * PRTP_HEADER; where sequence number can be used as a statistical analysis and data of packet loss rate Flow recovery. Timestamp can be used for synchronization of video audio streams. Such a reception process can be conceivable: double buffer (or multi-buffer) for receiving and playback of multimedia data streams. The advantage of using the buffer is the delay jitter filter that can bring the network, the buffer size to be determined. Send a thread: audio and video sets different PAYLOAD types of different RTP headers. Key to ensure the continuity of the audio. Video, use VFW or DirectShow to capture the external image to the data frame (or data stream), with the Video Codec compressed single frame image as a smaller buffer. Press a length cut, RTP package, send it. Audio, collect, compress, RTP package, and send it according to the principle of VAD (Voice Activity Detection). The receiving thread is in accordance with the received packet by the TimeStampe group, sequence sort, fill it into the buffer. Accepting the Marker Bit flag to receive a data completion. After the limited N group data reception (N is determined by experiment), if the set of packets are complete, displayed or play it, if it is not disappeared, the buffer is discarded, and the buffer is set to receive the data state. . The above method can solve the packet loss, repetition, and discontinuation of multimedia data streams. The following is a packet format design with reference to the RTP protocol. Reliable receiving a complete LDU (one frame image, a piece of sound), replace another air buffer to the recipient, let the playing thread in the waiting state play the received buffer.
There are two design schemes: multi-threaded and single threaded thread part design: Basic methods of multi-threaded design: list all the works of the program, divide them into several threads, then consider synchronization, if thread cycle execution The time difference is too large, you must synchronize with synchronization objects (Mutex, criticalSection, Event, Semophere), must not be inquiry, otherwise a large number of CPU resources will be eaten. According to the completion time division, undoubtedly, the data collection, the data of the data is longest, and the compression transmission can be completed in the data acquisition idle time, and the data reception decompressed in the free time of playback. Double buffer resources use two mutual exclusive objects to ensure a single thread access, and the mutually exclusive object is responsible for creating and destroying the thread running. Therefore, the thread is arranged as follows: sending end: Data acquisition thread, compression sending thread sampling thread first run (mutex objects maintained by it), followed by compressing the transmission line. Synchronized by two mutual exclusion. The transfer of buffer data is passed to the compressed transmission thread by compressed transmission thread. Receiver: Data Play Thread, Receive Decompression Thread Receiver, Unzip Receive Culture Advance (Mutex Object Maintenance), the playing thread is suspended until the received decompression is completed and the data is sent to the empty buffer of the play thread. Solution 2 is a scheme that collects and sends two-threaded merges. Since the collection is an asynchronous process, the thread is just compressed and transmitted in the idle time of the recording. (Requirements to compress time and time short enough, short than one sound acquisition, in line with this conditional problem should not be large) Each time you send a audio LDU, detect that there is no video coding, if there is, while sending a video. Similarly, the sound reception is also the case, the thread is processed after starting the next frame of data, and the processing is completed and the next play is performed after the end is completed. Such a solution is delayed: and the delay time is a voice LDU play time network delay If the LDU is large enough, the buffer effect will be good (can wait for the arrival of the speech data without speech) However, the user will not endure the latency, so the appropriate LDU time should be selected according to the network quality. Both methods will be achieved and the effect is good or bad. The view is issued by the same port, which is different from the PayLoad Type, and the recipient puts the data in different buffers. Synchronous broadcast after the audio reception is complete. The principle of synchronization is the smallest video LDU and audio LDU for timestamp gaps. Others: The interface part uses a SKIN or ActiveSkin control, which can have a better look than Netmeeting. See the figure (VFW capture window, the characters are iron buddy nose beans, a guy who is not programmed to be old), in order to make the receiving end better use the program, free of installation software, should Put at least the receiving end in the web page. WebMaster uses the program for audio transmission, broadcast. Intranet users only need to open the browser to receive content. At this time, choose C Builder ActiveForm to embed the program code to the web page.
