Http://www-900.ibm.com/developerWorks/Cn/Linux/L-MDST/INDEX.SHTML
Real-time stream media programming under Linux
content:
First, streaming media introduction 2, streaming media agreement three, streaming media programming four, small knots five, reference resources about the author
In the Linux area:
Tutorial Tools & Product Codes and Component Project Articles
Xiao Wenpeng (xiaowp@263.net) Department of Computer, Beijing University of Technology, Master's 2004 February 2004
The streaming refers to the continuous time base media that uses streaming technology transmission in the network. It is characterized by not need to download the entire file before playing, but use the way to play back on the side, it is a video conference, IP phone and other applications. Technical foundation. RTP is standard protocols and key technologies for real-time streaming media, which describes how to use JRTPLIB to make real-time streaming media programming under Linux.
I. Introduction to the streaming media As the Internet is increasing, the data transmitted on the network is no longer limited to text and graphics, but gradually transition to multimedia format such as sound and video. At present, multiple media files such as audio / video (AUDIO / VIDEO, A / V) on the network are currently transmitted, and there are only two options for downloading and streaming. In general, the storage space occupied by the A / V file is relatively large, and the download may take several minutes or even hours in the bandwidth network environment, so this processing method has a large delay. If you switch, sound, image, animation and other multimedia files will be responsible for continuous and real-time by special streaming server, so that users can do not have to wait until all the files are downloaded, but only take a few seconds. The startup delay can be, when these multimedia data is played on the client, the remainder of the file will continue to download from the streaming server. Streaming is a new concept that appears on the Internet in recent years, which is very widely defined, mainly referring to the technical general name of multimedia data through the network. Streaming media contains broad and narrow connotations: a wide range of streaming media refers to a series of techniques, methods, and protocols that form stable and continuous transport streams and back discharge of audio and video, 即 flow media technologies; The streaming media is relative to the traditional download-playback mode, refers to a new method of acquiring multimedia data such as audio and video on the Internet, which can support real-time transmission and real-time playback of multimedia data streams. By using streaming media technology, the server can send stable and continuous multimedia data streams to the client, and the client is replaced with a stable rate while receiving data, without having to download all the data. Play back. Due to the restrictions on network bandwidth, computer processing power and protocol specification, you must download a large amount of audio and video data from the Internet, no matter how far from download time and storage space, streaming media technology The appearance solves this problem very well. There are currently two ways to realize streaming media transmission and real-time flow (RealTime streaming), which are suitable for different applications. Sequential flow transmission sequence flow transmission uses sequential download mode, while the user can play back multimedia data online while downloading, but can only watch the part of the downloaded part, can not jump to the un downloaded part, and cannot be in transmission Adjust the download speed based on the network condition. Since standard HTTP servers can send this form of streaming media, without support of other special protocols, it is often referred to as HTTP streaming. Sequential flow transfer is relatively suitable for high quality multimedia fragments such as tabs, tails or advertisements. Real-time streaming real-time flow transmission ensures that the media signal bandwidth can match the current network condition, so that streaming media data is always transmitted in real time, so particularly suitable for field events. Real-time streaming supports random access, that is, the user can view the front or later content through fast forward or back operation. In theory, the real-time streaming media will not pause, but in fact, there may still be periodic suspension phenomena, especially when the network is deteriorated. Different from sequential streaming, real-time streaming requires a specific streaming server, but also requires support for specific network protocols. Second, the Real-Time Transport Protocol, PRT) is a network protocol for processing multimedia data streams on the Internet, using it to be one-on-one (unicast, unicast) or one-to-many (MultiCast) , Real-time transmission of convection media data in a network environment of multicast).
RTP usually uses UDP to perform multiple media data transmission, but if needed, other protocols such as TCP or ATM can consist of two closely related parts: RTP data protocols and RTP control protocols. Real Time Streaming Protocol, RTSP is incorporated by reference in REAL NETWORKS and NetScape, which is located on RTP and RTCP, and its purpose is to efficiently transmit multimedia data through IP networks. 2.1 RTP Data Protocol The RTP data protocol is responsible for packet of streaming media data and implements real-time transmission of media streams. Each RTP datagram is composed of two parts of the header and load, including the first 12 words in the head. The meaning of the section is fixed, and the load can be audio or video data. The head format of the RTP datag report is shown in Figure 1: Figure 1 The more important and significant of the RTP header format and its significance are as follows:
The CSRC count (CC) represents the number of CSRC identities. After the CSRC identification is followed by the RTP fixed head, the RTP data source is used to represent the source of the RTP datagram, which allows multiple data sources to be present in the same session, which can be combined into a data source via the RTP mixer. For example, a CSRC list can be generated to represent a conference call, which combines all speakers into an RTP data source through an RTP mixer. Load Type (PT) indicates the format of the RTP load, including the encoding algorithm used, sampling frequency, bearer channel, and the like. For example, type 2 indicates that the voice data encoded by the ITU G.721 algorithm is carried out in the RTP packet, and the sampling frequency is 8000 Hz and monouround. The serial number is used to provide the recipient to provide the data loss method, but how to deal with lost data is the application own thing, the RTP protocol itself is not responsible for the retransmission of data. The timestamp records the sampling time of the first byte in the load, the recipient can determine whether the arrival of the data is affected by delayed jitter, but how specifically, how to compensate for the delay shake is the application's own business. It is not difficult to see from the RTP datagram, which contains information such as the type, format, serial number, timestamp, and whether there is additional data, which provides a corresponding basis for real-time streaming. The purpose of the RTP protocol is to provide end-to-end transport services for real-time data (such as interactive audio and video), so there is no connection in the RTP, which can be built on the underlying orientation or non-connection-oriented transmission protocol. RTP does not rely on the special network address format, but only requires the underlying transmission protocol support group frame (segmentation) is sufficient; the other RTP itself does not provide any reliability mechanism, these must Transfer protocols or applications to ensure. In a typical application, RTP is generally implemented as part of the application as part of the application, as shown in Figure 2: Figure 2 Relationship between RTP and various network protocols 2.2 RTCP control protocol RTCP control protocols need to be with RTP The data protocol is used together. When the application starts an RTP session, the two ports will be used at the same time, and the RTP and RTCP are used separately. RTP itself does not provide reliable guarantees for sequential transmission packets, nor does traffic control and congestion control, which are completed by RTCP. Usually the RTCP will use the same distribution mechanism as RTP, periodically send control information to all members in the session, the application receives information about the session participants, and the network conditions, packet loss probability, etc. The service quality can be controlled or the network condition is diagnosed. The function of the RTCP protocol is implemented through different RTCP datagrams, mainly with the following types:
The SR sender reports that the so-called transmitting end refers to an application or terminal of the RTP datagram, and the transmitting end can also be a receiving end. The RR receiving end reports that the so-called receiving end refers to an application or terminal that receives but does not transmit the RTP datagram. SDES source description, the main function is a carrier regarding identification information as a session member, such as user name, email address, telephone number, and the like, and has the function of conveying session control information to session members. BYE Notification is leaving, the main function is to indicate that a certain or several sources are no longer valid, ie other members in the notification session will exit the session. The app is defined by the application, solves the RTCP scalability problem and provides great flexibility for the implementation of the protocol. RTCP Data Reports With the necessary information of service quality monitoring, the service quality can be dynamically adjusted, and network congestion can be effectively controlled. Since the RTCP datagram is used in multicast mode, all members in the session can understand the current situation of other participants through the control information returned by the RTCP datagram. Under a typical application, the application that transmits the media stream will periodically generate the sender report SR, the RTCP datagram contains synchronization information between different media flows, and the number of data reports and bytes that have been sent. The actual data transmission rate can be estimated based on this information. On the other hand, the receiving end sends a receiving end report RR to all known transmitters, which contains important information such as the maximum serial number of the received data, lost data report, delay jitter, and timestamp. The sender application can estimate the round trip delay based on this information, and the transmission rate can be dynamically adjusted according to the data report loss probability and the time delay mode to improve the network congestion status, or smoothly adjust the service quality of the application based on the network condition. 2.3 RTSP Real-Time Flow Agreement As an application layer protocol, RTSP provides a framework for extended, which makes it possible to enable real-time streaming media data. In general, RTSP is a streaming media representation protocol, mainly used to control data sent with real-time characteristics, but it does not transfer data, but must rely on certain services provided by the lower transfer protocol. RTSP can provide a convection medium such as play, pause, fast forward, etc., which is responsible for defining a specific control message, operation method, status code, etc., further describing interaction with the RTP. RTSP refers to the HTTP / 1.1 protocol in the formulation, and even many describes the same as HTTP / 1.1. RTSP deliberately uses the syntax and operation similar to HTTP / 1.1, to a large extent, for compatibility with existing web infrastructure, because of this, HTTP / 1.1 expanded mechanism can be introduced directly into RTSP. The media stream collection controlled by RTSP can be defined by a representation description. The so-called representation is a collection of one or more media streams supplied to the client, while the description contains a representation of each medium. Information information, such as data encoding / decoding algorithms, network addresses, media streams, etc. Although the RTSP server also uses the identifier to distinguish each stream connection session (SESSION), the RTSP connection is not bound to the transport layer connection (such as TCP, etc.), that is, the RTSP user can open or Turns off multiple reliable transfer connections to the RTSP server to issue RTSP requests. In addition, the RTSP connection can also be based on a connectionless transmission protocol (such as UDP, etc.). The RTSP protocol currently supports the following:
The retrieval medium allows the user to submit a representation description to the media server via HTTP or other methods. As indicated by multicast, the description contains the multicast address and port number for the media stream; if it is indicated, it should only provide only the destination address in the representation description. Invitation to join the media server can be invited to participate in the ongoing conference, or in the middle playback media, or record all media or its subsets in the representation, which is ideal for distributed teaching. Adding a media to notify users of newly added media streams, which is especially useful for on-site lectures. Similar to HTTP / 1.1, RTSP requests can also be handled by agents, channels, or caches. Third, the streaming media programming RTP is the best way to solve the real-time transmission of streaming media. If you need real-time streaming media programming on the Linux platform, you can consider using some Open source code RTP libraries such as librtp, jrtplib, etc. JRTPLIB is an object-oriented RTP library that fully follows the RFC 1889 design, which is a very good choice in many cases. The following will use JRTPLIB as an example to describe real-time streaming of real-time streaming on the Linux platform. 3.1 Environmental Construction JRTPLIB is an RTP library implemented with C language, which is now run on a variety of operating systems such as Windows, Linux, FreeBSD, Solaris, UNIX, and VXWORKS. To install JRTPLIB for the Linux system, first download the latest source package from JRTPLIB's website (http://lumumba.luc.ac.be/jori/jrtplib/jrtplib.html), use JRTPLIB-2.7B.TAR .bz2. Assume that the downloaded source package is saved in the / usr / local / src directory, perform the following command can decompress it: [Root @ Linuxgam src] # bzip2 -dc jrtplib-2.7b.tar.bz2 | tar xvf -
Next, JRTPLIB needs to be configured and compiled:
[root @ linuxgam src] # CD JRTPLIB-2.7
[root @ linuxgam jrtplib-2.7b] # ./configure
[root @ linuxgam jrtplib-2.7b] # Make
Finally, you can complete the JRTPLIB installation:
[root @ linuxgam jrtplib-2.7b] # make install
3.2 Initialization Before using JRTPLIB to perform real-time streaming media data transmission, first generate an instance of the RTPSession class to represent this RTP session, then call the create () method to initialize it. The CREATE () method of the RTPSession class has only one parameter to indicate the port number used by the RTP session. Listing 1 gives a simplest initialization framework, which only completed the initialization of the RTP session, and does not have any actual features. Code List 1: Initial.cpp
#include "rpsession.h"
Int main (void)
{
RTPSession sess;
Sess.create (5000);
Return 0;
}
If the RTP session creation process fails, the Create () method will return a negative number. Although it can easily determine the function call is successful or fail, it is difficult to understand what is wrong. JRTPLIB uses a unified error handling mechanism, and all functions it provides If the return negative indicates some form of errors, and the specific error information can be obtained by calling the RTPGETERRORSTRING () function. The RTPGETERRORSTRING () function incorporates the error code as a parameter, and then returns the error message corresponding to the error code. Listing 2 gives a more complete initialization framework, which can better handle errors generated during the RTP session initialization process: code list 2: framework.cpp # include
#include "rpsession.h"
Int main (void)
{
RTPSession sess;
Int status;
Char * msg;
Sess.create (6000);
MSG = RTPGETERRORSTRING (STATUS);
Printf ("Error String:% S // N", MSG);
Return 0;
}
Setting the appropriate time stamp unit, is another important job to be performed by the RTP session initialization process, which is implemented by calling the setTimeStampUnit () method of the RTPSession class. This method also has only one parameter, indicating that in seconds The time stamp unit of the unit. For example, when the audio data samples sampled using the RTP session, since the timestamp will increase 8000 per second, the time stamp unit should be set to 1/8000:
Sess.SetTimeStampunit (1.0 / 8000.0);
3.3 Data Send When the RTP session is successfully established, the real-time transmission of streaming media data can be started. First, you need to set the target address sent by the data, and the RTP protocol allows the same session to exist multiple destination addresses, which can be done by calling the rtpsession class addStination (), deletedStination () and CleteStinationS () methods. For example, the following statement represents the 6000 port that allows the RTP session to send data to the local host:
Unsigned long addr = ntohl (INET_ADDR ("127.0.0.1"));
Sess.AddDestination (Addr, 6000);
After all the target addresses are specified, the SendPacket () method of the RTPSession class can then be called to send stream data to all target addresses. Sendpacket () is an overload function provided by the RTPSESSION class, which has the following various forms:
Int SendPacket (Void * Data, Int Len)
Int Sendpacket (Void * Data, Int Len, Unsigned Char PT, Bool Mark, unsigned long timestampinc)
Int SendPacket (Void * Data, Int Len, Unsigned Short HDRextid, Void * HDrextData, Int NumHDRextwords)
Int Sendpacket (Void * Data, Int Len, Unsigned Char PT, Bool Mark, Unsigned Long TimeStampinc, Unsigned Short Hdrextid, Void * HDRextData, Int NumHDrextwords)
Sendpacket () The most typical usage is similar to the following statement, where the first parameter is the data to be transmitted, and the second parameter indicates the length of the data to send, and then the RTP load type, identification and Time stamp increment. Sess.sendpacket (buffer, 5, 0, false, 10);
For the same RTP session, the load type, identification, and timestamp increments are usually the same. JRTPLIB allows them to set them as the default parameters of the session, which is a setDefaultPayLoadType (), setDefaultmark () It is done with the setDefaultTimeStampIncest () method. The advantage of setting these default parameters for RTP sessions is to simplify data transmission, for example, if the default parameters are set for the RTP session:
Sess.setDefaultPayloadType (0);
Sess.SetDefaultmark (false);
Sess.SetDefaultTimeStampIncrement (10);
Then only indicate the data to be sent and the length of the data to be sent and the length is:
Sess.sendpacket (Buffer, 5);
3.4 Data Receives the receiving end for streaming data, first requiring the POLLDATA () method of the RTPSession class to receive the transmitted RTP or RTCP datagram. Since multiple participants (sources) are allowed in the same RTP session, you can pass all the sources by calling the RTPSession class's gotofirstsource () and gotonextsource () methods, you can also call GotofirstSourceWithdata () and GotonextsourceWithData by calling the RTPSession class. () Methods to traverse those sources carrying data. After detecting a valid data source from the RTP session, you can call the RTP datagram in the RTPSession class. When the received RTP datagram is processed, it must be remembered in time. The following code demonstrates how to process the RTP datagram that is received:
IF (sess.gotofirstsourcewithdata ()) {
Do {
RTPPACKET * PACK;
Pack = sess.getnextpacket ();
// Process the received data
Delete Pack;
While (sess.gotonextsourcewithdata ());
}
JRTPLIB defines three reception modes for the RTP datagram, where each receiving mode specifies which arrival RTP datagram will be accepted, and the arriving RTP datagram will be rejected. These receiving modes can be set by calling the setRecEession () method of the RTPSession class:
ReceMode_all default reception mode, all arriving RTP datagrams will be accepted; ReceiveMode_ignoreSome In addition to certain senders, all arriving RTP datagrams will be accepted, while the rejected sender list can be called AddToIgnoreList (), deleteFromignoreList () and ClearignoreList () method for settings; all reached RTP datagrans will be rejected in addition to certain senders, and the received sender list can be called by calling AddToAcceptlist ( The deleteFromacceptList and ClearAcceptList () methods are set. 3.5 Control Information JRTPLIB is a highly encapsulated RTP library. Many times the programmer does not have to care about how the RTCP datagram is sent and received, because these can be done by JRTPLIB. As long as the POLLDATA () or sendpacket () method is successfully called, JRTPLIB can automatically process the reached RTCP datagram, and will also send RTCP datagrams when needed, so that the correctness of the entire RTP session process can be ensured. On the other hand, by calling the setLocalName (), setLocalTool (), setLocAlTool (), setLocAlTool (), and setLocalNote () methods provided by calling the RTPSession class, and the programmers allow programmers to set control information for the RTP session. All of these methods have two parameters in the call, where the first parameter is a CHAR type pointer, pointing to the data to be set; and the second parameter is an INT type value, indicating the data How many characters in front will be used. For example, the following statement can be used to set the email address in the control information: SESS.SETLOCALEMAIL ("xiaowp@linuxgam.com", 19);
During the RTP session, not all control information need to be sent, encrysendname (), enablesendemail (), enablesendlocation (), enablesendlocation (), enablesendphone (), enablesendTool (), and enablesendnote () method, can be sent by calling the enablesendname (), enablesendemail (), enablesendlocation (), enablesendlocation (), enablesendphone (). The RTP session selection will be sent control information. 3.6 Actual Application Finally, through a simple streaming-receiving example, describes how to use JRTPLIB to program the real-time streaming. Listing 3 gives the full code of the data sender, which is responsible for the IP address and port specified to the user, constantly send RTP packets: code list 3: sender.cpp
#include
#include
#include "rpsession.h"
// Error handler
Void Checkerror (int ERR)
{
IF (Err <0) {
Char * errstr = rtpGeterstring (ERR);
Printf ("Error:% s // n", errstr);
EXIT (-1);
}
}
INT main (int Argc, char ** argv)
{
RTPSession sess;
Unsigned long destip;
Int destport;
INT portbase = 6000;
Int status, index;
CHAR BUFFER [128];
IF (argc! = 3) {
Printf ("usage: ./sender destip destport ///); Return -1;
}
/ / Get the IP address and port number of the receiving end
destip = inet_addr (argv [1]);
IF (destip == INADDR_NONE) {
Printf ("Bad IP Address Specified.//N");
Return -1;
}
Destip = NTOHL (Destip);
Destport = ATOI (Argv [2]);
// Create an RTP session
STATUS = sess.create (portbase);
Checkerror (status);
/ / Specify the RTP data receiver
Status = sess.adddstination (destip, destport);
Checkerror (status);
/ / Set the default parameter of the RTP session
Sess.setDefaultPayloadType (0);
Sess.SetDefaultmark (false);
Sess.SetDefaultTimeStampIncrement (10);
// Send streaming data
INDEX = 1;
Do {
Sprintf (buffer, "% d: RTP Packet", Index );
Sess.sendPacket (Buffer, Strlen (Buffer);
Printf ("Send Packet! // N");
WHILE (1);
Return 0;
}
Listing 4 gives full code for the data receiving end, which is responsible for constantly reading RTP packets from the specified port: Code List 4: Receiver.cpp
#include
#include "rpsession.h"
#include "rpppacket.h"
// Error handler
Void Checkerror (int ERR)
{
IF (Err <0) {
Char * errstr = rtpGeterstring (ERR);
Printf ("Error:% s // n", errstr);
EXIT (-1);
}
}
INT main (int Argc, char ** argv)
{
RTPSession sess;
INT localport;
Int status;
IF (argc! = 2) {
Printf ("usage: ./sender localport /// n");
Return -1;
}
/ / Get the port number specified by the user
LocalPort = ATOI (Argv [1]);
// Create an RTP session
STATUS = sess.create (localport);
Checkerror (status);
Do {
// Accept RTP data
STATUS = sess.polldata ();
// Retrieve the RTP data source
IF (sess.gotofirstsourcewithdata ()) {
Do {
RTPPACKET * PACKET;
/ / Get RTP Data Report
While ((packet = sess.getnextpacket ())! = null) {
Printf ("got packet! // n");
/ / Delete RTP Datashers
Delete packet;
}
While (sess.gotonextsourcewithdata ());
}
WHILE (1);
Return 0;
}
This article Source Code Download 4, Summary With the role of multimedia data on the Internet, it is increasingly important, and it will become more and more applications in real-time transmission of audio and video, such as IP phones, video on demand. , Online conference, etc. RTP is an agreement used to perform real-time streaming media transmission on the Internet. It is currently widely used in various occasions. JRTPLIB is an object-oriented RTP package library, which can easily complete real-time on the Linux platform. Stream media programming. 5. Reference resources 1. On the JRTPLIB website http://lumumba.luc.ac.be/jori/jrtplib/jrtplib.html, you can download the latest source package for JRTPLIB, and you can find some resources related to RTP. . 2. Gu Shizhen et al, Beijing: Machinery Industry Press, 2002 3. Huang Yongfeng, etc., IP Network Multimedia Communication Technology, Beijing: People's Posts and Telecommunications Press, 2003 About the author
This article Xiao Wenpeng is a free software enthusiast, mainly engaged in the research of operating system and distributed computing environment, love Linux and Python. You can contact him through xiaowp@263.net, you can also do further communication with him on the website http://www.linuxgam.com. He is currently committed to the development of OpenEC project, aims to push Linux to home entertainment, you can learn from the website http://www.openec.org.cn to the current progress and the latest results of the project.
