Communication between Linux Environment Processes (1): Pipelines and famous pipes
content:
Pipeline Overview and Related API Applications Amount Overview and Related API App Number Reference Information About Authors
In the Linux area:
Tutorial Tools & Product Codes & Component Articles
Zheng Yanxing (mlinux@163.com) November 2002 pipeline and famous pipeline
In this series
The author of the sequence outlines several major means of communication between Linux processes. Among them, the pipeline and a well-known pipe are one of the earliest processes, and the pipeline can be used to communicate between the processes of the proximity relationship. The famous pipe overcomes the limitations of the pipeline without the name, and therefore, in addition to the function of the pipe, it also allows There is no communication between processes. The reading and writing rules that recognize the pipeline and the famous pipeline are the key to the application of their programs. This paper discusses the communication mechanism of the pipeline and a famous pipeline in detail, and uses instances to verify its read and write rules. It is conducive to enhance readers to read and write the sense of understanding, and also provide an application example.
1, the key concept of pipeline overview and related API applications 1.1 Pipeline is one of the original UNIX IPC forms supported by Linux, with the following features:
The pipe is half-duplex, and the data can only flow in one direction; two channels are required to establish two pipes; only for the parent-child process or the process of the brother process (with a pro-affinity process); Independent file system: Pipeline is a file, but it is not an ordinary file, it does not belong to a file system, but a self-reliable portal, a file system, and only exists with memory in. The data reads and writes: A process read from the content written in the pipe to the other end of the pipe. The written content is added at the end of the pipe buffer each time, and each time it is read from the header of the buffer. 1.2 Creation of the pipeline: #include
INT PIPE (int FD [2])
The two ends of the duct created in the process are in the middle of the process, there is not much sense in practical applications, so a process is in a pro process after creating a pipe by the PIPE (), and then achieves the parent process through the pipeline. Communication (therefore it is not difficult to introduce, as long as there is a relative relationship in both processes, the relationship here refers to a common ancestor, and can communicate with pipelines. 1.3 Pipe reading and writing rules: The two ends of the pipe can be described by describing the word FD [0] and FD [1], it should be noted that the two ends of the pipe are fixed. On one end, only one end can only be read, represented by the description word FD [0], referred to as a pipe reading end; the other end can only be written, represented by the descriptor FD [1], referred to as a pipeline. If you try to read data from the pipeline, you will cause an error to occur. The I / O function of the general file can be used for pipes such as Close, Read, Write, and more. Read data from the pipe:
If the write end of the pipe does not exist, it is considered that the end of the data has been read, and the number of readout bytes returned by the read function is 0; when the write end of the pipe is present, if the number of bytes of the request is greater than the PIPE_BUF, return to the pipe The number of existing data bytes, if the number of bytes requested is not more than PIPE_BUF, then return the number of existing data bytes in the pipe (at this time, the amount of data in the pipe is less than the requested data); or return the number of bytes of requests (At this time, the amount of data in the pipe is not less than the amount of data requested). Note: (PIPE_BUF is defined in Include / Linux / Limits.h, different kernel versions may vary. Posix.1 requires that PIPE_BUF is at least 512 bytes, and 4096 in Red Hat 7.2). About the reading rule of the pipeline: / **************
* ReadTest.c *
************* /
#include
#include
#include
Main ()
{
INT PIPE_FD [2];
PID_T PID;
Char r_buf [100];
Char w_buf [4];
CHAR * P_WBUF;
int r_num;
INT CMD;
MEMSET (r_buf, 0, sizeof (r_buf));
MEMSET (W_BUF, 0, SIZEOF (R_BUF));
P_WBUF = W_BUF;
IF (PIPE_FD) <0)
{
Printf ("Pipe Create Error / N);
Return -1;
}
IF ((pID = fork ()) == 0)
{
Printf ("/ n");
Close (PIPE_FD [1]);
Sleep (3); // Make sure the parent process closes the write
R_num = read (PIPE_FD [0], R_BUF, 100);
Printf ("Read Num IS% D the Data Read from The Pipe IS% D / N", R_NUM, ATOI (R_BUF));
Close (PIPE_FD [0]);
exit ();
}
Else IF (PID> 0)
{
Close (PIPE_FD [0]); // read
STRCPY (W_BUF, "111");
IF (Write (PIPE_FD [1], W_BUF, 4)! = - 1)
Printf ("Parent Write over / n");
Close (PIPE_FD [1]); // Write
Printf ("Parent Close FD [1] OVER / N");
Sleep (10);
}
}
/ ************************************************** *
* Program output results:
* Parent Write over Over
* Parent Close FD [1] OVER
* Read Num IS 4 The Data Read from The Pipe IS 111
* Additional conclusions:
* After the pipeline is closed, the written data will always exist until it is read.
*********************************************************** ** /
Write data to the pipe:
When writing data in the pipe, Linux will not guarantee the written atomicity, the pipe buffer has an idle area, and the writing process will try to write data to the pipe. If the read process does not read the data in the pipe buffer, the write operation will be blocked. Note: Only when the read end of the pipe is present, it is meaningful to write data to the pipe. Otherwise, the process writes data to the pipe will receive the SIFPIPE signal from the kernel. The application can handle the signal or ignore (the default action is the application termination). Verification of writing rules for pipeline 1: Write the dependency of the read end exist #include
#include
Main ()
{
INT PIPE_FD [2];
PID_T PID;
Char r_buf [4];
CHAR * W_BUF;
Int writenum;
INT CMD;
MEMSET (r_buf, 0, sizeof (r_buf));
IF (PIPE_FD) <0)
{
Printf ("Pipe Create Error / N);
Return -1;
}
IF ((pID = fork ()) == 0)
{
Close (PIPE_FD [0]);
Close (PIPE_FD [1]);
Sleep (10);
exit ();
}
Else IF (PID> 0)
{
Sleep (1); // Waiting for the sub-process to complete the operation of the read end
Close (PIPE_FD [0]); // Write
W_buf = "111";
IF ((WRITENUM = Write (PIPE_FD [1], W_BUF, 4)) == - 1)
Printf ("Write to Pipe Error / N);
Else
Printf ("The Bytes Write to Pipe IS% D / N", WRITENUM);
Close (PIPE_FD [1]);
}
}
The output results are: Broken Pipe, the reason is that the pipeline and the read end of all of its FORK () products have been closed. If you keep the reading end in the parent process, that is, after writing the PIPE, turn off the read end of the parent process, will also write to PIPE, readers can verify this conclusion. Therefore, when writing data to the pipe, there should be at least a process, wherein the pipe read end is not closed, otherwise the above error (pipe break, the process receives the SIGPIPE signal, the default action is the process termination) to the pipe Verification of the rules 2: Linux does not guarantee atomic verification of the write pipe
#include
#include
#include
Main (int Argc, char ** argv)
{
INT PIPE_FD [2];
PID_T PID;
Char r_buf [4096];
Char w_buf [4096 * 2];
Int writenum;
int RNUM;
MEMSET (r_buf, 0, sizeof (r_buf));
IF (PIPE_FD) <0)
{
Printf ("Pipe Create Error / N);
Return -1;
}
IF ((pID = fork ()) == 0)
{
Close (PIPE_FD [1]);
While (1)
{
Sleep (1);
RNUM = Read (PIPE_FD [0], R_BUF, 1000);
Printf ("CHILD: READNUM IS% D / N", RNUM);
}
Close (PIPE_FD [0]); exit ();
}
Else IF (PID> 0)
{
Close (PIPE_FD [0]); // Write
MEMSET (r_buf, 0, sizeof (r_buf));
IF ((WRITENUM = Write (PIPE_FD [1], W_BUF, 1024) == - 1)
Printf ("Write to Pipe Error / N);
Else
Printf ("The Bytes Write to Pipe IS% D / N", WRITENUM);
WriteNum = Write (PIPE_FD [1], W_BUF, 4096);
Close (PIPE_FD [1]);
}
}
Output results:
The bytes write to pipe 1000
THETES WRITE TO PIPE 1000 // Note that this line outputs a non-incoming written
The bytes write to pipe 1000
The bytes write to pipe 1000
The bytes write to pipe 1000
THETES WRITE TO PIPE 120 // Note that this row outputs a non-incomingly written
The bytes write to pipe 0
The bytes write to pipe 0
......
Conclusion: Writing is non-atomic when writing less than 4096! If the number of writes in the parent process is changed to 5000, it is easy to obtain the following conclusions: When the amount of data written is greater than 4096 bytes, the idle space of the buffer will be written to the data (supplement) Allowed) until all data is written, if there is no process read data, it has been blocked. 1.4 Pipe Application Example: Example 1: Used for the shell pipe to be used to input an output redirection, which directs the output of a command directly to the input of another command. For example, when you type WHELL programs (Bourne Shell or C shell, the corresponding shell program will create WHO and WC two processes and pipelines between the two processes. Consider the following command line: $ KILL -L running results are attached. $ KILL -L | GREP SIGRTMIN Run as follows:
30) Sigpwr 31) Sigsys 32) SigRTMIN 33) Sigrtmin 1
34) Sigrtmin 2 35) Sigrtmin 3 36) Sigrtmin 4 37) SigRTMIN 5
38) Sigrtmin 6 39) Sigrtmin 7 40) Sigrtmin 8 41) SigRTMIN 9
42) Sigrtmin 10 43) Sigrtmin 11 44) Sigrtmin 12 45) SigRTMIN 13
46) Sigrtmin 14 47) SigrtMin 15 48) Sigrtmax-15 49) Sigrtmax-14
Example 2: Inter-process communication between processes with a relative relationship gives the specific application of the pipe, and the parent process sends some commands to the child process, the child process parsing command through the pipe, and processes according to the command.
#include
#include
Main ()
{
INT PIPE_FD [2];
PID_T PID;
Char r_buf [4];
Char ** w_buf [256];
INT childExit = 0;
INT I;
INT CMD;
MEMSET (r_buf, 0, sizeof (r_buf));
IF (PIPE_FD) <0)
{
Printf ("Pipe Create Error / N);
Return -1;
}
IF ((pID = fork ()) == 0)
// Sub process: parse the command acquired from the pipe and make corresponding processing
{
Printf ("/ n");
Close (PIPE_FD [1]);
Sleep (2);
While (! CHildexit)
{
Read (PIPE_FD [0], R_BUF, 4);
CMD = ATOI (r_buf);
IF (cmd == 0)
{
Printf ("Child: Receive Command from Parent over / n");
Childexit = 1;
}
Else IF (Handle_CMD (CMD)! = 0)
Return;
Sleep (1);
}
Close (PIPE_FD [0]);
exit ();
}
Else IF (PID> 0)
// parent: Send Commands to Child
{
Close (PIPE_FD [0]);
W_buf [0] = "003";
W_buf [1] = "005";
W_buf [2] = "777";
W_buf [3] = "000";
For (i = 0; i <4; i )
Write (PIPE_FD [1], W_BUF [I], 4);
Close (PIPE_FD [1]);
}
}
// The following is the command processing function of the child process (specific to application):
INT HANDLE_CMD (INT CMD)
{
IF ((cmd <0) || (cmd> 256)))
// suppose child Only Support 256 CommANDS
{
Printf ("Child: Invalid Command / N);
Return -1;
}
Printf ("Child: THE CMD from Parent IS% D / N", CMD);
Return 0;
}
1.5 The main limitations of limitations of pipelines are reflected in it:
Only one-way data stream; only between processes having a relative relationship; no name; the buffer of the pipe is limited (the pipeline is present in memory, and when the pipe is created, the buffer allocates a page size) The pipe is transmitted is unformatted by no format, which requires the reader of the pipe and the write party necessary to set the format of the data in advance, such as how many bytes are counted as a message (or command, or record), etc.; 2, a famous pipeline overview and related API applications 2. A major restriction in the key concept of a well-known pipeline is that it has no name, so it can only be used to communicate between processes with the affinity relationship, in the famous pipe (NAMED PIPE or FIFO) This restriction is overcome after it is proposed. The FIFO is different from the pipeline in that it provides a path name and associated with FIFO, exists in the file system. Thus, even if there is no processes with the FIFO's creation process, as long as the path can be accessed, it is possible to communicate with each other (which can access the path of the path and between the process of the FIFO), so that the FIFO is not related The process can also exchange data. It is worth noting that FIFO strictly follows First In First Out, the Pipe and FIFO reads always return data from the beginning, and the data is added to the end. They do not support file location operations such as LSeek () (). 2.2 Creating a famous pipe #include
#include
INT MKFIFO (const char * pathname, mode_t mode)
The first parameter of this function is a normal path name, which is the name of the FIFO. The second parameter is the same as the MODE parameter in the open () function that opens a normal file. If the first parameter of MKFIFO is an existing path name, the Eexist error is returned, so the typical call code first checks if the error is returned, and if it is indeed returns the error, then as long as the function is called to open the FIFO can be called . The I / O function of the general file can be used for FIFO, such as Close, Read, Write, and more. 2.3 Opening Rules of the famous pipeline A well-known pipe is more open than the pipeline: Open. Open Rule of FIFO If the current open operation is open to the FIFO, if there is already a corresponding process to open the FIFO, the current open operation will be successfully returned; otherwise, it may block until there is a corresponding process to open FIFO (current open operation sets the blocking flag); or successfully returns (the current open operation is not set up.). If the current open operation is to open the FIFO, if there is already a corresponding process to read the FIFO, the current open operation will be successfully returned; otherwise, it may blocked until the corresponding process is read to open the FIFO (currently open operation Set the blocking flag); or return an Enxio error (the current open operation does not set the blocking flag). See attaching 2 for verification of open rules. 2.4 Read and write rules of the famous pipeline from the FIFO: A process: If a process is blocked from the FIFO to read the data from the FIFO, the read operation in this process is called a read operation of the blocking flag.
If there is a process write open FIFO, and there is no data in the current FIFO, it will be blocked for the read operation of the blocking flag. Returns -1 for unsetting the blocking logo read operation, the current errno value is eagain, reminds you will try again. For read operations that set the blocking flag, there are two reasons why the blocking: there is data in the current FIFO, but there are other processes in the FIFO, there is no data. The reason for solving the plug is the new data writing in the FIFO, regardless of the size of the read operation request, regardless of the size of the read operation request. The read open blocking flag is only applied to the first read operation of the process. If there is a plurality of read operation sequences in this process, the read operations to be executed will not be executed after the first read operation is awakened and completes the read operation. Blocking, even if the read operation is performed, there is no data in the FIFO (at this time, the read operation returns 0). If there is no process write to open the FIFO, the read operation of the blocking flag will be blocked. Note: If there is data in the FIFO, the read operation of the blocking flag is set because the number of bytes in the FIFO is smaller than the number of bytes of the request read. At this time, the read operation returns an existing amount of data in the FIFO. Write the data to the FIFO: A process: If a process is blocked to open the FIFO in order to write data in the FIFO, the write operation in this process is a write operation that sets the blocking flag. For write operations set up the blocking mark: When the amount of data to be written is not more than the PIPE_BUF, Linux will ensure the written atomicity. If the pipeline idle buffer is insufficient to accommodate the number of bytes to be written, then the sleep is entered until the one-time write operation begins when the buffer can hold the number of bytes to be written. When the amount of data to be written is greater than PIPE_BUF, Linux will no longer guarantee the atomicity written. The FIFO buffer has an idle area, and the writing process will try to write data to the pipeline, and the write operation returns after writing all the requested data. For write operations that do not set the blocking flag:
When the amount of data to be written is greater than PIPE_BUF, Linux will no longer guarantee the atomicity written. After writing all FIFO idle buffers, the write operation returns. When the amount of data to be written is not more than PIPE_BUF, Linux will guarantee the atomicity written. If the current FIFO idle buffer can accommodate the number of bytes requested, it will be successfully returned after writing; if the current FIFO idle buffer cannot accommodate the number of bytes written, returns an Eagain error, reminding the future; FiFo read and write rules verification: Below with two read and write programs for FIFO, fewerly or programs in the appropriate regulatory program, or the command line parameters of the program can be verified for various FIFO read and write rules. Program 1: Write the program of FIFO
#include
#include
#include
#include
#define FIFO_SERVER "/ TMP / FIFOSERVER"
Main (int Argc, char ** argv)
/ / Parameter is the number of bytes that is about to be written
{
Int fd;
Char w_buf [4096 * 2];
INT REAL_WNUM;
MEMSET (W_BUF, 0,4096 * 2);
IF ((Mkfifo_Server, O_CREAT | O_EXCL) <0) && (Errno! = EEXIST))
Printf ("Cannot Create FifoServer / N);
IF (fd == - 1)
IF (errno == enxi)
Printf ("Open Error; No Reading Process / N"); FD = Open (FIFO_SERVER, O_WRONLY | O_NONBLOCK, 0);
/ / Set a non-blocking sign
// fd = open (FIFO_SERVER, O_WRONLY, 0);
// Set the blocking sign
REAL_WNUM = Write (FD, W_BUF, 2048);
IF (REAL_WNUM == - 1)
{
IF (errno == EAGAIN)
Printf ("Write to FIFO ERROR; TRY LATER / N");
}
Else
Printf ("Real Write Num IS% D / N", REAL_WNUM);
REAL_WNUM = Write (FD, W_BUF, 5000);
// 5000 is used to test non-atomicity when the writing bytes is greater than 4096
// REAL_WNUM = Write (fd, w_buf, 4096);
// 4096 Atomicity used to test when writing bytes is not more than 4096
IF (REAL_WNUM == - 1)
IF (errno == EAGAIN)
Printf ("Try Later / N");
}
Program 2: Test the rules of writing FIFO with the program 1, the first command line parameter is the number of bytes requesting from FIFO
#include
#include
#include
#include
#define FIFO_SERVER "/ TMP / FIFOSERVER"
Main (int Argc, char ** argv)
{
Char r_buf [4096 * 2];
Int fd;
INT r_SIZE;
int RET_SIZE;
R_SIZE = ATOI (Argv [1]);
Printf ("Requred Real Read Bytes% D / N", R_SIZE);
MEMSET (r_buf, 0, sizeof (r_buf));
FD = Open (FIFO_SERVER, O_RDONLY | O_NONBLOCK, 0);
// fd = open (FIFO_SERVER, O_RDONLY, 0);
/ / You can compile the reader into two different versions: blocking version and non-blocking version
IF (fd == - 1)
{
Printf ("Open% S for Read Error / N");
exit ();
}
While (1)
{
MEMSET (r_buf, 0, sizeof (r_buf));
RET_SIZE = Read (FD, R_BUF, R_SIZE);
IF (Ret_Size == - 1)
IF (errno == EAGAIN)
Printf ("NO Data Avlaible / N");
Printf ("Real Read Bytes% D / N", RET_SIZE);
Sleep (1);
}
Pause ();
Unlink (FIFO_SERVER);
}
Program Application Description: Compile the reader into two different versions:
Blocking read version: BR and non-blocking read version NBR compiles writing to two four versions:
Non-blocking and requested by the number of bytes greater than PIPE_BUF version: NBWG non-blocking and requested write number of bytes is not larger than PIPE_BUF version: version NBW blocking and request written bytes are greater than PIPE_BUF version: BWG blocking and written bytes Number is not more than PIPE_BUF version: Version BW will use BR, NBR, W instead of block read in the corresponding program, non-blocking read verification blockwright: Non-incubated data when the request written is greater than the non-incoming time of PIPE_BUF:
NBR 1000 BWG When the amount of data requested is not more than PIPE_BUF:
NBR 1000 BW verifies non-blocking write operations:
When the amount of data requested is greater than the PIPE_BUF:
NBR 1000 NBWG Requests The amount of data written is not greater than the atomicity of PIPE_BUF:
The NBR 1000 NBW does not care whether the number of bytes written is greater than 4096 when the write-open blocking flag is greater than 4096, and is not guaranteed. But the difference between the two: For blocking, write operations will wait until all data is written until all data is written, and the data requested will eventually write FIFO; rather than blocking Then return (the number of bytes that is actually written) after the idle area filled with FIFO, so some data can eventually be written. The verification of the read operation is relatively simple, no longer discussed. 2.5 A famous pipe application example After verifying the corresponding read and write rules, the application instance does not seem to be necessary. Summary: Pipes are often used in two aspects: (1) When the pipe is used in the shell (as an input input redirection), in this application mode, the creation of the pipe is transparent to the user; (2) Inter-process communication with the proximity relationship, the user creates a pipe, and completes the read and write operation. FIFO can be said to be the promotion of the pipeline, overcoming the limitations of the pipeline without name, so that the process of non-physical relationship can also use advanced communication mechanisms to communicate. The data of the pipe and FIFO is byte stream, and the specific transmission "protocol" must be determined in advance, and a message with a particular meaning is used. To flexibly apply pipes and FIFO, understand their read and write rules is the key. Attached 1: Kill -l operation results show all the signals supported by the current system:
1) SIGHUP 2) SIGINT 3) SIGQUIT 4) SIGILL
5) SigTrap 6) Sigabrt 7) Sigbus 8) SIGFPE
9) Sigkill 10) SIGUSR1 11) SIGSEGV 12) SIGUSR2
13) SIGPIPE 14) SIGALRM 15) Sigterm 17) Sigchld
18) Sigcont 19) Sigstop 20) Sigtstp 21) Sigttin
22) Sigttou 23) Sigurg 24) SIGXCPU 25) SIGXFSZ
26) SigvTalRM 27) SigProf 28) Sigwinch 29) SIGIO
30) Sigpwr 31) Sigsys 32) SigRTMIN 33) Sigrtmin 1
34) Sigrtmin 2 35) Sigrtmin 3 36) Sigrtmin 4 37) SigRTMIN 5
38) Sigrtmin 6 39) Sigrtmin 7 40) Sigrtmin 8 41) SigRTMIN 9
42) Sigrtmin 11 44) Sigrtmin 12 45) Sigrtmin 14 47) SigrtMAX 15 49) Sigrtmax-15 49) Sigrtmax-14
50) SigrtMAX-13 51) Sigrtmax-12 52) Sigrtmax-11 53) Sigrtmax-10
54) Sigrtmax-9 55) Sigrtmax-8 56) Sigrtmax-7 57) SigrtMAX-6
58) SigrtMAX-5 59) Sigrtmax-4 60) Sigrtmax-3 61) Sigrtmax-2
62) Sigrtmax-1 63) Sigrtmax
In addition to explaining the pipeline here, the next topic also discusses these signals. Attachment 2: Verification of FiFo Open Rules (mainly verified written on-site-open dependence)
#include
#include
#include
#include
#define FIFO_SERVER "/ TMP / FIFOSERVER"
INT HANDLE_CLIENT (CHAR *);
Main (int Argc, char ** argv)
{
int R_rd;
INT W_FD;
PID_T PID;
IF ((Mkfifo_Server, O_CREAT | O_EXCL) <0) && (Errno! = EEXIST))
Printf ("Cannot Create FifoServer / N);
Handle_client (FIFO_SERVER);
}
INT HANDLE_CLIENT (CHAR * ARG)
{
int R;
RET = W_Open (arg);
Switch (re)
{
Case 0:
{
Printf ("Open% S Error / N", ARG);
Printf ("NO Process Has The FIFO Open for Reading / N");
Return -1;
}
Case -1:
{
Printf ("Something Wrong With Open THE FIFO EXCEPT for Enxio";
Return -1;
}
Case 1:
{
Printf ("Open Server OK / N");
Return 1;
}
DEFAULT:
{
Printf ("W_NO_R RETURN ???? / n");
Return 0;
}
}
Unlink (FIFO_SERVER);
}
INT W_OPEN (Char * Arg)
// 0 Open error for no reading
// - 1 Open Error for other reasons
// 1 Open OK
{
IF (Open (Arg, O_Wronly | o_nonblock, 0) == - 1)
{IF (errno == enxio)
{
Return 0;
}
Else
Return -1;
}
Return 1;
}
references:
