Creation Of an InterProcess Pipe in python, the Unix (Gnu/Linux) way

The most simple way is to use the shell to create a pipeline. But if you want to create any other shape you will have to do some programming. Some languages such as Python, TCL, purl … have built in functions to create task harnesses. Here is how it is done at the lowest level. Using Unix primitive system calls.

1. Create a Task.
   

   #!/usr/bin/python3
   

2. Open Pipe
   

   #!/usr/bin/python3
   import os
   stdout_id=1
   
   pipe_out,pipe_in = os.pipe()
   print ("pipe is: ",(pipe_in,pipe_out))
   
   

3. Fork
   

   #!/usr/bin/python3
   import os
   stdout_id=1
   
   pipe_out,pipe_in = os.pipe()
   print ("pipe is: ",(pipe_in,pipe_out))
   
   pid=os.fork()
   #print("forked")
   
   if pid<0:
       print ("fork error")
       exit()
   elif pid == 0:
       #print ("I am child:", str(os.getpid()))
   else:
       #print ("I am parent:", str(os.getpid()))
       #print ("parent: child's pid="+str(pid))
   

4. Trim Pipe
   

   #!/usr/bin/python3
   import os
   stdout_id=1
   
   pipe_out,pipe_in = os.pipe()
   print ("pipe is: ",(pipe_in,pipe_out))
   
   pid=os.fork()
   #print("forked")
   
   if pid<0:
       print ("fork error")
       exit()
   elif pid == 0:
       #print ("I am child:", str(os.getpid()))
       os.close(pipe_in)
       del pipe_in
   else:
       #print ("I am parent:", str(os.getpid()))
       #print ("parent: child's pid="+str(pid))
       os.close(pipe_out)
       del pipe_out
   

5. Plumb Pipe
   

   #!/usr/bin/python3
   import os
   stdout_id=1
   stdin_id=0
   
   pipe_out,pipe_in = os.pipe()
   print ("pipe is: ",(pipe_in,pipe_out))
   
   pid=os.fork()
   print("forked")
   
   if pid<0:
       print ("fork error")
       exit()
   elif pid == 0:
       #print ("I am child:", str(os.getpid()))
       os.close(pipe_in)
       del pipe_in
       os.dup2(pipe_out, stdin_id)
   else:
       #print ("I am parent:", str(os.getpid()))
       #print ("parent: child's pid="+str(pid))
       os.close(pipe_out)
       del pipe_out
       pipe=os.fdopen(pipe_in,"r")
       pipe.write("hello, child\n")
       pipe.write("goodbye, child\n")
   

6. Exec
   

   #!/usr/bin/python3
   import os
   stdout_id=1
   stdin_id=0
   
   pipe_out,pipe_in = os.pipe()
   #print ("pipe is: ",(pipe_in,pipe_out))
   
   pid=os.fork()
   #print("forked")
   
   if pid<0:
       #print ("fork error")
       exit()
   elif pid == 0:
       #print ("I am child:", str(os.getpid()))
       os.close(pipe_in)
       del pipe_in
       os.dup2(pipe_out, stdin_id)
       os.execvp("grep",("grep", "hello" ))
   else:
       #print ("I am parent:", str(os.getpid()))
       #print ("parent: child's pid="+str(pid))
       os.close(pipe_out)
       del pipe_out
       pipe=os.fdopen(pipe_in,"w")
       pipe.write("hello, child\n")
       pipe.write("goodbye, child\n")
   

   See creation-of-an-interprocess-pipe for how to do it in C.

Setting up sudo on Gnu/Linux

If sudo is not set up properly you will have to:

• Add line %sudo ALL=(ALL) ALL to /etc/sudoers
• Add users that can use unrestricted(but need to authenticate) sudo to group sudo
• Each Gnome user to run gksu-properties and change authentication mode to sudo, to tell gnome to use sudo instead of su

Creation Of an InterProcess Pipe in C Unix/Linux

The most simple way is to use the shell to create a pipeline. But if you want to create any other shape you will have to do some programming. Some languages such as Python, TCL, purl … have built in functions to create task harnesses. Here is how to do it in C.

Creation Of InterProcess Pipes in C on Unix/Linux

1. Create a Task.
   

   int main (void) {
   
   } 

2. Open Pipe
   

   #include <unistd.h>
   int main (void) {
      
       int MyPipe[2];
       if ( pipe( MyPipe) != 0 ){
           Error("Could not open pipe");
       } 
       
   }

3. Fork
   

   #include <unistd.h>
   int main (void) {
       int MyPipe[2];
   
       int Pid;
   
       if ( pipe( MyPipe) != 0 ){
           Error("Could not open pipe");
       }
   
       Pid=fork();
       if (-1 == Pid) // Error
       {
           Error("Could not Fork");
       } 
   }

4. Trim Pipe
   

   #include <unistd.h>
   
   enum  PipeDirection_i {PipeRead=0,PipeWrite=1};
   
   int main (void) {
       int MyPipe[2];
   
       int Pid;
   
       if ( pipe( MyPipe) != 0 ){
           Error("Could not open pipe");
       }
   
       Pid=fork();
       if (-1 == Pid) // Error
       {
           Error("Could not Fork");
       }
   
       else if (0 == Pid) // Child Process....
       {
           /*Close parent ends*/
           CHK( close( My[PipeWrite]   ) ); //Close parent ends
       }
   
       // Parent Process....
       CHK( close( MyPipe[PipeRead] ) );    //Close child ends
    }

5. Plumb Pipe
   

   #include <unistd.h>
   
   enum  PipeDirection_i {PipeRead=0,PipeWrite=1};
   
   FILE* PipeOut;
   
   int main (void) {
       int MyPipe[2];
   
       int Pid;
   
       if ( pipe( MyPipe) != 0 ){
           Error("Could not open pipe");
       }
   
       Pid=fork();
       if (-1 == Pid) // Error
       {
           Error("Could not Fork");
       } 
   
       else if (0 == Pid) // Child Process....
       {
           /*Close parent ends*/
           CHK( close( My[PipeWrite]   ) ); //Close parent ends
   
           /**************************/
           /*Plumbing of Destination */
           CHK( dup2(  MyPipe[PipeRead],0 ) ); //attach stdin to pipe
           CHK( close( MyPipe[PipeRead] ) ); //close old descriptors in child 
       }
   
       // Parent Process....
       CHK( close( MyPipe[PipeRead] ) );    //Close child ends
   
       /*Plumbing of Parent */
       PipeOut = fdopen ( MyPipe[PipeWrite] ,"w"); //attach parent ends
   
       if ( NULL == PipeOut ) {
         Error("error in fdopen opening pipe");
       }
   }

6. Exec
   

   #include <unistd.h>
   
   enum  PipeDirection_i {PipeRead=0,PipeWrite=1};
   
   FILE* PipeOut;
   
   int main (void) {
       int MyPipe[2];
   
       int Pid;
   
       if ( pipe( MyPipe) != 0 ){
           Error("Could not open pipe");
       }
   
       Pid=fork();
       if (-1 == Pid) // Error
       {
           Error("Could not Fork");
       } 
   
       else if (0 == Pid) // Child Process....
       {
           /*Close parent ends*/
           CHK( close( My[PipeWrite]   ) ); //Close parent ends 
   
           /**************************/
           /*Plumbing of Destination */
           CHK( dup2(  MyPipe[PipeRead],0 ) ); //attach stdin to pipe
           CHK( close( MyPipe[PipeRead] ) ); //close old descriptors in child
   
       execl( "/bin/bash","--verbose","-c",ExecName,NULL );
       //or: execl( ExecName,NULL );
       //or: execl( ExecName,...,NULL ); 
   
       //Cant get here unless exec fails.
       Error("Did not exec");
       }
   
       // Parent Process....
       CHK( close( MyPipe[PipeRead] ) );    //Close child ends  
   
       /*Plumbing of Parent */
       PipeOut = fdopen ( MyPipe[PipeWrite] ,"w"); //attach parent ends
   
       if ( NULL == PipeOut ) {
         Error("error in fdopen opening pipe");
       }
   }

Many Connected Tasks

Introduction

The Problem

You can put all the functionality in a single process but this may not be easy. You may be limited to one language. A bug in one part can affect another. You have to debug it all at the same time. If it has a GUI as its interface it will be difficult to do automated tests.

Therefore

To help to separate concerns, break the program in to multiple processes and connect them with inter-process pipes.

A single process task

A Task is usually single stand alone process. It is protected from other tasks. Each process can be written in a different language, by different people, at different times. A Task can talk to the outside world via files, and std in, out and error. A Task has one Std in, out and error.

Here is a task just showing the standard in,out and error. All Tasks Have these input/outputs. Any additional I/O must be created by the Task.

Task PipeLine

A PipeLine is probably the most common way to connect processes. It is easy to do this using a shell. You just type the following (Where A,B,C are the names of the programs): A | B | C . This can be done interactively of from a shell script. As you can see the new Task is made of 3 processes, but it still has one std in, out, and error. Processes A,B,C do not need to know about each other, they just talk via std in, out, and error. The shell does not know How A,B,or C works, it just connects them up.

Task Harness

In a Task Harness task A Starts Task B and connects to it via B’s std in/out, and additional FILE descriptors on A. As you can see the new Task is made of 2 processes, but it still has one std in, out, and error. B does not Know about A. But A knows about B and must do the connecting. A does not know HOW B works. The shell only knows about A. A could open many others processes in the same way.

Tasks

So tasks don’t have to be single processes. In the diagrams above the tasks may be multi process tasks (Pipeline, Task-Harness, or another pattern). No matter how you connect tasks you will have many processes but you can think of the result as a task and then connect this to produce a still more complex task.

Next: Creating Inter process pipes