It would be nice if the \exec command from FluidShell would support the pipe feature as Bash does.
A simple example would be:
$ echo simple | exec cat
with the expected output to be:
simple
Or other uses that extend the functionalities offered by FluidShell:
$ echo "Luke
Dan
Mary" | exec sort
with the expected output:
Dan
Luke
Mary
|
73 KB
This feature request implies users are allowed to mix fluid commands and host commands together in a pipeline as examples shown below:
prompt$ \cat file | \exec host-cmd-1 | \tee | \exec host-cmd-2
prompt$ \exec host-cmd-1 | \exec host-cmd-2 | \exec host-cmd-3 > file
Perhaps we could defer this to a later release.
A new FluidShell user has asked for pipeline integration for PowerShell and Bash. So that he can do something like :
:$ sqlexport | exec powershell myscript.ps
:$ exec powershell myscript.ps | sqlimport
A new FluidShell user has asked for pipeline integration for PowerShell and Bash. So that he can do something like :
:$ sqlexport | exec powershell myscript.ps
:$ exec powershell myscript.ps | sqlimport
Sachin was attempting to use "sed" or "awk" for a comand pipeline like this :
:$ sqlexport -k server1 -f insert | exec sed -e 'x/y/z' | source
Supporting "exec sed" and "exec awk" in pipelines would be very useful.
http://en.wikipedia.org/wiki/Sed
http://en.wikipedia.org/wiki/Awk
Sachin was attempting to use "sed" or "awk" for a comand pipeline like this :
:$ sqlexport -k server1 -f insert | exec sed -e 'x/y/z' | source
Supporting "exec sed" and "exec awk" in pipelines would be very useful.
http://en.wikipedia.org/wiki/Sed
http://en.wikipedia.org/wiki/Awk
Initial implementation is done - SVN r29776. Need to test \exec on UNIX.
The following problems yet to be addressed, no solutions at the moment:
(1) Executing \exec command will generate a 'Write end dead' error most of time.
(2) Executing a pipeline using \exec might create a deadlock.
Initial implementation is done - SVN r29776. Need to test \exec on UNIX.
The following problems yet to be addressed, no solutions at the moment:
(1) Executing \exec command will generate a 'Write end dead' error most of time.
(2) Executing a pipeline using \exec might create a deadlock.
Here is a good post on "Write end dead" ..
http://techtavern.wordpress.com/2008/07/16/whats-this-ioexception-write-end-dead/
In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish. The ExecRedirectMonitor should be interrupted if either consumer or producer thread has exited, while right now it only exits if the user cancels or the producer closes the "from" stream. In the example below we should interrupt the _stdin when "exec" completes, and not when "sleep" completes :
sleep 5 | exec echo Niels
Here is a good post on "Write end dead" ..
http://techtavern.wordpress.com/2008/07/16/whats-this-ioexception-write-end-dead/
In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish. The ExecRedirectMonitor should be interrupted if either consumer or producer thread has exited, while right now it only exits if the user cancels or the producer closes the "from" stream. In the example below we should interrupt the _stdin when "exec" completes, and not when "sleep" completes :
sleep 5 | exec echo Niels
'Write end dead' message is generated by shell's Term object which has something to do with ExecStdout (_stdout) and ExecStderr (_stderr) threads. This error likely will be seen if the destination of _stdout and _stderr is shell's terminal (i.e. data will be sent to Term via ShellPipe). I think the root cause of this problem is ShellPipe never closes (it has a paired PipedInputStream and PipedOutputStream where PipedInputStream is used by the Term object if I remember correctly), if data reached Term after _stdout/_stderr threads terminated, the 'Write end dead' message is generated. If this is the case, then we cannot prevent Term from logging this error; I likely will add a sleep(N) at the end of run() in ExecInputMonitor to delay thread termination for a while, this would not solve the problem, though.
> In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish.
I need to do some more test based on your test case to see whether _stdin.interrupt() would help.
We currently don't join/wait for those threads now, on Windows, that will cause deadlock if 2 or more \exec commands involved in a pipeline; doing so, all of command threads will be blocked by _process.waitFor() call. I probably will try some more test to see if we can work around this.
'Write end dead' message is generated by shell's Term object which has something to do with ExecStdout (_stdout) and ExecStderr (_stderr) threads. This error likely will be seen if the destination of _stdout and _stderr is shell's terminal (i.e. data will be sent to Term via ShellPipe). I think the root cause of this problem is ShellPipe never closes (it has a paired PipedInputStream and PipedOutputStream where PipedInputStream is used by the Term object if I remember correctly), if data reached Term after _stdout/_stderr threads terminated, the 'Write end dead' message is generated. If this is the case, then we cannot prevent Term from logging this error; I likely will add a sleep(N) at the end of run() in ExecInputMonitor to delay thread termination for a while, this would not solve the problem, though.
> In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish.
I need to do some more test based on your test case to see whether _stdin.interrupt() would help.
We currently don't join/wait for those threads now, on Windows, that will cause deadlock if 2 or more \exec commands involved in a pipeline; doing so, all of command threads will be blocked by _process.waitFor() call. I probably will try some more test to see if we can work around this.
--- SVN r29792 ---
(1) Executing \exec command will generate a 'Write end dead' error most of time.
We currently work around this problem by delaying the termination of _stdout/_stderr threads by 0.2 seconds.
(2) Executing a pipeline using \exec might create a deadlock.
If problem (1) did occur, there might be a chance to create a deadlock. We don't have a solution at the moment, user needs to hit Control-C to kill the pipeline.
(3) > In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish.
For test case 'sleep 5 | exec echo Niels': test shows command thread, and _stdout/_stderr threads would terminate right away but not _stdin thread. Code is added so that once _process.waitFor() is done, _stdin thread is interrupted as well. Note that this test case will generate a 'Pipe broken' exception (because '\exec' is terminated before '\sleep') followed by some 'Write end dead' messages.
--- SVN r29798 ---
Updated \exec man page.
--- SVN r29792 ---
(1) Executing \exec command will generate a 'Write end dead' error most of time.
We currently work around this problem by delaying the termination of _stdout/_stderr threads by 0.2 seconds.
(2) Executing a pipeline using \exec might create a deadlock.
If problem (1) did occur, there might be a chance to create a deadlock. We don't have a solution at the moment, user needs to hit Control-C to kill the pipeline.
(3) > In ExecProcess.waitFor() I think you may actually need to call _stdin.interrupt() after _process.waitFor(), while you seem to be trying to join/wait for them to finish.
For test case 'sleep 5 | exec echo Niels': test shows command thread, and _stdout/_stderr threads would terminate right away but not _stdin thread. Code is added so that once _process.waitFor() is done, _stdin thread is interrupted as well. Note that this test case will generate a 'Pipe broken' exception (because '\exec' is terminated before '\sleep') followed by some 'Write end dead' messages.
--- SVN r29798 ---
Updated \exec man page.
Verified pipe support with simple test cases. See attached picture.
Verified pipe support with simple test cases. See attached picture.
Added regression tests for the above mentioned scenarios.
Additional bugs or improvements regarding the exec cmd, and or pipe & exec cmd should be added separately.
Closed.
Added regression tests for the above mentioned scenarios.
Additional bugs or improvements regarding the exec cmd, and or pipe & exec cmd should be added separately.
Closed.
Issue #7574 |
Closed |
Fixed |
Resolved |
Completion |
No due date |
Fixed Build trunk/29776, 29792, 29798 |
No time estimate |
This feature request implies users are allowed to mix fluid commands and host commands together in a pipeline as examples shown below:
prompt$ \cat file | \exec host-cmd-1 | \tee | \exec host-cmd-2
prompt$ \exec host-cmd-1 | \exec host-cmd-2 | \exec host-cmd-3 > file
Perhaps we could defer this to a later release.