echo -n or sleep 0 needed to unblock Bourne shell script

echo -n or sleep 0 needed to unblock Bourne shell script

[Matthijs Nescio]

Hi,

Sorry for replying in a new thread.

        Hi,

        I have the strangest problem. I am running some complicated advanced Bourne shell scripting. These scripts work fine under Ubuntu 14.04 LTS. However, in Cygwin I have to add some strange commands with no obvious effect; without these the scripts just hang. I can not disclose the scripts completely because they are company property. I realise that this does not comply to the rules of smart questions, but if anybody else has this problem the Cygwinners need to know about it. I do not exactly know when these problems started. I reinstalled Cygwin twice, several weeks ago and yesterday evening (25-2-2016).

        I have had this problem with several complicated, lengthy scripts. Debugging using echo debug statements is hard because the echo makes the problem go away. When I discovered similar problems in multiple scripts, I reinstalled Cygwin, but the problem persists. Also rebooting my Windows 7 Enterprise edition machine does not solve the problem. I have had some experienced colleagues to look at this and they were as flabbergasted as I was.

        Examples:

           TTP_FILE_DR_KML=`echo $TTP_FILE.dr.kml | sed -s 's#\.txt\.#\.#g'`
           #echo -n> /dev/null
           sleep 0
           if [ "`convert_needed "$TTP_FILE_DR_KML"`" = "convert" ]
           then

        In the examples above and below, either the sleep or the echo has to be commented in, otherwise the script just hangs. In the example below the doit function needs the echo or sleep, otherwise it does not return.

        doit()
        {
           # 8< code removed
           #echo -n
           sleep 0
        }

        In
          order to investigate this problem the Cywinners will need more information. Feel free to ask, but it may not be legal for me to share.

        Questions:
        * Was a similar issue ever reported? I have looked, but did not find anything.
        * Is there a solution?

        Thank you, kind regards, Matthijs.

Marco Atzeri answered:

    I suspect you hit a corner case of resource race.
    May be as the time for execution is very short :

    $ time echo -n> /dev/null 0

    real    0m0.000s
    user    0m0.000s
    sys     0m0.000s

    $ time sleep 0

    real    0m0.070s
    user    0m0.015s
    sys     0m0.062s


    As both the two commands are just supposed to waste minimal time,
    what was the original need of both of them ?

The explaination of the issue may not be clear. I have to add the sleep 0 or the echo in order to keep my scripts running. Without these the scripts hang. Sometimes they can be interrupted by CTRL-C, sometimes they do not respond. The doit() function fails to end all together without the sleep. So the sleep 0 was not there before, it is needed for the flow of the script all of a sudden.

Obviously, if adding an echo to a script changes the behaviour, it is kind of hard to prove that an empty echo has an effect. Again, I showed it to two colleagues who were surprised too. Together there was over 30 years of Bourne shell experience looking at the problem.

The script containing the doit function is single-threaded, so there is no race condition in the script. Actually, it should just end itself after the curly brace. Without either the sleep or the echo it just hangs. Note also that the reproducibility is 100%, which is not expected from a race condition of marginal time differences. With reproducibility here I mean: adding the sleep/echo gives the normal flow, removing the sleep/echo gives a hanging script.

Questions:
* Were there changes in the Bourne shell interpreter in 2015/2016?
* Are these interpreters well-tested for regression?
* Are similar issues reported?

Thank you, kind regards, Matthijs.





 		 	   		  
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: echo -n or sleep 0 needed to unblock Bourne shell script

[Marco Atzeri]

On 27/02/2016 23:28, Matthijs Nescio wrote:
> Hi,
>
> Sorry for replying in a new thread.
>
>          Hi,
>
>          I have the strangest problem. I am running some complicated advanced Bourne shell scripting. These scripts work fine under Ubuntu 14.04 LTS. However, in Cygwin I have to add some strange commands with no obvious effect; without these the scripts just hang. I can not disclose the scripts completely because they are company property. I realise that this does not comply to the rules of smart questions, but if anybody else has this problem the Cygwinners need to know about it. I do not exactly know when these problems started. I reinstalled Cygwin twice, several weeks ago and yesterday evening (25-2-2016).
>
>          I have had this problem with several complicated, lengthy scripts. Debugging using echo debug statements is hard because the echo makes the problem go away. When I discovered similar problems in multiple scripts, I reinstalled Cygwin, but the problem persists. Also rebooting my Windows 7 Enterprise edition machine does not solve the problem. I have had some experienced colleagues to look at this and they were as flabbergasted as I was.
>
>          Examples:
>
>             TTP_FILE_DR_KML=`echo $TTP_FILE.dr.kml | sed -s 's#\.txt\.#\.#g'`
>             #echo -n> /dev/null
>             sleep 0
>             if [ "`convert_needed "$TTP_FILE_DR_KML"`" = "convert" ]
>             then
>
>          In the examples above and below, either the sleep or the echo has to be commented in, otherwise the script just hangs. In the example below the doit function needs the echo or sleep, otherwise it does not return.
>
>          doit()
>          {
>             # 8< code removed
>             #echo -n
>             sleep 0
>          }
>

>
> The explaination of the issue may not be clear. I have to add the sleep 0 or the echo in order to keep my scripts running. Without these the scripts hang. Sometimes they can be interrupted by CTRL-C, sometimes they do not respond. The doit() function fails to end all together without the sleep. So the sleep 0 was not there before, it is needed for the flow of the script all of a sudden.
>
> Obviously, if adding an echo to a script changes the behaviour, it is kind of hard to prove that an empty echo has an effect. Again, I showed it to two colleagues who were surprised too. Together there was over 30 years of Bourne shell experience looking at the problem.
>
> The script containing the doit function is single-threaded, so there is no race condition in the script. Actually, it should just end itself after the curly brace. Without either the sleep or the echo it just hangs. Note also that the reproducibility is 100%, which is not expected from a race condition of marginal time differences. With reproducibility here I mean: adding the sleep/echo gives the normal flow, removing the sleep/echo gives a hanging script.
>
> Questions:
> * Were there changes in the Bourne shell interpreter in 2015/2016?

see the announcements
https://www.cygwin.com/ml/cygwin-announce/

I suspect the list is very long.

> * Are these interpreters well-tested for regression?

Yes.

> * Are similar issues reported?

Not that I personally remember.

> Thank you, kind regards, Matthijs.
>

I misunderstood your first explanation.
So the script hang if you don't add one of the two commands.


 >          doit()
 >          {
 >             8< code removed
 >             #echo -n
 >             #sleep 0
 >          }

more than bash, the output redirection or the piping is a
likely source of hang.

For example this command serie, works differently on
cygwin than on Linux

   echo "prova" > prova.txt
   cat prova.txt
   cat prova.txt > prova.txt
   cat prova.txt

I suspect something similar is happening in your case and it is not
due to bash.

Regards
Marco





--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

Re: echo -n or sleep 0 needed to unblock Bourne shell script

[Marco Atzeri]

On 26/02/2016 14:04, Matthijs Nescio wrote:
> Hi,
>
> I have the strangest problem. I am running some complicated advanced Bourne shell scripting. These scripts work fine under Ubuntu 14.04 LTS. However, in Cygwin I have to add some strange commands with no obvious effect; without these the scripts just hang. I can not disclose the scripts completely because they are company property. I realise that this does not comply to the rules of smart questions, but if anybody else has this problem the Cygwinners need to know about it. I do not exactly know when these problems started. I reinstalled Cygwin twice, several weeks ago and yesterday evening.
>
> I have had this problem with several complicated, lengthy scripts. Debugging using echo debug statements is hard because the echo makes the problem go away. When I discovered similar problems in multiple scripts, I reinstalled Cygwin, but the problem persists. Also rebooting my Windows 7 Enterprise edition machine does not solve the problem. I have had some experienced colleagues to look at this and they were as flabbergasted as I was.
>
> Examples:
>
>    TTP_FILE_DR_KML=`echo $TTP_FILE.dr.kml | sed -s 's#\.txt\.#\.#g'`
>    #echo -n> /dev/null
>    sleep 0
>    if [ "`convert_needed "$TTP_FILE_DR_KML"`" = "convert" ]
>    then
>
> In the examples above and below, either the sleep or the echo has to be commented in, otherwise the script just hangs. In the example below the doit function needs the echo or sleep, otherwise it does not return.
>
> doit()
> {
>    # 8< code removed
>    #echo -n
>    sleep 0
> }
>
> In
>   order to investigate this problem the Cywinners will need more information. Feel free to ask, but it may not be legal for me to share.
>
> Questions:
> * Was a similar issue ever reported? I have looked, but did not find anything.
> * Is there a solution?
>
> Thank you, kind regards, Matthijs.


I suspect you hit a corner case of resource race.
May be as the time for execution is very short :

$ time echo -n > /dev/null 0

real    0m0.000s
user    0m0.000s
sys     0m0.000s

$ time sleep 0

real    0m0.070s
user    0m0.015s
sys     0m0.062s


As both the two commands are just supposed to waste minimal time,
what was the original need of both of them ?



--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple

echo -n or sleep 0 needed to unblock Bourne shell script

[Matthijs Nescio]

Hi,

I have the strangest problem. I am running some complicated advanced Bourne shell scripting. These scripts work fine under Ubuntu 14.04 LTS. However, in Cygwin I have to add some strange commands with no obvious effect; without these the scripts just hang. I can not disclose the scripts completely because they are company property. I realise that this does not comply to the rules of smart questions, but if anybody else has this problem the Cygwinners need to know about it. I do not exactly know when these problems started. I reinstalled Cygwin twice, several weeks ago and yesterday evening.

I have had this problem with several complicated, lengthy scripts. Debugging using echo debug statements is hard because the echo makes the problem go away. When I discovered similar problems in multiple scripts, I reinstalled Cygwin, but the problem persists. Also rebooting my Windows 7 Enterprise edition machine does not solve the problem. I have had some experienced colleagues to look at this and they were as flabbergasted as I was.

Examples:

  TTP_FILE_DR_KML=`echo $TTP_FILE.dr.kml | sed -s 's#\.txt\.#\.#g'`
  #echo -n> /dev/null
  sleep 0
  if [ "`convert_needed "$TTP_FILE_DR_KML"`" = "convert" ]
  then

In the examples above and below, either the sleep or the echo has to be commented in, otherwise the script just hangs. In the example below the doit function needs the echo or sleep, otherwise it does not return.

doit()
{
  # 8< code removed
  #echo -n
  sleep 0
}

In
 order to investigate this problem the Cywinners will need more information. Feel free to ask, but it may not be legal for me to share.

Questions:
* Was a similar issue ever reported? I have looked, but did not find anything.
* Is there a solution?

Thank you, kind regards, Matthijs.
 		 	   		  
--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple