* bash shell script: recently running, now failing @ 2023-04-06 4:43 Fergus Daly 2023-04-06 8:03 ` Corinna Vinschen ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Fergus Daly @ 2023-04-06 4:43 UTC (permalink / raw) To: 'cygwin@cygwin.com'; +Cc: Fergus Daly I have a "hash bang" bash shell script i.e. first line #! /bin/sh or equivalently #! /bin/bash For various reasons I want this file to be identified as binary so its second line is the single character null \x00 showing up in some editors e.g. nano as ^@ This does not prevent the script from running to a successful conclusion. Or not until recently. Now the script fails with /home/user/bin/file.old.sh: cannot execute binary file Q1 - was bash recently updated? Would this explain the changed behaviour? Q2 - if so, is this newly introduced "glitch" known and presumably intended? Or an unintended consequence that will be retracted in a later update? I then altered the first line to #! /bin/dash whilst retaining the null character at line 2 and subsequent content also unaltered.. The altered script file.new.sh runs as previously to a successful conclusion. Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. Should the edit (dash replacing bash/sh) be incorporated elsewhere or would this be a bad idea (and retained only locally in what is indeed an eccentric and one-off context)? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 4:43 bash shell script: recently running, now failing Fergus Daly @ 2023-04-06 8:03 ` Corinna Vinschen 2023-04-06 12:23 ` Scott Smith 2023-04-06 12:21 ` Andrey Repin 2023-04-06 17:18 ` Adam Dinwoodie 2 siblings, 1 reply; 8+ messages in thread From: Corinna Vinschen @ 2023-04-06 8:03 UTC (permalink / raw) To: cygwin On Apr 6 04:43, Fergus Daly via Cygwin wrote: > I have a "hash bang" bash shell script i.e. first line > #! /bin/sh > or equivalently > #! /bin/bash > For various reasons I want this file to be identified as binary so its second line > is the single character null \x00 showing up in some editors e.g. nano as > ^@ > This does not prevent the script from running to a successful conclusion. > Or not until recently. Now the script fails with > /home/user/bin/file.old.sh: cannot execute binary file > Q1 - was bash recently updated? Would this explain the changed behaviour? bash was recently updated from 4.4.12 to 5.2.15. The behaviour is the same in bash on Linux. Take this file with a \0 in line 2: $ cat -v x.sh #! /bin/bash ^@ echo foo $ bash --version | head -1 GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu) $ ./x.sh ./x.sh: ./x.sh: cannot execute binary file While dash on Linux runs the script: $ sed -i -e 's/bash/dash/' x.sh $ ./x.sh foo > Q2 - if so, is this newly introduced "glitch" known and presumably > intended? Or an unintended consequence that will be retracted in a > later update? Bash follows the POSIX standard: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html#tag_20_117_07 https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403 So I don't expect this will change any time soon. > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. Dash is a minimal shell with no bells and whistles. It loads ands runs slightly faster than bash. If you only need bare minimum bourne shell behaviour, it's a good choice for scripts. Corinna ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 8:03 ` Corinna Vinschen @ 2023-04-06 12:23 ` Scott Smith 0 siblings, 0 replies; 8+ messages in thread From: Scott Smith @ 2023-04-06 12:23 UTC (permalink / raw) To: cygwin; +Cc: Corinna Vinschen [-- Attachment #1: Type: text/plain, Size: 2284 bytes --] Place the nul on the third line. For example: #!/bin/bash # # ^@ identify as a binary file ... On Thu, Apr 6, 2023 at 4:03 AM Corinna Vinschen via Cygwin < cygwin@cygwin.com> wrote: > On Apr 6 04:43, Fergus Daly via Cygwin wrote: > > I have a "hash bang" bash shell script i.e. first line > > #! /bin/sh > > or equivalently > > #! /bin/bash > > For various reasons I want this file to be identified as binary so its > second line > > is the single character null \x00 showing up in some editors e.g. nano as > > ^@ > > This does not prevent the script from running to a successful conclusion. > > Or not until recently. Now the script fails with > > /home/user/bin/file.old.sh: cannot execute binary file > > Q1 - was bash recently updated? Would this explain the changed behaviour? > > bash was recently updated from 4.4.12 to 5.2.15. > > The behaviour is the same in bash on Linux. Take this file with > a \0 in line 2: > > $ cat -v x.sh > #! /bin/bash > ^@ > echo foo > $ bash --version | head -1 > GNU bash, version 5.2.15(1)-release (x86_64-redhat-linux-gnu) > $ ./x.sh > ./x.sh: ./x.sh: cannot execute binary file > > While dash on Linux runs the script: > > $ sed -i -e 's/bash/dash/' x.sh > $ ./x.sh > foo > > > Q2 - if so, is this newly introduced "glitch" known and presumably > > intended? Or an unintended consequence that will be retracted in a > > later update? > > Bash follows the POSIX standard: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sh.html#tag_20_117_07 > > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap03.html#tag_03_403 > > So I don't expect this will change any time soon. > > > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role > and reach of dash. > > Dash is a minimal shell with no bells and whistles. It loads ands runs > slightly faster than bash. If you only need bare minimum bourne shell > behaviour, it's a good choice for scripts. > > > Corinna > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 4:43 bash shell script: recently running, now failing Fergus Daly 2023-04-06 8:03 ` Corinna Vinschen @ 2023-04-06 12:21 ` Andrey Repin 2023-04-07 19:34 ` Brian Inglis 2023-04-06 17:18 ` Adam Dinwoodie 2 siblings, 1 reply; 8+ messages in thread From: Andrey Repin @ 2023-04-06 12:21 UTC (permalink / raw) To: Fergus Daly, cygwin [-- Attachment #1: Type: text/plain, Size: 760 bytes --] Greetings, Fergus Daly! > I have a "hash bang" bash shell script i.e. first line > #! /bin/sh > or equivalently > #! /bin/bash By default, sh is bash in base Cygwin installation. > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. > Should the edit (dash replacing bash/sh) be incorporated elsewhere or would this be a > bad idea (and retained only locally in what is indeed an eccentric and one-off context)? I'm replacing /bin/sh with dash as I've found that even in POSIX mode, bash allows for a lot of bash'izms in scripts, which would not otherwise run under (d?a)?sh. See the post-install script attached. -- With best regards, Andrey Repin Thursday, April 6, 2023 15:18:19 Sorry for my terrible english... [-- Attachment #2: zp_dash-as-bin-sh.dash --] [-- Type: application/octet-stream, Size: 279 bytes --] #!/bin/dash [ -f /bin/dash.exe ] || exit _f="$(mktemp /bin/sh.prelink.XXXXXX)" [ -L /bin/sh.exe ] || mv /bin/sh.exe "$_f" ln -fsT /bin/dash.exe /bin/sh.exe if [ $? -gt 0 ]; then echo "Got a problem creating link for /bin/sh !" >&2 exit 1 else rm "$_f" fi exit 0 ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 12:21 ` Andrey Repin @ 2023-04-07 19:34 ` Brian Inglis 2023-04-08 8:37 ` Andrey Repin 0 siblings, 1 reply; 8+ messages in thread From: Brian Inglis @ 2023-04-07 19:34 UTC (permalink / raw) To: cygwin On 2023-04-06 06:21, Andrey Repin via Cygwin wrote: >> I have a "hash bang" bash shell script i.e. first line >> #! /bin/sh >> or equivalently >> #! /bin/bash > By default, sh is bash in base Cygwin installation. >> Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. >> Should the edit (dash replacing bash/sh) be incorporated elsewhere or would this be a >> bad idea (and retained only locally in what is indeed an eccentric and one-off context)? > I'm replacing /bin/sh with dash as I've found that even in POSIX mode, bash > allows for a lot of bash'izms in scripts, which would not otherwise run under > (d?a)?sh. > See the post-install script attached. You should use either a hard link or copy of d/ash from/to /bin/ to allow it (or any shell) to be used from Windows cmd shells or Scheduled Tasks, which do not recognise Cygwin /usr mounts or symlinks, but can be run with elevated privileges and at sometimes more useful points than user cron jobs. For example, "manually" restarting Cygwin services after delayed startup, stopping Cygwin services and processes before upgrades, and cleaning up cron jobs that have not finished. You should also consider changing the sh man page. -- Take care. Thanks, Brian Inglis Calgary, Alberta, Canada La perfection est atteinte Perfection is achieved non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut -- Antoine de Saint-Exupéry ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-07 19:34 ` Brian Inglis @ 2023-04-08 8:37 ` Andrey Repin 0 siblings, 0 replies; 8+ messages in thread From: Andrey Repin @ 2023-04-08 8:37 UTC (permalink / raw) To: Brian Inglis via Cygwin, cygwin Greetings, Brian Inglis via Cygwin! > On 2023-04-06 06:21, Andrey Repin via Cygwin wrote: >>> I have a "hash bang" bash shell script i.e. first line >>> #! /bin/sh >>> or equivalently >>> #! /bin/bash >> By default, sh is bash in base Cygwin installation. >>> Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. >>> Should the edit (dash replacing bash/sh) be incorporated elsewhere or would this be a >>> bad idea (and retained only locally in what is indeed an eccentric and one-off context)? >> I'm replacing /bin/sh with dash as I've found that even in POSIX mode, bash >> allows for a lot of bash'izms in scripts, which would not otherwise run under >> (d?a)?sh. >> See the post-install script attached. > You should use either a hard link or copy of d/ash from/to /bin/ to allow > it (or any shell) to be used from Windows cmd shells or Scheduled Tasks, That depends on your setup. /usr/bin/env is a real executable and it is a VERY good idea to handle Windows associations through it. But on my end it is covered by CYGWIN=winsymlinks:nativestrict and a more elaborate TCC-RT wrapper script which is again falling down to /usr/bin/env for the purposes of actually starting the program, be it a script or binary or… > which do not recognise Cygwin /usr mounts or symlinks, but can be run with > elevated privileges and at sometimes more useful points than user cron jobs. > For example, "manually" restarting Cygwin services after delayed startup, > stopping Cygwin services and processes before upgrades, and cleaning up cron > jobs that have not finished. That part is handled elsewhere already. > You should also consider changing the sh man page. That's a good idea, thanks. I never did "man sh" so never stumbled across that discrepancy. -- With best regards, Andrey Repin Saturday, April 8, 2023 11:31:34 Sorry for my terrible english... ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 4:43 bash shell script: recently running, now failing Fergus Daly 2023-04-06 8:03 ` Corinna Vinschen 2023-04-06 12:21 ` Andrey Repin @ 2023-04-06 17:18 ` Adam Dinwoodie 2023-04-06 19:26 ` Scott Smith 2 siblings, 1 reply; 8+ messages in thread From: Adam Dinwoodie @ 2023-04-06 17:18 UTC (permalink / raw) To: Fergus Daly; +Cc: 'cygwin@cygwin.com' On Thu, Apr 06, 2023 at 04:43:51AM +0000, Fergus Daly via Cygwin wrote: > I have a "hash bang" bash shell script i.e. first line > #! /bin/sh > or equivalently > #! /bin/bash > For various reasons I want this file to be identified as binary so its second line > is the single character null \x00 showing up in some editors e.g. nano as > ^@ > This does not prevent the script from running to a successful conclusion. > Or not until recently. Now the script fails with > /home/user/bin/file.old.sh: cannot execute binary file > Q1 - was bash recently updated? Would this explain the changed behaviour? > Q2 - if so, is this newly introduced "glitch" known and presumably intended? Or > an unintended consequence that will be retracted in a later update? > I then altered the first line to > #! /bin/dash > whilst retaining the null character at line 2 and subsequent content also unaltered.. > The altered script file.new.sh runs as previously to a successful conclusion. > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role and reach of dash. > Should the edit (dash replacing bash/sh) be incorporated elsewhere or would this be a > bad idea (and retained only locally in what is indeed an eccentric and one-off context)? Dash is smaller and much less feature-rich than Bash. Whether Dash is a suitable replacement for Bash depends on how much (if at all) you're relying on Bash-specific functions. For very simple scripts, the only difference is likely that Dash will be very slightly faster, but working out whether your script is using any "Bashisms" isn't always a trivial job. (I have previously been involved work in migrating scripts between Ksh and Bash, which is a similar-but-different problem, and there were *a lot* of surprises in how the two differed.) Depending on why you want the file to be identified as a binary, and how that identification is being done, you could move your null byte later in the file. In particular, a pattern I've seen several times in Bash is to have a normal Bash script, finishing with an explicit `exit`, followed by an actual binary blob; this can be used to create things like self-extracting bundles, where the binary blob is a tarball and the script at the top of the file has the instructions for extracting the tarball. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: bash shell script: recently running, now failing 2023-04-06 17:18 ` Adam Dinwoodie @ 2023-04-06 19:26 ` Scott Smith 0 siblings, 0 replies; 8+ messages in thread From: Scott Smith @ 2023-04-06 19:26 UTC (permalink / raw) To: Adam Dinwoodie; +Cc: Fergus Daly, cygwin [-- Attachment #1: Type: text/plain, Size: 3882 bytes --] Coming from an enterprise and supercomputing background, we were able to control what shell was available, so bashisms weren't a problem any more than dashisms, fishisms, kornisms, perl or python versionisms, etc, might be. But, when I was in a commercial environment, everything - shell, perl, C, text tools, etc - had to be tested and usage adjusted accordingly. It wasn't uncommon to have a starter script that performed compatibility tests and forked to the appropriate version... of the same code, functionally. Back to the OP's problem. I haven't seen the bash source, but if I had to guess, based on behavior, bash is only checking the line after the she-bang. With a little more testing, I've concluded that the nul ^@ can appear anywhere other than in line #2. My guess is that, after finding the she-bang, bash is reading one more line to make the "is or is not binary" determination. I imagine that is oodles less overhead than scanning the entire file. I have not tested the behavior if the first line is not a she-bang, such as if the script is run via argument to bash. On Thu, Apr 6, 2023 at 1:19 PM Adam Dinwoodie via Cygwin <cygwin@cygwin.com> wrote: > On Thu, Apr 06, 2023 at 04:43:51AM +0000, Fergus Daly via Cygwin wrote: > > I have a "hash bang" bash shell script i.e. first line > > #! /bin/sh > > or equivalently > > #! /bin/bash > > For various reasons I want this file to be identified as binary so its > second line > > is the single character null \x00 showing up in some editors e.g. nano as > > ^@ > > This does not prevent the script from running to a successful conclusion. > > Or not until recently. Now the script fails with > > /home/user/bin/file.old.sh: cannot execute binary file > > Q1 - was bash recently updated? Would this explain the changed behaviour? > > Q2 - if so, is this newly introduced "glitch" known and presumably > intended? Or > > an unintended consequence that will be retracted in a later update? > > I then altered the first line to > > #! /bin/dash > > whilst retaining the null character at line 2 and subsequent content > also unaltered.. > > The altered script file.new.sh runs as previously to a successful > conclusion. > > Q3 - at 1/8 the size of bash and sh, I am not at all sure of the role > and reach of dash. > > Should the edit (dash replacing bash/sh) be incorporated elsewhere or > would this be a > > bad idea (and retained only locally in what is indeed an eccentric and > one-off context)? > > Dash is smaller and much less feature-rich than Bash. Whether Dash is a > suitable replacement for Bash depends on how much (if at all) you're > relying on Bash-specific functions. For very simple scripts, the only > difference is likely that Dash will be very slightly faster, but working > out whether your script is using any "Bashisms" isn't always a trivial > job. > > (I have previously been involved work in migrating scripts between Ksh > and Bash, which is a similar-but-different problem, and there were *a > lot* of surprises in how the two differed.) > > Depending on why you want the file to be identified as a binary, and how > that identification is being done, you could move your null byte later > in the file. In particular, a pattern I've seen several times in Bash > is to have a normal Bash script, finishing with an explicit `exit`, > followed by an actual binary blob; this can be used to create things > like self-extracting bundles, where the binary blob is a tarball and the > script at the top of the file has the instructions for extracting the > tarball. > > -- > Problem reports: https://cygwin.com/problems.html > FAQ: https://cygwin.com/faq/ > Documentation: https://cygwin.com/docs.html > Unsubscribe info: https://cygwin.com/ml/#unsubscribe-simple > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-04-08 8:50 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-04-06 4:43 bash shell script: recently running, now failing Fergus Daly 2023-04-06 8:03 ` Corinna Vinschen 2023-04-06 12:23 ` Scott Smith 2023-04-06 12:21 ` Andrey Repin 2023-04-07 19:34 ` Brian Inglis 2023-04-08 8:37 ` Andrey Repin 2023-04-06 17:18 ` Adam Dinwoodie 2023-04-06 19:26 ` Scott Smith
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).