* port contrib/download_prerequisites script to macOS
@ 2017-04-05 1:11 Damian Rouson
2017-04-07 1:04 ` [PATCH, contrib] " Jerry DeLisle
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Damian Rouson @ 2017-04-05 1:11 UTC (permalink / raw)
To: gcc patches; +Cc: Jerry DeLisle
[-- Attachment #1: Type: text/plain, Size: 659 bytes --]
All,
The attached patch modifies the contrib/download_prerequisites script to work on macOS.
The revised script detects the operating system and adjusts the shasum and md5 commands
to their expected name and arguments on macOS. The revised script also uses curl if
wget is not present. macOS ships with curl but not wget.
Tested on macOS and Lubuntu and Fedora Linux distributions.
Ok for trunk?
Damian
2017-04-05 Damian Rouson <damian@sourceryinstitute.org>
* download_prerequisites (md5_check): New function emulates Linux
'md5 --check' on macOS. Modified script for macOS compatibility.
[-- Attachment #2: macOS-download_prerequisites.diff --]
[-- Type: application/octet-stream, Size: 3495 bytes --]
diff --git a/contrib/download_prerequisites b/contrib/download_prerequisites
index a9eac67..a3918cb 100755
--- a/contrib/download_prerequisites
+++ b/contrib/download_prerequisites
@@ -20,7 +20,6 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see http://www.gnu.org/licenses/.
-
program='download_prerequisites'
version='(unversioned)'
@@ -45,7 +44,23 @@ echo_archives() {
graphite=1
verify=1
force=0
-chksum='sha512'
+OS=$(uname)
+
+case $OS in
+ "Darwin")
+ chksum='shasum -a 512 --check'
+ ;;
+ *)
+ chksum='sha512sum --check'
+ ;;
+esac
+
+if type wget > /dev/null ; then
+ fetch='wget'
+else
+ fetch='curl -LO -u anonymous:'
+fi
+chksum_extension='sha512'
directory='.'
helptext="usage: ${program} [OPTION...]
@@ -95,6 +110,19 @@ do
done
unset arg
+# Emulate Linux's 'md5 --check' on macOS
+md5_check() {
+ md5_checksum_line=$(cat -) # Store the standard input: a line from contrib/prerequisites.md5:
+ md5_checksum_expected="${md5_checksum_line%% *}" # Grab the text before the first space
+ file_to_check="${md5_checksum_line##* }" # Grab the text after the first space
+
+ md5_checksum_output=$(md5 -r "${file_to_check}") # Calculate the md5 checksum for the downloaded file
+ md5_checksum_detected="${md5_checksum_output%% *}" # Grab the text before the first space
+ [ "${md5_checksum_expected}" == "${md5_checksum_detected}" ] || die "Cannot verify integrity of possibly corrupted file ${file_to_check}"
+ echo "${file_to_check}: OK"
+}
+
+
argnext=
for arg in "$@"
do
@@ -126,11 +154,27 @@ do
verify=0
;;
--sha512)
- chksum='sha512'
+ case $OS in
+ "Darwin")
+ chksum='shasum -a 512 --check'
+ ;;
+ *)
+ chksum='sha512sum --check'
+ ;;
+ esac
+ chksum_extension='sha512'
verify=1
;;
--md5)
- chksum='md5'
+ case $OS in
+ "Darwin")
+ chksum='md5_check'
+ ;;
+ *)
+ chksum='md5 --check'
+ ;;
+ esac
+ chksum_extension='md5'
verify=1
;;
-*)
@@ -170,19 +214,19 @@ for ar in $(echo_archives)
do
if [ ${force} -gt 0 ]; then rm -f "${directory}/${ar}"; fi
[ -e "${directory}/${ar}" ] \
- || wget --no-verbose -O "${directory}/${ar}" "${base_url}${ar}" \
+ || ${fetch} --no-verbose -O "${directory}/${ar}" "${base_url}${ar}" \
|| die "Cannot download ${ar} from ${base_url}"
done
unset ar
if [ ${verify} -gt 0 ]
then
- chksumfile="contrib/prerequisites.${chksum}"
+ chksumfile="contrib/prerequisites.${chksum_extension}"
[ -r "${chksumfile}" ] || die "No checksums available"
for ar in $(echo_archives)
do
grep "${ar}" "${chksumfile}" \
- | ( cd "${directory}" && "${chksum}sum" --check ) \
+ | ( cd "${directory}" && ${chksum} ) \
|| die "Cannot verify integrity of possibly corrupted file ${ar}"
done
unset chksumfile
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH, contrib] Re: port contrib/download_prerequisites script to macOS
2017-04-05 1:11 port contrib/download_prerequisites script to macOS Damian Rouson
@ 2017-04-07 1:04 ` Jerry DeLisle
2017-04-10 16:48 ` Mike Stump
2017-04-12 22:03 ` Jeff Law
2 siblings, 0 replies; 9+ messages in thread
From: Jerry DeLisle @ 2017-04-07 1:04 UTC (permalink / raw)
To: Damian Rouson, gcc patches
ping
On 04/04/2017 06:10 PM, Damian Rouson wrote:
> All,
>
> The attached patch modifies the contrib/download_prerequisites script to work on macOS.
> The revised script detects the operating system and adjusts the shasum and md5 commands
> to their expected name and arguments on macOS. The revised script also uses curl if
> wget is not present. macOS ships with curl but not wget.
>
> Tested on macOS and Lubuntu and Fedora Linux distributions.
>
> Ok for trunk?
>
> Damian
>
>
> 2017-04-05 Damian Rouson <damian@sourceryinstitute.org>
>
> * download_prerequisites (md5_check): New function emulates Linux
> 'md5 --check' on macOS. Modified script for macOS compatibility.
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: port contrib/download_prerequisites script to macOS
2017-04-05 1:11 port contrib/download_prerequisites script to macOS Damian Rouson
2017-04-07 1:04 ` [PATCH, contrib] " Jerry DeLisle
@ 2017-04-10 16:48 ` Mike Stump
[not found] ` <etPan.58ebd61c.3bd0724f.164e@sourceryinstitute.org>
2017-04-12 22:03 ` Jeff Law
2 siblings, 1 reply; 9+ messages in thread
From: Mike Stump @ 2017-04-10 16:48 UTC (permalink / raw)
To: Damian Rouson; +Cc: gcc patches, Jerry DeLisle
On Apr 4, 2017, at 6:10 PM, Damian Rouson <damian@sourceryinstitute.org> wrote:
>
> The attached patch modifies the contrib/download_prerequisites script to work on macOS.
> Ok for trunk?
Don't know if others would prefer to review... assuming no strong feelings against...
Ok.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: port contrib/download_prerequisites script to macOS
2017-04-05 1:11 port contrib/download_prerequisites script to macOS Damian Rouson
2017-04-07 1:04 ` [PATCH, contrib] " Jerry DeLisle
2017-04-10 16:48 ` Mike Stump
@ 2017-04-12 22:03 ` Jeff Law
2017-04-13 0:40 ` Martin Sebor
[not found] ` <etPan.58f001c5.14409c5f.c453@sourceryinstitute.org>
2 siblings, 2 replies; 9+ messages in thread
From: Jeff Law @ 2017-04-12 22:03 UTC (permalink / raw)
To: Damian Rouson, gcc patches; +Cc: Jerry DeLisle
On 04/04/2017 07:10 PM, Damian Rouson wrote:
> All,
>
> The attached patch modifies the contrib/download_prerequisites script to work on macOS.
> The revised script detects the operating system and adjusts the shasum and md5 commands
> to their expected name and arguments on macOS. The revised script also uses curl if
> wget is not present. macOS ships with curl but not wget.
>
> Tested on macOS and Lubuntu and Fedora Linux distributions.
>
> Ok for trunk?
>
> Damian
>
>
> 2017-04-05 Damian Rouson <damian@sourceryinstitute.org>
>
> * download_prerequisites (md5_check): New function emulates Linux
> 'md5 --check' on macOS. Modified script for macOS compatibility.
I wonder if we should just switch to curl from wget in general rather
than conditionalizing the code at all.
For the sums, rather than doing a check of the OS, just see if
sha512/md5sum exists. If not, then fallback to the Darwin defaults.
Jeff
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: port contrib/download_prerequisites script to macOS
2017-04-12 22:03 ` Jeff Law
@ 2017-04-13 0:40 ` Martin Sebor
2017-04-13 3:19 ` Jerry DeLisle
[not found] ` <etPan.58f001c5.14409c5f.c453@sourceryinstitute.org>
1 sibling, 1 reply; 9+ messages in thread
From: Martin Sebor @ 2017-04-13 0:40 UTC (permalink / raw)
To: Jeff Law, Damian Rouson, gcc patches; +Cc: Jerry DeLisle
On 04/12/2017 04:03 PM, Jeff Law wrote:
> On 04/04/2017 07:10 PM, Damian Rouson wrote:
>> All,
>>
>> The attached patch modifies the contrib/download_prerequisites script
>> to work on macOS.
>> The revised script detects the operating system and adjusts the shasum
>> and md5 commands
>> to their expected name and arguments on macOS. The revised script
>> also uses curl if
>> wget is not present. macOS ships with curl but not wget.
>>
>> Tested on macOS and Lubuntu and Fedora Linux distributions.
>>
>> Ok for trunk?
>>
>> Damian
>>
>>
>> 2017-04-05 Damian Rouson <damian@sourceryinstitute.org>
>>
>> * download_prerequisites (md5_check): New function emulates Linux
>> 'md5 --check' on macOS. Modified script for macOS compatibility.
> I wonder if we should just switch to curl from wget in general rather
> than conditionalizing the code at all.
That was going to be my suggestion as well. It will make updating
the script easier.
Martin
>
> For the sums, rather than doing a check of the OS, just see if
> sha512/md5sum exists. If not, then fallback to the Darwin defaults.
>
> Jeff
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: port contrib/download_prerequisites script to macOS
2017-04-13 0:40 ` Martin Sebor
@ 2017-04-13 3:19 ` Jerry DeLisle
0 siblings, 0 replies; 9+ messages in thread
From: Jerry DeLisle @ 2017-04-13 3:19 UTC (permalink / raw)
To: Martin Sebor, Jeff Law, Damian Rouson, gcc patches
On 04/12/2017 05:40 PM, Martin Sebor wrote:
> On 04/12/2017 04:03 PM, Jeff Law wrote:
>> On 04/04/2017 07:10 PM, Damian Rouson wrote:
>>> All,
>>>
>>> The attached patch modifies the contrib/download_prerequisites script
>>> to work on macOS.
>>> The revised script detects the operating system and adjusts the shasum
>>> and md5 commands
>>> to their expected name and arguments on macOS. The revised script
>>> also uses curl if
>>> wget is not present. macOS ships with curl but not wget.
>>>
>>> Tested on macOS and Lubuntu and Fedora Linux distributions.
>>>
>>> Ok for trunk?
>>>
>>> Damian
>>>
>>>
>>> 2017-04-05 Damian Rouson <damian@sourceryinstitute.org>
>>>
>>> * download_prerequisites (md5_check): New function emulates Linux
>>> 'md5 --check' on macOS. Modified script for macOS compatibility.
>> I wonder if we should just switch to curl from wget in general rather
>> than conditionalizing the code at all.
>
> That was going to be my suggestion as well. It will make updating
> the script easier.
>
> Martin
>
>>
>> For the sums, rather than doing a check of the OS, just see if
>> sha512/md5sum exists. If not, then fallback to the Darwin defaults.
>>
>> Jeff
>
I did not wait long enough for your comments and comitted it already. We can
certainly adjust it.
Jerry
^ permalink raw reply [flat|nested] 9+ messages in thread
[parent not found: <etPan.58f001c5.14409c5f.c453@sourceryinstitute.org>]
* Re: port contrib/download_prerequisites script to macOS
[not found] ` <etPan.58f001c5.14409c5f.c453@sourceryinstitute.org>
@ 2017-04-13 23:29 ` Damian Rouson
2017-04-17 22:11 ` Martin Sebor
0 siblings, 1 reply; 9+ messages in thread
From: Damian Rouson @ 2017-04-13 23:29 UTC (permalink / raw)
To: Jeff Law, gcc patches; +Cc: Jerry DeLisle
Resending as plain text:
On April 12, 2017 at 3:03:19 PM, Jeff Law (law@redhat.com(mailto:law@redhat.com)) wrote:
> >
> > 2017-04-05 Damian Rouson
> >
> > * download_prerequisites (md5_check): New function emulates Linux
> > 'md5 --check' on macOS. Modified script for macOS compatibility.
> I wonder if we should just switch to curl from wget in general rather
> than conditionalizing the code at all.
Hi Jeff,
Thanks for your comments. The conditionals are more portable than hardwiring one choice. On macOS, curl is always present but not wget. On the Linux distribution that I use (Lubuntu), wget is always present but curl isn’t installed by default. My goal was to support as many users as possible without requiring them to install prerequisites just to download the prerequisites. :) As my first contribution of a patch to GCC, I took the baby step of allowing for the use of curl. If acceptable, my next step would be to allow for the use of ftp when neither curl nor wget is present. That’s what I do in the scripts from which I culled the code in the patch. In those scripts, I invoke ftp via the following function:
https://github.com/sourceryinstitute/OpenCoarrays/blob/master/prerequisites/build-functions/ftp_url.sh
>
>
> For the sums, rather than doing a check of the OS, just see if
> sha512/md5sum exists. If not, then fallback to the Darwin defaults.
I think I tried that first and ran into some difficulties. I’ll make another attempt. My recent shell programming experience is with the bash shell and I have some experience with the C-shell from years ago, but I had no experience with writing Bourne shell scripts before writing this patch and I think the problem I encountered was related to differences between Bourne and bash.
Damian
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: port contrib/download_prerequisites script to macOS
2017-04-13 23:29 ` Damian Rouson
@ 2017-04-17 22:11 ` Martin Sebor
0 siblings, 0 replies; 9+ messages in thread
From: Martin Sebor @ 2017-04-17 22:11 UTC (permalink / raw)
To: Damian Rouson, Jeff Law, gcc patches; +Cc: Jerry DeLisle
On 04/13/2017 05:29 PM, Damian Rouson wrote:
> Resending as plain text:
>
> On April 12, 2017 at 3:03:19 PM, Jeff Law (law@redhat.com(mailto:law@redhat.com)) wrote:
>
> > >
> > > 2017-04-05 Damian Rouson
> > >
> > > * download_prerequisites (md5_check): New function emulates Linux
> > > 'md5 --check' on macOS. Modified script for macOS compatibility.
> > I wonder if we should just switch to curl from wget in general rather
> > than conditionalizing the code at all.
>
> Hi Jeff,
>
>
> Thanks for your comments. The conditionals are more portable than hardwiring one choice. On macOS, curl is always present but not wget. On the Linux distribution that I use (Lubuntu), wget is always present but curl isnât installed by default.
It sounds like using the first of the two utilities that's in PATH,
similarly to the sha512/md5sum approach discussed below, would be
a more robust solution. That way it will work on any distribution
that provides either utility, and it will also be possible to test
the script both ways on either kind of system without having to make
changes to it.
Martin
> My goal was to support as many users as possible without requiring them to install prerequisites just to download the prerequisites. :) As my first contribution of a patch to GCC, I took the baby step of allowing for the use of curl. If acceptable, my next step would be to allow for the use of ftp when neither curl nor wget is present. Thatâs what I do in the scripts from which I culled the code in the patch. In those scripts, I invoke ftp via the following function:
>
> https://github.com/sourceryinstitute/OpenCoarrays/blob/master/prerequisites/build-functions/ftp_url.sh
>
> >
> >
> > For the sums, rather than doing a check of the OS, just see if
> > sha512/md5sum exists. If not, then fallback to the Darwin defaults.
>
> I think I tried that first and ran into some difficulties. Iâll make another attempt. My recent shell programming experience is with the bash shell and I have some experience with the C-shell from years ago, but I had no experience with writing Bourne shell scripts before writing this patch and I think the problem I encountered was related to differences between Bourne and bash.
>
>
> Damian
>
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-04-17 21:42 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-05 1:11 port contrib/download_prerequisites script to macOS Damian Rouson
2017-04-07 1:04 ` [PATCH, contrib] " Jerry DeLisle
2017-04-10 16:48 ` Mike Stump
[not found] ` <etPan.58ebd61c.3bd0724f.164e@sourceryinstitute.org>
2017-04-11 16:40 ` Jerry DeLisle
2017-04-12 22:03 ` Jeff Law
2017-04-13 0:40 ` Martin Sebor
2017-04-13 3:19 ` Jerry DeLisle
[not found] ` <etPan.58f001c5.14409c5f.c453@sourceryinstitute.org>
2017-04-13 23:29 ` Damian Rouson
2017-04-17 22:11 ` Martin Sebor
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).