[PATCH] manual: scripts/documented.sh updated

[PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

[-- Attachment #1: Type: text/plain, Size: 381 bytes --]

Hello,
As per the Missing Functions section of master Todo list[1],
scripts/documented.sh needs to be updated.

Could someone please review this patch?
It removes the reference to linuxthreads/*.taxi files from the script
and removes the individual email address from the output.

Thanks!
Girish Joshi
girishjoshi.io

[1]: https://sourceware.org/glibc/wiki/Development_Todo/Master

[-- Attachment #2: 0001-manual-scripts-documented.sh-updated.patch --]
[-- Type: text/x-patch, Size: 2160 bytes --]

From 8f589f12be85655d5bb5f69b07753ca7aa87bc16 Mon Sep 17 00:00:00 2001
From: Girish Joshi <girish946@gmail.com>
Date: Fri, 24 Feb 2023 19:50:38 +0530
Subject: [PATCH] manual: scripts/documented.sh updated

As mentioned in the master todo list,
References for linuxthreads are removed from scripts/documented.sh.
Also removed the individual email address from the html output.
---
 scripts/documented.sh | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/scripts/documented.sh b/scripts/documented.sh
index 7d1ffd4a92..7d9a044295 100644
--- a/scripts/documented.sh
+++ b/scripts/documented.sh
@@ -1,9 +1,17 @@
 #!/bin/sh
+
+# usage: scripts/documented.sh <path to glibc build directory>
+# for example:
+# scripts/documented.sh $HOME/build/glibc
+
+
 bindir=$1
 
 VERSION=1.0
 
-grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
+# Collect the documented functions from manual/*.texi files.
+
+grep -E -h @deftypefu?nx? manual/*.texi |
 sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
 sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
 sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
@@ -14,7 +22,9 @@ sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
 sed -e '/^alloca$/d' |
 sort -u > DOCUMENTED
 
-nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
+# Generate the HTML doc
+
+nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
 grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
 sed 's/\(@.*\)//' |
 cut -b 12- |
@@ -95,7 +105,6 @@ cat <<EOF
     </table></center>
 
     <hr>
-    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
 Generated on $(date) with documented.sh version $VERSION
   </body>
 </html>
-- 
2.39.1

Re: [PATCH] manual: scripts/documented.sh updated

[Florian Weimer]

* Girish Joshi via Libc-alpha:

> Hello,
> As per the Missing Functions section of master Todo list[1],
> scripts/documented.sh needs to be updated.
>
> Could someone please review this patch?
> It removes the reference to linuxthreads/*.taxi files from the script
> and removes the individual email address from the output.

The script still doesn't seem to work, probably because the symbol names
are not extract properly:

43120 T _dl_mcount_wrapper 	43140 T _dl_mcount_wrapper_c 	43320 W clogf 	43320 W clogf32
435c0 T _dl_find_object 	43740 T realpath 	43770 T quick_exit 	43790 T _IO_vfscanf
437c0 T _pthread_cleanup_pus 	437d0 T _pthread_cleanup_pop 	437d0 W cprojf 	437d0 W cprojf32

_dl_find_object is documented.

_IO_vfscanf should be recognized as an alias of vfscanf, but this is
more of an RFE than a bug fix.

Thanks,
Florian

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

[-- Attachment #1: Type: text/plain, Size: 1317 bytes --]

Hi Florian,
Thanks for the review.
Turns out `diff` was not working with the given input files, this
patch fixes that.
I'm not sure if there is a list of people who volunteered to document
some specific functions.
So that comment from the output html is removed.

Thanks!
Girish Joshi

On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
>
> * Girish Joshi via Libc-alpha:
>
> > Hello,
> > As per the Missing Functions section of master Todo list[1],
> > scripts/documented.sh needs to be updated.
> >
> > Could someone please review this patch?
> > It removes the reference to linuxthreads/*.taxi files from the script
> > and removes the individual email address from the output.
>
> The script still doesn't seem to work, probably because the symbol names
> are not extract properly:
>
> 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
>
> _dl_find_object is documented.
>
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.
>
> Thanks,
> Florian
>

[-- Attachment #2: 0001-manual-scripts-documented.sh-updated.patch --]
[-- Type: text/x-patch, Size: 4244 bytes --]

From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
From: Girish Joshi <girish946@gmail.com>
Date: Sun, 5 Mar 2023 11:15:00 +0530
Subject: [PATCH] manual: scripts/documented.sh updated

References for linuxthreads are removed from scripts/documented.sh.
Also removed the individual email address from the html output.

Fixed the steps to retrive undocumented functions
---
 scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
 1 file changed, 32 insertions(+), 12 deletions(-)
 mode change 100644 => 100755 scripts/documented.sh

diff --git a/scripts/documented.sh b/scripts/documented.sh
old mode 100644
new mode 100755
index 7d1ffd4a92..dda6a32283
--- a/scripts/documented.sh
+++ b/scripts/documented.sh
@@ -1,9 +1,15 @@
 #!/bin/sh
 bindir=$1
 
-VERSION=1.0
+# usage: scripts/documented.sh <path to glibc build directory>
+# for example:
+# scripts/documented.sh $HOME/build/glibc
 
-grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
+VERSION=1.1
+
+# Collect the documented functions from manual/*.texi files.
+
+grep -E -h @deftypefu?nx? manual/*.texi |
 sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
 sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
 sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
@@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
 sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
 sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
 sed -e '/^alloca$/d' |
+sed -r '/^\s*$/d' |
 sort -u > DOCUMENTED
 
-nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
+
+# extract the exported symbols from the libraries.
+nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
 grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
 sed 's/\(@.*\)//' |
 cut -b 12- |
@@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
 sed -e '/^_nss/d' -e '/^_setjmp$/d' |
 sort -u > AVAILABLE
 
+# remove offset details from nm output
+cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
+
 cat <<EOF
 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
 <html>
@@ -39,9 +51,7 @@ cat <<EOF
     which are not yet documented in the manual.  This list is
     automatically created and therefore might contain errors.  Please
     check the latest manual (available from the CVS archive) before
-    starting to work.  It might also be good to let me know in
-    advanace on which functions you intend to work to avoid
-    duplication.</p>
+    starting to work.</p>
 
     <p>A few comments:</p>
 
@@ -52,15 +62,23 @@ cat <<EOF
       <li>Similarly for the LFS functions (those ending in 64).</li>
     </ul>
 
-    <p>The function sombody already volunteered to document are marked
-    with a reference to the person.</p>
 
     <center><table>
 EOF
 
 n=0
-diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
-expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
+
+# get the list of undocumented functions
+grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
+
+# get undocumented functions with offset info
+for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
+do
+  grep " $func$" AVAILABLE >> UNDOCUMENTED
+done
+
+diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
+expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
 while read name; do
   line="$line
 <td><tt>$name</tt></td>"
@@ -95,8 +113,10 @@ cat <<EOF
     </table></center>
 
     <hr>
-    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
-Generated on $(date) with documented.sh version $VERSION
+    Generated on $(date) with documented.sh version $VERSION
   </body>
 </html>
 EOF
+
+# cleanup the files
+rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
-- 
2.39.2

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

Also,
> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.
I'll try to figure out a way to mark such functions as aliases, as of
now the script does not do that.

Thanks,
Girish Joshi


On Sun, Mar 5, 2023 at 11:44 AM Girish Joshi <girish946@gmail.com> wrote:
>
> Hi Florian,
> Thanks for the review.
> Turns out `diff` was not working with the given input files, this
> patch fixes that.
> I'm not sure if there is a list of people who volunteered to document
> some specific functions.
> So that comment from the output html is removed.
>
> Thanks!
> Girish Joshi
>

> >
> > * Girish Joshi via Libc-alpha:
> >
> > > Hello,
> > > As per the Missing Functions section of master Todo list[1],
> > > scripts/documented.sh needs to be updated.
> > >
> > > Could someone please review this patch?
> > > It removes the reference to linuxthreads/*.taxi files from the script
> > > and removes the individual email address from the output.
> >
> > The script still doesn't seem to work, probably because the symbol names
> > are not extract properly:
> >
> > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> >
> > _dl_find_object is documented.
> >
> > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > more of an RFE than a bug fix.
> >
> > Thanks,
> > Florian
> >

Re: [PATCH] manual: scripts/documented.sh updated

[Joe Simmons-Talbott]

On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> Hi Florian,
> Thanks for the review.
> Turns out `diff` was not working with the given input files, this
> patch fixes that.
> I'm not sure if there is a list of people who volunteered to document
> some specific functions.
> So that comment from the output html is removed.
> 
> Thanks!
> Girish Joshi
> 
> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> >
> > * Girish Joshi via Libc-alpha:
> >
> > > Hello,
> > > As per the Missing Functions section of master Todo list[1],
> > > scripts/documented.sh needs to be updated.
> > >
> > > Could someone please review this patch?
> > > It removes the reference to linuxthreads/*.taxi files from the script
> > > and removes the individual email address from the output.
> >
> > The script still doesn't seem to work, probably because the symbol names
> > are not extract properly:
> >
> > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> >
> > _dl_find_object is documented.
> >
> > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > more of an RFE than a bug fix.
> >
> > Thanks,
> > Florian
> >

> From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> From: Girish Joshi <girish946@gmail.com>
> Date: Sun, 5 Mar 2023 11:15:00 +0530
> Subject: [PATCH] manual: scripts/documented.sh updated
> 
> References for linuxthreads are removed from scripts/documented.sh.
> Also removed the individual email address from the html output.
> 
> Fixed the steps to retrive undocumented functions
> ---
>  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
>  1 file changed, 32 insertions(+), 12 deletions(-)
>  mode change 100644 => 100755 scripts/documented.sh
> 
> diff --git a/scripts/documented.sh b/scripts/documented.sh
> old mode 100644
> new mode 100755
> index 7d1ffd4a92..dda6a32283
> --- a/scripts/documented.sh
> +++ b/scripts/documented.sh
> @@ -1,9 +1,15 @@
>  #!/bin/sh
>  bindir=$1
>  
> -VERSION=1.0
> +# usage: scripts/documented.sh <path to glibc build directory>
> +# for example:
> +# scripts/documented.sh $HOME/build/glibc
>  
> -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> +VERSION=1.1
> +
> +# Collect the documented functions from manual/*.texi files.
> +
> +grep -E -h @deftypefu?nx? manual/*.texi |
>  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
>  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
>  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
>  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
>  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
>  sed -e '/^alloca$/d' |
> +sed -r '/^\s*$/d' |
>  sort -u > DOCUMENTED
>  
> -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> +
> +# extract the exported symbols from the libraries.
> +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
>  sed 's/\(@.*\)//' |
>  cut -b 12- |

You can change this to 'cut -v 20-' and avoid the extra
AVAILABLE_WITHOUT_OFFSET part.

Thanks,
Joe
> @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
>  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
>  sort -u > AVAILABLE
>  
> +# remove offset details from nm output
> +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> +
>  cat <<EOF
>  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
>  <html>
> @@ -39,9 +51,7 @@ cat <<EOF
>      which are not yet documented in the manual.  This list is
>      automatically created and therefore might contain errors.  Please
>      check the latest manual (available from the CVS archive) before
> -    starting to work.  It might also be good to let me know in
> -    advanace on which functions you intend to work to avoid
> -    duplication.</p>
> +    starting to work.</p>
>  
>      <p>A few comments:</p>
>  
> @@ -52,15 +62,23 @@ cat <<EOF
>        <li>Similarly for the LFS functions (those ending in 64).</li>
>      </ul>
>  
> -    <p>The function sombody already volunteered to document are marked
> -    with a reference to the person.</p>
>  
>      <center><table>
>  EOF
>  
>  n=0
> -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> +
> +# get the list of undocumented functions
> +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> +
> +# get undocumented functions with offset info
> +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> +do
> +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> +done
> +
> +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
>  while read name; do
>    line="$line
>  <td><tt>$name</tt></td>"
> @@ -95,8 +113,10 @@ cat <<EOF
>      </table></center>
>  
>      <hr>
> -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> -Generated on $(date) with documented.sh version $VERSION
> +    Generated on $(date) with documented.sh version $VERSION
>    </body>
>  </html>
>  EOF
> +
> +# cleanup the files
> +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> -- 
> 2.39.2
> 

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

[-- Attachment #1: Type: text/plain, Size: 6812 bytes --]

Thanks Joe, I've updated the script.
It's just that the output now does not contain the addresses and the
T/W info about the symbol is not present.
Do we need that info about each symbol in the output?

Thanks!
Girish Joshi
girishjoshi.io

On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
<joetalbott@gmail.com> wrote:
>
> On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> > Hi Florian,
> > Thanks for the review.
> > Turns out `diff` was not working with the given input files, this
> > patch fixes that.
> > I'm not sure if there is a list of people who volunteered to document
> > some specific functions.
> > So that comment from the output html is removed.
> >
> > Thanks!
> > Girish Joshi
> >
> > On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> > >
> > > * Girish Joshi via Libc-alpha:
> > >
> > > > Hello,
> > > > As per the Missing Functions section of master Todo list[1],
> > > > scripts/documented.sh needs to be updated.
> > > >
> > > > Could someone please review this patch?
> > > > It removes the reference to linuxthreads/*.taxi files from the script
> > > > and removes the individual email address from the output.
> > >
> > > The script still doesn't seem to work, probably because the symbol names
> > > are not extract properly:
> > >
> > > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> > >
> > > _dl_find_object is documented.
> > >
> > > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > > more of an RFE than a bug fix.
> > >
> > > Thanks,
> > > Florian
> > >
>
> > From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> > From: Girish Joshi <girish946@gmail.com>
> > Date: Sun, 5 Mar 2023 11:15:00 +0530
> > Subject: [PATCH] manual: scripts/documented.sh updated
> >
> > References for linuxthreads are removed from scripts/documented.sh.
> > Also removed the individual email address from the html output.
> >
> > Fixed the steps to retrive undocumented functions
> > ---
> >  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
> >  1 file changed, 32 insertions(+), 12 deletions(-)
> >  mode change 100644 => 100755 scripts/documented.sh
> >
> > diff --git a/scripts/documented.sh b/scripts/documented.sh
> > old mode 100644
> > new mode 100755
> > index 7d1ffd4a92..dda6a32283
> > --- a/scripts/documented.sh
> > +++ b/scripts/documented.sh
> > @@ -1,9 +1,15 @@
> >  #!/bin/sh
> >  bindir=$1
> >
> > -VERSION=1.0
> > +# usage: scripts/documented.sh <path to glibc build directory>
> > +# for example:
> > +# scripts/documented.sh $HOME/build/glibc
> >
> > -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> > +VERSION=1.1
> > +
> > +# Collect the documented functions from manual/*.texi files.
> > +
> > +grep -E -h @deftypefu?nx? manual/*.texi |
> >  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
> >  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
> >  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> > @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
> >  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
> >  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
> >  sed -e '/^alloca$/d' |
> > +sed -r '/^\s*$/d' |
> >  sort -u > DOCUMENTED
> >
> > -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > +
> > +# extract the exported symbols from the libraries.
> > +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> >  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
> >  sed 's/\(@.*\)//' |
> >  cut -b 12- |
>
> You can change this to 'cut -v 20-' and avoid the extra
> AVAILABLE_WITHOUT_OFFSET part.
>
> Thanks,
> Joe
> > @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
> >  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
> >  sort -u > AVAILABLE
> >
> > +# remove offset details from nm output
> > +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> > +
> >  cat <<EOF
> >  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
> >  <html>
> > @@ -39,9 +51,7 @@ cat <<EOF
> >      which are not yet documented in the manual.  This list is
> >      automatically created and therefore might contain errors.  Please
> >      check the latest manual (available from the CVS archive) before
> > -    starting to work.  It might also be good to let me know in
> > -    advanace on which functions you intend to work to avoid
> > -    duplication.</p>
> > +    starting to work.</p>
> >
> >      <p>A few comments:</p>
> >
> > @@ -52,15 +62,23 @@ cat <<EOF
> >        <li>Similarly for the LFS functions (those ending in 64).</li>
> >      </ul>
> >
> > -    <p>The function sombody already volunteered to document are marked
> > -    with a reference to the person.</p>
> >
> >      <center><table>
> >  EOF
> >
> >  n=0
> > -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> > -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> > +
> > +# get the list of undocumented functions
> > +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> > +
> > +# get undocumented functions with offset info
> > +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> > +do
> > +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> > +done
> > +
> > +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> > +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
> >  while read name; do
> >    line="$line
> >  <td><tt>$name</tt></td>"
> > @@ -95,8 +113,10 @@ cat <<EOF
> >      </table></center>
> >
> >      <hr>
> > -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> > -Generated on $(date) with documented.sh version $VERSION
> > +    Generated on $(date) with documented.sh version $VERSION
> >    </body>
> >  </html>
> >  EOF
> > +
> > +# cleanup the files
> > +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> > --
> > 2.39.2
> >
>

[-- Attachment #2: 0001-manual-scripts-documented.sh-updated.patch --]
[-- Type: text/x-patch, Size: 4100 bytes --]

From a564f1afee1e1f5f74a5852882e8c582cf0ae97d Mon Sep 17 00:00:00 2001
From: Girish Joshi <girish946@gmail.com>
Date: Sun, 31 Mar 2024 00:17:34 +0530
Subject: [PATCH] manual: scripts/documented.sh updated

References for linuxthreads are removed from scripts/documented.sh.
Fixed the steps to retrive undocumented functions

Also removed the individual email address from the html output.

Signed-off-by: Girish Joshi <girish946@gmail.com>
---
 scripts/documented.sh | 41 ++++++++++++++++++++++++++++-------------
 1 file changed, 28 insertions(+), 13 deletions(-)
 mode change 100644 => 100755 scripts/documented.sh

diff --git a/scripts/documented.sh b/scripts/documented.sh
old mode 100644
new mode 100755
index 24bb7b3e54..372a28ebf0
--- a/scripts/documented.sh
+++ b/scripts/documented.sh
@@ -1,9 +1,15 @@
 #!/bin/sh
 bindir=$1
 
-VERSION=1.0
+# usage: scripts/documented.sh <path to glibc build directory>
+# for example:
+# scripts/documented.sh $HOME/build/glibc
 
-grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
+VERSION=1.1
+
+# Collect the documented functions from manual/*.texi files.
+
+grep -E -h @deftypefu?nx? manual/*.texi |
 sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
 sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
 sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
@@ -12,12 +18,14 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
 sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
 sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
 sed -e '/^alloca$/d' |
+sed -r '/^\s*$/d' |
 sort -u > DOCUMENTED
 
-nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
+# extract the exported symbols from the libraries.
+nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
 grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
 sed 's/\(@.*\)//' |
-cut -b 12- |
+cut -b 20- |
 sed -e '/^_IO/d' -e '/^_dl/d' -e '/^_pthread/d' -e '/^_obstack/d' |
 sed -e '/^_argp/d' -e '/^_authenticate$/d' -e '/^_environ$/d' |
 sed -e '/^_errno$/d' -e '/^_h_errno$/d' -e '/^_longjmp$/d' |
@@ -39,9 +47,7 @@ cat <<EOF
     which are not yet documented in the manual.  This list is
     automatically created and therefore might contain errors.  Please
     check the latest manual (available from the CVS archive) before
-    starting to work.  It might also be good to let me know in
-    advanace on which functions you intend to work to avoid
-    duplication.</p>
+    starting to work.</p>
 
     <p>A few comments:</p>
 
@@ -52,15 +58,22 @@ cat <<EOF
       <li>Similarly for the LFS functions (those ending in 64).</li>
     </ul>
 
-    <p>The function sombody already volunteered to document are marked
-    with a reference to the person.</p>
 
     <center><table>
 EOF
 
 n=0
-diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
-expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
+# get the list of undocumented functions
+grep -Fvx -f DOCUMENTED AVAILABLE> UNDOCUMENTED_WITHOUT_OFFSET
+
+# get undocumented functions with offset info
+for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
+do
+  grep "$func$" AVAILABLE >> UNDOCUMENTED
+done
+
+diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
+expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
 while read name; do
   line="$line
 <td><tt>$name</tt></td>"
@@ -95,8 +108,10 @@ cat <<EOF
     </table></center>
 
     <hr>
-    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
-Generated on $(date) with documented.sh version $VERSION
+    Generated on $(date) with documented.sh version $VERSION
   </body>
 </html>
 EOF
+
+# cleanup the files
+rm -rf UNDOCUMENTED_WITHOUT_OFFSET UNDOCUMENTED AVAILABLE DOCUMENTED
-- 
2.43.0

Re: [PATCH] manual: scripts/documented.sh updated

[Joe Talbott]

[-- Attachment #1: Type: text/plain, Size: 7517 bytes --]

On Sat, Mar 30, 2024 at 2:54 PM Girish Joshi <girish946@gmail.com> wrote:
>
> Thanks Joe, I've updated the script.
> It's just that the output now does not contain the addresses and the
> T/W info about the symbol is not present.
> Do we need that info about each symbol in the output?

I guess that information is useful.  I'm attaching a patch that
reverts to 'cut -b 12' and does the grep and diff with a copy of
AVAILABLE that does not have the offsets and waits until it builds the
HTML table to get the matching line with the offset.

Thanks,
Joe
>
> Thanks!
> Girish Joshi
> girishjoshi.io
>
> On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
> <joetalbott@gmail.com> wrote:
> >
> > On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> > > Hi Florian,
> > > Thanks for the review.
> > > Turns out `diff` was not working with the given input files, this
> > > patch fixes that.
> > > I'm not sure if there is a list of people who volunteered to document
> > > some specific functions.
> > > So that comment from the output html is removed.
> > >
> > > Thanks!
> > > Girish Joshi
> > >
> > > On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> > > >
> > > > * Girish Joshi via Libc-alpha:
> > > >
> > > > > Hello,
> > > > > As per the Missing Functions section of master Todo list[1],
> > > > > scripts/documented.sh needs to be updated.
> > > > >
> > > > > Could someone please review this patch?
> > > > > It removes the reference to linuxthreads/*.taxi files from the script
> > > > > and removes the individual email address from the output.
> > > >
> > > > The script still doesn't seem to work, probably because the symbol names
> > > > are not extract properly:
> > > >
> > > > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > > > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > > > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> > > >
> > > > _dl_find_object is documented.
> > > >
> > > > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > > > more of an RFE than a bug fix.
> > > >
> > > > Thanks,
> > > > Florian
> > > >
> >
> > > From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> > > From: Girish Joshi <girish946@gmail.com>
> > > Date: Sun, 5 Mar 2023 11:15:00 +0530
> > > Subject: [PATCH] manual: scripts/documented.sh updated
> > >
> > > References for linuxthreads are removed from scripts/documented.sh.
> > > Also removed the individual email address from the html output.
> > >
> > > Fixed the steps to retrive undocumented functions
> > > ---
> > >  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
> > >  1 file changed, 32 insertions(+), 12 deletions(-)
> > >  mode change 100644 => 100755 scripts/documented.sh
> > >
> > > diff --git a/scripts/documented.sh b/scripts/documented.sh
> > > old mode 100644
> > > new mode 100755
> > > index 7d1ffd4a92..dda6a32283
> > > --- a/scripts/documented.sh
> > > +++ b/scripts/documented.sh
> > > @@ -1,9 +1,15 @@
> > >  #!/bin/sh
> > >  bindir=$1
> > >
> > > -VERSION=1.0
> > > +# usage: scripts/documented.sh <path to glibc build directory>
> > > +# for example:
> > > +# scripts/documented.sh $HOME/build/glibc
> > >
> > > -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> > > +VERSION=1.1
> > > +
> > > +# Collect the documented functions from manual/*.texi files.
> > > +
> > > +grep -E -h @deftypefu?nx? manual/*.texi |
> > >  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
> > >  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
> > >  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> > > @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
> > >  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
> > >  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
> > >  sed -e '/^alloca$/d' |
> > > +sed -r '/^\s*$/d' |
> > >  sort -u > DOCUMENTED
> > >
> > > -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > > +
> > > +# extract the exported symbols from the libraries.
> > > +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > >  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
> > >  sed 's/\(@.*\)//' |
> > >  cut -b 12- |
> >
> > You can change this to 'cut -v 20-' and avoid the extra
> > AVAILABLE_WITHOUT_OFFSET part.
> >
> > Thanks,
> > Joe
> > > @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
> > >  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
> > >  sort -u > AVAILABLE
> > >
> > > +# remove offset details from nm output
> > > +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> > > +
> > >  cat <<EOF
> > >  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
> > >  <html>
> > > @@ -39,9 +51,7 @@ cat <<EOF
> > >      which are not yet documented in the manual.  This list is
> > >      automatically created and therefore might contain errors.  Please
> > >      check the latest manual (available from the CVS archive) before
> > > -    starting to work.  It might also be good to let me know in
> > > -    advanace on which functions you intend to work to avoid
> > > -    duplication.</p>
> > > +    starting to work.</p>
> > >
> > >      <p>A few comments:</p>
> > >
> > > @@ -52,15 +62,23 @@ cat <<EOF
> > >        <li>Similarly for the LFS functions (those ending in 64).</li>
> > >      </ul>
> > >
> > > -    <p>The function sombody already volunteered to document are marked
> > > -    with a reference to the person.</p>
> > >
> > >      <center><table>
> > >  EOF
> > >
> > >  n=0
> > > -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> > > -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> > > +
> > > +# get the list of undocumented functions
> > > +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> > > +
> > > +# get undocumented functions with offset info
> > > +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> > > +do
> > > +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> > > +done
> > > +
> > > +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> > > +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
> > >  while read name; do
> > >    line="$line
> > >  <td><tt>$name</tt></td>"
> > > @@ -95,8 +113,10 @@ cat <<EOF
> > >      </table></center>
> > >
> > >      <hr>
> > > -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> > > -Generated on $(date) with documented.sh version $VERSION
> > > +    Generated on $(date) with documented.sh version $VERSION
> > >    </body>
> > >  </html>
> > >  EOF
> > > +
> > > +# cleanup the files
> > > +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> > > --
> > > 2.39.2
> > >
> >

[-- Attachment #2: documented-jst.patch --]
[-- Type: text/x-patch, Size: 3759 bytes --]

diff --git a/scripts/documented.sh b/scripts/documented.sh
old mode 100644
new mode 100755
index 24bb7b3e54..94633a78ea
--- a/scripts/documented.sh
+++ b/scripts/documented.sh
@@ -1,9 +1,15 @@
 #!/bin/sh
 bindir=$1
 
-VERSION=1.0
+# usage: scripts/documented.sh <path to glibc build directory>
+# for example:
+# scripts/documented.sh $HOME/build/glibc
 
-grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
+VERSION=1.1
+
+# Collect the documented functions from manual/*.texi files.
+
+grep -E -h @deftypefu?nx? manual/*.texi |
 sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
 sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
 sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
@@ -12,9 +18,11 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
 sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
 sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
 sed -e '/^alloca$/d' |
+sed -r '/^\s*$/d' |
 sort -u > DOCUMENTED
 
-nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
+# extract the exported symbols from the libraries.
+nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
 grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
 sed 's/\(@.*\)//' |
 cut -b 12- |
@@ -23,7 +31,10 @@ sed -e '/^_argp/d' -e '/^_authenticate$/d' -e '/^_environ$/d' |
 sed -e '/^_errno$/d' -e '/^_h_errno$/d' -e '/^_longjmp$/d' |
 sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
 sed -e '/^_nss/d' -e '/^_setjmp$/d' |
-sort -u > AVAILABLE
+sort -u > AVAILABLE_WITH_OFFSET
+
+# Build a list of available functions with out the offsets
+cat AVAILABLE_WITH_OFFSET | cut -d ' ' -f 3 >AVAILABLE
 
 cat <<EOF
 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
@@ -39,9 +50,7 @@ cat <<EOF
     which are not yet documented in the manual.  This list is
     automatically created and therefore might contain errors.  Please
     check the latest manual (available from the CVS archive) before
-    starting to work.  It might also be good to let me know in
-    advanace on which functions you intend to work to avoid
-    duplication.</p>
+    starting to work.</p>
 
     <p>A few comments:</p>
 
@@ -52,18 +61,20 @@ cat <<EOF
       <li>Similarly for the LFS functions (those ending in 64).</li>
     </ul>
 
-    <p>The function sombody already volunteered to document are marked
-    with a reference to the person.</p>
 
     <center><table>
 EOF
 
 n=0
-diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
-expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
+# get the list of undocumented functions
+grep -Fvx -f DOCUMENTED AVAILABLE> UNDOCUMENTED
+
+diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
+expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
 while read name; do
+  with_offset=`grep -w $name AVAILABLE_WITH_OFFSET`;
   line="$line
-<td><tt>$name</tt></td>"
+<td><tt>$with_offset</tt></td>"
   n=$(expr $n + 1)
   if [ $n -eq 4 ]; then
     echo "<tr>
@@ -95,8 +106,10 @@ cat <<EOF
     </table></center>
 
     <hr>
-    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
-Generated on $(date) with documented.sh version $VERSION
+    Generated on $(date) with documented.sh version $VERSION
   </body>
 </html>
 EOF
+
+# cleanup the files
+rm -rf UNDOCUMENTED_WITHOUT_OFFSET UNDOCUMENTED AVAILABLE AVAILABLE_WITH_OFFSET DOCUMENTED

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
>
> I guess that information is useful.  I'm attaching a patch that
> reverts to 'cut -b 12' and does the grep and diff with a copy of
> AVAILABLE that does not have the offsets and waits until it builds the
> HTML table to get the matching line with the offset.

Thanks Joe,
I Just realized that this patch is archived on the patchwork. So the
latest changes are not reflected there.

Girish Joshi
girishjoshi.io

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

Hello,
Could someone please review this patch?

Thanks!
Girish Joshi
girishjoshi.io

On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
>
> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
> >
> > I guess that information is useful.  I'm attaching a patch that
> > reverts to 'cut -b 12' and does the grep and diff with a copy of
> > AVAILABLE that does not have the offsets and waits until it builds the
> > HTML table to get the matching line with the offset.
>
> Thanks Joe,
> I Just realized that this patch is archived on the patchwork. So the
> latest changes are not reflected there.
>
> Girish Joshi
> girishjoshi.io

Re: [PATCH] manual: scripts/documented.sh updated

[Adhemerval Zanella Netto]

On 03/04/24 12:44, Girish Joshi wrote:
> Hello,
> Could someone please review this patch?
> 
> Thanks!
> Girish Joshi
> girishjoshi.io
> 
> On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
>>
>> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
>>>
>>> I guess that information is useful.  I'm attaching a patch that
>>> reverts to 'cut -b 12' and does the grep and diff with a copy of
>>> AVAILABLE that does not have the offsets and waits until it builds the
>>> HTML table to get the matching line with the offset.
>>
>> Thanks Joe,
>> I Just realized that this patch is archived on the patchwork. So the
>> latest changes are not reflected there.
>>
>> Girish Joshi
>> girishjoshi.io

I will take a look.

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
> I will take a look.
Thanks Adhemerval.

Girish Joshi
girishjoshi.io

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

[-- Attachment #1: Type: text/plain, Size: 792 bytes --]

Hello Florian,
On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> more of an RFE than a bug fix.

I was able to write a small python script that gets the list of
aliases in the given binaries and the html.
Not sure if this can be a separate script or should it be a part of
the same thing?
Please let me know if it can be a separate script, I'll post a patch for it.

Thanks!
Girish Joshi
girishjoshi.io

On Thu, Apr 4, 2024 at 7:28 PM Girish Joshi <girish946@gmail.com> wrote:
>
> On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
> <adhemerval.zanella@linaro.org> wrote:
> >
> > I will take a look.
> Thanks Adhemerval.
>
> Girish Joshi
> girishjoshi.io

[-- Attachment #2: aliases.py --]
[-- Type: text/x-python, Size: 3095 bytes --]

#!/bin/env python
import sys
import subprocess

# list of binaries to check for aliases
input_bins = [
    "libc.so",
    "math/libm.so",
    "rt/librt.so",
    "nptl/libpthread.so",
    "dlfcn/libdl.so",
    "login/libutil.so",
]
base_cmd = "readelf -Ws "

function_identifier = "FUNC"

base_html_body = """
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>glibc functions aliases</title>
  </head>

  <body>
    <center><h1>aliases <tt>glibc</tt> functions</h1></center>

    <center><p>The following table includes names of the function in glibc
    which are aliases of each other.  This list is
    automatically created and therefore might contain errors.</center>
"""


def get_aliases(bin, glibc_build_dir):
    """
    Get aliases of functions in the given binary.
    Aliases are functions that are mapped to the same address.

    Parameters:
        bin (str): binary to check for aliases
        glibc_build_dir (str): path to glibc build directory

    Returns:
        dict: dictionary of aliases.
    """

    # prepare the command string
    cmd = f"{base_cmd} {glibc_build_dir}/{bin}"

    # get the output of readelf
    output = subprocess.check_output(cmd, shell=True)
    lines = output.decode().split("\n")
    aliases = {}
    # parse the output to get the aliases
    for i in lines:
        details = [j for j in i.split(" ") if j]
        if len(details) > 7 and function_identifier in details[3]:
            if details[1] in aliases:
                aliases[details[1]].append(details[7])
            else:
                aliases[details[1]] = [details[7]]
    aliases = {i: aliases[i] for i in aliases if len(aliases[i]) > 1}
    return aliases


def generate_html(aliases, binary, html_body):
    """
    Generate html for the given aliases.

    Parameters:
        aliases (dict): dictionary of aliases
        binary (str): binary name
        html_body (str): html body

    Returns:
        str: html body
    """
    row = (
        """
    <br>
    <center><h1><b>Binary:</b> <tt>
    """
        + binary
        + """</tt><br></h1></center>
        <center><table  border="1 | 0" > 
            <td><tt>location</tt></td>
            <td><tt>funtions</tt></td>
    """
    )
    for i in aliases:
        row += f"    <tr><td>0x{i}</td>\n"
        row += "    <td>\n"
        for j in aliases[i]:
            row += f"        {j}<br>\n"
        row += "    </td>\n"
    row += "    </tr>\n    </table></center>\n"

    html_body = row
    return html_body


if __name__ == "__main__":
    if len(sys.argv) == 1:
        print("Usage: aliases.py <glibc-build-dir>")
        sys.exit(1)

    # remove trailing slash if present
    glib_build_dir = sys.argv[1].strip("/") 
    try:
        html_body = base_html_body
        for bin in input_bins:
            aliases = get_aliases(bin, glib_build_dir)
            html_body += generate_html(aliases, bin, html_body)
        html_body += "  </body>\n</html>"
        print(html_body)
    except subprocess.CalledProcessError as ex:
        print(f"error: {ex}")
        sys.exit(1)

Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi]

ping.

Girish Joshi
girishjoshi.io

On Wed, Apr 3, 2024 at 10:41 PM Adhemerval Zanella Netto
<adhemerval.zanella@linaro.org> wrote:
>
>
>
> On 03/04/24 12:44, Girish Joshi wrote:
> > Hello,
> > Could someone please review this patch?
> >
> > Thanks!
> > Girish Joshi
> > girishjoshi.io
> >
> > On Tue, Apr 2, 2024 at 9:31 AM Girish Joshi <girish946@gmail.com> wrote:
> >>
> >> On Mon, Apr 1, 2024 at 10:03 PM Joe Talbott <joetalbott@gmail.com> wrote:
> >>>
> >>> I guess that information is useful.  I'm attaching a patch that
> >>> reverts to 'cut -b 12' and does the grep and diff with a copy of
> >>> AVAILABLE that does not have the offsets and waits until it builds the
> >>> HTML table to get the matching line with the offset.
> >>
> >> Thanks Joe,
> >> I Just realized that this patch is archived on the patchwork. So the
> >> latest changes are not reflected there.
> >>
> >> Girish Joshi
> >> girishjoshi.io
>
> I will take a look.

Re: [PATCH] manual: scripts/documented.sh updated

[Adhemerval Zanella Netto]

On 01/04/24 13:33, Joe Talbott wrote:
> On Sat, Mar 30, 2024 at 2:54 PM Girish Joshi <girish946@gmail.com> wrote:
>>
>> Thanks Joe, I've updated the script.
>> It's just that the output now does not contain the addresses and the
>> T/W info about the symbol is not present.
>> Do we need that info about each symbol in the output?
> 
> I guess that information is useful.  I'm attaching a patch that
> reverts to 'cut -b 12' and does the grep and diff with a copy of
> AVAILABLE that does not have the offsets and waits until it builds the
> HTML table to get the matching line with the offset.
> 
> Thanks,
> Joe
>>
>> Thanks!
>> Girish Joshi
>> girishjoshi.io
>>
>> On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
>> <joetalbott@gmail.com> wrote:
>>>
>>> On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
>>>> Hi Florian,
>>>> Thanks for the review.
>>>> Turns out `diff` was not working with the given input files, this
>>>> patch fixes that.
>>>> I'm not sure if there is a list of people who volunteered to document
>>>> some specific functions.
>>>> So that comment from the output html is removed.
>>>>
>>>> Thanks!
>>>> Girish Joshi
>>>>
>>>> On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
>>>>>
>>>>> * Girish Joshi via Libc-alpha:
>>>>>
>>>>>> Hello,
>>>>>> As per the Missing Functions section of master Todo list[1],
>>>>>> scripts/documented.sh needs to be updated.
>>>>>>
>>>>>> Could someone please review this patch?
>>>>>> It removes the reference to linuxthreads/*.taxi files from the script
>>>>>> and removes the individual email address from the output.
>>>>>
>>>>> The script still doesn't seem to work, probably because the symbol names
>>>>> are not extract properly:
>>>>>
>>>>> 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
>>>>> 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
>>>>> 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
>>>>>
>>>>> _dl_find_object is documented.
>>>>>
>>>>> _IO_vfscanf should be recognized as an alias of vfscanf, but this is
>>>>> more of an RFE than a bug fix.
>>>>>
>>>>> Thanks,
>>>>> Florian
>>>>>
>>>
>>>> From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
>>>> From: Girish Joshi <girish946@gmail.com>
>>>> Date: Sun, 5 Mar 2023 11:15:00 +0530
>>>> Subject: [PATCH] manual: scripts/documented.sh updated
>>>>
>>>> References for linuxthreads are removed from scripts/documented.sh.
>>>> Also removed the individual email address from the html output.
>>>>
>>>> Fixed the steps to retrive undocumented functions
>>>> ---
>>>>  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
>>>>  1 file changed, 32 insertions(+), 12 deletions(-)
>>>>  mode change 100644 => 100755 scripts/documented.sh
>>>>
>>>> diff --git a/scripts/documented.sh b/scripts/documented.sh
>>>> old mode 100644
>>>> new mode 100755
>>>> index 7d1ffd4a92..dda6a32283
>>>> --- a/scripts/documented.sh
>>>> +++ b/scripts/documented.sh
>>>> @@ -1,9 +1,15 @@
>>>>  #!/bin/sh
>>>>  bindir=$1
>>>>
>>>> -VERSION=1.0
>>>> +# usage: scripts/documented.sh <path to glibc build directory>
>>>> +# for example:
>>>> +# scripts/documented.sh $HOME/build/glibc
>>>>
>>>> -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
>>>> +VERSION=1.1
>>>> +
>>>> +# Collect the documented functions from manual/*.texi files.
>>>> +
>>>> +grep -E -h @deftypefu?nx? manual/*.texi |
>>>>  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
>>>>  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
>>>>  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
>>>> @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
>>>>  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
>>>>  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
>>>>  sed -e '/^alloca$/d' |
>>>> +sed -r '/^\s*$/d' |
>>>>  sort -u > DOCUMENTED
>>>>
>>>> -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>>>> +
>>>> +# extract the exported symbols from the libraries.
>>>> +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
>>>>  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
>>>>  sed 's/\(@.*\)//' |
>>>>  cut -b 12- |
>>>
>>> You can change this to 'cut -v 20-' and avoid the extra
>>> AVAILABLE_WITHOUT_OFFSET part.
>>>
>>> Thanks,
>>> Joe
>>>> @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
>>>>  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
>>>>  sort -u > AVAILABLE
>>>>
>>>> +# remove offset details from nm output
>>>> +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
>>>> +
>>>>  cat <<EOF
>>>>  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
>>>>  <html>
>>>> @@ -39,9 +51,7 @@ cat <<EOF
>>>>      which are not yet documented in the manual.  This list is
>>>>      automatically created and therefore might contain errors.  Please
>>>>      check the latest manual (available from the CVS archive) before
>>>> -    starting to work.  It might also be good to let me know in
>>>> -    advanace on which functions you intend to work to avoid
>>>> -    duplication.</p>
>>>> +    starting to work.</p>
>>>>
>>>>      <p>A few comments:</p>
>>>>
>>>> @@ -52,15 +62,23 @@ cat <<EOF
>>>>        <li>Similarly for the LFS functions (those ending in 64).</li>
>>>>      </ul>
>>>>
>>>> -    <p>The function sombody already volunteered to document are marked
>>>> -    with a reference to the person.</p>
>>>>
>>>>      <center><table>
>>>>  EOF
>>>>
>>>>  n=0
>>>> -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
>>>> -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
>>>> +
>>>> +# get the list of undocumented functions
>>>> +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
>>>> +
>>>> +# get undocumented functions with offset info
>>>> +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
>>>> +do
>>>> +  grep " $func$" AVAILABLE >> UNDOCUMENTED
>>>> +done
>>>> +
>>>> +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
>>>> +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
>>>>  while read name; do
>>>>    line="$line
>>>>  <td><tt>$name</tt></td>"
>>>> @@ -95,8 +113,10 @@ cat <<EOF
>>>>      </table></center>
>>>>
>>>>      <hr>
>>>> -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
>>>> -Generated on $(date) with documented.sh version $VERSION
>>>> +    Generated on $(date) with documented.sh version $VERSION
>>>>    </body>
>>>>  </html>
>>>>  EOF
>>>> +
>>>> +# cleanup the files
>>>> +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
>>>> --
>>>> 2.39.2
>>>>
>>>

The issues from previous reviews seems not to be present and the output is
an improvement over the current status. LGTM, thanks.

Reviewed-by: Adhemerval Zanella  <adhemerval.zanella@linaro.org>

PS: next time please send the patch as inline as well (it is easier to
    comment). Also for newer version send a new message with a vX.