Re: [PATCH] manual: scripts/documented.sh updated

[Girish Joshi <girish946@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6812 bytes --]

Thanks Joe, I've updated the script.
It's just that the output now does not contain the addresses and the
T/W info about the symbol is not present.
Do we need that info about each symbol in the output?

Thanks!
Girish Joshi
girishjoshi.io

On Sat, Mar 11, 2023 at 8:18 PM Joe Simmons-Talbott
<joetalbott@gmail.com> wrote:
>
> On Sun, Mar 05, 2023 at 11:44:22AM +0530, Girish Joshi via Libc-alpha wrote:
> > Hi Florian,
> > Thanks for the review.
> > Turns out `diff` was not working with the given input files, this
> > patch fixes that.
> > I'm not sure if there is a list of people who volunteered to document
> > some specific functions.
> > So that comment from the output html is removed.
> >
> > Thanks!
> > Girish Joshi
> >
> > On Mon, Feb 27, 2023 at 11:26 PM Florian Weimer <fweimer@redhat.com> wrote:
> > >
> > > * Girish Joshi via Libc-alpha:
> > >
> > > > Hello,
> > > > As per the Missing Functions section of master Todo list[1],
> > > > scripts/documented.sh needs to be updated.
> > > >
> > > > Could someone please review this patch?
> > > > It removes the reference to linuxthreads/*.taxi files from the script
> > > > and removes the individual email address from the output.
> > >
> > > The script still doesn't seem to work, probably because the symbol names
> > > are not extract properly:
> > >
> > > 43120 T _dl_mcount_wrapper      43140 T _dl_mcount_wrapper_c    43320 W clogf   43320 W clogf32
> > > 435c0 T _dl_find_object         43740 T realpath        43770 T quick_exit      43790 T _IO_vfscanf
> > > 437c0 T _pthread_cleanup_pus    437d0 T _pthread_cleanup_pop    437d0 W cprojf  437d0 W cprojf32
> > >
> > > _dl_find_object is documented.
> > >
> > > _IO_vfscanf should be recognized as an alias of vfscanf, but this is
> > > more of an RFE than a bug fix.
> > >
> > > Thanks,
> > > Florian
> > >
>
> > From 5148c65ac07688dc5c3ee79a9b0f054232d64df3 Mon Sep 17 00:00:00 2001
> > From: Girish Joshi <girish946@gmail.com>
> > Date: Sun, 5 Mar 2023 11:15:00 +0530
> > Subject: [PATCH] manual: scripts/documented.sh updated
> >
> > References for linuxthreads are removed from scripts/documented.sh.
> > Also removed the individual email address from the html output.
> >
> > Fixed the steps to retrive undocumented functions
> > ---
> >  scripts/documented.sh | 44 +++++++++++++++++++++++++++++++------------
> >  1 file changed, 32 insertions(+), 12 deletions(-)
> >  mode change 100644 => 100755 scripts/documented.sh
> >
> > diff --git a/scripts/documented.sh b/scripts/documented.sh
> > old mode 100644
> > new mode 100755
> > index 7d1ffd4a92..dda6a32283
> > --- a/scripts/documented.sh
> > +++ b/scripts/documented.sh
> > @@ -1,9 +1,15 @@
> >  #!/bin/sh
> >  bindir=$1
> >
> > -VERSION=1.0
> > +# usage: scripts/documented.sh <path to glibc build directory>
> > +# for example:
> > +# scripts/documented.sh $HOME/build/glibc
> >
> > -grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
> > +VERSION=1.1
> > +
> > +# Collect the documented functions from manual/*.texi files.
> > +
> > +grep -E -h @deftypefu?nx? manual/*.texi |
> >  sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
> >  sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
> >  sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
> > @@ -12,9 +18,12 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
> >  sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
> >  sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
> >  sed -e '/^alloca$/d' |
> > +sed -r '/^\s*$/d' |
> >  sort -u > DOCUMENTED
> >
> > -nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> > +
> > +# extract the exported symbols from the libraries.
> > +nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/crypt/libcrypt.so $bindir/login/libutil.so |
> >  grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
> >  sed 's/\(@.*\)//' |
> >  cut -b 12- |
>
> You can change this to 'cut -v 20-' and avoid the extra
> AVAILABLE_WITHOUT_OFFSET part.
>
> Thanks,
> Joe
> > @@ -25,6 +34,9 @@ sed -e '/^_mcleanup$/d' -e '/^_rpc_dtablesize$/d' -e '/^_seterr_reply$/d' |
> >  sed -e '/^_nss/d' -e '/^_setjmp$/d' |
> >  sort -u > AVAILABLE
> >
> > +# remove offset details from nm output
> > +cat AVAILABLE | cut -b 9-  | sort -u  > AVAILABLE_WITHOUT_OFFSET
> > +
> >  cat <<EOF
> >  <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
> >  <html>
> > @@ -39,9 +51,7 @@ cat <<EOF
> >      which are not yet documented in the manual.  This list is
> >      automatically created and therefore might contain errors.  Please
> >      check the latest manual (available from the CVS archive) before
> > -    starting to work.  It might also be good to let me know in
> > -    advanace on which functions you intend to work to avoid
> > -    duplication.</p>
> > +    starting to work.</p>
> >
> >      <p>A few comments:</p>
> >
> > @@ -52,15 +62,23 @@ cat <<EOF
> >        <li>Similarly for the LFS functions (those ending in 64).</li>
> >      </ul>
> >
> > -    <p>The function sombody already volunteered to document are marked
> > -    with a reference to the person.</p>
> >
> >      <center><table>
> >  EOF
> >
> >  n=0
> > -diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
> > -expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
> > +
> > +# get the list of undocumented functions
> > +grep -Fvx -f DOCUMENTED AVAILABLE_WITHOUT_OFFSET > UNDOCUMENTED_WITHOUT_OFFSET
> > +
> > +# get undocumented functions with offset info
> > +for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
> > +do
> > +  grep " $func$" AVAILABLE >> UNDOCUMENTED
> > +done
> > +
> > +diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
> > +expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
> >  while read name; do
> >    line="$line
> >  <td><tt>$name</tt></td>"
> > @@ -95,8 +113,10 @@ cat <<EOF
> >      </table></center>
> >
> >      <hr>
> > -    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
> > -Generated on $(date) with documented.sh version $VERSION
> > +    Generated on $(date) with documented.sh version $VERSION
> >    </body>
> >  </html>
> >  EOF
> > +
> > +# cleanup the files
> > +rm -rf UNDOCUMENTED_WITHOUT_OFFSET AVAILABLE_WITHOUT_OFFSET AVAILABLE
> > --
> > 2.39.2
> >
>

[-- Attachment #2: 0001-manual-scripts-documented.sh-updated.patch --]
[-- Type: text/x-patch, Size: 4100 bytes --]

From a564f1afee1e1f5f74a5852882e8c582cf0ae97d Mon Sep 17 00:00:00 2001
From: Girish Joshi <girish946@gmail.com>
Date: Sun, 31 Mar 2024 00:17:34 +0530
Subject: [PATCH] manual: scripts/documented.sh updated

References for linuxthreads are removed from scripts/documented.sh.
Fixed the steps to retrive undocumented functions

Also removed the individual email address from the html output.

Signed-off-by: Girish Joshi <girish946@gmail.com>
---
 scripts/documented.sh | 41 ++++++++++++++++++++++++++++-------------
 1 file changed, 28 insertions(+), 13 deletions(-)
 mode change 100644 => 100755 scripts/documented.sh

diff --git a/scripts/documented.sh b/scripts/documented.sh
old mode 100644
new mode 100755
index 24bb7b3e54..372a28ebf0
--- a/scripts/documented.sh
+++ b/scripts/documented.sh
@@ -1,9 +1,15 @@
 #!/bin/sh
 bindir=$1
 
-VERSION=1.0
+# usage: scripts/documented.sh <path to glibc build directory>
+# for example:
+# scripts/documented.sh $HOME/build/glibc
 
-grep -E -h @deftypefu?nx? *.texi ../linuxthreads/*.texi |
+VERSION=1.1
+
+# Collect the documented functions from manual/*.texi files.
+
+grep -E -h @deftypefu?nx? manual/*.texi |
 sed -e 's/@deftypefunx*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e 's/@deftypefn {[^}]*function}*[[:space:]]*\({[^{]*}\|[[:alnum:]_]*\)[[:space:]]*\([[:alnum:]_]*\).*/\2/' -e '/^@/d' |
 sed -e '/^obstack_/d' -e '/^\([lf]\|\)stat\(\|64\)$/d' -e '/^mknod$/d' |
 sed -e '/^signbit$/d' -e '/^sigsetjmp$/d' |
@@ -12,12 +18,14 @@ sed -e '/^__fwriting$/d' -e '/^__fwritable$/d' -e '/^__fsetlocking$/d' |
 sed -e '/^__freading$/d' -e '/^__freadable$/d' -e '/^__fpurge$/d' |
 sed -e '/^__fpending$/d' -e '/^__flbf$/d' -e '/^__fbufsize$/d' |
 sed -e '/^alloca$/d' |
+sed -r '/^\s*$/d' |
 sort -u > DOCUMENTED
 
-nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/linuxthreads/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
+# extract the exported symbols from the libraries.
+nm --extern --define $bindir/libc.so $bindir/math/libm.so $bindir/rt/librt.so $bindir/nptl/libpthread.so $bindir/dlfcn/libdl.so $bindir/login/libutil.so |
 grep -E " [TW] ([[:alpha:]]|_[[:alpha:]])" |
 sed 's/\(@.*\)//' |
-cut -b 12- |
+cut -b 20- |
 sed -e '/^_IO/d' -e '/^_dl/d' -e '/^_pthread/d' -e '/^_obstack/d' |
 sed -e '/^_argp/d' -e '/^_authenticate$/d' -e '/^_environ$/d' |
 sed -e '/^_errno$/d' -e '/^_h_errno$/d' -e '/^_longjmp$/d' |
@@ -39,9 +47,7 @@ cat <<EOF
     which are not yet documented in the manual.  This list is
     automatically created and therefore might contain errors.  Please
     check the latest manual (available from the CVS archive) before
-    starting to work.  It might also be good to let me know in
-    advanace on which functions you intend to work to avoid
-    duplication.</p>
+    starting to work.</p>
 
     <p>A few comments:</p>
 
@@ -52,15 +58,22 @@ cat <<EOF
       <li>Similarly for the LFS functions (those ending in 64).</li>
     </ul>
 
-    <p>The function sombody already volunteered to document are marked
-    with a reference to the person.</p>
 
     <center><table>
 EOF
 
 n=0
-diff -y --width=60 --suppress-common-lines DOCUMENTED AVAILABLE |
-expand | cut -b 33- | sed '/^[[:space:]]*$/d' |
+# get the list of undocumented functions
+grep -Fvx -f DOCUMENTED AVAILABLE> UNDOCUMENTED_WITHOUT_OFFSET
+
+# get undocumented functions with offset info
+for func in `cat UNDOCUMENTED_WITHOUT_OFFSET`
+do
+  grep "$func$" AVAILABLE >> UNDOCUMENTED
+done
+
+diff -y --width=120 --suppress-common-lines DOCUMENTED UNDOCUMENTED |
+expand | cut -b 61- | sed '/^[[:space:]]*$/d' |
 while read name; do
   line="$line
 <td><tt>$name</tt></td>"
@@ -95,8 +108,10 @@ cat <<EOF
     </table></center>
 
     <hr>
-    <address><a href="mailto:drepper@redhat.com">Ulrich Drepper</a></address>
-Generated on $(date) with documented.sh version $VERSION
+    Generated on $(date) with documented.sh version $VERSION
   </body>
 </html>
 EOF
+
+# cleanup the files
+rm -rf UNDOCUMENTED_WITHOUT_OFFSET UNDOCUMENTED AVAILABLE DOCUMENTED
-- 
2.43.0