[PATCH] set/show python dont-write-bytecode fixes

[PATCH] set/show python dont-write-bytecode fixes

[Kevin Buettner]

GDB uses the environment variable PYTHONDONTWRITEBYTECODE to
determine whether or not to write the result of byte-compiling
python modules when the "python dont-write-bytecode" setting
is "auto".  Simon noticed that GDB's implementation doesn't
follow the Python documentation.

At present, GDB only checks for the existence of this environment
variable.  That is not sufficient though.  Regarding
PYTHONDONTWRITEBYTECODE, this document...

    https://docs.python.org/3/using/cmdline.html

...says:

    If this is set to a non-empty string, Python won't try to write
    .pyc files on the import of source modules.

This commit fixes GDB's handling of PYTHONDONTWRITEBYTECODE by adding
an empty string check.

This commit also corrects the set/show command documentation for
"python dont-write-bytecode".  The current doc was just a copy
of that for set/show python ignore-environment.
---
 gdb/python/python.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/gdb/python/python.c b/gdb/python/python.c
index c719e3dc90c..2b85bbb6301 100644
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -1865,8 +1865,15 @@ python_write_bytecode ()
   int wbc = 0;
 
   if (python_dont_write_bytecode == AUTO_BOOLEAN_AUTO)
-    wbc = (!python_ignore_environment
-	    && getenv ("PYTHONDONTWRITEBYTECODE") != nullptr) ? 0 : 1;
+    {
+      if (python_ignore_environment)
+	wbc = 1;
+      else
+	{
+	  const char *pdwbc = getenv ("PYTHONDONTWRITEBYTECODE");
+	  wbc = (pdwbc == nullptr || pdwbc[0] == '\0') ? 1 : 0;
+	}
+    }
   else
     wbc = python_dont_write_bytecode == AUTO_BOOLEAN_TRUE ? 0 : 1;
 
@@ -2341,11 +2348,11 @@ python executable."),
 
   add_setshow_auto_boolean_cmd ("dont-write-bytecode", no_class,
 				&python_dont_write_bytecode, _("\
-Set whether the Python interpreter should ignore environment variables."), _(" \
-Show whether the Python interpreter showlist ignore environment variables."), _(" \
-When enabled GDB's Python interpreter will ignore any Python related\n	\
-flags in the environment.  This is equivalent to passing `-E' to a\n	\
-python executable."),
+Set whether the Python interpreter won't byte-compile python modules."), _("\
+Show whether the Python interpreter won't byte-compile python modules."), _("\
+When enabled GDB's Python interpreter won't byte-compile python modules.\n\
+In order to take effect, this setting must be enabled before python\n\
+intialization."),
 				set_python_dont_write_bytecode,
 				show_python_dont_write_bytecode,
 				&user_set_python_list,
-- 
2.36.1

Re: [PATCH] set/show python dont-write-bytecode fixes

[Pedro Alves]

Hi Kevin,

I just skimmed the patch without paying much attention, but I noticed a typo, below,
in "intialization".

On 2022-07-20 8:14 p.m., Kevin Buettner via Gdb-patches wrote:
> +In order to take effect, this setting must be enabled before python\n\
> +intialization."),

Re: [PATCH] set/show python dont-write-bytecode fixes

[Kevin Buettner]

On Thu, 21 Jul 2022 18:55:35 +0100
Pedro Alves <pedro@palves.net> wrote:

> Hi Kevin,
> 
> I just skimmed the patch without paying much attention, but I noticed a typo, below,
> in "intialization".
> 
> On 2022-07-20 8:14 p.m., Kevin Buettner via Gdb-patches wrote:
> > +In order to take effect, this setting must be enabled before python\n\
> > +intialization."),  
> 

Thanks for spotting this!

Fixed - see below:

GDB uses the environment variable PYTHONDONTWRITEBYTECODE to
determine whether or not to write the result of byte-compiling
python modules when the "python dont-write-bytecode" setting
is "auto".  Simon noticed that GDB's implementation doesn't
follow the Python documentation.

At present, GDB only checks for the existence of this environment
variable.  That is not sufficient though.  Regarding
PYTHONDONTWRITEBYTECODE, this document...

    https://docs.python.org/3/using/cmdline.html

...says:

    If this is set to a non-empty string, Python won't try to write
    .pyc files on the import of source modules.

This commit fixes GDB's handling of PYTHONDONTWRITEBYTECODE by adding
an empty string check.

This commit also corrects the set/show command documentation for
"python dont-write-bytecode".  The current doc was just a copy
of that for set/show python ignore-environment.
---
 gdb/python/python.c | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/gdb/python/python.c b/gdb/python/python.c
index c7d4157b70c..eef7946f28d 100644
--- a/gdb/python/python.c
+++ b/gdb/python/python.c
@@ -1866,8 +1866,15 @@ python_write_bytecode ()
   int wbc = 0;
 
   if (python_dont_write_bytecode == AUTO_BOOLEAN_AUTO)
-    wbc = (!python_ignore_environment
-	    && getenv ("PYTHONDONTWRITEBYTECODE") != nullptr) ? 0 : 1;
+    {
+      if (python_ignore_environment)
+	wbc = 1;
+      else
+	{
+	  const char *pdwbc = getenv ("PYTHONDONTWRITEBYTECODE");
+	  wbc = (pdwbc == nullptr || pdwbc[0] == '\0') ? 1 : 0;
+	}
+    }
   else
     wbc = python_dont_write_bytecode == AUTO_BOOLEAN_TRUE ? 0 : 1;
 
@@ -2346,11 +2353,11 @@ python executable."),
 
   add_setshow_auto_boolean_cmd ("dont-write-bytecode", no_class,
 				&python_dont_write_bytecode, _("\
-Set whether the Python interpreter should ignore environment variables."), _(" \
-Show whether the Python interpreter showlist ignore environment variables."), _(" \
-When enabled GDB's Python interpreter will ignore any Python related\n	\
-flags in the environment.  This is equivalent to passing `-E' to a\n	\
-python executable."),
+Set whether the Python interpreter won't byte-compile python modules."), _("\
+Show whether the Python interpreter won't byte-compile python modules."), _("\
+When enabled GDB's Python interpreter won't byte-compile python modules.\n\
+In order to take effect, this setting must be enabled before python\n\
+initialization."),
 				set_python_dont_write_bytecode,
 				show_python_dont_write_bytecode,
 				&user_set_python_list,

Re: [PATCH] set/show python dont-write-bytecode fixes

[Eli Zaretskii]

> Date: Thu, 21 Jul 2022 16:08:32 -0700
> From: Kevin Buettner via Gdb-patches <gdb-patches@sourceware.org>
> Cc: Simon Marchi <simark@simark.ca>, gdb-patches@sourceware.org
> 
> @@ -2346,11 +2353,11 @@ python executable."),
>  
>    add_setshow_auto_boolean_cmd ("dont-write-bytecode", no_class,
>  				&python_dont_write_bytecode, _("\
> -Set whether the Python interpreter should ignore environment variables."), _(" \
> -Show whether the Python interpreter showlist ignore environment variables."), _(" \
> -When enabled GDB's Python interpreter will ignore any Python related\n	\
> -flags in the environment.  This is equivalent to passing `-E' to a\n	\
> -python executable."),
> +Set whether the Python interpreter won't byte-compile python modules."), _("\
> +Show whether the Python interpreter won't byte-compile python modules."), _("

"Set whether ... won't byte-compile" is a kind-of double negation.
How about

 Set whether the Python interpreter should avoid byte-compiling python modules.

instead (and similar change in the "Show" line)?

> +When enabled GDB's Python interpreter won't byte-compile python modules.\n\
               ^
Comma missing there.

> +In order to take effect, this setting must be enabled before python\n\
> +initialization."),

Will GDB users understand clearly what that means in practical terms?
Should we say instead "before invoking the Python interpreter for the
first time"?

Finally should we mention the PYTHONDONTWRITEBYTECODE environment
variable?

Thanks.

Re: [PATCH] set/show python dont-write-bytecode fixes

[Kevin Buettner]

On Fri, 22 Jul 2022 09:40:07 +0300
Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Thu, 21 Jul 2022 16:08:32 -0700
> > From: Kevin Buettner via Gdb-patches <gdb-patches@sourceware.org>
> > Cc: Simon Marchi <simark@simark.ca>, gdb-patches@sourceware.org
> > 
> > @@ -2346,11 +2353,11 @@ python executable."),
> >  
> >    add_setshow_auto_boolean_cmd ("dont-write-bytecode", no_class,
> >  				&python_dont_write_bytecode, _("\
> > -Set whether the Python interpreter should ignore environment variables."), _(" \
> > -Show whether the Python interpreter showlist ignore environment variables."), _(" \
> > -When enabled GDB's Python interpreter will ignore any Python related\n	\
> > -flags in the environment.  This is equivalent to passing `-E' to a\n	\
> > -python executable."),
> > +Set whether the Python interpreter won't byte-compile python modules."), _("\
> > +Show whether the Python interpreter won't byte-compile python modules."), _("  
> 
> "Set whether ... won't byte-compile" is a kind-of double negation.
> How about
> 
>  Set whether the Python interpreter should avoid byte-compiling python modules.
> 
> instead (and similar change in the "Show" line)?
> 
> > +When enabled GDB's Python interpreter won't byte-compile python modules.\n\  
>                ^
> Comma missing there.

I've made each of your suggested changes above in my local tree.  I'll
send a v2 patch once we work out the rest (below).

> > +In order to take effect, this setting must be enabled before python\n\
> > +initialization."),  
>
> Will GDB users understand clearly what that means in practical terms?

The GDB manual contains similar (though not identical) wording.  I
referenced the manual while writing the help text.  However, the
manual does contain additional explanation to help the reader
understand how to use the command so that it will actually take
effect.

> Should we say instead "before invoking the Python interpreter for the
> first time"?

We could, but I don't think that your suggested wording is accurate.

My understanding of things is that in order to effectively use 'set
python dont-write-bytecode', it must be placed in an early
initialization file, i.e. those run via the --early-init-eval-command
or -eix options.  If the command were placed in some other
initialization file, it might well be run before the embedded Python
interpreter is run for the first time, yet still not take effect due
to the command being executed after Python initialization.

Another way to make sure that the command takes effect is to
use one of the following options on the GDB command line:

  --early-init-eval-command 'set python dont-write-bytecode on'

or:

  -eiex 'set python dont-write-bytecode on'

> Finally should we mention the PYTHONDONTWRITEBYTECODE environment
> variable?

It is already documented in the GDB manual.  Should it also be
mentioned in the help text?

Another fly in the ointment is that the 'python ignore-environment'
setting also affects how dont-write-bytecode behaves when it's set to
'auto'.  If ignore-environment is 'on', then PYTHONDONTWRITEBYTECODE
is ignored.

This seems like a lot of detail to place in the help text.

IMO, this is an pretty obscure command.  I'm okay with a GDB user
having to ferret out the details from the manual.  However, I am
(also) willing to expand the help text if you think it useful.

I just noticed that, regarding the "auto" setting, the GDB manual says
"... in this mode Python will check the environment variable
PYTHONDONTWRITEBYTECODE to see if it should write out byte-code or
not."  What it doesn't say is what settings the environment variable
must have in order to cause byte-code compilation to be suppressed.
It turns out that any value other than non-existence or the
empty string will enable the supression of byte-code compilation.
I.e. using "PYTHONDONTWRITEBYTECODE=no" might not do what the user
expects.  So, it seems to me that we might update the manual to
make this clear.  (I can include that in the v2 patch if you agree.)

Kevin

Re: [PATCH] set/show python dont-write-bytecode fixes

[Eli Zaretskii]

> Date: Fri, 22 Jul 2022 18:37:22 -0700
> From: Kevin Buettner <kevinb@redhat.com>
> Cc: gdb-patches@sourceware.org, pedro@palves.net, simark@simark.ca
> 
> > > +In order to take effect, this setting must be enabled before python\n\
> > > +initialization."),  
> >
> > Will GDB users understand clearly what that means in practical terms?
> 
> The GDB manual contains similar (though not identical) wording.  I
> referenced the manual while writing the help text.  However, the
> manual does contain additional explanation to help the reader
> understand how to use the command so that it will actually take
> effect.
> 
> > Should we say instead "before invoking the Python interpreter for the
> > first time"?
> 
> We could, but I don't think that your suggested wording is accurate.
> 
> My understanding of things is that in order to effectively use 'set
> python dont-write-bytecode', it must be placed in an early
> initialization file, i.e. those run via the --early-init-eval-command
> or -eix options.  If the command were placed in some other
> initialization file, it might well be run before the embedded Python
> interpreter is run for the first time, yet still not take effect due
> to the command being executed after Python initialization.
> 
> Another way to make sure that the command takes effect is to
> use one of the following options on the GDB command line:
> 
>   --early-init-eval-command 'set python dont-write-bytecode on'
> 
> or:
> 
>   -eiex 'set python dont-write-bytecode on'

Maybe we should say in the doc string that these are the only ways of
changing this setting so it has effect?

> > Finally should we mention the PYTHONDONTWRITEBYTECODE environment
> > variable?
> 
> It is already documented in the GDB manual.  Should it also be
> mentioned in the help text?

That was my question.

> Another fly in the ointment is that the 'python ignore-environment'
> setting also affects how dont-write-bytecode behaves when it's set to
> 'auto'.  If ignore-environment is 'on', then PYTHONDONTWRITEBYTECODE
> is ignored.
> 
> This seems like a lot of detail to place in the help text.

Maybe we should try saying all that, and then decide whether the
resulting text is too long?

> I just noticed that, regarding the "auto" setting, the GDB manual says
> "... in this mode Python will check the environment variable
> PYTHONDONTWRITEBYTECODE to see if it should write out byte-code or
> not."  What it doesn't say is what settings the environment variable
> must have in order to cause byte-code compilation to be suppressed.
> It turns out that any value other than non-existence or the
> empty string will enable the supression of byte-code compilation.
> I.e. using "PYTHONDONTWRITEBYTECODE=no" might not do what the user
> expects.  So, it seems to me that we might update the manual to
> make this clear.  (I can include that in the v2 patch if you agree.)

I agree.  Thanks.

Re: [PATCH] set/show python dont-write-bytecode fixes

[Kevin Buettner]

On Sat, 23 Jul 2022 09:54:33 +0300
Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Fri, 22 Jul 2022 18:37:22 -0700
> > From: Kevin Buettner <kevinb@redhat.com>
> > 
> > My understanding of things is that in order to effectively use 'set
> > python dont-write-bytecode', it must be placed in an early
> > initialization file, i.e. those run via the --early-init-eval-command
> > or -eix options.  If the command were placed in some other
> > initialization file, it might well be run before the embedded Python
> > interpreter is run for the first time, yet still not take effect due
> > to the command being executed after Python initialization.
> > 
> > Another way to make sure that the command takes effect is to
> > use one of the following options on the GDB command line:
> > 
> >   --early-init-eval-command 'set python dont-write-bytecode on'
> > 
> > or:
> > 
> >   -eiex 'set python dont-write-bytecode on'  
> 
> Maybe we should say in the doc string that these are the only ways of
> changing this setting so it has effect?

Done.

> > > Finally should we mention the PYTHONDONTWRITEBYTECODE environment
> > > variable?  
> > 
> > It is already documented in the GDB manual.  Should it also be
> > mentioned in the help text?  
> 
> That was my question.
> 
> > Another fly in the ointment is that the 'python ignore-environment'
> > setting also affects how dont-write-bytecode behaves when it's set to
> > 'auto'.  If ignore-environment is 'on', then PYTHONDONTWRITEBYTECODE
> > is ignored.
> > 
> > This seems like a lot of detail to place in the help text.  
> 
> Maybe we should try saying all that, and then decide whether the
> resulting text is too long?

Done, though hopefully in a concise manner.
 
> > I just noticed that, regarding the "auto" setting, the GDB manual says
> > "... in this mode Python will check the environment variable
> > PYTHONDONTWRITEBYTECODE to see if it should write out byte-code or
> > not."  What it doesn't say is what settings the environment variable
> > must have in order to cause byte-code compilation to be suppressed.
> > It turns out that any value other than non-existence or the
> > empty string will enable the supression of byte-code compilation.
> > I.e. using "PYTHONDONTWRITEBYTECODE=no" might not do what the user
> > expects.  So, it seems to me that we might update the manual to
> > make this clear.  (I can include that in the v2 patch if you agree.)  
> 
> I agree.  Thanks.

I've made some changes to the GDB manual too.

These changes, including the ones you asked for in your earlier email,
are in the v2 patch.  See:

https://sourceware.org/pipermail/gdb-patches/2022-July/191030.html

Kevin

Re: [PATCH] set/show python dont-write-bytecode fixes

[Eli Zaretskii]

> Date: Sat, 23 Jul 2022 14:30:25 -0700
> From: Kevin Buettner <kevinb@redhat.com>
> Cc: gdb-patches@sourceware.org, pedro@palves.net, simark@simark.ca
> 
> I've made some changes to the GDB manual too.
> 
> These changes, including the ones you asked for in your earlier email,
> are in the v2 patch.  See:
> 
> https://sourceware.org/pipermail/gdb-patches/2022-July/191030.html

Thanks, I approved them.