public inbox for crossgcc@sourceware.org
 help / color / mirror / Atom feed
* Storing toolchain configurations and toolchains inside the crosstool-ng tree
@ 2013-07-25 17:33 patrickdepinguin
  2013-07-25 17:58 ` Yann E. MORIN
  0 siblings, 1 reply; 3+ messages in thread
From: patrickdepinguin @ 2013-07-25 17:33 UTC (permalink / raw)
  To: crossgcc maillist, Yann E. MORIN

Hi Yann, all,

Now that crosstool-ng is deprecated inside buildroot, I'm
transitioning to standalone crosstool-ng for building toolchains.
In buildroot, I had two defconfigs for a given project, one for
building the crosstool-ng toolchain, and one for the real project. The
toolchain defconfig would refer to a crosstool-ng config file. After
building, I would create a tarball of the contents of output/host/usr.
This tarball would then be used as an external toolchain in the second
defconfig file.
The defconfig files were stored in the buildroot configs/
subdirectory, while the crosstool-ng configs were stored inside
board/<manufacturer>/<project>.

All this is done in a version-controlled buildroot repository. A user
wanting to reproduce the toolchain simply does 'make foo_defconfig'
followed by 'make'. I'm looking for a similar simplicity in the new
approach.

I have now set up a clone of crosstool-ng. In it, I would first run
'make --enable-local', instead of 'make --prefix=/foo' because I want
to avoid extra locations outside the repository. In fact, all paths
involved in the creation of the toolchain are temporary, as the
toolchain will still be tarballed and used as an external toolchain
for buildroot later on.
After setting up crosstool-ng with --enable-local, I need to set a
configuration. At first sight, using the existing 'samples' directory
to store my project-specific configurations is most convenient. After
all, there already are provisions to store such configurations using
'ct-ng saveconfig', and to set an existing configuration. However, I
faced three issues with this:

1. the samples are named after the tuple. This means that if I make a
project-specific toolchain that happens to have the same tuple as an
existing sample, saving it would overwrite the existing sample.
Because I want to make sure I can easily upgrade to newer versions of
crosstool-ng without too much merging, I want to avoid overwriting
existing samples.
One solution I could think of is to use a custom vendor string, so
that the tuple is unique for my project. This certainly works, but if
there are other solutions I'd be glad to hear about them.

2. As I want to keep the toolchain installation inside the
crosstool-ng repository, because it will be removed after tarballing,
I had set CT_PREFIX_DIR to ${CT_TOP_DIR}/foo/${CT_TARGET}. (more about
'foo' later). Unfortunately, the saveconfig command filters out
CT_PREFIX_DIR from the saved configuration file. Hence, when
re-setting this configuration, the toolchain ends up in the default
${HOME}/x-tools/${CT_TARGET}.
The only way I see to fix this, is to make a modification to
crosstool-ng, either to stop filtering out CT_PREFIX_DIR from the
saved config (although I understand that it may not be ok for other
use cases), or providing another way to easily save and set a
configuration.
I have thought of the using 'ct-ng savedefconfig' and storing the
configuration elsewhere inside the tree, but this seems a bit
duplication of the 'samples'.

3. Regarding the location 'foo' where the toolchain can be temporarily
saved inside the tree: at first I put the toolchain in a subdirectory
output/, in analogy of what buildroot does. Unfortunately, there is no
existing way of cleaning that directory. Neither 'ct-ng clean' nor
'ct-ng distclean' cleans the installed toolchain, and didn't
immediately find a way to do this using ct-ng (unless of course
manually removing the directory with rm).
Then I saw that distclean does remove the legacy location 'targets',
so I figured I could use this directory for storing the toolchains.
It's a bit of a hack, though, and when you remove this legacy code,
I'll have a problem again.
Therefore, I'd like to know if we can come up with another way to meet
my need. One possibility is to add a new command to remove the
generated toolchain(s), another is to foresee a specific directory
where users can put their toolchains if they want to and have this be
removed with distclean. There surely are other solutions as well.

Looking forward to your thoughts...

Thomas

--
For unsubscribe information see http://sourceware.org/lists.html#faq

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Storing toolchain configurations and toolchains inside the crosstool-ng tree
  2013-07-25 17:33 Storing toolchain configurations and toolchains inside the crosstool-ng tree patrickdepinguin
@ 2013-07-25 17:58 ` Yann E. MORIN
  2013-07-26  7:57   ` patrickdepinguin
  0 siblings, 1 reply; 3+ messages in thread
From: Yann E. MORIN @ 2013-07-25 17:58 UTC (permalink / raw)
  To: patrickdepinguin; +Cc: crossgcc maillist

Thomas, All,

On 2013-07-25 19:33 +0200, patrickdepinguin spake thusly:
[--SNIP--]
> I have now set up a clone of crosstool-ng. In it, I would first run
> 'make --enable-local', instead of 'make --prefix=/foo' because I want
> to avoid extra locations outside the repository. In fact, all paths
> involved in the creation of the toolchain are temporary, as the
> toolchain will still be tarballed and used as an external toolchain
> for buildroot later on.
> After setting up crosstool-ng with --enable-local, I need to set a
> configuration. At first sight, using the existing 'samples' directory
> to store my project-specific configurations is most convenient. After
> all, there already are provisions to store such configurations using
> 'ct-ng saveconfig', and to set an existing configuration. However, I
> faced three issues with this:
> 
> 1. the samples are named after the tuple. This means that if I make a
> project-specific toolchain that happens to have the same tuple as an
> existing sample, saving it would overwrite the existing sample.
> Because I want to make sure I can easily upgrade to newer versions of
> crosstool-ng without too much merging, I want to avoid overwriting
> existing samples.
> One solution I could think of is to use a custom vendor string, so
> that the tuple is unique for my project. This certainly works, but if
> there are other solutions I'd be glad to hear about them.

I think the custom vendor string is the correct way to differentiate two
otherwise identical toolchains. After all, that its only reason to exist
in the first place! ;-)

Otherwise, what about:
    ct-ng savedefconfig DEFCONFIG=samples/my-project/my-beloved-toolchain
    ct-ng defconfig DEFCONFIG=samples/my-project/my-beloved-toolchain

You do not even have to name the toolchains with a tuple this way (but
arguably, if you want to use the tupple, it is a bit more comples this
way.)

(Note: the DEFCONFIG variable name was recently changed in the
repository; previously, it was called CONFIG, but the doc and the code
did not agree on the naming, so the doc won, and the code now uses
DECONFIG.)

> 2. As I want to keep the toolchain installation inside the
> crosstool-ng repository, because it will be removed after tarballing,
> I had set CT_PREFIX_DIR to ${CT_TOP_DIR}/foo/${CT_TARGET}. (more about
> 'foo' later). Unfortunately, the saveconfig command filters out
> CT_PREFIX_DIR from the saved configuration file. Hence, when
> re-setting this configuration, the toolchain ends up in the default
> ${HOME}/x-tools/${CT_TARGET}.

This is expected, so that when I do saveconfig, the sample does not
include my local path. This way, when a user recalls a sample, we are
sure the install dir will be in a writeable location. And the only
location we are sure is writeable by the user is in a sub-dir of his
${HOME}.

> The only way I see to fix this, is to make a modification to
> crosstool-ng, either to stop filtering out CT_PREFIX_DIR from the
> saved config (although I understand that it may not be ok for other
> use cases),

Not possible, as explained above.

> or providing another way to easily save and set a
> configuration.
> I have thought of the using 'ct-ng savedefconfig' and storing the
> configuration elsewhere inside the tree, but this seems a bit
> duplication of the 'samples'.

See above, that's what I do suggest.

> 3. Regarding the location 'foo' where the toolchain can be temporarily
> saved inside the tree: at first I put the toolchain in a subdirectory
> output/, in analogy of what buildroot does. Unfortunately, there is no
> existing way of cleaning that directory. Neither 'ct-ng clean' nor
> 'ct-ng distclean' cleans the installed toolchain, and didn't
> immediately find a way to do this using ct-ng (unless of course
> manually removing the directory with rm).
> Then I saw that distclean does remove the legacy location 'targets',
> so I figured I could use this directory for storing the toolchains.
> It's a bit of a hack, though, and when you remove this legacy code,
> I'll have a problem again.
> Therefore, I'd like to know if we can come up with another way to meet
> my need. One possibility is to add a new command to remove the
> generated toolchain(s), another is to foresee a specific directory
> where users can put their toolchains if they want to and have this be
> removed with distclean. There surely are other solutions as well.

Care to send a patch to change the comment from "remove legacy build
dir" to "remove legacy build-dir *and* localy installed toolchains" ?
Plus a nit in the doc about this new feature, of course. ;-)

That way, we treat targets/ as the legacy build-dir, *and* the new
locally-installed toolchains at the same time, and the code won't be
removed in the future.

Note: I'm not happy with the phrase "locally-installed". Feel free to
come up with a better wording.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

--
For unsubscribe information see http://sourceware.org/lists.html#faq

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Storing toolchain configurations and toolchains inside the crosstool-ng tree
  2013-07-25 17:58 ` Yann E. MORIN
@ 2013-07-26  7:57   ` patrickdepinguin
  0 siblings, 0 replies; 3+ messages in thread
From: patrickdepinguin @ 2013-07-26  7:57 UTC (permalink / raw)
  To: Yann E. MORIN; +Cc: crossgcc maillist

Hi Yann,

On Thu, Jul 25, 2013 at 7:58 PM, Yann E. MORIN <yann.morin.1998@free.fr> wrote:
> Thomas, All,
>
> On 2013-07-25 19:33 +0200, patrickdepinguin spake thusly:
> [--SNIP--]
>> I have now set up a clone of crosstool-ng. In it, I would first run
>> 'make --enable-local', instead of 'make --prefix=/foo' because I want
>> to avoid extra locations outside the repository. In fact, all paths
>> involved in the creation of the toolchain are temporary, as the
>> toolchain will still be tarballed and used as an external toolchain
>> for buildroot later on.
>> After setting up crosstool-ng with --enable-local, I need to set a
>> configuration. At first sight, using the existing 'samples' directory
>> to store my project-specific configurations is most convenient. After
>> all, there already are provisions to store such configurations using
>> 'ct-ng saveconfig', and to set an existing configuration. However, I
>> faced three issues with this:
>>
>> 1. the samples are named after the tuple. This means that if I make a
>> project-specific toolchain that happens to have the same tuple as an
>> existing sample, saving it would overwrite the existing sample.
>> Because I want to make sure I can easily upgrade to newer versions of
>> crosstool-ng without too much merging, I want to avoid overwriting
>> existing samples.
>> One solution I could think of is to use a custom vendor string, so
>> that the tuple is unique for my project. This certainly works, but if
>> there are other solutions I'd be glad to hear about them.
>
> I think the custom vendor string is the correct way to differentiate two
> otherwise identical toolchains. After all, that its only reason to exist
> in the first place! ;-)
>
> Otherwise, what about:
>     ct-ng savedefconfig DEFCONFIG=samples/my-project/my-beloved-toolchain
>     ct-ng defconfig DEFCONFIG=samples/my-project/my-beloved-toolchain
>
> You do not even have to name the toolchains with a tuple this way (but
> arguably, if you want to use the tupple, it is a bit more comples this
> way.)

Thanks for your feedback.
I'm gonna go for the savedefconfig/defconfig alternative, as it
immediately fixes the CT_PREFIX_DIR removal issue.
As you suggest, I'll put it in a subdirectory of samples.

One new minor issue with this: if the subdir does not yet exist, the
savedefconfig fails without clear warning:

GEN   savedefconfig
*** Error while saving defconfig to: samples/my-project/my-beloved-toolchain

make: *** [savedefconfig] Error 1


I'm not sure whether it's kconfig or crosstool-ng that should attempt
to create the directory, or whether it's at all necessary to fix this.
What do you think?


>
> (Note: the DEFCONFIG variable name was recently changed in the
> repository; previously, it was called CONFIG, but the doc and the code
> did not agree on the naming, so the doc won, and the code now uses
> DECONFIG.)
>
>> 2. As I want to keep the toolchain installation inside the
>> crosstool-ng repository, because it will be removed after tarballing,
>> I had set CT_PREFIX_DIR to ${CT_TOP_DIR}/foo/${CT_TARGET}. (more about
>> 'foo' later). Unfortunately, the saveconfig command filters out
>> CT_PREFIX_DIR from the saved configuration file. Hence, when
>> re-setting this configuration, the toolchain ends up in the default
>> ${HOME}/x-tools/${CT_TARGET}.
>
> This is expected, so that when I do saveconfig, the sample does not
> include my local path. This way, when a user recalls a sample, we are
> sure the install dir will be in a writeable location. And the only
> location we are sure is writeable by the user is in a sub-dir of his
> ${HOME}.
>
>> The only way I see to fix this, is to make a modification to
>> crosstool-ng, either to stop filtering out CT_PREFIX_DIR from the
>> saved config (although I understand that it may not be ok for other
>> use cases),
>
> Not possible, as explained above.
>
>> or providing another way to easily save and set a
>> configuration.
>> I have thought of the using 'ct-ng savedefconfig' and storing the
>> configuration elsewhere inside the tree, but this seems a bit
>> duplication of the 'samples'.
>
> See above, that's what I do suggest.
>
>> 3. Regarding the location 'foo' where the toolchain can be temporarily
>> saved inside the tree: at first I put the toolchain in a subdirectory
>> output/, in analogy of what buildroot does. Unfortunately, there is no
>> existing way of cleaning that directory. Neither 'ct-ng clean' nor
>> 'ct-ng distclean' cleans the installed toolchain, and didn't
>> immediately find a way to do this using ct-ng (unless of course
>> manually removing the directory with rm).
>> Then I saw that distclean does remove the legacy location 'targets',
>> so I figured I could use this directory for storing the toolchains.
>> It's a bit of a hack, though, and when you remove this legacy code,
>> I'll have a problem again.
>> Therefore, I'd like to know if we can come up with another way to meet
>> my need. One possibility is to add a new command to remove the
>> generated toolchain(s), another is to foresee a specific directory
>> where users can put their toolchains if they want to and have this be
>> removed with distclean. There surely are other solutions as well.
>
> Care to send a patch to change the comment from "remove legacy build
> dir" to "remove legacy build-dir *and* localy installed toolchains" ?
> Plus a nit in the doc about this new feature, of course. ;-)
>
> That way, we treat targets/ as the legacy build-dir, *and* the new
> locally-installed toolchains at the same time, and the code won't be
> removed in the future.
>
> Note: I'm not happy with the phrase "locally-installed". Feel free to
> come up with a better wording.

Ok, I will.
What about calling them 'in-tree-installed toolchains', or 'in-tree
toolchain installations'
I also don't have much better suggestions, I'm afraid...

Thanks,
Thomas

--
For unsubscribe information see http://sourceware.org/lists.html#faq

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2013-07-26  7:57 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-07-25 17:33 Storing toolchain configurations and toolchains inside the crosstool-ng tree patrickdepinguin
2013-07-25 17:58 ` Yann E. MORIN
2013-07-26  7:57   ` patrickdepinguin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).