[ECOS] Tracking eCos as a hg/git submodule

[ECOS] Tracking eCos as a hg/git submodule

[Øyvind Harboe]

I'm reading up on git submodules and it looks like just the ticket
to handle a typical eCos application which is pulled together
from specific modified versions of multiple projects and
libraries, therein eCos.

submodules importantly supports the concept of a modified eCos
repository for a project and a way to track and eventually pushing
those changes back up to the official eCos repository.

Importantly such a modified eCos repository can be shared
amongst several applications.

I would like to see that the choice of DVCS in eCos does not
put unnecessary stumbling blocks in the way working in this
fashion.

I'm sure mercurial supports submodules.

Mercurial can probably clone git repositories... http://hg-git.github.com/.

Vice versa is probably also possible....
http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#Mercurial


-- 
Øyvind Harboe
http://www.zylin.com/zy1000.html
ARM7 ARM9 ARM11 XScale Cortex
JTAG debugger and flash programmer

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Tracking eCos as a hg/git submodule

[Alex Schuilenburg]

Øyvind Harboe wrote on 2009-10-14 12:37:
> I'm reading up on git submodules and it looks like just the ticket
> to handle a typical eCos application which is pulled together
> from specific modified versions of multiple projects and
> libraries, therein eCos.
>
> submodules importantly supports the concept of a modified eCos
> repository for a project and a way to track and eventually pushing
> those changes back up to the official eCos repository.
>
> Importantly such a modified eCos repository can be shared
> amongst several applications.
>
> I would like to see that the choice of DVCS in eCos does not
> put unnecessary stumbling blocks in the way working in this
> fashion.
>   
That rules out bzr then because AFAIK it does not currently support
submodules.  But then I pretty much have ruled out bzr for other reasons.

However, while external developers may want to use submodules and indeed
have eCos as a submodule as you suggest, I dont really see that much of
a need for them within the public eCos repo myself - unless you really
want to break down individual eCos packages into separate submodules but
IMHO that is just taking things too far.

> I'm sure mercurial supports submodules.
>   
It does, as mentioned in a previous email sometime ago.

> Mercurial can probably clone git repositories... http://hg-git.github.com/.
>   
It does, as per my email a couple of days ago ;-)

> Vice versa is probably also possible....
> http://git.or.cz/gitwiki/InterfacesFrontendsAndTools#Mercurial
>   
I would be surprised if git could not clone from git, although the tools
you list there are conversion tools, not cross DRCS push/pull tools such
as the hg-git hg plugin as above.  The hg-git plugin allows you to treat
a git repo as if it were an hg repo from hg.  However, I would not be
keen for git to be selected because of a feature it lacked (i.e. being
able push/pull to/from an hg repo).

As someone pushing for git, maybe you can give me an answer on this
since I have not yet found a git equivalent.  Does git allow changesets
to follow copied files?  i.e. if you copy file a to file b, and then
modify b, but later someone finds a bug in a and fixes it, does git
propogate the fix automatically to b?  hg does this and IMHO it is a
very useful feature since many device drivers and BSPs in ecos are
clones of others.  It would cover not only bugs, but changes in APIs and
suchlike which means the risk of conflict would be small.  So for
example if a template device driver or API were provided in hg, updates
to the template would be propogated automatically to copies making major
API changes a lot easier.

FAOD, I am trying to track down a genuine set of useful technical
differences between git and hg to help the maintainers make their mind
up.  For example, git octopus merges have no current equivalent in hg,
though somehow I don't see that git super-power user feature will have
much, if any, use in eCos.  However, changeset tracking copies of files
IMHO is very useful.

The other difference I have found is that git repos tend to grow in size
and become inefficient if left unattended for long periods and require
packing, while hg repos dont.

I have ruled out speed as a technical measurement since IMHO the
differences are negligible and can be ignored.

-- Alex Schuilenburg

Managing Director/CEO                                eCosCentric Limited
www.ecoscentric.com


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Tracking eCos as a hg/git submodule

[Patrick Doyle]

On Wed, Oct 14, 2009 at 9:20 AM, Alex Schuilenburg
<alexs@ecoscentric.com> wrote:
> Øyvind Harboe wrote on 2009-10-14 12:37:
>
> However, while external developers may want to use submodules and indeed
> have eCos as a submodule as you suggest, I dont really see that much of
> a need for them within the public eCos repo myself - unless you really
> want to break down individual eCos packages into separate submodules but
> IMHO that is just taking things too far.
I have a very weak disagreement with that last point...

I think that it might make sense to package the different processors
as submodules.  At least, I thought that might make sense back in the
day when I ported eCos to run on the OMAP.  I did the port; published
it on the mailing list; Jonathan reviewed it; I changed jobs; updated
the port; ported it again to another OMAP variant as well as several
different boards custom designed at the new job; and never saw my
original port make it to the official repository.

I'm not complaining... there were only 2 or possible 3 other people in
the world who were slightly interested in the port, and I pointed them
to the patches on the mailing list and helped them out as best as I
could.  Besides, there was no reasonable way for the eCos maintainers
to test or integrate my ports into the mainline tree.

But I recall thinking at the time, "Gee, wouldn't it be nice if I
could just publish this package somewhere and let others use it."  If
I were to do this again today, I would probably place my ports on
GitHUB and just make an announcement on the mailing list that the
ports were available to anybody who wanted to use them.  Some of
Jonathan's criticisms with my original port had to do with concerns
regarding code that had, ahem, been too inspired by Linux.  For quite
valid reasons, he couldn't let that code into the official repository.
 If I had published it on my own (along with the disclaimers that
such-and-such modules were, ahem, inspired by similar code in the
Linux kernel), then the official eCos repository would not be tainted
by pure GPL code.

As I said, it's a weak disagreement, but I think that eCos' packaging
system lends itself quite well to grafting in submodules from Git (or
mercurial, with which I have no experience).  It could also streamline
the core eCos distribution and remove the need to carry around code
for ancient ports and processors.

That's my next $.02 contribution to the discussion.

--wpd

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Tracking eCos as a hg/git submodule

[Øyvind Harboe]

On Wed, Oct 14, 2009 at 3:39 PM, Patrick Doyle <wpdster@gmail.com> wrote:
> On Wed, Oct 14, 2009 at 9:20 AM, Alex Schuilenburg
> <alexs@ecoscentric.com> wrote:
>> Øyvind Harboe wrote on 2009-10-14 12:37:
>>
>> However, while external developers may want to use submodules and indeed
>> have eCos as a submodule as you suggest, I dont really see that much of
>> a need for them within the public eCos repo myself - unless you really
>> want to break down individual eCos packages into separate submodules but
>> IMHO that is just taking things too far.
> I have a very weak disagreement with that last point...
>
> I think that it might make sense to package the different processors
> as submodules.  At least, I thought that might make sense back in the
> day when I ported eCos to run on the OMAP.  I did the port; published
> it on the mailing list; Jonathan reviewed it; I changed jobs; updated
> the port; ported it again to another OMAP variant as well as several
> different boards custom designed at the new job; and never saw my
> original port make it to the official repository.

I think this is an excellent point.

If we can make this work well, including
FSF assignments, then eCos should see a real boost in hardware
support... There are *LOTS* of eCos HAL's there that never make it
into the official eCos repository, simply because there is no manpower.

Though I don't believe that eCos itself should use submodules, much,
rather that the application engineer would use one submodule for the offical
core eCos repository and another submodule for the OMAP parts.
There are, as you point out, a very small crowd that will be interested
in the OMAP HAL's and only for as long as it is in the job description.

(You can list several eCos repositories in ECOS_REPOSITORY environment
variable, a much undervalued, undermarketed feature...)


-- 
Øyvind Harboe
http://www.zylin.com/zy1000.html
ARM7 ARM9 ARM11 XScale Cortex
JTAG debugger and flash programmer

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Tracking eCos as a hg/git submodule

[Øyvind Harboe]

> Does git allow changesets to follow copied files?  i.e. if you copy file
> a to file b, and then modify b, but later someone finds a bug in a and
> fixes it, does git propogate the fix automatically to b?

I'm not at the point where I can answer this yet. :-) I'm sure a git
expert will volunteer information here.

Sounds neat though.



-- 
Øyvind Harboe
http://www.zylin.com/zy1000.html
ARM7 ARM9 ARM11 XScale Cortex
JTAG debugger and flash programmer

--
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] Re: Tracking eCos as a hg/git submodule

[Sergei Organov]

Alex Schuilenburg <alexs@ecoscentric.com> writes:
[...]
> As someone pushing for git, maybe you can give me an answer on this
> since I have not yet found a git equivalent.

As yet another git proponent, I'll try to answer this.

> Does git allow changesets to follow copied files?

I don't think it does, but neither does Mercurial, see below.

> i.e. if you copy file a to file b, and then modify b, but later
> someone finds a bug in a and fixes it, does git propogate the fix
> automatically to b? hg does this and IMHO it is a very useful feature
> since many device drivers and BSPs in ecos are clones of others. It
> would cover not only bugs, but changes in APIs and suchlike which
> means the risk of conflict would be small. So for example if a
> template device driver or API were provided in hg, updates to the
> template would be propogated automatically to copies making major API
> changes a lot easier.

It seems you've missed the point of "hg copy". (Caveat: I didn't use it,
it's just how I understand it from documentation.) It's not a way to
permanently automatically propagate changes made to a file to a (set of)
other file(s). A change will propagate to the copy only on the first
merge (never on commit) and then propagation will stop.

Here are quotes taken from here:
<http://hgbook.red-bean.com/read/mercurial-in-daily-use.html>

"First of all, remember that this propagation only happens when you
merge. So if you hg copy a file, and subsequently modify the original
file during the normal course of your work, nothing will happen."

"Once your change history has a record that the copy and subsequent merge
occurred, there's usually no further need to propagate changes from the
original file to the copied file, and that's why Mercurial only
propagates changes across copies at the first merge, and not
afterwards."

Based on the above, as soon as public repository has "the copy" and a
cloned repository see it, a change won't propagate anymore even on
merges.

The nice thing about "hg copy" is that "hg log" will be able to show
history back including the history of origin of the copy. And yes, "git
log" will also learn this soon (if not already, as I've already seen a
patch). However, git doesn't have "git copy" as it auto-detects the
copies. Yes, it will be able to figure them out even if you just applied
a raw patch to you working tree using 'patch' utility and committed the
result ;-) (this is a note to the question of what system is better
suited for patch-based workflows).

> FAOD, I am trying to track down a genuine set of useful technical
> differences between git and hg to help the maintainers make their mind
> up.  For example, git octopus merges have no current equivalent in hg,
> though somehow I don't see that git super-power user feature will have
> much, if any, use in eCos.  However, changeset tracking copies of files
> IMHO is very useful.

[Un]fortunately, there is no such thing in Mercurial either, see above.
"hg copy" seems to be useless for this purpose and is at the same
category of usefulness as git octopus merges. [BTW, do you really mean
you'd like a commit to be able to change a lot of files when you did
modification to only one? I seriously doubt it's a good idea.]

On the other hand, for example, "git blame" has an ability to
automatically detect and follow copied and moved pieces of code without
any additional steps taken at commit time. You see, power becomes
visible in deep details. Does Hg have 'blame'? Yes, it does, but that
doesn't mean it is as powerful as git's. The same is with named branches
inside repository. Does Hg have them? Yes, it does, but it is rather
recent addition, while git was born with this, so I suspect gits'
support for such branches is superior. Such things makes comparisons of
the tools even more difficult (for example, it could be the case that Hg
already has copy/move detection I'm not aware of).

Another thing is tagging. Both have tagging, but it's implemented very
differently. Me personally likes git's idea of tags more.

Yet another thing is copy/rename tracking/detection and merges
propagation over them. Its deep subtle details that actually make
difference in this field.

> The other difference I have found is that git repos tend to grow in size
> and become inefficient if left unattended for long periods and require
> packing, while hg repos dont.

Well, git manages Linux source trees just fine, you know. eCos is very
small compared to that. But even if git repos tend to grow indeed, a
cron job on public repositories should do the trick, I think.

-- Sergei.


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Tracking eCos as a hg/git submodule

[Alex Schuilenburg]

Sergei Organov wrote on 2009-10-15 13:42:
> Alex Schuilenburg <alexs@ecoscentric.com> writes:
> [...]
>   
>> As someone pushing for git, maybe you can give me an answer on this
>> since I have not yet found a git equivalent.
>>     
>
> As yet another git proponent, I'll try to answer this.
>
>   
>> Does git allow changesets to follow copied files?
>>     
>
> I don't think it does, but neither does Mercurial, see below.
>
>   
>> i.e. if you copy file a to file b, and then modify b, but later
>> someone finds a bug in a and fixes it, does git propogate the fix
>> automatically to b? hg does this and IMHO it is a very useful feature
>> since many device drivers and BSPs in ecos are clones of others. It
>> would cover not only bugs, but changes in APIs and suchlike which
>> means the risk of conflict would be small. So for example if a
>> template device driver or API were provided in hg, updates to the
>> template would be propogated automatically to copies making major API
>> changes a lot easier.
>>     
>
> It seems you've missed the point of "hg copy". (Caveat: I didn't use it,
> it's just how I understand it from documentation.) It's not a way to
> permanently automatically propagate changes made to a file to a (set of)
> other file(s). A change will propagate to the copy only on the first
> merge (never on commit) and then propagation will stop.
>   
[...]
Sorry, yes.  I did mean purely in the context of existing development
before a merge, not permanent propagation.  Sorry for the confusion - I
did not intend to mislead.

As this is all in the context of shared local development, you would
only expect changes in the merge and subsequent push back to the
official eCos repo when the contribution is to be made.  This is what
makes it a useful feature IMHO.

But similarly, I can see a point to octopus merges...

>   
>> FAOD, I am trying to track down a genuine set of useful technical
>> differences between git and hg to help the maintainers make their mind
>> up.  For example, git octopus merges have no current equivalent in hg,
>> though somehow I don't see that git super-power user feature will have
>> much, if any, use in eCos.  However, changeset tracking copies of files
>> IMHO is very useful.
>>     
>
> [Un]fortunately, there is no such thing in Mercurial either, see above.
> "hg copy" seems to be useless for this purpose and is at the same
> category of usefulness as git octopus merges.
I disagree.  During development stages, lets take for example NAND since
it is currently topical, it is our policy to use a template for device
drivers and develop at least one driver for a real platform before we
contribute.  So if any bugs were found in the template or changes to the
API were made, the merge would propogate these changes.  It is not
unheard of in our company for someone to make a change or fix in one
place and forget to tell everyone else ;-)


>  [BTW, do you really mean
> you'd like a commit to be able to change a lot of files when you did
> modification to only one? I seriously doubt it's a good idea.]
>   
I did, but only upto the first merge/commit which IMHO is a good place
to cut off further propagation.  After then I agree it is a bad idea.


> On the other hand, for example, "git blame" has an ability to
> automatically detect and follow copied and moved pieces of code without
> any additional steps taken at commit time. You see, power becomes
> visible in deep details. Does Hg have 'blame'? Yes, it does, but that
> doesn't mean it is as powerful as git's. The same is with named branches
> inside repository. Does Hg have them? Yes, it does, but it is rather
> recent addition, while git was born with this, so I suspect gits'
> support for such branches is superior.
I believe hg has always had named branches - I can google hg named
branches references back to 2006 where it is described as a feature, not
an addition.  Given both git and hg were started in 2005, that is hardly
recent.  For example, here is a article in Feb 2007 talking about
branches: http://blog.medallia.com/2007/02/a_guided_tour_of_mercurial.html


>  Such things makes comparisons of
> the tools even more difficult (for example, it could be the case that Hg
> already has copy/move detection I'm not aware of).
>   
I agree.  No sooner than I mention that you can refer locally to a
changesets by an alias (shorter id) instead of the SHA3 key, while git
AFAIK requires the full key, will somone in git-land go ahead and
implement it.  Or maybe they already have? ;-)

FWIW I found this alias very very useful when navigating our own
conversion from CVS to hg through branmches and CVS rebasing, especially
since I am mildly dyslexic, a poor typist and had two cats and a 5
year-old floating around while I was doing the work.  Incremental local
aliases good!!

> Another thing is tagging. Both have tagging, but it's implemented very
> differently. Me personally likes git's idea of tags more.
>
> Yet another thing is copy/rename tracking/detection and merges
> propagation over them. Its deep subtle details that actually make
> difference in this field.
>   
Will they really?  There is so much feeding off ideas between the two
camps I don't think so.  Nothing stands out as a definitive
deal-breaking feature-grabbing feature. IMHO we are really just talking
about two variants of the same class of car.

Of course this does make it a continual moving target when both are
playing catchup against each other which makes it harder to evaluate
between the two. The deciding factor IMHO for eCos will probably come
down to being a non-technical one, like maintenance cost or colour or
upholstery finish or drive ability or user manual.


>> The other difference I have found is that git repos tend to grow in size
>> and become inefficient if left unattended for long periods and require
>> packing, while hg repos dont.
>>     
>
> Well, git manages Linux source trees just fine, you know. eCos is very
> small compared to that. But even if git repos tend to grow indeed, a
> cron job on public repositories should do the trick, I think.
>   
:-)  Both git and hg were initially written to manage Linux source
trees. Linus should have thought of that when he started ;-P

IMHO the main reason why git is so popular is that it is pretty much
mandated if you want to develop and contribute to Linux. With so many
linux developers using it, when they come to use eCos it is not
surprising that they also insist on using git. It also should then come
as no surprise that so far I have found the interoperability between hg
and git a lot easier than git to hg since, as above, hg too was
initially written to manage Linux source trees and all the patches that
float around.

But in our experience, 80% of eCos developers are Windows based, and
roughly the % same prefer GUI development than CLI-based. This differs
largely from what the core eCos maintainers use as well as most of the
eCos experts on this list, so I suspect there may be a large leaning
towards git anyway.  I would hope not though because I still believe hg
is simpler, easier to use, is better documented and _currently_ has
better GUI and Windows support than git.  But that is just my subjective
experience having used both fairly deeply (no octopus merges, but wry
did one behind me not long ago). I also admit to having used hg more
than git, even though I started with git, but this was simply because I
could figure out how to do something on hg easier than how to do it on
git.  Otherwise I may be singing git's praises now as well since on a
fundamental level they are so similar.

Incidentally, I managed to get hg 1.3.1 running on RH7.2 (dont ask
why!!!) with little effort but have not yet succeeded getting all of git
built and think I may just give up for now.  So if anyone has already
done it, please let me know if you are willing to share.

-- Alex Schuilenburg

Managing Director/CEO                                eCosCentric Limited
www.ecoscentric.com


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

[ECOS] Re: Tracking eCos as a hg/git submodule

[Sergei Organov]

Alex Schuilenburg <alexs@ecoscentric.com> writes:

[...]

> I agree.  No sooner than I mention that you can refer locally to a
> changesets by an alias (shorter id) instead of the SHA3 key, while git
> AFAIK requires the full key, will somone in git-land go ahead and
> implement it.  Or maybe they already have? ;-)

No, they believe git doesn't need them and I tend to agree. Not only you
can use just first few digits of SHA to reference a revision in git, but
also git has nice syntax to refer to revisions back in history
relatively, i.e., HEAD~3 means 3 revisions back from HEAD (for those who
doesn't know git, HEAD is the symbolic name for the commit your current
working tree is based on).

> FWIW I found this alias very very useful when navigating our own
> conversion from CVS to hg through branmches and CVS rebasing, especially
> since I am mildly dyslexic, a poor typist and had two cats and a 5
> year-old floating around while I was doing the work.  Incremental local
> aliases good!!
>
>> Another thing is tagging. Both have tagging, but it's implemented very
>> differently. Me personally likes git's idea of tags more.
>>
>> Yet another thing is copy/rename tracking/detection and merges
>> propagation over them. Its deep subtle details that actually make
>> difference in this field.
>>   
> Will they really?

I believe yes.

> There is so much feeding off ideas between the two camps I don't think
> so. Nothing stands out as a definitive deal-breaking feature-grabbing
> feature. IMHO we are really just talking about two variants of the
> same class of car.

Hg insists on recording copies and renames at commit time, while git
detects copies and renames at the time of execution of particular
operation that needs this information. And yes, both have their pros and
cons.

From the user point of view it means that in hg you must remember to use
"hg rename" and "hg copy" when appropriate (and blame yourself should
you forget to do it), while in git you are free from this duty (now tell
me which one is easier to use ;))

From behavior point of view, the results of merge will be different in
corner cases.

-- Sergei.


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

Re: [ECOS] Re: Tracking eCos as a hg/git submodule

[Alex Schuilenburg]

Sergei Organov wrote:
> Alex Schuilenburg <alexs@ecoscentric.com> writes:
>
> [...]
>
>   
>> I agree.  No sooner than I mention that you can refer locally to a
>> changesets by an alias (shorter id) instead of the SHA3 key, while git
>> AFAIK requires the full key, will somone in git-land go ahead and
>> implement it.  Or maybe they already have? ;-)
>>     
>
> No, they believe git doesn't need them and I tend to agree. Not only you
> can use just first few digits of SHA to reference a revision in git, but
> also git has nice syntax to refer to revisions back in history
> relatively, i.e., HEAD~3 means 3 revisions back from HEAD (for those who
> doesn't know git, HEAD is the symbolic name for the commit your current
> working tree is based on).
>   
hg equivalent to HEAD is tip, and I just discovered that hg also accepts
the first few digits of SHA.  Go figure...  Learn something new everyday...
>
>> There is so much feeding off ideas between the two camps I don't think
>> so. Nothing stands out as a definitive deal-breaking feature-grabbing
>> feature. IMHO we are really just talking about two variants of the
>> same class of car.
>>     
>
> Hg insists on recording copies and renames at commit time, while git
> detects copies and renames at the time of execution of particular
> operation that needs this information. And yes, both have their pros and
> cons.
>
> From the user point of view it means that in hg you must remember to use
> "hg rename" and "hg copy" when appropriate (and blame yourself should
> you forget to do it), while in git you are free from this duty (now tell
> me which one is easier to use ;))
>   
Depends whether you trust gits ability to detect renames and copies.  I
also prefer knowing what my DRCS is going to do by telling it, rather
than it guessing and doing it, so I am not a fan of the "hg
guessrenames" extension written to match git's behaviour you describe
above, so I choose not to install/enable it.  So I say hg is easier,
because I can choose whether I want this this behaviour ;-P  Oh, wait,
git...

I am going to stop now with the feature bashing because I dont think we
are getting anywhere, other than proving exactly how similar git's and
hg's features are.  I have a bugzilla upgrade to get on with.

-- Alex Schuilenburg

Managing Director/CEO                                eCosCentric Limited
www.ecoscentric.com


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss