public inbox for ecos-discuss@sourceware.org
 help / color / mirror / Atom feed
* [ECOS] Why does sending multicast UDP require a gateway?
@ 2011-03-30 14:03 Grant Edwards
  2011-03-30 14:14 ` [ECOS] " Grant Edwards
  2011-03-30 14:27 ` [ECOS] " Andrew Lunn
  0 siblings, 2 replies; 10+ messages in thread
From: Grant Edwards @ 2011-03-30 14:03 UTC (permalink / raw)
  To: ecos-discuss

Somebody I work with has spent the last three days trying to send UDP
multicast packets using eCos and the FreeBSD network stack.

After quite a bit of trial-and-error we've discovered that you can't
send UDP multicast packets without configuring a valid gateway
address.

That seems broken to me.

What are you supposed to do if you're on a strictly local network that
doesn't _have_ a gateway?  [That is the case for many of our products.]

UDP multicast (by definition) won't use the gateway, so why require
that one is configured?

-- 
Grant Edwards               grant.b.edwards        Yow! It's some people
                                  at               inside the wall!  This is
                              gmail.com            better than mopping!


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 14:03 [ECOS] Why does sending multicast UDP require a gateway? Grant Edwards
@ 2011-03-30 14:14 ` Grant Edwards
  2011-03-30 14:24   ` Gary Thomas
  2011-03-30 14:27 ` [ECOS] " Andrew Lunn
  1 sibling, 1 reply; 10+ messages in thread
From: Grant Edwards @ 2011-03-30 14:14 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-29, Grant Edwards <grant.b.edwards@gmail.com> wrote:

> Somebody I work with has spent the last three days trying to send UDP
> multicast packets using eCos and the FreeBSD network stack.

AFAICT, the FreeBSD stack will refuse to send a UDP multicast packet
unless it can find a valid route to the multicast destination IP
address with that IP _treated_as_a_unicast_address_.

The broken code is in sys/netinet/ip_output.c, and the error is
asserted at line 269:

   123	ip_output(m0, opt, ro, flags, imo)
[...]
   239	        /*
   240	         * If routing to interface only,
   241	         * short circuit routing lookup.
   242	         */
   243	#define ifatoia(ifa)    ((struct in_ifaddr *)(ifa))
   244	#define sintosa(sin)    ((struct sockaddr *)(sin))
   245	        if (flags & IP_ROUTETOIF) {
   246	                if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == 0 &&
   247	                    (ia = ifatoia(ifa_ifwithnet(sintosa(dst)))) == 0) {
   248	                        ipstat.ips_noroute++;
   249	                        error = ENETUNREACH;
   250	                        goto bad;
   251	                }
   252	                ifp = ia->ia_ifp;
   253	                ip->ip_ttl = 1;
   254	                isbroadcast = in_broadcast(dst->sin_addr, ifp);
   255	        } else {
   256	                /*
   257	                 * If this is the case, we probably don't want to allocate
   258	                 * a protocol-cloned route since we didn't get one from the
   259	                 * ULP.  This lets TCP do its thing, while not burdening
   260	                 * forwarding or ICMP with the overhead of cloning a route.
   261	                 * Of course, we still want to do any cloning requested by
   262	                 * the link layer, as this is probably required in all cases
   263	                 * for correct operation (as it is for ARP).
   264	                 */
   265	                if (ro->ro_rt == 0)
   266	                        rtalloc_ign(ro, RTF_PRCLONING);
   267	                if (ro->ro_rt == 0) {
   268	                        ipstat.ips_noroute++;
   269	                        error = EHOSTUNREACH;
   270	                        goto bad;
   271	                }
   272	                ia = ifatoia(ro->ro_rt->rt_ifa);
   273	                ifp = ro->ro_rt->rt_ifp;
   274	                ro->ro_rt->rt_use++;
   275	                if (ro->ro_rt->rt_flags & RTF_GATEWAY)
   276	                        dst = (struct sockaddr_in *)ro->ro_rt->rt_gateway;
   277	                if (ro->ro_rt->rt_flags & RTF_HOST)
   278	                        isbroadcast = (ro->ro_rt->rt_flags & RTF_BROADCAST);
   279	                else
   280	                        isbroadcast = in_broadcast(dst->sin_addr, ifp);
   281	        }
   282	        if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr))) {
   283	                struct in_multi *inm;
   284	
   285	                m->m_flags |= M_MCAST;
   286	                /*
   287	                 * IP destination address is multicast.  Make sure "dst"
   288	                 * still points to the address in "ro".  (It may have been
   289	                 * changed to point to a gateway address, above.)
   290	                 */

As you can see, the output code rejects the packet for lack of a valid
route to the destination IP address before it checks to see if it's a
multicast IP address.

That doesn't look right to me...

If it's a multicast destination, shouldn't that short-circuit the
route-checking stuff?

The docs I can find are pretty clear: if it's a multicast IP
destination you ship it out to the corresponding multicast MAC
address. Period. No messing about with routes and gateway addresses.

-- 
Grant Edwards               grant.b.edwards        Yow! I selected E5 ... but
                                  at               I didn't hear "Sam the Sham
                              gmail.com            and the Pharoahs"!


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 14:14 ` [ECOS] " Grant Edwards
@ 2011-03-30 14:24   ` Gary Thomas
  2011-03-30 14:33     ` Grant Edwards
  0 siblings, 1 reply; 10+ messages in thread
From: Gary Thomas @ 2011-03-30 14:24 UTC (permalink / raw)
  To: Grant Edwards; +Cc: ecos-discuss

On 03/29/2011 05:09 PM, Grant Edwards wrote:
> On 2011-03-29, Grant Edwards<grant.b.edwards@gmail.com>  wrote:
>
>> Somebody I work with has spent the last three days trying to send UDP
>> multicast packets using eCos and the FreeBSD network stack.
>
> AFAICT, the FreeBSD stack will refuse to send a UDP multicast packet
> unless it can find a valid route to the multicast destination IP
> address with that IP _treated_as_a_unicast_address_.
>
> The broken code is in sys/netinet/ip_output.c, and the error is
> asserted at line 269:
>
>     123	ip_output(m0, opt, ro, flags, imo)
> [...]
>     239	        /*
>     240	         * If routing to interface only,
>     241	         * short circuit routing lookup.
>     242	         */
>     243	#define ifatoia(ifa)    ((struct in_ifaddr *)(ifa))
>     244	#define sintosa(sin)    ((struct sockaddr *)(sin))
>     245	        if (flags&  IP_ROUTETOIF) {
>     246	                if ((ia = ifatoia(ifa_ifwithdstaddr(sintosa(dst)))) == 0&&
>     247	                    (ia = ifatoia(ifa_ifwithnet(sintosa(dst)))) == 0) {
>     248	                        ipstat.ips_noroute++;
>     249	                        error = ENETUNREACH;
>     250	                        goto bad;
>     251	                }
>     252	                ifp = ia->ia_ifp;
>     253	                ip->ip_ttl = 1;
>     254	                isbroadcast = in_broadcast(dst->sin_addr, ifp);
>     255	        } else {
>     256	                /*
>     257	                 * If this is the case, we probably don't want to allocate
>     258	                 * a protocol-cloned route since we didn't get one from the
>     259	                 * ULP.  This lets TCP do its thing, while not burdening
>     260	                 * forwarding or ICMP with the overhead of cloning a route.
>     261	                 * Of course, we still want to do any cloning requested by
>     262	                 * the link layer, as this is probably required in all cases
>     263	                 * for correct operation (as it is for ARP).
>     264	                 */
>     265	                if (ro->ro_rt == 0)
>     266	                        rtalloc_ign(ro, RTF_PRCLONING);
>     267	                if (ro->ro_rt == 0) {
>     268	                        ipstat.ips_noroute++;
>     269	                        error = EHOSTUNREACH;
>     270	                        goto bad;
>     271	                }
>     272	                ia = ifatoia(ro->ro_rt->rt_ifa);
>     273	                ifp = ro->ro_rt->rt_ifp;
>     274	                ro->ro_rt->rt_use++;
>     275	                if (ro->ro_rt->rt_flags&  RTF_GATEWAY)
>     276	                        dst = (struct sockaddr_in *)ro->ro_rt->rt_gateway;
>     277	                if (ro->ro_rt->rt_flags&  RTF_HOST)
>     278	                        isbroadcast = (ro->ro_rt->rt_flags&  RTF_BROADCAST);
>     279	                else
>     280	                        isbroadcast = in_broadcast(dst->sin_addr, ifp);
>     281	        }
>     282	        if (IN_MULTICAST(ntohl(ip->ip_dst.s_addr))) {
>     283	                struct in_multi *inm;
>     284	
>     285	                m->m_flags |= M_MCAST;
>     286	                /*
>     287	                 * IP destination address is multicast.  Make sure "dst"
>     288	                 * still points to the address in "ro".  (It may have been
>     289	                 * changed to point to a gateway address, above.)
>     290	                 */
>
> As you can see, the output code rejects the packet for lack of a valid
> route to the destination IP address before it checks to see if it's a
> multicast IP address.
>
> That doesn't look right to me...
>
> If it's a multicast destination, shouldn't that short-circuit the
> route-checking stuff?
>
> The docs I can find are pretty clear: if it's a multicast IP
> destination you ship it out to the corresponding multicast MAC
> address. Period. No messing about with routes and gateway addresses.

I agree that this doesn't seem right.  You might try looking at the
current BSD networking code to see if this has changed since I merged
the stack last - some *ten* years ago :-)

If you find a fix/work-around, send a patch!

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [ECOS] Why does sending multicast UDP require a gateway?
  2011-03-30 14:03 [ECOS] Why does sending multicast UDP require a gateway? Grant Edwards
  2011-03-30 14:14 ` [ECOS] " Grant Edwards
@ 2011-03-30 14:27 ` Andrew Lunn
  2011-03-30 14:34   ` [ECOS] " Grant Edwards
  1 sibling, 1 reply; 10+ messages in thread
From: Andrew Lunn @ 2011-03-30 14:27 UTC (permalink / raw)
  To: Grant Edwards; +Cc: ecos-discuss

On Tue, Mar 29, 2011 at 10:33:09PM +0000, Grant Edwards wrote:
> Somebody I work with has spent the last three days trying to send UDP
> multicast packets using eCos and the FreeBSD network stack.
> 
> After quite a bit of trial-and-error we've discovered that you can't
> send UDP multicast packets without configuring a valid gateway
> address.
> 
> That seems broken to me.
> 
> What are you supposed to do if you're on a strictly local network that
> doesn't _have_ a gateway?  [That is the case for many of our products.]

Hi Grant

In the Linux world you have to do something like:

ip route add 224.0.0.0/4 dev eth0

before multicast works. The kernel has no idea which interface to send
the multicast packet out, unless you add a route like this. Maybe in
the BSD world, instead of defining the interface directly, you give
the local IP address on the interface?

Maybe this helps:

http://www.freebsd.org/doc/handbook/network-routing.html

In the example routing table is:
224              link#1             UC          0        0

	Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 14:24   ` Gary Thomas
@ 2011-03-30 14:33     ` Grant Edwards
  0 siblings, 0 replies; 10+ messages in thread
From: Grant Edwards @ 2011-03-30 14:33 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-29, Gary Thomas <gary@mlbassoc.com> wrote:
>
>> As you can see, the output code rejects the packet for lack of a valid
>> route to the destination IP address before it checks to see if it's a
>> multicast IP address.
>>
>> That doesn't look right to me...
>>
>> If it's a multicast destination, shouldn't that short-circuit the
>> route-checking stuff?
>>
>> The docs I can find are pretty clear: if it's a multicast IP
>> destination you ship it out to the corresponding multicast MAC
>> address. Period. No messing about with routes and gateway addresses.
>
> I agree that this doesn't seem right.  You might try looking at the
> current BSD networking code

That's on the list of things to do...

> to see if this has changed since I merged the stack last - some *ten*
> years ago :-)

Ten years?  Cripes. Maybe it's time I stopped referring to it as the
"new stack".

> If you find a fix/work-around, send a patch!

-- 
Grant Edwards               grant.b.edwards        Yow! TAILFINS!! ... click
                                  at               ...
                              gmail.com            


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 14:27 ` [ECOS] " Andrew Lunn
@ 2011-03-30 14:34   ` Grant Edwards
  2011-03-30 15:00     ` Andrew Lunn
  0 siblings, 1 reply; 10+ messages in thread
From: Grant Edwards @ 2011-03-30 14:34 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-30, Andrew Lunn <andrew@lunn.ch> wrote:
> On Tue, Mar 29, 2011 at 10:33:09PM +0000, Grant Edwards wrote:
>> Somebody I work with has spent the last three days trying to send UDP
>> multicast packets using eCos and the FreeBSD network stack.
>> 
>> After quite a bit of trial-and-error we've discovered that you can't
>> send UDP multicast packets without configuring a valid gateway
>> address.
>> 
>> That seems broken to me.
>> 
>> What are you supposed to do if you're on a strictly local network that
>> doesn't _have_ a gateway?  [That is the case for many of our products.]
>
> Hi Grant
>
> In the Linux world you have to do something like:
>
> ip route add 224.0.0.0/4 dev eth0
>
> before multicast works. The kernel has no idea which interface to
> send the multicast packet out, unless you add a route like this.

It's being sent via a socket that's bound to a specific interface, so
I wouldn't think that would be a problem.

> Maybe in the BSD world, instead of defining the interface directly,
> you give the local IP address on the interface?
>
> Maybe this helps:
>
> http://www.freebsd.org/doc/handbook/network-routing.html
>
> In the example routing table is:
> 224              link#1             UC          0        0

I can see that if you're sending from an unbound socket, you might
want a way to specify which interface to use for sending multicast
packets.  But without such a route, the only thing that makes sense to
me is "send it out all of them".

Perhaps one work-around would be to always configure a "multicast
route" for 224.0.0.0/28.

-- 
Grant Edwards               grant.b.edwards        Yow! I need to discuss
                                  at               BUY-BACK PROVISIONS
                              gmail.com            with at least six studio
                                                   SLEAZEBALLS!!


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 14:34   ` [ECOS] " Grant Edwards
@ 2011-03-30 15:00     ` Andrew Lunn
  2011-03-30 21:04       ` Grant Edwards
  0 siblings, 1 reply; 10+ messages in thread
From: Andrew Lunn @ 2011-03-30 15:00 UTC (permalink / raw)
  To: Grant Edwards; +Cc: ecos-discuss

> It's being sent via a socket that's bound to a specific interface, so
> I wouldn't think that would be a problem.

Ah, O.K. 

Maybe try it anyway?

> I can see that if you're sending from an unbound socket, you might
> want a way to specify which interface to use for sending multicast
> packets.  But without such a route, the only thing that makes sense to
> me is "send it out all of them".

And on a multi homed machine that might result in the group receiving
two copies of the packet when both multicast routers pick it up and
send to the RP.

     Andrew

-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 15:00     ` Andrew Lunn
@ 2011-03-30 21:04       ` Grant Edwards
  2011-03-31 12:30         ` Grant Edwards
  0 siblings, 1 reply; 10+ messages in thread
From: Grant Edwards @ 2011-03-30 21:04 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-30, Andrew Lunn <andrew@lunn.ch> wrote:
>> It's being sent via a socket that's bound to a specific interface, so
>> I wouldn't think that would be a problem.
>
> Ah, O.K. 
>
> Maybe try it anyway?
>
>> I can see that if you're sending from an unbound socket, you might
>> want a way to specify which interface to use for sending multicast
>> packets.  But without such a route, the only thing that makes sense to
>> me is "send it out all of them".
>
> And on a multi homed machine that might result in the group receiving
> two copies of the packet when both multicast routers pick it up and
> send to the RP.

True.

According to RFC1112, you only send a multicast packet on one
interface. If the application doesn't specify an interface, a default
interface is used.  IOW, discarding the packet is not correct:

   Second, for hosts that may be attached to more than one network,
   the service interface should provide a way for the upper-layer
   protocol to identify which network interface is be used for the
   multicast transmission.  Only one interface is used for the initial
   transmission; multicast routers are responsible for forwarding to
   any other networks, if necessary.  If the upper-layer protocol
   chooses not to identify an outgoing interface, a default interface
   should be used, preferably under the control of system management.

It turns out this was fixed in the FreeBSD sources in 2001:
   
  http://svn.freebsd.org/viewvc/base?view=revision&revision=79830   
  http://svn.freebsd.org/viewvc/base?view=revision&revision=79836

I'll work up a patch...
  
-- 
Grant Edwards               grant.b.edwards        Yow! I'm having an
                                  at               emotional outburst!!
                              gmail.com            


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-30 21:04       ` Grant Edwards
@ 2011-03-31 12:30         ` Grant Edwards
  2011-03-31 18:24           ` Grant Edwards
  0 siblings, 1 reply; 10+ messages in thread
From: Grant Edwards @ 2011-03-31 12:30 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-30, Grant Edwards <grant.b.edwards@gmail.com> wrote:
> On 2011-03-30, Andrew Lunn <andrew@lunn.ch> wrote:
>>> It's being sent via a socket that's bound to a specific interface, so
>>> I wouldn't think that would be a problem.
>>
>> Ah, O.K. 
>>
>> Maybe try it anyway?
>>
>>> I can see that if you're sending from an unbound socket, you might
>>> want a way to specify which interface to use for sending multicast
>>> packets.  But without such a route, the only thing that makes sense to
>>> me is "send it out all of them".
>>
>> And on a multi homed machine that might result in the group receiving
>> two copies of the packet when both multicast routers pick it up and
>> send to the RP.
>
> True.
>
> According to RFC1112, you only send a multicast packet on one
> interface. If the application doesn't specify an interface, a default
> interface is used.  IOW, discarding the packet is not correct:
>
>    Second, for hosts that may be attached to more than one network,
>    the service interface should provide a way for the upper-layer
>    protocol to identify which network interface is be used for the
>    multicast transmission.  Only one interface is used for the initial
>    transmission; multicast routers are responsible for forwarding to
>    any other networks, if necessary.  If the upper-layer protocol
>    chooses not to identify an outgoing interface, a default interface
>    should be used, preferably under the control of system management.
>
> It turns out this was fixed in the FreeBSD sources in 2001:
>    
>   http://svn.freebsd.org/viewvc/base?view=revision&revision=79830   
>   http://svn.freebsd.org/viewvc/base?view=revision&revision=79836
>
> I'll work up a patch...

It turns out that doesn't fix the problem.  That only handles the case
where you've explicitly asked the multicast packets to go out a
specific interface.  Binding a socket to an interface and sending
multicast packets still doesn't work unless there's a route to the
"destination".

One would think that binding the socket to a particular interface
would cause sent packets to go out that interface, but I guess not...

-- 
Grant Edwards               grant.b.edwards        Yow! Here I am in 53
                                  at               B.C. and all I want is a
                              gmail.com            dill pickle!!


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

* [ECOS] Re: Why does sending multicast UDP require a gateway?
  2011-03-31 12:30         ` Grant Edwards
@ 2011-03-31 18:24           ` Grant Edwards
  0 siblings, 0 replies; 10+ messages in thread
From: Grant Edwards @ 2011-03-31 18:24 UTC (permalink / raw)
  To: ecos-discuss

On 2011-03-30, Grant Edwards <grant.b.edwards@gmail.com> wrote:
>>
>> It turns out this was fixed in the FreeBSD sources in 2001:
>>    
>>   http://svn.freebsd.org/viewvc/base?view=revision&revision=79830   
>>   http://svn.freebsd.org/viewvc/base?view=revision&revision=79836
>>
>> I'll work up a patch...
>
> It turns out that doesn't fix the problem.  That only handles the
> case where you've explicitly asked the multicast packets to go out a
> specific interface.  Binding a socket to an interface and sending
> multicast packets still doesn't work unless there's a route to the
> "destination".
>
> One would think that binding the socket to a particular interface
> would cause sent packets to go out that interface, but I guess not...

After more Googling, it appears that unless you want a "default
interface" chosen (via the normal routing table, I guess), you are
indeed expected to explicitly set the sending interface independently
of what interface the sending socket is bound to:

   bind(sock, (struct sockaddr *)&local, sizeof local);
         
   setsockopt(sock, IPPROTO_IP, IP_MULTICAST_IF, &local.sin_addr, sizeof local.sin_addr);

It appears that if you don't bind the socket, ip_output will fill in
the source IP field with the IP address of the sending interface, but
if you want to specify the source port, you still have to bind() the
socket.  [Whether you can bind the socket to one interface but send
packets via a different one, I don't know.]

I'll have a patch tomorrow.
   
-- 
Grant Edwards               grant.b.edwards        Yow! An air of FRENCH FRIES
                                  at               permeates my nostrils!!
                              gmail.com            


-- 
Before posting, please read the FAQ: http://ecos.sourceware.org/fom/ecos
and search the list archive: http://ecos.sourceware.org/ml/ecos-discuss

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2011-03-30 22:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-03-30 14:03 [ECOS] Why does sending multicast UDP require a gateway? Grant Edwards
2011-03-30 14:14 ` [ECOS] " Grant Edwards
2011-03-30 14:24   ` Gary Thomas
2011-03-30 14:33     ` Grant Edwards
2011-03-30 14:27 ` [ECOS] " Andrew Lunn
2011-03-30 14:34   ` [ECOS] " Grant Edwards
2011-03-30 15:00     ` Andrew Lunn
2011-03-30 21:04       ` Grant Edwards
2011-03-31 12:30         ` Grant Edwards
2011-03-31 18:24           ` Grant Edwards

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).