Re: RFC: A new MIPS64 ABI

Re: RFC: A new MIPS64 ABI

[Richard Sandiford]

David Daney <ddaney@caviumnetworks.com> writes:
> Background:
>
> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space.  This is due the way MIPS32 memory space is
> segmented.  Only the range from 0..2^31-1 is available.  Pointer
> values are always sign extended.
>
> Because there are not already enough MIPS ABIs, I present the ...
>
> Proposal: A new ABI to support 4GB of address space with 32-bit
> pointers.

FWIW, I'd be happy to see this go into GCC.

Richard

Re: RFC: A new MIPS64 ABI

[Ralf Baechle]

On Mon, Feb 21, 2011 at 07:45:41PM +0000, Richard Sandiford wrote:

> David Daney <ddaney@caviumnetworks.com> writes:
> > Background:
> >
> > Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> > user virtual memory space.  This is due the way MIPS32 memory space is
> > segmented.  Only the range from 0..2^31-1 is available.  Pointer
> > values are always sign extended.
> >
> > Because there are not already enough MIPS ABIs, I present the ...
> >
> > Proposal: A new ABI to support 4GB of address space with 32-bit
> > pointers.
> 
> FWIW, I'd be happy to see this go into GCC.

So am I for the kernel primarily because it's not really a new ABI but
an enhancement of the existing N32 ABI.

  Ralf

Re: RFC: A new MIPS64 ABI

[David Daney]

On 05/09/2011 07:28 AM, Ralf Baechle wrote:
> On Mon, Feb 21, 2011 at 07:45:41PM +0000, Richard Sandiford wrote:
>
>> David Daney<ddaney@caviumnetworks.com>  writes:
>>> Background:
>>>
>>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>>> user virtual memory space.  This is due the way MIPS32 memory space is
>>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>>> values are always sign extended.
>>>
>>> Because there are not already enough MIPS ABIs, I present the ...
>>>
>>> Proposal: A new ABI to support 4GB of address space with 32-bit
>>> pointers.
>>
>> FWIW, I'd be happy to see this go into GCC.
>
> So am I for the kernel primarily because it's not really a new ABI but
> an enhancement of the existing N32 ABI.
>

The patches are resting peacefully on my laptop.  Now with endorcements 
like these, I may be forced to actually finish them...

David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 05/06/2011 01:29 AM, Alexandre Oliva wrote:
> On Feb 15, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:
>
>> On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
>>> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>   wrote:
>
>>> So, sorry if this is a dumb question, but wouldn't it be much easier to
>>> keep on using sign-extended addresses, and just make sure the kernel
>>> never allocates a virtual memory range that crosses a sign-bit change,
>
>> No, it is not possible.  The MIPS (and MIPS64) hardware architecture
>> does not allow userspace access to addresses with the high bit (two
>> bits for mips64) set.
>
> Interesting.  I guess this makes it far easier to transition to the u32
> ABI: n32 addresses all have the 32-bit MSB bit clear, so n32 binaries
> can be used within u32 environments, as long as the environment refrains
> from using addresses that have the MSB bit set.
>

Correct.

> So we could switch lib32 to u32, have a machine-specific bit set for u32
> binaries, and if the kernel starts an executable or interpreter that has
> that bit clear, it will refrain from allocating any n32-invalid address
> for that process.  Furthermore, libc, upon loading a library, should be
> able to notify the kernel when an n32 library is to be loaded, to which
> the kernel would respond either with failure (if that process already
> uses u32-valid but n32-invalid addresses) or success (switching to n32
> mode if not in it already).
>
> Am I missing any other issues?

No, this is pretty much what Ralf and I came up with on IRC.

We tag u32 objects (in a similar manner to how non-executable stack is 
done).  The linker will propagate the u32 tag as it links things together.

u32 shared libraries are compatible with legacy n32 binaries as long as 
the OS doesn't map any memory where the address has bit 31 set.

When the OS loads an n32 executable it would check the u32 tag (both of 
the executable and ld.so) and adjust its memory allocation strategy.

The OS will continue to map the VDSO at the 2GB point.  This will cause 
the maximum size of any object to be compatible with the 32-bit n32 
ptrdiff_t.

I think once the OS puts a process into u32 mode, there is no going 
back.  We would just have ld.so refuse to load any shared objects that 
were not compatible with the current mode.

We would continue to place libraries in /lib32, /usr/lib32, 
/usr/local/lib32, etc.


David Daney

Re: RFC: A new MIPS64 ABI

[Alexandre Oliva]

On Feb 15, 2011, David Daney <ddaney@caviumnetworks.com> wrote:

> On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
>> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:

>> So, sorry if this is a dumb question, but wouldn't it be much easier to
>> keep on using sign-extended addresses, and just make sure the kernel
>> never allocates a virtual memory range that crosses a sign-bit change,

> No, it is not possible.  The MIPS (and MIPS64) hardware architecture
> does not allow userspace access to addresses with the high bit (two
> bits for mips64) set.

Interesting.  I guess this makes it far easier to transition to the u32
ABI: n32 addresses all have the 32-bit MSB bit clear, so n32 binaries
can be used within u32 environments, as long as the environment refrains
from using addresses that have the MSB bit set.

So we could switch lib32 to u32, have a machine-specific bit set for u32
binaries, and if the kernel starts an executable or interpreter that has
that bit clear, it will refrain from allocating any n32-invalid address
for that process.  Furthermore, libc, upon loading a library, should be
able to notify the kernel when an n32 library is to be loaded, to which
the kernel would respond either with failure (if that process already
uses u32-valid but n32-invalid addresses) or success (switching to n32
mode if not in it already).

Am I missing any other issues?

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 12:29 PM, David Daney wrote:
> Background:
>
> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space. This is due the way MIPS32 memory space is
> segmented. Only the range from 0..2^31-1 is available. Pointer
> values are always sign extended.
>
> Because there are not already enough MIPS ABIs, I present the ...
>
> Proposal: A new ABI to support 4GB of address space with 32-bit
> pointers.
>
> The proposed new ABI would only be available on MIPS64 platforms. It
> would be identical to the current MIPS n32 ABI *except* that pointers
> would be zero-extended rather than sign-extended when resident in
> registers. In the remainder of this document I will call it
> 'n32-big'. As a result, applications would have access to a full 4GB
> of virtual address space. The operating environment would be
> configured such that the entire lower 4GB of the virtual address space
> was available to the program.
>
>
> At a low level here is how it would work:
>
> 1) Load a pointer to a register from memory:
>
> n32:
> LW $reg, offset($reg)
>
> n32-big:
> LWU $reg, offset($reg)
>
> 2) Load an address constant into a register:
>
> n32:
> LUI $reg, high_part
> ORI $reg, low_part

That is not reality.  Really it is:

LUI $reg, R_MIPS_HI16
ADDIU $reg, R_MIPS_LO16


>
> n32-big:
> ORI $reg, high_part
> DSLL $reg, $reg, 16
> ORI $reg, low_part
>

This one would really be:

ORI $reg, R_MIPS_HI16
DSLL $reg, $reg, 16
ADDIU $reg, R_MIPS_LO16


>
> Q: What would have to change to make this work?
>
> o A new ELF header flag to denote the ABI.
>
> o Linker support to use proper library search paths, and linker scrips
> to set the INTERP program header, etc.
>
> o GCC has to emit code for the new ABI.
>
> o Could all existing n32 relocation types be used? I think so.
>
> o Runtime libraries would have to be placed in a new location
> (/lib32big, /usr/lib32big ...)
>
> o The C library's ld.so would have to use a distinct LD_LIBRARY_PATH
> for n32-big code.
>
> o What would the Linux system call interface be? I would propose
> using the existing Linux n32 system call interface. Most system
> calls would just work. Some, that pass pointers in in-memory
> structures, might require kernel modifications (sigaction() for
> example).
>

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/15/2011 09:32 AM, Joseph S. Myers wrote:
> On Mon, 14 Feb 2011, Joe Buck wrote:
>
>> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
>>> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
>>>
>>> This seems like a microscopically small market segment.  I can't see any sense in such an effort.
>>
>> I remember the RHEL hugemem patch being a big deal for lots of their
>> customers, so a process could address the full 4GB instead of only 3GB
>> on a 32-bit machine.  If I recall correctly, upstream didn't want it
>> (get a 64-bit machine!) but lots of paying customers clamored for it.
>>
>> (I personally don't have an opinion on whether it's worth bothering with).
>
> As I've been warning recently in the context of the "operator new[]"
> overflow checks discussion, even if your process is addressing 4GB in such
> circumstances it can't safely use single objects of 2GB or more and it's a
> security problem when malloc/calloc/etc. allow such objects to be created.
> See PR 45779.  (There could well be issues with pointer comparisons as
> well as pointer differences, although there at least it's possible to be
> consistent if you don't allow objects to wrap around both in the middle
> and at the end of the address space.)
>

Thanks Joseph,

My idea for n32-big is that there would never be wraparound issues in 
the middle.  The address space is contiguous from 0 to 4GB.  Typically 
the area around the 4GB limit would be occupied by the stack and perhaps 
a several other regions reserved by the OS (vdso, etc. ).

At the ends there could be wraparound/truncation issues, but this is no 
different than with the ABIs of most 32-bit targets.

I don't know how hard it would be to make ptrdiff_t a signed 64-bit 
type.  That would certainly complicate things somewhat.

David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/15/2011 09:56 AM, Alexandre Oliva wrote:
> On Feb 14, 2011, David Daney<ddaney@caviumnetworks.com>  wrote:
>
>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>> user virtual memory space.  This is due the way MIPS32 memory space is
>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>> values are always sign extended.
>
>> The proposed new ABI would only be available on MIPS64 platforms.  It
>> would be identical to the current MIPS n32 ABI *except* that pointers
>> would be zero-extended rather than sign-extended when resident in
>> registers.
>
> FTR, I don't really know why my Yeeloong is limited to 31-bit addresses,
> and I kind of hoped an n32 userland would improve that WRT o32, without
> wasting memory with longer pointers like n64 would.
>
> So, sorry if this is a dumb question, but wouldn't it be much easier to
> keep on using sign-extended addresses, and just make sure the kernel
> never allocates a virtual memory range that crosses a sign-bit change,
> or whatever other reason there is for addresses to be limited to the
> positive 2GB range in n32?
>

No, it is not possible.  The MIPS (and MIPS64) hardware architecture 
does not allow userspace access to addresses with the high bit (two bits 
for mips64) set.

Your complaint is a good summary of why I am thinking about n32-big.

David Daney

Re: RFC: A new MIPS64 ABI

[Alexandre Oliva]

On Feb 14, 2011, David Daney <ddaney@caviumnetworks.com> wrote:

> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space.  This is due the way MIPS32 memory space is
> segmented.  Only the range from 0..2^31-1 is available.  Pointer
> values are always sign extended.

> The proposed new ABI would only be available on MIPS64 platforms.  It
> would be identical to the current MIPS n32 ABI *except* that pointers
> would be zero-extended rather than sign-extended when resident in
> registers.

FTR, I don't really know why my Yeeloong is limited to 31-bit addresses,
and I kind of hoped an n32 userland would improve that WRT o32, without
wasting memory with longer pointers like n64 would.

So, sorry if this is a dumb question, but wouldn't it be much easier to
keep on using sign-extended addresses, and just make sure the kernel
never allocates a virtual memory range that crosses a sign-bit change,
or whatever other reason there is for addresses to be limited to the
positive 2GB range in n32?

-- 
Alexandre Oliva, freedom fighter    http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/   FSF Latin America board member
Free Software Evangelist      Red Hat Brazil Compiler Engineer

Re: RFC: A new MIPS64 ABI

[Paul Koning]

On Feb 15, 2011, at 12:41 PM, David Daney wrote:

> ...
>> 
>>> The main work would be in the compiler toolchain and runtime libraries.
>> 
>> You'd also need to update gas for la and dla expansion.
>> 
> 
> I am counting gas, ld and libc as part of the 'compiler toolchain'

Don't forget GDB.

	paul

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 07:00 PM, Matt Thomas wrote:
>
> On Feb 14, 2011, at 6:50 PM, David Daney wrote:
>
>> On 02/14/2011 06:33 PM, Matt Thomas wrote:
>>>
>>> On Feb 14, 2011, at 6:22 PM, David Daney wrote:
>>>
>>>> On 02/14/2011 04:15 PM, Matt Thomas wrote:
>>>>>
>>>>> I have to wonder if it's worth the effort.  The primary problem I see
>>>>> is that this new ABI requires a 64bit kernel since faults through the
>>>>> upper 2G will go through the XTLB miss exception vector.
>>>>>
>>>>
>>>> Yes, that is correct.  It is a 64-bit ABI, and like the existing n32 ABI requires a 64-bit kernel.
>>>
>>> N32 doesn't require a LP64 kernel, just a 64-bit register aware kernel.
>>> Your N32-big does require a LP64 kernel.
>>>
>>
>> But using 'official' kernel sources the only way to get a 64-bit register aware kernel is for it to also be LP64.  So effectively, you do in fact need a 64-bit kernel to run n32 userspace code.
>
> Not all the world is Linux. :)  NetBSD supports N32 kernels.
>

Use of LP32 in the kernel is only really appropiate in systems with a 
small amount of memory.  The proposed n32-big would run on such systems, 
but would probably *not* have an advantage over standard n32.



>> My proposed ABI would need trivial kernel changes:
>>
>> o Fix a couple of places where pointers are sign extended instead of zero extended.
>
> I think you'll find there are more of these than you'd expect.

You could be right, but to date in auditing the Linux kernel, 
sigaction() is the only place I have found.


>
>> o Change the stack address and address ranges returned by mmap().
>
> My biggest concern is that many many mips opcodes expect properly
> sign-extended value for registers.  Thusly N32-big will require
> using daddu/dadd/dsub/dsubu for addresses.  So that's yet another
> departure from N32 which can use addu/add/sub/subu.


That's right.  Which is why I said...

>
>> The main work would be in the compiler toolchain and runtime libraries.
>
> You'd also need to update gas for la and dla expansion.
>

I am counting gas, ld and libc as part of the 'compiler toolchain'

David Daney

Re: RFC: A new MIPS64 ABI

[Joseph S. Myers]

On Mon, 14 Feb 2011, Joe Buck wrote:

> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
> > It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
> > 
> > This seems like a microscopically small market segment.  I can't see any sense in such an effort.
> 
> I remember the RHEL hugemem patch being a big deal for lots of their
> customers, so a process could address the full 4GB instead of only 3GB
> on a 32-bit machine.  If I recall correctly, upstream didn't want it
> (get a 64-bit machine!) but lots of paying customers clamored for it.
> 
> (I personally don't have an opinion on whether it's worth bothering with).

As I've been warning recently in the context of the "operator new[]" 
overflow checks discussion, even if your process is addressing 4GB in such 
circumstances it can't safely use single objects of 2GB or more and it's a 
security problem when malloc/calloc/etc. allow such objects to be created.  
See PR 45779.  (There could well be issues with pointer comparisons as 
well as pointer differences, although there at least it's possible to be 
consistent if you don't allow objects to wrap around both in the middle 
and at the end of the address space.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFC: A new MIPS64 ABI

[Matt Thomas]

On Feb 14, 2011, at 6:50 PM, David Daney wrote:

> On 02/14/2011 06:33 PM, Matt Thomas wrote:
>> 
>> On Feb 14, 2011, at 6:22 PM, David Daney wrote:
>> 
>>> On 02/14/2011 04:15 PM, Matt Thomas wrote:
>>>> 
>>>> I have to wonder if it's worth the effort.  The primary problem I see
>>>> is that this new ABI requires a 64bit kernel since faults through the
>>>> upper 2G will go through the XTLB miss exception vector.
>>>> 
>>> 
>>> Yes, that is correct.  It is a 64-bit ABI, and like the existing n32 ABI requires a 64-bit kernel.
>> 
>> N32 doesn't require a LP64 kernel, just a 64-bit register aware kernel.
>> Your N32-big does require a LP64 kernel.
>> 
> 
> But using 'official' kernel sources the only way to get a 64-bit register aware kernel is for it to also be LP64.  So effectively, you do in fact need a 64-bit kernel to run n32 userspace code.

Not all the world is Linux. :)  NetBSD supports N32 kernels.  

> My proposed ABI would need trivial kernel changes:
> 
> o Fix a couple of places where pointers are sign extended instead of zero extended.

I think you'll find there are more of these than you'd expect.

> o Change the stack address and address ranges returned by mmap().

My biggest concern is that many many mips opcodes expect properly 
sign-extended value for registers.  Thusly N32-big will require 
using daddu/dadd/dsub/dsubu for addresses.  So that's yet another
departure from N32 which can use addu/add/sub/subu.

> The main work would be in the compiler toolchain and runtime libraries.

You'd also need to update gas for la and dla expansion.

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 06:33 PM, Matt Thomas wrote:
>
> On Feb 14, 2011, at 6:22 PM, David Daney wrote:
>
>> On 02/14/2011 04:15 PM, Matt Thomas wrote:
>>>
>>> I have to wonder if it's worth the effort.  The primary problem I see
>>> is that this new ABI requires a 64bit kernel since faults through the
>>> upper 2G will go through the XTLB miss exception vector.
>>>
>>
>> Yes, that is correct.  It is a 64-bit ABI, and like the existing n32 ABI requires a 64-bit kernel.
>
> N32 doesn't require a LP64 kernel, just a 64-bit register aware kernel.
> Your N32-big does require a LP64 kernel.
>

But using 'official' kernel sources the only way to get a 64-bit 
register aware kernel is for it to also be LP64.  So effectively, you do 
in fact need a 64-bit kernel to run n32 userspace code.

My proposed ABI would need trivial kernel changes:

o Fix a couple of places where pointers are sign extended instead of 
zero extended.

o Change the stack address and address ranges returned by mmap().


The main work would be in the compiler toolchain and runtime libraries.


David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 06:34 PM, Matt Thomas wrote:
>
> On Feb 14, 2011, at 6:26 PM, David Daney wrote:
>
>> On 02/14/2011 06:14 PM, Joe Buck wrote:
>>> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
>>>> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
>>>>
>>>> This seems like a microscopically small market segment.  I can't see any sense in such an effort.
>>>
>>> I remember the RHEL hugemem patch being a big deal for lots of their
>>> customers, so a process could address the full 4GB instead of only 3GB
>>> on a 32-bit machine.  If I recall correctly, upstream didn't want it
>>> (get a 64-bit machine!) but lots of paying customers clamored for it.
>>>
>>> (I personally don't have an opinion on whether it's worth bothering with).
>>>
>>
>> Also look at the new x86_64 ABI (See all those X32 psABI messages) that the Intel folks are actively working on.  This proposal is very similar to what they are doing.
>
> untrue.  N32 is closer to the X32 ABI since it is limited to 2GB.
>

It would only be 'untrue' if I had said it was *exactly like* the X32 thing.

Really n32 is, as you note, already quite similar to what X32 is trying 
to do.  My proposal is really for a small improvement to n32 to allow 
doubling the size of the virtual address space to 4GB.

David Daney

Re: RFC: A new MIPS64 ABI

[Matt Thomas]

On Feb 14, 2011, at 6:26 PM, David Daney wrote:

> On 02/14/2011 06:14 PM, Joe Buck wrote:
>> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
>>> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
>>> 
>>> This seems like a microscopically small market segment.  I can't see any sense in such an effort.
>> 
>> I remember the RHEL hugemem patch being a big deal for lots of their
>> customers, so a process could address the full 4GB instead of only 3GB
>> on a 32-bit machine.  If I recall correctly, upstream didn't want it
>> (get a 64-bit machine!) but lots of paying customers clamored for it.
>> 
>> (I personally don't have an opinion on whether it's worth bothering with).
>> 
> 
> Also look at the new x86_64 ABI (See all those X32 psABI messages) that the Intel folks are actively working on.  This proposal is very similar to what they are doing.

untrue.  N32 is closer to the X32 ABI since it is limited to 2GB.

Re: RFC: A new MIPS64 ABI

[Matt Thomas]

On Feb 14, 2011, at 6:22 PM, David Daney wrote:

> On 02/14/2011 04:15 PM, Matt Thomas wrote:
>> 
>> I have to wonder if it's worth the effort.  The primary problem I see
>> is that this new ABI requires a 64bit kernel since faults through the
>> upper 2G will go through the XTLB miss exception vector.
>> 
> 
> Yes, that is correct.  It is a 64-bit ABI, and like the existing n32 ABI requires a 64-bit kernel.

N32 doesn't require a LP64 kernel, just a 64-bit register aware kernel.
Your N32-big does require a LP64 kernel.

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 06:14 PM, Joe Buck wrote:
> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
>> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
>>
>> This seems like a microscopically small market segment.  I can't see any sense in such an effort.
>
> I remember the RHEL hugemem patch being a big deal for lots of their
> customers, so a process could address the full 4GB instead of only 3GB
> on a 32-bit machine.  If I recall correctly, upstream didn't want it
> (get a 64-bit machine!) but lots of paying customers clamored for it.
>
> (I personally don't have an opinion on whether it's worth bothering with).
>

Also look at the new x86_64 ABI (See all those X32 psABI messages) that 
the Intel folks are actively working on.  This proposal is very similar 
to what they are doing.

David Daney

Re: RFC: A new MIPS64 ABI

[David Daney]

On 02/14/2011 04:15 PM, Matt Thomas wrote:
>
> On Feb 14, 2011, at 12:29 PM, David Daney wrote:
>
>> Background:
>>
>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>> user virtual memory space.  This is due the way MIPS32 memory space is
>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>> values are always sign extended.
>>
>> Because there are not already enough MIPS ABIs, I present the ...
>>
>> Proposal: A new ABI to support 4GB of address space with 32-bit
>> pointers.
>>
>> The proposed new ABI would only be available on MIPS64 platforms.  It
>> would be identical to the current MIPS n32 ABI *except* that pointers
>> would be zero-extended rather than sign-extended when resident in
>> registers.  In the remainder of this document I will call it
>> 'n32-big'.  As a result, applications would have access to a full 4GB
>> of virtual address space.  The operating environment would be
>> configured such that the entire lower 4GB of the virtual address space
>> was available to the program.
>
> I have to wonder if it's worth the effort.  The primary problem I see
> is that this new ABI requires a 64bit kernel since faults through the
> upper 2G will go through the XTLB miss exception vector.
>

Yes, that is correct.  It is a 64-bit ABI, and like the existing n32 ABI 
requires a 64-bit kernel.


>> At a low level here is how it would work:
>>
>> 1) Load a pointer to a register from memory:
>>
>> n32:
>> 	LW $reg, offset($reg)
>>
>> n32-big:
>> 	LWU $reg, offset($reg)
>
>
> That might be sufficient for userland, but the kernel will need
> to do similar things (even if a 64bit kernel) when accessing
> structures supplied by 32-bit syscalls.
>

It is a userspace ABI.  The MIPS64 kernel already uses something similar 
(the -msym32 option).  There would be no change to the kernel.


> It seems to be workable but if you need the additional address space
> why not use N64?

In n64 pointers are 64-bits wide.  Programs that use many pointer laden 
data structures have a much larger cache/memory footprint than their n32 
versions.

Also the number of instructions required to load a 64-bit constant is 
much larger than that needed to load a 32-bit constant.

David Daney

Re: RFC: A new MIPS64 ABI

[Paul Koning]

On Feb 14, 2011, at 9:14 PM, Joe Buck wrote:

> On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
>> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
>> 
>> This seems like a microscopically small market segment.  I can't see any sense in such an effort.
> 
> I remember the RHEL hugemem patch being a big deal for lots of their
> customers, so a process could address the full 4GB instead of only 3GB
> on a 32-bit machine.  If I recall correctly, upstream didn't want it
> (get a 64-bit machine!) but lots of paying customers clamored for it.

Fair enough, but that's a very different case.  As Matt points out, the proposal here requires a 64 bit machine; the only difference is in the user process space limit when that process is running in 32 bit mode.

	paul

Re: RFC: A new MIPS64 ABI

[Joe Buck]

On Mon, Feb 14, 2011 at 05:57:13PM -0800, Paul Koning wrote:
> It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.
> 
> This seems like a microscopically small market segment.  I can't see any sense in such an effort.

I remember the RHEL hugemem patch being a big deal for lots of their
customers, so a process could address the full 4GB instead of only 3GB
on a 32-bit machine.  If I recall correctly, upstream didn't want it
(get a 64-bit machine!) but lots of paying customers clamored for it.

(I personally don't have an opinion on whether it's worth bothering with).

Re: RFC: A new MIPS64 ABI

[Paul Koning]

On Feb 14, 2011, at 7:15 PM, Matt Thomas wrote:

> 
> On Feb 14, 2011, at 12:29 PM, David Daney wrote:
> 
>> Background:
>> 
>> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
>> user virtual memory space.  This is due the way MIPS32 memory space is
>> segmented.  Only the range from 0..2^31-1 is available.  Pointer
>> values are always sign extended.
>> 
>> Because there are not already enough MIPS ABIs, I present the ...
>> 
>> Proposal: A new ABI to support 4GB of address space with 32-bit
>> pointers....
> 
> I have to wonder if it's worth the effort.  The primary problem I see
> is that this new ABI requires a 64bit kernel since faults through the
> upper 2G will go through the XTLB miss exception vector.  

It seems a very large amount of work for a very small benefit.

> 
>> At a low level here is how it would work:
>> 
>> 1) Load a pointer to a register from memory:
>> 
>> n32:
>> 	LW $reg, offset($reg)
>> 
>> n32-big:
>> 	LWU $reg, offset($reg)
> 
> 
> That might be sufficient for userland, but the kernel will need
> to do similar things (even if a 64bit kernel) when accessing 
> structures supplied by 32-bit syscalls.  

Right, which creates amazing opportunities for bugs.
> 
> It seems to be workable but if you need the additional address space
> why not use N64?

It seems that this proposal would benefit programs that need more than 2 GB but less than 4 GB, and for some reason really don't want 64 bit pointers.

This seems like a microscopically small market segment.  I can't see any sense in such an effort.

	paul

Re: RFC: A new MIPS64 ABI

[Matt Thomas]

On Feb 14, 2011, at 12:29 PM, David Daney wrote:

> Background:
> 
> Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
> user virtual memory space.  This is due the way MIPS32 memory space is
> segmented.  Only the range from 0..2^31-1 is available.  Pointer
> values are always sign extended.
> 
> Because there are not already enough MIPS ABIs, I present the ...
> 
> Proposal: A new ABI to support 4GB of address space with 32-bit
> pointers.
> 
> The proposed new ABI would only be available on MIPS64 platforms.  It
> would be identical to the current MIPS n32 ABI *except* that pointers
> would be zero-extended rather than sign-extended when resident in
> registers.  In the remainder of this document I will call it
> 'n32-big'.  As a result, applications would have access to a full 4GB
> of virtual address space.  The operating environment would be
> configured such that the entire lower 4GB of the virtual address space
> was available to the program.

I have to wonder if it's worth the effort.  The primary problem I see
is that this new ABI requires a 64bit kernel since faults through the
upper 2G will go through the XTLB miss exception vector.  

> At a low level here is how it would work:
> 
> 1) Load a pointer to a register from memory:
> 
> n32:
> 	LW $reg, offset($reg)
> 
> n32-big:
> 	LWU $reg, offset($reg)


That might be sufficient for userland, but the kernel will need
to do similar things (even if a 64bit kernel) when accessing 
structures supplied by 32-bit syscalls.  

It seems to be workable but if you need the additional address space
why not use N64?

RFC: A new MIPS64 ABI

[David Daney]

Background:

Current MIPS 32-bit ABIs (both o32 and n32) are restricted to 2GB of
user virtual memory space.  This is due the way MIPS32 memory space is
segmented.  Only the range from 0..2^31-1 is available.  Pointer
values are always sign extended.

Because there are not already enough MIPS ABIs, I present the ...

Proposal: A new ABI to support 4GB of address space with 32-bit
pointers.

The proposed new ABI would only be available on MIPS64 platforms.  It
would be identical to the current MIPS n32 ABI *except* that pointers
would be zero-extended rather than sign-extended when resident in
registers.  In the remainder of this document I will call it
'n32-big'.  As a result, applications would have access to a full 4GB
of virtual address space.  The operating environment would be
configured such that the entire lower 4GB of the virtual address space
was available to the program.


At a low level here is how it would work:

1) Load a pointer to a register from memory:

n32:
	LW $reg, offset($reg)

n32-big:
	LWU $reg, offset($reg)

2) Load an address constant into a register:

n32:
	LUI $reg, high_part
	ORI $reg, low_part

n32-big:
	ORI $reg, high_part
	DSLL $reg, $reg, 16
	ORI $reg, low_part


Q: What would have to change to make this work?

o A new ELF header flag to denote the ABI.

o Linker support to use proper library search paths, and linker scrips
   to set the INTERP program header, etc.

o GCC has to emit code for the new ABI.

o Could all existing n32 relocation types be used?  I think so.

o Runtime libraries would have to be placed in a new location
   (/lib32big, /usr/lib32big ...)

o The C library's ld.so would have to use a distinct LD_LIBRARY_PATH
   for n32-big code.

o What would the Linux system call interface be?  I would propose
   using the existing Linux n32 system call interface.  Most system
   calls would just work.  Some, that pass pointers in in-memory
   structures, might require kernel modifications (sigaction() for
   example).