Re: GSOC Proposal

[nick <xerofoify@gmail.com>]

On 2019-04-01 9:47 a.m., Richard Biener wrote:
> On Mon, 1 Apr 2019, nick wrote:
> 
>>
>>
>> On 2019-04-01 5:56 a.m., Richard Biener wrote:
>>> On Fri, 29 Mar 2019, nick wrote:
>>>
>>>>
>>>>
>>>> On 2019-03-29 10:28 a.m., nick wrote:
>>>>>
>>>>>
>>>>> On 2019-03-29 5:08 a.m., Richard Biener wrote:
>>>>>> On Thu, 28 Mar 2019, nick wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 2019-03-28 4:59 a.m., Richard Biener wrote:
>>>>>>>> On Wed, Mar 27, 2019 at 6:31 PM nick <xerofoify@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Greetings All,
>>>>>>>>>
>>>>>>>>> I've already done most of the work required for signing up for GSoC
>>>>>>>>> as of last year i.e. reading getting started, being signed up legally
>>>>>>>>> for contributions.
>>>>>>>>>
>>>>>>>>> My only real concern would be the proposal which I started writing here:
>>>>>>>>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit?usp=sharing
>>>>>>>>>
>>>>>>>>> The biography and success section I'm fine with my bigger concern would be the project and roadmap
>>>>>>>>> section. The roadmap is there and I will go into more detail about it in the projects section as
>>>>>>>>> need be. Just wanted to known if the roadmap is detailed enough or can I just write out a few
>>>>>>>>> paragraphs discussing it in the Projects Section.
>>>>>>>>
>>>>>>>> I'm not sure I understand either the problem analysis nor the project
>>>>>>>> goal parts.  What
>>>>>>>> shared state with respect to garbage collection are you talking about?
>>>>>>>>
>>>>>>>> Richard.
>>>>>>>>
>>>>>>> I just fixed it. Seems we were discussing RTL itself. I edited it to 
>>>>>>> reflect those changes. Let me know if it's unclear or you would actually 
>>>>>>> like me to discuss some changes that may occur in the RTL layer itself.
>>>>>>>
>>>>>>>
>>>>>>> I'm glad to be more exact if that's better but seems your confusion was 
>>>>>>> just what layer we were touching.
>>>>>>
>>>>>> Let me just throw in some knowledge here.  The issue with RTL
>>>>>> is that we currently can only have a single function in this
>>>>>> intermediate language state since a function in RTL has some
>>>>>> state in global variables that would differ if it were another
>>>>>> function.  We can have multiple functions in GIMPLE intermediate
>>>>>> language state since all such state is in a function-specific
>>>>>> data structure (struct function).  The hard thing about moving
>>>>>> all this "global" state of RTL into the same place is that
>>>>>> there's global state in the various backends (and there's
>>>>>> already a struct funtion 'machine' part for such state, so there's
>>>>>> hope the issue isn't as big as it could be) and that some of
>>>>>> the global state is big and only changes very rarely.
>>>>>> That said, I'm not sure if anybody knows the full details here.
>>>>>>
>>>>>> So as far as I understand you'd like to tackle this as project
>>>>>> with the goal to be able to have multiple functions in RTL
>>>>>> state.
>>>>>>
>>>>>> That's laudable but IMHO also quite ambitious for a GSoC
>>>>>> project.  It's also an area I am not very familiar with so
>>>>>> I opt out of being a mentor for this project.
>>>>>>
>>>>> While I'm aware of three areas where the shared state is an issue
>>>>> currently:
>>>>> 1, Compiler's Proper
>>>>> 2. The expand_functions 
>>>>> 3. RTL
>>>>> 4.Garbage Collector
>>>>>
>>>>> Or maybe a project to be more
>>>>> explicit about regions of the code that assume that the garbage-
>>>>> collector can't run within them?[3] (since the GC is state that would
>>>>> be shared by the threads).
>>>>>
>>>>> This is what we were discussing previously and I wrote my proposal for
>>>>> that. You however seem confused about what parts of the garbage collector
>>>>> would be touched. That's fine with me, however seems you want be to
>>>>> be more exact about which part  is touched.
>>>>>
>>>>> My questions would be as it's changed back to the garbage collector project:
>>>>> https://docs.google.com/document/d/1BKVeh62IpigsQYf_fJqkdu_js0EeGdKtXInkWZ-DtU0/edit
>>>>>
>>>>> 1. Your confusion about which part of the garbage collector is touched doesn't
>>>>> really make sense s it's for the whole garbage collector as related to shared
>>>>> state?
>>>>> 2. Injection was my code here in phase 3 for the callers of the new functions or
>>>>> macros, perhaps this is not needed as the work with the garbage collector is enough?
>>>>> 3. Am I not understanding this project as I thought I was in the proposal I wrote?
>>>>>
>>>>> Seems your more confusing my wording probably so I'm going to suggest one of 
>>>>> two things here:
>>>>> a) I'm going to allow you to make comments with what's confusing you and
>>>>> it needs that's the issue here more than anything else so I sent you 
>>>>> a link and please comment where you are having issues with this not
>>>>> be clear for you:
>>>>> Or maybe a project to be more
>>>>> explicit about regions of the code that assume that the garbage-
>>>>> collector can't run within them?[3] (since the GC is state that would
>>>>> be shared by the threads).
>>>>> as that's the actual project
>>>>>
>>>>> b) Just comment here about the wording that's an issue for you or
>>>>> where you want more exact wording
>>>>>
>>>>> Sorry and hopefully this is helps you understand where I'm going,
>>>>> Nick
>>>>>
>>>>>> Richard.
>>>>>>
>>>>>>> Nick
>>>>>>>>> Any other comments are welcome as well as I write it there,
>>>>>>>>> Nick
>>>>>>>
>>>>
>>>> Richard,
>>>>
>>>> Seems your right touching the complete garbage collector is too much. I'm
>>>> just looking at the users of the garbage collector and it seems one of the
>>>> major ones is pre compiled headers.
>>>>
>>>> I've narrowed it down to that. My own real final concern is two things:
>>>> 1. Does it make sense to you in my writing?
>>>> 2. Should callers inject the information for state sharing as required
>>>> as that seems better or is it better for the garbage collector to  store
>>>> the state sharing flags,marcos and functions internally for this.
>>>>
>>>> Thanks and seems I was over thinking the last proposal it's too much:),
>>>
>>> Nick,
>>>
>>> as I've previously explained the garbage collector (and 
>>> precompiled-headers) workings are understood well and its state
>>> is already annotated everywhere.  What I do not understand is
>>> what "global state" with respect to parallelization in GCC you
>>> refer to when you write about the garbage collector and how
>>> annotating helps parallelization.  The garbage collector itself
>>> is only an issue for parallelization as far as it is not
>>> thread-safe at the moment.  Collection already only happens
>>> at very specific points where it is known to be safe.
>>>
>>> Richard.
>>>
>> Well I'm talking about the shared roots of this garbage collector core state 
>> data structure or just struct ggc_root_tab.
>>
>> But also this seems that this to be no longer shared globally if I'm not mistaken 
>> or this:
>> static vec<const_ggc_root_tab_t> extra_root_vec;
>>
>> Not sure after reading the code which is a bigger deal through so I wrote
>> my proposal not just asking which is a better issue for not being thread
>> safe. Sorry about that.
>>
>> As for the second question injection seems to not be the issue or outside
>> callers but just internal so phase 3 or step 3 would now be:
>> Find internal callers or users of x where x is one of the above rather
>> than injecting outside callers. Which answers my second question about
>> external callers being a issue still.
>>
>> Let me know which  of the two is a better issue:
>> 1. struct ggc_root_tabs being shared
>> 2.static vec<const_ggc_root_tab_t> extra_root_vec; as a shared heap or
>> vector of root nodes for each type of allocation
>>
>> and I will gladly rewrite my proposal sections for that
>> as needs to be reedited.
> 
> I don't think working on the garbage collector as a separate
> GSoC project is useful at this point.  Doing locking around
> allocation seems like a good short-term solution and if that
> turns out to be a performance issue for the threaded part
> using per-thread freelists is likely an easy to deploy
> solution.
> 
> Richard.
> 
I agree but we were discussing this:
Or maybe a project to be more
explicit about regions of the code that assume that the garbage-
collector can't run within them?[3] (since the GC is state that would
be shared by the threads).

In addition I moved my paper back to our discussion about garbage collector
state with outside callers.Seems we really need to do something about
my wording as the idea of my project in a nutshell was to figure
out how to mark shared state by callers and inject it into the
garbage collector letting it known that the state was not shared between
threads or shared. Seems that was on the GSoc page and in our discussions the issue
is marking outside code for shared state. If that's correct then my
wording of outside callers is incorrect it should have been shared
state between threads on outside callers to the garbage collector.
If the state is that in your wording above then great as I understand
where we are going and will gladly change my wording.

Also freelists don't work here as the state is shared at the caller's end which
would need two major issues:
1. Locking on nodes of the freelists when two threads allocate at the same thing
which can be a problem if the shared state is shared a lot
2. Locking allocation with large numbers of callers can starve threads

Seems that working on the garbage collector itself isn't the issue but the callers
as I just figured out as related to your state idea. Let me know if that's correct
and if the wording change I mentioned is fine with you as that's the state it
seems that needs to be changed.
Nick