* [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] @ 2019-02-10 5:45 sameeran joshi 2019-02-18 18:22 ` Martin Jambor 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-02-10 5:45 UTC (permalink / raw) To: gcc; +Cc: Andi Kleen Hi,I am an undergraduate student currently in final year of computer science and engineering degree course from Pune University, India. I and Shubham have been working on Last year's GSoC project idea : Implement a fuzzer leveraging GCC extensions. Fuzzers like csmith are fairly good at finding compiler bugs. But they only generate standard C, but no extensions. GCC has many extensions, which are not covered. It would be good to extend a fuzzer like csmith to fuzz extensions like OpenMP, attributes, vector extensions, etc. Then run the fuzzer and report compiler bugs. since June 2018 under the guidance of mentor Andi Kleen. I worked on generating GCC C language extensions here is the link (coverage reports,implemented extension's list,bugs found,test cases, and usage are in README file on github) github Link: https://github.com/Sameeranjoshi/csmith/tree/gcc-extensions We choose this as our university project as well, and are still fuzzing the extensions on compiler farm. Based on the previous work I would like to propose the following idea for GSoC 2019: Extending Csmith for OpenMP extensions. I would implement following constructs of OpenMP 1.PARALLEL CONSTRUCT 2.WORKSHARING CONSTRUCTS - 2.1 sections 2.2 single 2.3 loop constructs 2.4 master construct 3.TEAMS CONSTRUCT 4.TASKING CONSTRUCT - 4.1 task 4.2 taskloop 4.3 taskloop simd 4.4 taskyield 5.SYNCHRONIZATION CONSTRUCTS - 5.1 critical 5.2 atomic 5.3 barrier 5.4 taskwait 5.5 taskgroup 6.DATA SHARING ATTRIBUTES - 6.1 private 6.2 public 6.3 firstprivate 6.4 lastprivate Also, I would like to work on the target constrains if time permits. The main challenge what I think would be to ensure that there aren't any data races and data conflicts so that the parallelized program is not undefined. Usage for the GCC community : 1. It might have slight large increments in code coverage and trigger a lot of unique code . I have watched A "Hands-on" Introduction to OpenMP | Tim Mattson, Intel all 4 parts https://www.youtube.com/watch?v=pRtTIW9-Nr0 I have started reading the specification of latest 5.0 standard. Please suggest if this could be an interesting idea for upcoming GSoC ? Thanks, Sameeran Joshi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-02-10 5:45 [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] sameeran joshi @ 2019-02-18 18:22 ` Martin Jambor 2019-03-23 18:19 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Martin Jambor @ 2019-02-18 18:22 UTC (permalink / raw) To: sameeran joshi, gcc; +Cc: Andi Kleen Hello Sameeran, On Sun, Feb 10 2019, sameeran joshi wrote: > Hi,I am an undergraduate student currently in final year of computer > science and engineering degree course from Pune University, India. I > and Shubham have been working on Last year's GSoC project idea : > > Implement a fuzzer leveraging GCC extensions. Fuzzers like csmith are > fairly good at finding compiler bugs. But they only generate standard > C, but no extensions. GCC has many extensions, which are not covered. > It would be good to extend a fuzzer like csmith to fuzz extensions > like OpenMP, attributes, vector extensions, etc. Then run the fuzzer > and report compiler bugs. > > since June 2018 under the guidance of mentor Andi Kleen. > I worked on generating GCC C language extensions here is the link > (coverage reports,implemented extension's list,bugs found,test cases, > and usage are in README file on github) > github Link: https://github.com/Sameeranjoshi/csmith/tree/gcc-extensions > > We choose this as our university project as well, and are still > fuzzing the extensions on compiler farm. > > Based on the previous work I would like to propose the following idea > for GSoC 2019: > Extending Csmith for OpenMP extensions. > > I would implement following constructs of OpenMP > > 1.PARALLEL CONSTRUCT > 2.WORKSHARING CONSTRUCTS - > 2.1 sections > 2.2 single > 2.3 loop constructs > 2.4 master construct > 3.TEAMS CONSTRUCT > 4.TASKING CONSTRUCT - > 4.1 task > 4.2 taskloop > 4.3 taskloop simd > 4.4 taskyield > 5.SYNCHRONIZATION CONSTRUCTS - > 5.1 critical > 5.2 atomic > 5.3 barrier > 5.4 taskwait > 5.5 taskgroup > 6.DATA SHARING ATTRIBUTES - > 6.1 private > 6.2 public > 6.3 firstprivate > 6.4 lastprivate > Also, I would like to work on the target constrains if time permits. > The main challenge what I think would be to ensure that there aren't > any data races and data conflicts so that the parallelized program is > not undefined. > > Usage for the GCC community : > 1. It might have slight large increments in code coverage and trigger > a lot of unique code . > > I have watched > A "Hands-on" Introduction to OpenMP | Tim Mattson, Intel all 4 parts > https://www.youtube.com/watch?v=pRtTIW9-Nr0 > > I have started reading the specification of latest 5.0 standard. > Please suggest if this could be an interesting idea for upcoming GSoC ? > Indeed it is, you clearly have it thought out very well. I have noted it down and am looking forward to your project submission (assuming Google approves us as a participating organization, of course). Meanwhile, if you have any technical questions, regarding the GCC extensions you listed above, feel free to ask here on the list. Good luck diving into the OpenMP spec and thank you, Martin ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-02-18 18:22 ` Martin Jambor @ 2019-03-23 18:19 ` sameeran joshi 2019-03-24 2:55 ` Andi Kleen 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-03-23 18:19 UTC (permalink / raw) To: gcc; +Cc: Andi Kleen Hi, I tried doing a basic/incomplete patch(needs more detailed conditions check) for implementing a structured block in Csmith code base. The structured block is almost needed for most of the constructs which would be further implemented. 0) The patch identifies and labels each block(after block created) as leaf and non leaf block. example: for (){ //block 1 if(){ //block 2 if(){ //block 3 } else{ //block4 } } } leaf blocks: block3, block4 non-leaf:block1 1) check_structured_block_conditions() checks for the conditions related to a structured block 1.no returns in block 2.no gotos 3.no breaks and accordingly labels the blocks as structured block, for example for (){ //unstructured //block 1 if(){ //unstructured //block 2 if(){ //block 3 //structured 1.no gotos 2.no breaks 3.no return 4.Do I need to check continue as well? } else{ //block4 //unstructured return ; } } } structured:block3 unstructured: block1,2(one of the childs is unstructured), block4 This applies mostly when the break,goto,return statements have some probability of generation. Another workaround I think(which would increase the generation of more OpenMP constructs)is to make probabilities of above statements to '0' so code generated will have all structured blocks, which aids in implementing the future OpenMP pargmas. I have some questions regarding OpenMP: For the following code: struct S1 { int f0; }; global variable: static struct S1 g_371 = {0x54L}; void main ( ){ #pragma omp parallel for for (g_371.f0 = (-3); (g_371.f0 >= (-27)) ; g_371.f0 = safe_sub_func_int32_t_s_s(g_371.f0, 2)) {/* block id: 1 */ structured block } } I have following 3 questions in regards to usage of openmp. 0.Can't I use a '(test)' a 'bracket' for a 'test' expression? error:invalid controlling predicate If I try removing the brackets the program works fine. 1.Can I use a struct field variable for initialization?: Whereas the 5.0 specification states: var operator lb var := variable of signed or unsigned type | variable of pointer type which obeys the specification rules. error:expected iteration declaration/initialization before 'g_371' 2.Consider a case where I need to increment the variable: Considering the grammar of Csmith, in increment expression I need to use function safe_sub_func_int32_t_s_s(g_371.f0, 2) the function decrements the variable(same as g_371.f0 = g_371.f0-1 ) but it does some checks for the bounds, which are essential for checking the undefined conditions. but openmp compiler gives following error: error: invalid increment expression Spec 5.0 needs something like var++, var = var +incr, etc... How do I overcome this case, any workaround for this, but still using the safe_sub_func_int32_t_s_s() ? What do you think about adding command lines so as to disable generation of such increment expressions so it follows spec 5.0 Thanks. ====================== diff --git a/src/Block.cpp b/src/Block.cpp index ab2fe14..0928ad3 100644 --- a/src/Block.cpp +++ b/src/Block.cpp @@ -186,18 +186,60 @@ Block::make_random(CGContext &cg_context, bool looping) } // perform DFA analysis after creation + //for ex. 4 statements were created inside a block, but + //after DFA analysis they were removed + //function checks for leaf block, if yes then sets 'leaf_block' = true + b->set_leaf_or_nonleaf_block(b); + //check for structuredness + //function blocks are not structured, due to return + //need a more detailed conditions for above case + //arrayOp is in itself a structured block statement + if (curr_func->blocks[0] == b){//optimizes a bit + b->IsStructured = false; + } + else{ + if (b->is_leaf_block ()){ + if (b->check_structured_block_conditions() ){ // if true + //logic of struc block + b->IsStructured = true; + } + else{//not a structured block + b->IsStructured = false; + } + } + else{//a parent block + if (b->check_structured_block_conditions() ){ // if true + //logic of struc block + //wait the internal blocks remain to check + //1.get internal blocks + vector<const Block*> internal_blks; + for (size_t i =0 ; i< b->stms.size() ; i++) + b->stms[i]->get_blocks(internal_blks); + + bool internal_block_fails=false; + for (size_t k = 0; k < internal_blks.size(); k++){ + if(!internal_blks[k]->IsStructured){ + internal_block_fails = true; + break; + } + } + (internal_block_fails == true) ? (b->IsStructured = false ) : (b->IsStructured = true); + } + else{//not a structured block + b->IsStructured = false; + } + + } + } curr_func->stack.pop_back(); return NULL; @@ -248,7 +290,49 @@ Block::make_random(CGContext &cg_context, bool looping) } return b; } +//need to call after DFA analysis,else some statements removed +//we check for for and if statements ( as have blocks to these statements ) +//if above statements not present it's a leaf node else a parent one + +void Block::set_leaf_or_nonleaf_block(Block *b){ + b->leaf_block = true;//assume block is leaf,rule out later + for (int i =0 ; i<b->stms.size() ; i++){ + if ( (b->stms[i]->eType == eFor) || (b->stms[i]->eType == eIfElse) ){ //if a for/ if statement.do we need to check any more? + b->leaf_block = false; + } + } +} +//checks whether block is a leaf +bool Block::is_leaf_block(){ + if (this->leaf_block) + return true; + else + return false; +} +bool Block::check_structured_block_conditions(){ + //write here + size_t i ; + for (i = 0 ; i < this->stms.size(); i++){ + if (this->stms[i]->eType == eReturn) + return false; + if (this->stms[i]->eType == eBreak) + return false; + if (this->stms[i]->eType == eGoto) + return false; + //continue is alowed,right? + //1.if no return + //2.no break + //3. no statement goto + //if no label present-hard + } + return true;//still need more detailed analysis +} +bool Block::is_structured_block(){ + if (this->IsStructured) + return true; + else + return false; +} /* * */ @@ -296,7 +380,7 @@ Block::~Block(void) local_vars.clear(); macro_tmp_vars.clear(); } - +//new temp variable created and stored in macro_tmp_vars(mentioned below) std::string Block::create_new_tmp_var(enum eSimpleType type) const @@ -368,6 +452,17 @@ Block::Output(std::ostream &out, FactMgr* fm, int indent) const //blockid std::ostringstream ss; ss << "block id: " << stm_id; + if (IsStructured){ + outputln(out); + output_tab(out, indent); + output_comment_line (out , "structured"); + } + else{ + outputln(out); + output_tab(out, indent); + output_comment_line (out , "un structured"); + } + output_tab(out, indent); output_comment_line(out, ss.str()); if(CGOptions::stmt_expr()){ diff --git a/src/Block.h b/src/Block.h index 23c939f..79f95a6 100644 --- a/src/Block.h +++ b/src/Block.h @@ -132,6 +132,14 @@ public: bool func_start_stmt_expr = false; +//data variables + bool leaf_block ;//is it good if not initialized in constructor? + bool IsStructured ; +//member functions + void set_leaf_or_nonleaf_block(Block *b); + bool is_leaf_block(); + bool check_structured_block_conditions(); + bool is_structured_block(); private: bool depth_protect; On 2/18/19, Martin Jambor <mjambor@suse.cz> wrote: > Hello Sameeran, > > On Sun, Feb 10 2019, sameeran joshi wrote: >> Hi,I am an undergraduate student currently in final year of computer >> science and engineering degree course from Pune University, India. I >> and Shubham have been working on Last year's GSoC project idea : >> >> Implement a fuzzer leveraging GCC extensions. Fuzzers like csmith are >> fairly good at finding compiler bugs. But they only generate standard >> C, but no extensions. GCC has many extensions, which are not covered. >> It would be good to extend a fuzzer like csmith to fuzz extensions >> like OpenMP, attributes, vector extensions, etc. Then run the fuzzer >> and report compiler bugs. >> >> since June 2018 under the guidance of mentor Andi Kleen. >> I worked on generating GCC C language extensions here is the link >> (coverage reports,implemented extension's list,bugs found,test cases, >> and usage are in README file on github) >> github Link: >> https://github.com/Sameeranjoshi/csmith/tree/gcc-extensions >> >> We choose this as our university project as well, and are still >> fuzzing the extensions on compiler farm. >> >> Based on the previous work I would like to propose the following idea >> for GSoC 2019: >> Extending Csmith for OpenMP extensions. >> >> I would implement following constructs of OpenMP >> >> 1.PARALLEL CONSTRUCT >> 2.WORKSHARING CONSTRUCTS - >> 2.1 sections >> 2.2 single >> 2.3 loop constructs >> 2.4 master construct >> 3.TEAMS CONSTRUCT >> 4.TASKING CONSTRUCT - >> 4.1 task >> 4.2 taskloop >> 4.3 taskloop simd >> 4.4 taskyield >> 5.SYNCHRONIZATION CONSTRUCTS - >> 5.1 critical >> 5.2 atomic >> 5.3 barrier >> 5.4 taskwait >> 5.5 taskgroup >> 6.DATA SHARING ATTRIBUTES - >> 6.1 private >> 6.2 public >> 6.3 firstprivate >> 6.4 lastprivate >> Also, I would like to work on the target constrains if time permits. >> The main challenge what I think would be to ensure that there aren't >> any data races and data conflicts so that the parallelized program is >> not undefined. >> >> Usage for the GCC community : >> 1. It might have slight large increments in code coverage and trigger >> a lot of unique code . >> >> I have watched >> A "Hands-on" Introduction to OpenMP | Tim Mattson, Intel all 4 >> parts >> https://www.youtube.com/watch?v=pRtTIW9-Nr0 >> >> I have started reading the specification of latest 5.0 standard. >> Please suggest if this could be an interesting idea for upcoming GSoC ? >> > > Indeed it is, you clearly have it thought out very well. I have noted > it down and am looking forward to your project submission (assuming > Google approves us as a participating organization, of course). > > Meanwhile, if you have any technical questions, regarding the GCC > extensions you listed above, feel free to ask here on the list. > > Good luck diving into the OpenMP spec and thank you, > > Martin > > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-23 18:19 ` sameeran joshi @ 2019-03-24 2:55 ` Andi Kleen 2019-03-24 19:33 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Andi Kleen @ 2019-03-24 2:55 UTC (permalink / raw) To: sameeran joshi; +Cc: gcc On Sat, Mar 23, 2019 at 11:49:11PM +0530, sameeran joshi wrote: > 1) check_structured_block_conditions() > checks for the conditions related to a structured block > 1.no returns in block returns should be allowed inside statement expressions. > 2.no gotos > 3.no breaks > and accordingly labels the blocks as structured block, for example > for (){ > //unstructured > //block 1 > if(){ > //unstructured > //block 2 > if(){ > //block 3 > //structured > 1.no gotos > 2.no breaks > 3.no return > 4.Do I need to check continue as well? Yes check for continue too. > This applies mostly when the break,goto,return statements have some > probability of generation. > Another workaround I think(which would increase the generation of more > OpenMP constructs)is to make probabilities of above statements to '0' Sure, of course only within the structured expressions. > For the following code: > struct S1 { > int f0; > }; > global variable: > static struct S1 g_371 = {0x54L}; > > void main ( ){ > #pragma omp parallel for > for (g_371.f0 = (-3); (g_371.f0 >= (-27)) ; g_371.f0 = > safe_sub_func_int32_t_s_s(g_371.f0, 2)) > {/* block id: 1 */ > structured block > } > } > I have following 3 questions in regards to usage of openmp. > > 0.Can't I use a '(test)' a 'bracket' for a 'test' expression? > error:invalid controlling predicate > If I try removing the brackets the program works fine. You mean it errors for for (g_371.f0 = (-3); (g_371.f0 >= (-27)) ; g_371.f0 = safe_sub_func_int32_t_s_s(g_371.f0, 2)) and works for for (g_371.f0 = (-3); g_371.f0 >= (-27) ; g_371.f0 = safe_sub_func_int32_t_s_s(g_371.f0, 2)) ? If yes that would seem like a gcc bug. I would file in bugzilla with a simple test case. > > > 1.Can I use a struct field variable for initialization?: > > Whereas the 5.0 specification states: > var operator lb > var := variable of signed or unsigned type | variable of pointer type > which obeys the specification rules. > > error:expected iteration declaration/initialization before 'g_371' Ok I guess it's not supported, so you can't. > > > 2.Consider a case where I need to increment the variable: > > Considering the grammar of Csmith, in increment expression I need to > use function safe_sub_func_int32_t_s_s(g_371.f0, 2) > the function decrements the variable(same as g_371.f0 = g_371.f0-1 ) > but it does some checks for the bounds, which are essential for > checking the undefined conditions. Maybe could make the index only unsigned, then ++ would work. We definitely need increment expressions to be useful. And perhaps have an command line option to allow unchecked signed increment for this. > > What do you think about adding command lines so as to disable > generation of such increment expressions so it follows spec 5.0 We need them, otherwise too much useful OpenMP won't be generated. -Andi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-24 2:55 ` Andi Kleen @ 2019-03-24 19:33 ` sameeran joshi 2019-03-26 0:41 ` Andi Kleen 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-03-24 19:33 UTC (permalink / raw) To: gcc; +Cc: Andi Kleen On 3/24/19, Andi Kleen <ak@linux.intel.com> wrote: > On Sat, Mar 23, 2019 at 11:49:11PM +0530, sameeran joshi wrote: >> 1) check_structured_block_conditions() >> checks for the conditions related to a structured block >> 1.no returns in block > > returns should be allowed inside statement expressions. If I am correct, we would create a new branch "COMPsmith"(C OpenMP smith)(name will this work?) from the master of Csmith and work on it, statement expression are in the "gcc c extensions" branch I guess which would be a separate branch. So it shouldn't allow return as well, right? > >> 2.no gotos >> 3.no breaks >> and accordingly labels the blocks as structured block, for example >> for (){ >> //unstructured >> //block 1 >> if(){ >> //unstructured >> //block 2 >> if(){ >> //block 3 >> //structured >> 1.no gotos >> 2.no breaks >> 3.no return >> 4.Do I need to check continue as well? > > Yes check for continue too. referring this: http://forum.openmp.org/forum/viewtopic.php?f=3&t=458 continue can be used but with some restriction as: "innermost associated loop may be curtailed by a continue statement " Can you please elaborate this with some simple example? IIUC , we can use continue for innermost loop blocks and their sibling nodes, but not the parent blocks. > >> This applies mostly when the break,goto,return statements have some >> probability of generation. >> Another workaround I think(which would increase the generation of more >> OpenMP constructs)is to make probabilities of above statements to '0' > > Sure, of course only within the structured expressions. > >> For the following code: >> struct S1 { >> int f0; >> }; >> global variable: >> static struct S1 g_371 = {0x54L}; >> >> void main ( ){ >> #pragma omp parallel for >> for (g_371.f0 = (-3); (g_371.f0 >= (-27)) ; g_371.f0 = >> safe_sub_func_int32_t_s_s(g_371.f0, 2)) >> {/* block id: 1 */ >> structured block >> } >> } >> I have following 3 questions in regards to usage of openmp. >> >> 0.Can't I use a '(test)' a 'bracket' for a 'test' expression? >> error:invalid controlling predicate >> If I try removing the brackets the program works fine. > > > You mean it errors for > for (g_371.f0 = (-3); (g_371.f0 >= (-27)) ; g_371.f0 = > safe_sub_func_int32_t_s_s(g_371.f0, 2)) > > and works for > > for (g_371.f0 = (-3); g_371.f0 >= (-27) ; g_371.f0 = > safe_sub_func_int32_t_s_s(g_371.f0, 2)) > > ? yes > > If yes that would seem like a gcc bug. I would file in bugzilla with a > simple test case. > Did you file it? can you please send me the bug id? > >> >> >> 1.Can I use a struct field variable for initialization?: >> >> Whereas the 5.0 specification states: >> var operator lb >> var := variable of signed or unsigned type | variable of pointer type >> which obeys the specification rules. >> >> error:expected iteration declaration/initialization before 'g_371' > > Ok I guess it's not supported, so you can't. Any specific reason for not supporting struct variables as initialization, as the spec. doesn't impose any restrictions. > >> >> >> 2.Consider a case where I need to increment the variable: >> >> Considering the grammar of Csmith, in increment expression I need to >> use function safe_sub_func_int32_t_s_s(g_371.f0, 2) >> the function decrements the variable(same as g_371.f0 = g_371.f0-1 ) >> but it does some checks for the bounds, which are essential for >> checking the undefined conditions. > > Maybe could make the index only unsigned, then ++ would work. > We definitely need increment expressions to be useful. > > And perhaps have an command line option to allow unchecked signed increment > for this. >> >> What do you think about adding command lines so as to disable >> generation of such increment expressions so it follows spec 5.0 > > We need them, otherwise too much useful OpenMP won't be generated. I was thinking about say, --no-func-as-incr will not generate safe_* functions in increment expressions of for statement, but rather generate simple and valid increment statements. I am working on proposal currently and the timeline, will try sending in couple of days. > > -Andi > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-24 19:33 ` sameeran joshi @ 2019-03-26 0:41 ` Andi Kleen 2019-03-26 7:29 ` Jakub Jelinek 0 siblings, 1 reply; 16+ messages in thread From: Andi Kleen @ 2019-03-26 0:41 UTC (permalink / raw) To: sameeran joshi; +Cc: gcc sameeran joshi <gsocsameeran@gmail.com> writes: > On 3/24/19, Andi Kleen <ak@linux.intel.com> wrote: >> On Sat, Mar 23, 2019 at 11:49:11PM +0530, sameeran joshi wrote: >>> 1) check_structured_block_conditions() >>> checks for the conditions related to a structured block >>> 1.no returns in block >> >> returns should be allowed inside statement expressions. > If I am correct, we would create a new branch "COMPsmith"(C OpenMP > smith)(name will this work?) from the master of Csmith and work on > it, statement expression are in the "gcc c extensions" branch I guess > which would be a separate branch. > > So it shouldn't allow return as well, right? Yes >> Yes check for continue too. > referring this: http://forum.openmp.org/forum/viewtopic.php?f=3&t=458 > continue can be used but with some restriction as: > > "innermost associated loop may be curtailed by a continue statement " Ah you're right. Better don't do continue then. >> If yes that would seem like a gcc bug. I would file in bugzilla with a >> simple test case. >> > Did you file it? can you please send me the bug id? I didn't. Can you show some simple example that errors first? Perhaps Jakub or some other OpenMP expert can also chime in. >> >>> >>> >>> 1.Can I use a struct field variable for initialization?: >>> >>> Whereas the 5.0 specification states: >>> var operator lb >>> var := variable of signed or unsigned type | variable of pointer type >>> which obeys the specification rules. >>> >>> error:expected iteration declaration/initialization before 'g_371' >> >> Ok I guess it's not supported, so you can't. > Any specific reason for not supporting struct variables as > initialization, as the spec. doesn't impose any restriction Yes it seems like a gcc restriction. I would file a bug (feature request) that one. It's good, these are the kind of things the OpenMP fuzzing project should be finding. -Andi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-26 0:41 ` Andi Kleen @ 2019-03-26 7:29 ` Jakub Jelinek 2019-03-26 8:00 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Jakub Jelinek @ 2019-03-26 7:29 UTC (permalink / raw) To: Andi Kleen; +Cc: sameeran joshi, gcc On Mon, Mar 25, 2019 at 05:41:26PM -0700, Andi Kleen wrote: > sameeran joshi <gsocsameeran@gmail.com> writes: > > > On 3/24/19, Andi Kleen <ak@linux.intel.com> wrote: > >> On Sat, Mar 23, 2019 at 11:49:11PM +0530, sameeran joshi wrote: > >>> 1) check_structured_block_conditions() > >>> checks for the conditions related to a structured block > >>> 1.no returns in block > >> > >> returns should be allowed inside statement expressions. > > If I am correct, we would create a new branch "COMPsmith"(C OpenMP > > smith)(name will this work?) from the master of Csmith and work on > > it, statement expression are in the "gcc c extensions" branch I guess > > which would be a separate branch. > > > > So it shouldn't allow return as well, right? > > Yes Yeah, break, return, throw that isn't caught within the body again and not rethrown are not valid, similarly say longjmp out of it, goto too. exit, abort are ok (the block is then single entry, no exit rather than single entry, single exit, but that is fine). > >> Yes check for continue too. > > referring this: http://forum.openmp.org/forum/viewtopic.php?f=3&t=458 > > continue can be used but with some restriction as: > > > > "innermost associated loop may be curtailed by a continue statement " > > Ah you're right. Better don't do continue then. Why? continue is just fine if it is in the body of the innermost loop. In OpenMP 4.5 the loops if collapsed had to be closely nested with no code at all in between (which is what GCC 9 still implements), so the only way to add invalid continue is to add it into statement expressions in the b, lb and incr expressions. > >> If yes that would seem like a gcc bug. I would file in bugzilla with a > >> simple test case. > >> > > Did you file it? can you please send me the bug id? > > I didn't. Can you show some simple example that errors first? > > Perhaps Jakub or some other OpenMP expert can also chime in. I'd need to see an example of what you are talking about. > >>> 1.Can I use a struct field variable for initialization?: > >>> > >>> Whereas the 5.0 specification states: > >>> var operator lb > >>> var := variable of signed or unsigned type | variable of pointer type > >>> which obeys the specification rules. > >>> > >>> error:expected iteration declaration/initialization before 'g_371' > >> > >> Ok I guess it's not supported, so you can't. > > Any specific reason for not supporting struct variables as > > initialization, as the spec. doesn't impose any restriction > > Yes it seems like a gcc restriction. I would file a bug (feature > request) that one. It's good, these are the kind of things > the OpenMP fuzzing project should be finding. I can raise it on omp-lang, but I'm 100% sure the intent is to have only iterators which actually can be privatized, so: struct S { int s; void foo (); }; void S::foo () { #pragma omp for for (s = 0; s < 24; s++) ; } is fine (as one can in methods use non-static data members in private (s), but your example with obj.field iterator is rejected by all the compilers I've tried (gcc, icc, clang), so I guess we are just missing a restriction in the canonical loop form that \it{var} may not be part of another variable (as an array or structure element). for C and for C++ the with the exception for methods. Or say that var must be a base language identifier. Jakub ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-26 7:29 ` Jakub Jelinek @ 2019-03-26 8:00 ` sameeran joshi 2019-03-26 8:05 ` Jakub Jelinek 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-03-26 8:00 UTC (permalink / raw) To: gcc; +Cc: Andi Kleen, Jakub Jelinek On 3/26/19, Jakub Jelinek <jakub@redhat.com> wrote: > On Mon, Mar 25, 2019 at 05:41:26PM -0700, Andi Kleen wrote: >> sameeran joshi <gsocsameeran@gmail.com> writes: >> >> > On 3/24/19, Andi Kleen <ak@linux.intel.com> wrote: >> >> On Sat, Mar 23, 2019 at 11:49:11PM +0530, sameeran joshi wrote: >> >>> 1) check_structured_block_conditions() >> >>> checks for the conditions related to a structured block >> >>> 1.no returns in block >> >> >> >> returns should be allowed inside statement expressions. >> > If I am correct, we would create a new branch "COMPsmith"(C OpenMP >> > smith)(name will this work?) from the master of Csmith and work on >> > it, statement expression are in the "gcc c extensions" branch I guess >> > which would be a separate branch. >> > >> > So it shouldn't allow return as well, right? >> >> Yes > > Yeah, break, return, throw that isn't caught within the body again and not > rethrown are not valid, similarly say longjmp out of it, goto too. > exit, abort are ok (the block is then single entry, no exit rather than > single entry, single exit, but that is fine). > >> >> Yes check for continue too. >> > referring this: http://forum.openmp.org/forum/viewtopic.php?f=3&t=458 >> > continue can be used but with some restriction as: >> > >> > "innermost associated loop may be curtailed by a continue statement " >> >> Ah you're right. Better don't do continue then. > > Why? continue is just fine if it is in the body of the innermost loop. > In OpenMP 4.5 the loops if collapsed had to be closely nested with no code > at all in between (which is what GCC 9 still implements), so the only way > to add invalid continue is to add it into statement expressions in the b, > lb > and incr expressions. > >> >> If yes that would seem like a gcc bug. I would file in bugzilla with a >> >> simple test case. >> >> >> > Did you file it? can you please send me the bug id? >> >> I didn't. Can you show some simple example that errors first? >> >> Perhaps Jakub or some other OpenMP expert can also chime in. > > I'd need to see an example of what you are talking about. int i; #pragma omp parallel for for (i = (0) ; (i< (20)) ; i++) { printf ("\ntest expression fails due to brackets"); printf ("\n i< (20) works "); printf ("\n (i< (20)) fails "); } commands: ~$ gcc fail_for.c -Wall -Wextra -fopenmp fail_for.c: In function ‘main’: fail_for.c:5:17: error: invalid controlling predicate for (i = (0) ; (i< (20)) ; i++) { ^ swamimauli@swamimauli:~$ gcc --version gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0 > >> >>> 1.Can I use a struct field variable for initialization?: >> >>> >> >>> Whereas the 5.0 specification states: >> >>> var operator lb >> >>> var := variable of signed or unsigned type | variable of pointer type >> >>> which obeys the specification rules. >> >>> >> >>> error:expected iteration declaration/initialization before 'g_371' >> >> >> >> Ok I guess it's not supported, so you can't. >> > Any specific reason for not supporting struct variables as >> > initialization, as the spec. doesn't impose any restriction >> >> Yes it seems like a gcc restriction. I would file a bug (feature >> request) that one. It's good, these are the kind of things >> the OpenMP fuzzing project should be finding. > > I can raise it on omp-lang, but I'm 100% sure the intent is to have only > iterators which actually can be privatized, so: > struct S { int s; void foo (); }; > > void > S::foo () > { > #pragma omp for > for (s = 0; s < 24; s++) > ; > } > > is fine (as one can in methods use non-static data members in private (s), > but your example with obj.field iterator is rejected by all the compilers > I've tried (gcc, icc, clang), so I guess we are just missing a restriction > in the canonical loop form that > \it{var} may not be part of another variable (as an array or structure > element). > for C and for C++ the with the exception for methods. > Or say that var must be a base language identifier. > struct s{ int f0; }; void main (){ int i; struct s var1 ; #pragma omp parallel for for (var1.f0 = 0 ; var1.f0< (20); var1.f0++) { } } swamimauli@swamimauli:~$ gcc fail_struct.c -Wall -Wextra -fopenmp fail_struct.c: In function ‘main’: fail_struct.c:9:14: error: expected iteration declaration or initialization before ‘var1’ for (var1.f0 = 0 ; var1.f0< (20); var1.f0++) { ^~~~ Thanks, Sameeran Joshi > Jakub > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-26 8:00 ` sameeran joshi @ 2019-03-26 8:05 ` Jakub Jelinek 2019-03-26 13:32 ` Andi Kleen 0 siblings, 1 reply; 16+ messages in thread From: Jakub Jelinek @ 2019-03-26 8:05 UTC (permalink / raw) To: sameeran joshi; +Cc: gcc, Andi Kleen On Tue, Mar 26, 2019 at 01:30:28PM +0530, sameeran joshi wrote: > > I'd need to see an example of what you are talking about. > > int i; > #pragma omp parallel for > for (i = (0) ; (i< (20)) ; i++) { > printf ("\ntest expression fails due to brackets"); > printf ("\n i< (20) works "); > printf ("\n (i< (20)) fails "); > } > > commands: ~$ gcc fail_for.c -Wall -Wextra -fopenmp > > fail_for.c: In function âmainâ: > fail_for.c:5:17: error: invalid controlling predicate > for (i = (0) ; (i< (20)) ; i++) { > ^ That is a correct diagnostics. See Canonical loop form. test-expr One of the following: var relational-op b b relational-op var ( var relational-op b ) is neither of those. > struct s{ > int f0; > }; > void main (){ > int i; > struct s var1 ; > #pragma omp parallel for > for (var1.f0 = 0 ; var1.f0< (20); var1.f0++) { > > } > } > > swamimauli@swamimauli:~$ gcc fail_struct.c -Wall -Wextra -fopenmp > fail_struct.c: In function âmainâ: > fail_struct.c:9:14: error: expected iteration declaration or > initialization before âvar1â > for (var1.f0 = 0 ; var1.f0< (20); var1.f0++) { > ^~~~ Yeah, as I said, I believe the intent is that this is not valid, otherwise the iterator var1.f0 could not be predetermined private, you can't write private(var1.f0) and all the compilers I've tried agree with me. Jakub ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-26 8:05 ` Jakub Jelinek @ 2019-03-26 13:32 ` Andi Kleen 2019-04-01 16:52 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Andi Kleen @ 2019-03-26 13:32 UTC (permalink / raw) To: Jakub Jelinek; +Cc: sameeran joshi, gcc > That is a correct diagnostics. > > See Canonical loop form. > > test-expr One of the following: > var relational-op b > b relational-op var > > ( var relational-op b ) > is neither of those. Still seems strange to fail for some meaningless brackets. But ok. -Andi ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-03-26 13:32 ` Andi Kleen @ 2019-04-01 16:52 ` sameeran joshi 2019-04-01 17:02 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-04-01 16:52 UTC (permalink / raw) Cc: Jakub Jelinek, gcc On 3/26/19, Andi Kleen <ak@linux.intel.com> wrote: >> That is a correct diagnostics. >> >> See Canonical loop form. >> >> test-expr One of the following: >> var relational-op b >> b relational-op var >> >> ( var relational-op b ) >> is neither of those. > > Still seems strange to fail for some meaningless brackets. > > But ok. > > -Andi > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-04-01 16:52 ` sameeran joshi @ 2019-04-01 17:02 ` sameeran joshi 2019-04-03 16:50 ` Martin Jambor 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-04-01 17:02 UTC (permalink / raw) To: gcc; +Cc: Andi Kleen HI, Discussing the project with Andi, I have drafted a proposal, please review and suggest necessary changes. If some OpenMP experts from GCC have some ideas or changes please suggest. https://docs.google.com/document/d/1axElw-I5pTwcjI4iMle5NhLeRCcshIjH5ZHm3GGwsZU/edit?usp=sharing Thanks, Sameeran Joshi. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-04-01 17:02 ` sameeran joshi @ 2019-04-03 16:50 ` Martin Jambor 2019-04-03 17:46 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Martin Jambor @ 2019-04-03 16:50 UTC (permalink / raw) To: sameeran joshi, gcc; +Cc: Andi Kleen Hello Joshi, On Mon, Apr 01 2019, sameeran joshi wrote: > HI, > Discussing the project with Andi, I have drafted a proposal, please > review and suggest > necessary changes. > If some OpenMP experts from GCC have some ideas or changes please suggest. > > https://docs.google.com/document/d/1axElw-I5pTwcjI4iMle5NhLeRCcshIjH5ZHm3GGwsZU/edit?usp=sharing I will leave most of the evaluation on Andi (and assume he likes the proposal, by the way). However, I have read the proposal, I like it and I consider it very useful but also possibly quite ambitious. But we can adjust expectations as we go forward. Please consider fixing some of the formatting in the document, especially of the code snippets, sometimes they are a bit difficult to read without any indentation. That may be also the reason why I don't quite understand what is the relationship of omp tasks and loops or uninitialized local arrays. But apart from that it is nice, thanks for applying, Martin ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-04-03 16:50 ` Martin Jambor @ 2019-04-03 17:46 ` sameeran joshi 2019-04-04 11:20 ` Martin Jambor 0 siblings, 1 reply; 16+ messages in thread From: sameeran joshi @ 2019-04-03 17:46 UTC (permalink / raw) To: gcc; +Cc: Martin Jambor, Andi Kleen On 4/3/19, Martin Jambor <mjambor@suse.cz> wrote: > Hello Joshi, > > On Mon, Apr 01 2019, sameeran joshi wrote: >> HI, >> Discussing the project with Andi, I have drafted a proposal, please >> review and suggest >> necessary changes. >> If some OpenMP experts from GCC have some ideas or changes please >> suggest. >> >> https://docs.google.com/document/d/1axElw-I5pTwcjI4iMle5NhLeRCcshIjH5ZHm3GGwsZU/edit?usp=sharing > > I will leave most of the evaluation on Andi (and assume he likes the > proposal, by the way). However, I have read the proposal, I like it and > I consider it very useful but also possibly quite ambitious. But we can > adjust expectations as we go forward. > > Please consider fixing some of the formatting in the document, > especially of the code snippets, sometimes they are a bit difficult to Thanks for pointing it, I have indented them. > read without any indentation. That may be also the reason why I don't > quite understand what is the relationship of omp tasks and loops or > uninitialized local arrays. I have removed the location to insert for OMP tasks at 'uninitialized local array initialization'. Only kept it for as it was getting more complex. 1. while loops 2. nested for loops If you could suggest which constructs do you feel more ambitious?, so I could work on it to modify them. > > But apart from that it is nice, thanks for applying, Thanks, Sameeran Joshi > > Martin > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-04-03 17:46 ` sameeran joshi @ 2019-04-04 11:20 ` Martin Jambor 2019-04-04 13:45 ` sameeran joshi 0 siblings, 1 reply; 16+ messages in thread From: Martin Jambor @ 2019-04-04 11:20 UTC (permalink / raw) To: sameeran joshi, gcc; +Cc: Andi Kleen Hi, On Wed, Apr 03 2019, sameeran joshi wrote: > On 4/3/19, Martin Jambor <mjambor@suse.cz> wrote: >> Hello Joshi, >> >> On Mon, Apr 01 2019, sameeran joshi wrote: >>> HI, >>> Discussing the project with Andi, I have drafted a proposal, please >>> review and suggest >>> necessary changes. >>> If some OpenMP experts from GCC have some ideas or changes please >>> suggest. >>> >>> https://docs.google.com/document/d/1axElw-I5pTwcjI4iMle5NhLeRCcshIjH5ZHm3GGwsZU/edit?usp=sharing >> >> I will leave most of the evaluation on Andi (and assume he likes the >> proposal, by the way). However, I have read the proposal, I like it and >> I consider it very useful but also possibly quite ambitious. But we can >> adjust expectations as we go forward. >> >> Please consider fixing some of the formatting in the document, >> especially of the code snippets, sometimes they are a bit difficult to > > Thanks for pointing it, I have indented them. Great, it indeed is now much better. > >> read without any indentation. That may be also the reason why I don't >> quite understand what is the relationship of omp tasks and loops or >> uninitialized local arrays. > > I have removed the location to insert for OMP tasks at 'uninitialized > local array initialization'. > Only kept it for as it was getting more complex. > > 1. while loops > 2. nested for loops OK. > > If you could suggest which constructs do you feel more ambitious?, so The one thing I am quite intrigued by is how exactly you will detect data races, but I assume a conservative but useful approach will not be too difficult to devise. > I could work on it to modify them. Don't worry about the remark too much. The proposal looks solid. Martin ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] 2019-04-04 11:20 ` Martin Jambor @ 2019-04-04 13:45 ` sameeran joshi 0 siblings, 0 replies; 16+ messages in thread From: sameeran joshi @ 2019-04-04 13:45 UTC (permalink / raw) To: gcc; +Cc: Martin Jambor, Andi Kleen On 4/4/19, Martin Jambor <mjambor@suse.cz> wrote: > Hi, > > On Wed, Apr 03 2019, sameeran joshi wrote: >> On 4/3/19, Martin Jambor <mjambor@suse.cz> wrote: >>> Hello Joshi, >>> >>> On Mon, Apr 01 2019, sameeran joshi wrote: >>>> HI, >>>> Discussing the project with Andi, I have drafted a proposal, please >>>> review and suggest >>>> necessary changes. >>>> If some OpenMP experts from GCC have some ideas or changes please >>>> suggest. >>>> >>>> https://docs.google.com/document/d/1axElw-I5pTwcjI4iMle5NhLeRCcshIjH5ZHm3GGwsZU/edit?usp=sharing >>> >>> I will leave most of the evaluation on Andi (and assume he likes the >>> proposal, by the way). However, I have read the proposal, I like it and >>> I consider it very useful but also possibly quite ambitious. But we can >>> adjust expectations as we go forward. >>> >>> Please consider fixing some of the formatting in the document, >>> especially of the code snippets, sometimes they are a bit difficult to >> >> Thanks for pointing it, I have indented them. > > Great, it indeed is now much better. > >> >>> read without any indentation. That may be also the reason why I don't >>> quite understand what is the relationship of omp tasks and loops or >>> uninitialized local arrays. >> >> I have removed the location to insert for OMP tasks at 'uninitialized >> local array initialization'. >> Only kept it for as it was getting more complex. >> >> 1. while loops >> 2. nested for loops > > OK. > >> >> If you could suggest which constructs do you feel more ambitious?, so > > The one thing I am quite intrigued by is how exactly you will detect > data races, There could be 2 approaches to it: 1. Not changing the current generation grammar : How? //This is done after statement is generated and after fact analysis of statement is done, inside Statement::make_random() //whenever statement is generated, can set flag to indicate a data race in it. check_and set_data_race_in_statement(){ 1.retrieve write vars from current statement (Effect.h) 2.loop through all write vars and check is_shared(write_var) if true its a data race set some flag to indicate } This labels each statement with a flag to indicate data race in the statement, now after generation of a particular block, looping through the statements and checking whether they have a data race, if YES:(possible solutions ) 1.Neglect the block for parallel region ( may not be efficient as may neglect most of the blocks) 2. May be apply some synchronization constructs to the block. if NO race: good to go with the block to be parallel. Andi suggested "that will be hard after the fact", But I found Csmith doing some Fact analysis (DFA analysis) in it's code base. Which I didn't explore during the previous project. Will need some help to investigate about Fact Analysis (Fact*.h) which Csmith already implements, also they maintain the current generation context (CG_Context.h). As I think would solve most of the problems? 2. Decide before you generate the block that is parallel( a block would be explicitly selected as a parallel block (based on probability) ) and then avoid generating any data races. (Andi suggested ) So, how do you avoid them? e.g. LHS OP EXPRESSION Whenever a LHS variable is selected, care needs to be taken it's not a shared variable.This avoids the data races during generation itself. But this approach restricts the generation grammar, so only writes to local variables from that block will be present in the block . Will this work? The above function could now become something like Statement::check_and_avoid_data_races_in_statement() ----------------------------------------------------------------------------------------------- > but I assume a conservative but useful approach will not be > too difficult to devise. I feel the most time consuming and challenging part would be adding the first extension, i.e the ' #pragma omp parallel' pragma. Do you have more suggestions ? > >> I could work on it to modify them. > > Don't worry about the remark too much. The proposal looks solid. > > Martin > Thanks, Sameeran Joshi ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2019-04-04 13:45 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-02-10 5:45 [GSoC 2019] [extending Csmith for fuzzing OpenMp extensions] sameeran joshi 2019-02-18 18:22 ` Martin Jambor 2019-03-23 18:19 ` sameeran joshi 2019-03-24 2:55 ` Andi Kleen 2019-03-24 19:33 ` sameeran joshi 2019-03-26 0:41 ` Andi Kleen 2019-03-26 7:29 ` Jakub Jelinek 2019-03-26 8:00 ` sameeran joshi 2019-03-26 8:05 ` Jakub Jelinek 2019-03-26 13:32 ` Andi Kleen 2019-04-01 16:52 ` sameeran joshi 2019-04-01 17:02 ` sameeran joshi 2019-04-03 16:50 ` Martin Jambor 2019-04-03 17:46 ` sameeran joshi 2019-04-04 11:20 ` Martin Jambor 2019-04-04 13:45 ` sameeran joshi
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).