RFA: Fix other/44566

RFA: Fix other/44566

[Joern Rennecke]

[-- Attachment #1: Type: text/plain, Size: 2072 bytes --]

This depends on fixing PR bootstrap/44512 and PR bootstrap/44637 (for ppc),
and on the factored-out patches I posted earlier:

http://gcc.gnu.org/ml/gcc-patches/2010-06/msg02492.html
http://gcc.gnu.org/ml/gcc-patches/2010-06/msg02452.html
http://gcc.gnu.org/ml/gcc-patches/2010-06/msg02495.html
http://gcc.gnu.org/ml/gcc-patches/2010-06/msg02618.html
http://gcc.gnu.org/ml/gcc-patches/2010-06/msg02645.html

Bootstrap testing for the patch for PR bootstrap/44637 is delayed because of
issues with rs6000 test platform availability, performance, and either
machine or toolchain (java) configuration.

I have checked all the patches in order into the multi-target-20100625-branch,
and sucessfully bootstrapped and regression tested (against the  
branchpoint, trunk
revision 161392/161394) with and without --enable-build-with-cxx
on i686-pc-linux-gnu.

I have also confirmed that an i686-pc-linux-gnu X ppc-elf+spu-elf compiler
can be built using configure options:
  --target=ppc-elf --with-extra-target-list=spu-elf
and building the makefile target 'all-gcc', and that the resulting cc1
can compile a simple function for both architectures:

[amylaar@laria gcc]$ cat foo.c
int foo ()
{
   return 42;
}
[amylaar@laria gcc]$ ./cc1 -quiet -O2 foo.c
[amylaar@laria gcc]$ cat foo.s
         .file   "foo.c"
         .section        ".text"
         .align 2
         .globl foo
         .type   foo, @function
foo:
         li 3,42
         blr
         .size   foo, .-foo
         .ident  "GCC: (GNU) 4.6.0 20100625 (experimental)"
[amylaar@laria gcc]$ cat foop.c
int foo () __attribute__ ((target_arch ("spu-elf")));

int foo ()
{
   return 42;
}
[amylaar@laria gcc]$ ./cc1 -quiet -O2 foop.c
[amylaar@laria gcc]$ cat foop.s
         .file   "foop.c"
         .arch   "spu-elf"
.text
         .align  2
         .align  3
         .global foo
         .type   foo, @function
foo:
         il      $3,42
         bi      $lr
         .size   foo, .-foo
         .ident  "GCC: (GNU) 4.6.0 20100625 (experimental)"

[-- Attachment #2: ChangeLog.multi-target --]
[-- Type: text/plain, Size: 21773 bytes --]

2010-06-25  Joern Rennecke  <joern.rennecke@embecosm.com>

gcc:
	* configure.ac: When --with-extra-target-list is given, ensure
	--enable-build-with-cxx is active.
	* configure: Regenerate.
gcc/gcc:
	* multi-target.h, README-multi-target, any-lang.c: New files.
	* addresses.h: Include "multi-target.h" and add START_TARGET_SPECIFIC
	and END_TARGET_SPECIFC markers.
	* alias.c, alias.h, auto-inc-dec.c, basic-block.h: Likewise.
	* bb-reorder.c, bt-load.c, builtins.c, caller-save.c: Likewise.
	* calls.c, cfganal.c, cfgbuild.c, cfg.c, cfgcleanup.c: Likewise.
	* cfgexpand.c, cfghooks.c, cfghooks.h, cfglayout.c: Likewise.
	* cfglayout.h, cfgloopanal.c, cfgloop.h, cfgrtl.c, combine.c: Likewise.
	* combine-stack-adj.c, cse.c, cselib.c, cselib.h, dbxout.c: Likewise.
	* dbxout.h, dce.c, dce.h, ddg.c, ddg.h, df-byte-scan.c: Likewise.
	* df-core.c, df.h, df-problems.c, df-scan.c, dojump.c, dse.c: Likewise.
	* dwarf2asm.c, dwarf2asm.h, dwarf2out.c, dwarf2out.h: Likewise.
	* emit-rtl.c, except.c, except.h, explow.c, expmed.c, expr.c: Likewise.
	* expr.h, final.c, fixed-value.h, flags.h, function.c: Likewise.
	* function.h, fwprop.c, gcse.c, ggc.h, gimple.h, graph.c: Likewise.
	* haifa-sched.c, hard-reg-set.h, ifcvt.c, implicit-zee.c: Likewise.
	* init-regs.c, insn-addr.h, integrate.c, ira-build.c, ira.c: Likewise.
	* ira-color.c, ira-conflicts.c, ira-costs.c, ira-emit.c: Likewise.
	* ira.h, ira-int.h, ira-lives.c, jump.c, loop-doloop.c: Likewise.
	* loop-init.c, loop-invariant.c, loop-iv.c, loop-unroll.c: Likewise.
	* loop-unswitch.c, lower-subreg.c, machmode.h: Likewise.
	* mode-switching.c, modulo-sched.c, optabs.c, optabs.h: Likewise.
	* opts.c, output.h, passes.c, postreload.c: Likewise.
	* postreload-gcse.c, predict.c, predict.h, print-rtl.c: Likewise.
	* recog.c, recog.h, regcprop.c, reginfo.c, regmove.c: Likewise.
	* regrename.c, regset.h, regs.h, reg-stack.c, regstat.c: Likewise.
	* reload1.c, reload.c, reload.h, reorg.c, resource.c: Likewise.
	* resource.h, rtlanal.c, rtl.c, rtl.h, rtlhooks.c: Likewise.
	* rtlhooks-def.h, sched-deps.c, sched-ebb.c, sched-int.h: Likewise.
	* sched-rgn.c, sched-vis.c, sdbout.c, sel-sched.c: Likewise.
	* sel-sched-dump.c, sel-sched-dump.h, sel-sched-ir.c: Likewise.
	* sel-sched-ir.h, simplify-rtx.c, stack-ptr-mod.c, stmt.c: Likewise.
	* store-motion.c, stor-layout.c, target.h, targhooks.c: Likewise.
	* targhooks.h, toplev.c, toplev.h, tree-flow.h, tree.h: Likewise.
	* tree-pass.h, tree-ssa-address.c, tree-ssa-loop-manip.c: Likewise.
	* varasm.c, var-tracking.c, web.c, config/sh/sh.c: Likewise.
	* config/sh/sh.h, config/sh/sh-protos.h, config/spu/spu.c: Likewise.
	* config/spu/spu.h, config/rs6000/rs6000.h: Likewise.
	* config/rs6000/rs6000.c: Likewise.

	* calls.c: Add #ifndef EXTRA_TARGET / #endif directives.
	* cfgexpand.c, cfghooks.c, df-core.c, emit-rtl.c, explow.c: Likewise.
	* expr.c, function.c, gcse.c, gengtype-parse.c, optabs.c: Likewise.
	* opts.c, passes.c, reginfo.c, regstat.c, stor-layout.c: Likewise.
	* targhooks.c, toplev.c, tree-scalar-evolution.h: Likewise.
	* tree-ssa-address.c, varasm.c

	* T-extra.in: New file, broken out of:
	* Makefile.in.
	* T-extra.in: Use $(CURRENT_TARGET) prefix for names of variables
	that are set.

	* doc/extend.texi (target_arch): Document new attribute.
	* doc/tm.texi: Adjust struct machine_function documentation.

	* targhooks.c: Include diagnostic.h, tree-pass.h and
	tree-pretty-print.h.
	(default_branch_target_register_class): Remove enum reg_class from
	interface.
	(default_ira_cover_classes, default_secondary_reload): Likewise
	(default_override_options): New function.
	(default_task_ok_for_target): Likewise.
	(default_common_data_with_target): Likewise.
	(default_vectype_for_scalar_type): Likewise, using code from
	tree-vect-stmts.c:get_vectype_for_scalar_type.
	(default_addr_space_valid_pointer_mode): Use this_targetm.
	(default_addr_space_legitimate_address_p): Likewise.
	(default_target_option_valid_attribute_p): Look up name of target.
	(default_branch_target_register_class): Remove enum reg_class from
	interface.
	(default_ira_cover_classes, default_secondary_reload): Likewise.
	(default_override_options, default_vectype_for_scalar_type): Declare.
	(default_task_ok_for_target): Likewise.
	(default_common_data_with_target): Likewise.
	* regstat.c (setjmp_crosses): No longer static.  Declare extern.

	* doc/install.texi (Target specification): Document
	--with-extra-target-list.

	* genrecog.c (write_header, main): Make generated file include
	"multi-target.h" and have START_TARGET_SPECIFIC and END_TARGET_SPECIFC
	markers.
	* gengenrtl.c (genheader): Likewise.
	* genopinit (main): Likewise.
	* genoutput.c (output_prologue, main): Likewise.
	* genextract.c (print_header, main): Likewise.
	* genautomata.c (main): Likewise.
	* genemit.c (main): Likewise.
	* genpeep.c (main): Likewise.
	* genattrtab.c (main): Likewise.
	* genattr (gen_attr, main): Likewise.
	* optc-gen.awk: Likewise.
	* genmodes.c (emit_insn_modes_c_header, emit_insn_modes_c): Likewise.
	(emit_min_insn_modes_c_header, emit_min_insn_modes_c): Likewise.
	* genflags.c (main): Likewise.
	* genpreds.c (write_tm_constrs_h): Likewise.
	(write_tm_preds_h): Likewise.
	(write_insn_preds_c): Likewise.
	* opth-gen.awk: Likewise.

	* gcc/gcc/c-family/c-common.c (handle_target_arch_attribute):
	New function.
	(handle_caller_arch_attribute): Likewise.
	(c_common_attribute_table): Add target_arch and caller_arch.
	(handle_target_attribute): Call the valid_attribute_p of the
	target architecture the function belongs to.

	* gengtype.c (IS_EXTRA_TARGET): Define.
	(struct type): Move potentially narrow enums together.
	Add type_specific field.
	(string_type, scalar_nonchar, scalar_char): Update initializers.
	(ttags_file): New file-scope static variable.
	(read_input_list): For IS_EXTRA_TARGET, anticipate a [target_files]
	delimiter, to put all the target-specific files into 'language' 0.
	(new_structure, find_structure): Initialize target_specific field.
	(create_file): Update copyright notice.
	(write_rtx_next, open_base_files): Emit
	START_TARGET_SPECIFIC / END_TARGET_SPECIFIC for gtype_desc_c.
	(open_base_files): Add include of "multi-target.h" to ifiles.
	For IS_EXTRA_TARGET, create gtype-ttags.h.
	(relevant_type_p): New function.
	(write_types_process_field, output_type_enum): For IS_EXTRA_TARGET,
	augment mangling with target name.
	(write_enum_defn, write_splay_tree_allocator_def): Likewise.
	(write_types, write_local): Only perform output for relevant types.
	(write_enum_defn, write_splay_tree_allocators): Likewise.
	(write_func_for_structure): Likewise.  Remove FIXME kludge.
	(write_enum_defn): For IS_EXTRA_TARGET, place enumerators into
	separate file ttags_file, and include multi-target.h and
	../gtype-desc.h in header_file.
	(main): For IS_EXTRA_TARGET, wrap declarations / defintions in
	START_TARGET_SPECIFIC / END_TARGET_SPECIFIC.
	* mkconfig.sh <DEFINES processing>: Handle defines with parameters.
	<HEADERS processing>: Wrap config/FOO/FOO-protos.h in
	START_TARGET_SPECIFIC / END_TARGET_SPECIFIC.
	* gengtype.h (pp_nest, no_extra_nest, target_specific): Declare.
	(get_output_file_name): Likewise.
	* optabs.c [!EXTRA_TARGET] (optab_for_tree_code_array): New array.
	(optab_for_tree_code): Rename to...
	(optab_for_tree_code_1) ... this.
	* tree-ssa-loop-manip.c: Include target.h.
	(rewrite_phi_with_iv): Use targetm.sizetype_tab.
	* optabs.h: Include target.h.  Make types at start of file
	non-target- specific.
	(optab_for_tree_code_1, optab_for_tree_code_array): Declare.
	(optab_for_tree_code): Now a static inline function.
	* reload.c (push_secondary_reload): Remove enum reg_class from
	target hook interface.
	(secondary_reload_class): Likewise.
	* tree-pass.h (rtl_dispatch_pass): New struct.
	(TODO_arch_dispatch): New TODO flag.
	(pass_expand): Change type to rtl_dispatch_pass.
	* target.h (struct gimple_stmt_iterator_d): Add orward declaration.
	(enum task_type): New enum.
	(struct gcc_target): New members name, target_arch, sizetype_tab,
	optab_table, task_ok_for_target, common_data_with_target,
	copy_to_target, copy_from_target, alloc_task_on_target,
	build_call_on_target and override.
	Add new member new_arch to member asm_out.  Add new member
	vectype_for_scalar_type to member vectorize.
	Remove enum reg_class from target hook interface.
	(targetm_pnt, targetm_array, this_targetm): Declare.
	(targetm): Now a macro.
	* cfghooks.c (cfg_hooks): No longer static; shared across targets.
	* toplev.c [!EXTRA_TARGET] (targetm_array, targetm_pnt): Define and
	inititialize.
	(init_random_seed, process_options, backend_init): No longer static.
	(general_init): Add ATTRIBUTE_UNUSED to argv0.
	(process_options) [!EXTRA_TARGET]: Disable target-independent option
	processing.  Use target_option.override hook.
	(backend_init): Use EXTRA_TARGETS_CALL to backend_init on extra targets.
	(lang_dependent_init): No longer static.  Make EXTRA_TARGETS_CALL on
	initialize_sizetypes lang_dependent_init.
	[EXTRA_TARGET]: Fix up sizetype if it has wrong mode.
	(target_reinit): Do EXTRA_TARGETS_CALL to self.
	(do_compile): Do EXTRA_TARGETS_CALL to process_options.
	* toplev.h (init_random_seed): Declare.
	* tree-ssa-loop-ivopts.c (computation_cost): Use tree_expand_expr.
	(force_expr_to_var_cost): Use targetm.sizetype.
	(rewrite_use_address): Use tree_create_mem_ref.
	* gengtype-lex.l: Use extra options stack and noyy_top_state.
	(pp_nest, no_extra_nest, target_specific): New variables.
	(in_struct_comment): Remove state.
	(start_pp_directive, in_pp_directive, pp_scanning): New states.
	(*): Allow to skip #ifndef EXTRA_TARGET sections.
	(INITIAL): Recognize START_TARGET_SPECIFIC / END_TARGET_SPECIFIC
	to turn target_specific on and off.
	(TARGET_SCOPE/{EOID}): New pattern.
	* machmode.def (EXTRA_MODES_FILE): Replace with:
	("extra-modes.h")
	* expr.c (expand_expr_real_1): Use CRTL_ASM_CLOBBERS.
	[!EXTRA_TARGET] (tree_expand_expr): New function.
	(build_personality_function): Use a pointer type with the current
	targets mode.
	* expr.h (tree_expand_expr): Declare.
	* opts.c (decode_options): Use this_targetm for target-specific
	processing.  Do an EXTRA_TARGETS_CALL of self.
	* optc-gen.awk: Add an extra pass to scan for Init.
	* tree-ssa-address.c (tree_create_mem_ref): New function.
	* configure.ac (all_outputs): Add T-extra.
	(with_extra_target_list): New AC_SUBST.
	* configure: Regenerate.
	* function.c (allocate_struct_function_1): New function, broken out of:
	(allocate_struct_function).
	(free_after_compilation): Use MACHINE_FUNCTION.
	(allocate_struct_function_1): Likewise.
	(assign_parms): Use INCOMING_ARGS_INFO.
	(set_cfun): Update target_pnt.
	[!EXTRA_TARGET] (allocate_struct_function_1_array): New const array.
	(lookup_attr_target): New function, broken out of:
	(allocate_struct_function).
	[EXTRA_TARGET] (allocate_struct_function): Don't define.
	(allocate_struct_function): Initialize cfun->target_arch and set
	targetm_pnt.
	(init_emit_array): Declare.
	[!EXTRA_TARGET] (init_emit_array): Define and initialize.
	(prepare_function_start): Call the init_emit function for
	cfun->target_arch.
	(init_function_start): Do an EXTRA_TARGETS_CALL to self.
	* function.h: Include target-types / target-gtypes.h.
	(incoming_args): Use cumulative_args_u for info member.
	(INCOMING_ARGS_INFO, MACHINE_FUNCTION, CRTL_ASM_CLOBBERS): Define.
	(rtl_data): Define asm_clobbers as hard_reg_set_u.
	(struct function): Use machine_function_u for member machine.
	New member target_arch.
	* tree-vectorizer.h (target.h): Include.
	(get_vectype_for_scalar_type) Now a static incline function.
	* stor-layout.c (vector_type_mode_array): Declare.
	(vector_type_mode): Dispatch through vector_type_mode_array.
	[!EXTRA_TARGET]: (vector_type_mode_array): Define.
	* tree-affine.c (target.h): Include.
	(add_elt_to_tree): Use targetm.sizetype.
	(aff_combination_to_tree): Likewise.
	* gentargtype.cc: New file.
	* genmodes.c (struct mode_data): New member target.
	(blank_mode): Add initializer for new member.
	(struct mode_adjust): New member target.
	(target): New variable.
	(find_mode): Set target field for to-be searched mode.
	(new_mode): Set target member.
	(eq_mode): Also consider target.
	(new_adjust): Set target member.
	(drop_mode, fixup_target_modes): New functions.
	(make_complex_modes, make_vector_modes): When processing
	target-specific input, don't make modes for other targets.
	(make_complex_modes): Tag new modes with the same
	target field as the modes that they are made from.
	(make_partial_integer_mode): Make error message clearer.
	(calc_wider_mode, emit_mode_wider): Don't include modes from extra
	targets in the wider mode chain.
	(emit_insn_modes_h): Skip other target's modes in search
	For minimum/maximum modes.
	(emit_real_format_for_mode): Skip modes of other targets.
	(emit_mode_adjustments): Skip adjustments for other targets.
	(main): Call fixup_target_modes.
	* config/sh/sh.c (struct save_entry_s): Change type of mode to int.

	* bt-load.c (branch_target_load_optimize): Remove enum reg_class
	from target hook interface.
	* ggc-common.c (tm.h): Include.
	(gt_ggc_rtab, gt_ggc_deletable_rtab): Apply EXTRA_TARGETS_DECL.
	(gt_pch_cache_rtab, gt_pch_scalar_rtab): Likewise.
	(gt_ggc_cache_rtab): Likewise.
	(gt_ggc_rtabs, gt_ggc_deletable_rtabs): New arrays.
	(gt_pch_cache_rtabs, gt_pch_scalar_rtabs): Likewise.
	(gt_ggc_cache_rtabs): Likewise.
	(ggc_mark_roots, gt_pch_save): Iterate through new arrays.
	(gt_pch_restore): Likewise.
	* emit-rtl.c (label_num): No longer static.
	(const_int_htab, const_double_htab): Likewise.
	(gen_reg_rtx): Add assert.
	* cfgexpand.c (pass_expand): Change type to rtl_dispatch_pass.
	Add TODO_arch_dispatch flag and initializer for target_variants field.
        * tree-ssa-pre.c (next_expression_id): Initialize to 1.
        (eliminate): Check that lhsexpr has id > 0.
	* explow.c (hard_libcall_value): Use this_targetm.
        * common.opt (align_functions, align_jumps): Mark with AllTarget.
        (align_loops, flag_data_sections, flag_delayed_branch): Likewise.
        (flag_no_function_cse, flag_function_sections): Likewise.
        (flag_leading_underscore, flag_omit_frame_pointer): Likewise.
        (flag_rename_registers, flag_schedule_insns): Likewise.
        (flag_schedule_insns_after_reload): Likewise.
        * varasm.c (last_arch): New variable.
        (default_target_new_arch): New function.
        (assemble_start_function): Use asm_out.new_arch hook,
        and update last_arch.
	* ira.c (setup_cover_and_important_classes): Remove enum reg_class
	from target hook interface.  Use this_targetm and CRTL_ASM_CLOBBERS.
        * target-def.h (TARGET_ASM_NEW_ARCH): Define.
	(TARGET_VECTORIZE_VECTYPE_FOR_SCALAR_TYPE): Likewise.
	(TARGET_TASK_OK_FOR_TARGET, TARGET_COMMON_DATA_WITH_TARGET): Likewise.
	(TARGET_COPY_TO_TARGET, TARGET_COPY_FROM_TARGET): Likewise.
	(TARGET_ALLOC_TASK_ON_TARGET, TARGET_BUILD_CALL_ON_TARGET): Likewise.
	(TARGET_OVERRIDE_OPTIONS): Likewise.
	(TARGET_VECTORIZE): Add TARGET_VECTORIZE_VECTYPE_FOR_SCALAR_TYPE.
        (TARGET_OPTION_HOOKS): Include TARGET_OVERRIDE_OPTIONS.
	(TARGET_INITIALIZER): Initialize new members.
	* ira-costs.c (copy_cost): Remove enum reg_class from target hook
	interface.
	* rtl.h (struct function): Remove forward declaration.
	* tree-vect-stmts.c (vectorizable_store): Use targetm.optab_table.
	(vectorizable_load): Likewise.
	(get_vectype_for_scalar_type): Remove.
	* output.h (struct gcc_target): Add forward declaration.
	(default_target_new_arch): Declare.
	* reginfo.c (init_reg_sets): Do EXTRA_TARGETS_CALL to self.
	* Makefile.in (gcc_config_arguments, TA): New Makefile variables.
	(TARGET_NUM, OBJS-extra-target, EXTRA_TARGET_RULES): Likewise.
	(enumerate, empty, space, COMMON_ABI): Likewise.
	Include T-extra.
	(REGSET_H): Add multi-target.h.
	(OPTABS_H): Add $(TARGET_H).
	(GGC_H): Add s-extra-gty.
	(TREE_VECTORIZER_H): Remove duplicate.  Add target.h.
	[EXTRA_TARGET] (ALL_COMPILERFLAGS): Add -DEXTRA_TARGET.
	[EXTRA_TARGET] $(TA)%.o : %.c: New rule.
	(OBJS-rtl): New Makefile variable, broken out of...
	(OBJS-common).  Add $(OBJS-rtl).
	For each target, add members of $(OBJS-extra-target) and
	$($(TA)-out_object_file) with with target name prepended.
	(s-extra-gty, make-OBJS-rtl, $(TA)any-lang.o): New rules.
	(intl/config.intl, build/gentargtype.o): Likewise.
	(build/gentargtype$(build_exeext)): Likewise.
	[!EXTRA_TARGET]: For each extra target, evaluate EXTRA_TARGET_RULES,
	substituting the underscore-bearing / hyphened target name for
	any occurences of THIS_TARGET /  THIS-TARGET, and an ordinal number
	corresponding to an index into targetm_array for any occurences of
	THIS-TARGET-NUM.
	(MOSTLYCLEANFILES): Add $(TA) prefix to source files the object
	files for which appear in OBJS-rtl, and to insn-constants.h.
	(cs-tm.h): Add NUM_TARGETS, TARGET_NUM, EXTRA_TARGETS_EXPAND and
	TARGET_SCOPE_ to DEFINES.
	(cs-tm_p.h): Add $(tm_p_include_list) to HEADERS.
	(graph.o, c-family/c-opts.o): Prepend $(TA) to rule target basename.
	(options.o, gtype-desc.o): Likewise.
	(stor-layout.o, tree-ssa-address.o, opts.o, targhooks.o): Likewise.
	(toplev.o, passes.o, rtl.o, print-rtl.o, rtlanal.o): Likewise.
	(varasm.o, function.o, stmt.o, except.o, expr.o, dojump.o): Likewise.
	(builtins.o, calls.o, expmed.o, explow.o, optabs.o) : Likewise.
	(dbxout.o, sdbout.o, dwarf2out.o, dwarf2asm.o, vmsdbgout.o): Likewise.
	(emit-rtl.o, integrate.o, jump.o, simplify-rtx.o, cselib.o): Likewise.
	(cse.o, dce.o, dse.o, fwprop.o, web.o, implicit-zee.o): Likewise.
	(gcse.o, store-motion.o, resource.o, mode-switching.o): Likewise.
	(df-core.o, df-problems.o, df-scan.o, df-byte-scan.o): Likewise.
	(regstat.o, var-tracking.o, loop-doloop.o, auto-inc-dec.o): Likewise.
	(cfg.o, cfghooks.o, cfgexpand.o, cfgrtl.o, cfganal.o): Likewise.
	(cfgbuild.o, cfgcleanup.o, cfgloopanal.o, loop-iv.o): Likewise.
	(loop-invariant.o, loop-init.o, loop-unswitch.o): Likewise.
	(loop-unroll.o, combine.o, reginfo.o, reload.o, reload1.o): Likewise.
	(rtlhooks.o, postreload.o, postreload-gcse.o, caller-save.o): Likewise.
	(bt-load.o, reorg.o, alias.o, stack-ptr-mod.o, init-regs.o): Likewise.
	(ira-build.o, ira-costs.o, ira-conflicts.o, ira-color.o): Likewise.
	(ira-emit.o, ira-lives.o, ira.o, regmove.o): Likewise.
	(combine-stack-adj.o, ddg.o, modulo-sched.o, haifa-sched.o): Likewise.
	(sched-deps.o, sched-rgn.o, sched-ebb.o, sched-vis.o): Likewise.
	(sel-sched.o, sel-sched-dump.o, sel-sched-ir.o, final.o): Likewise.
	(recog.o, reg-stack.o, predict.o, bb-reorder.o, cfglayout.o): Likewise.
	(regcprop.o, regrename.o, ifcvt.o, lower-subreg.o): Likewise.
	($(out_object_file)), insn-attrtab.o, insn-automata.o): Likewise.
	(insn-emit.o, insn-enums.o, insn-extract.o, insn-modes.o): Likewise.
	(insn-opinit.o, insn-output.o, insn-peep.o, insn-preds.o): Likewise.
	(insn-recog.o, insn-modes.c, min-insn-modes.c, insn-preds.c): Likewise.
	(insn-emit.c, insn-recog.c, insn-extract.c, insn-output.c): Likewise.
	(insn-peep.c, insn-attrtab.c, insn-automata.c, insn-enums.c): Likewise.
	(insn-enums.c, insn-opinit.c): Likewise.
	($(simple_generated_c:insn-%.c=s-%)): Likewise.
	($(simple_rtl_generated_c:insn-%.c=s-%)): Likewise.
	($(simple_generated_c:insn-%.c=s-%)): Likewise.
	(ggc-common.o): Depend on $(TM_H).
	(tree-affine.o): Depend on $(TARGET_H).
	($(TA)targhooks.o): Depend on tree-pretty-print.h.
	($(TA)$(out_object_file)): Add compiler flags -Dtargetm and
	-DTARGET_NAME.
	($(simple_generated_c)): Add $(TA) prefix.
        (GTFILES-RAW): New makefile variable.  Use it to compute...
        (GTFILES) so that extra targets don't see *.c files the object files
        of which are not in $(OBJS-extra-target).
        [EXTRA_TARGET] (GTFILES): Bracket $(out_file) between [target_files]
        and [extra_target].
        (gtype-ttags.h): Depend on s-gtype.
	(build/%.o) [$(EXTRA_TARGET)]: Add -DEXTRA_TARGET to compiler flags.
        (build/gengtype.o): Add EXTRA_TARGETS_FILES and EXTRA_TARGET_STRING
        to BUILD_CPPFLAGS.
	Depend on extra-modes.h instead of $(extra_modes_file).
	[!TA] (extra-modes.h, target-types.h): New rules.
	[!TA] (EXTRA_OBJS): Include files from extra targets.
	* gimple.h (gimple_stmt_iterator_d): New struct tag.
	* passes.c (register_dump_files): No longer static.  Declare.
	(init_optimization_passes) [!EXTRA_TARGET]: Make EXTRA_TARGETS_CALL
	to self.
	(init_optimization_passes) [EXTRA_TARGET]: Start new pass list
	with pass_expand at &pass_expand.target_variants[TARGET_NUM-1];
	call register_dump_files for this pass list.
	[!EXTRA_TARGET] (execute_todo): Call df_finish_pass on target
	indicated by targetm.
	(execute_pass_list): Handle TODO_arch_dispatch.
	* reload1.c (emit_input_reload_insns): : Remove enum reg_class from
	target hook interface.

        * config/spu/spu-protos.h (spu_override_options): Don't declare.
        * config/spu/spu.c (spu_override_options): Now static.  Change
        signature to fit target hook.
        (TARGET_OVERRIDE_OPTIONS): Redefine.
        * config/spu/spu.h (OVERRIDE_OPTIONS): Don't define.

        * config/sh/sh.h (OVERRIDE_OPTIONS): Delete.
        * config/sh/sh.c (sh_override_options): Change type to match hook.
        Make manipulation of global flags dependent on main_target.
        (TARGET_OVERRIDE_OPTIONS): Redefine.

gcc/po:
	* EXCLUDES: Add gentargtype.cc.

[-- Attachment #3: mtgt-diff-20100625-2229.bz2 --]
[-- Type: application/x-bzip2, Size: 83908 bytes --]

Re: RFA: Fix other/44566

[Joseph S. Myers]

Could you give a general description of the approach you have taken, the 
limitations of it and the extent to which those limitations will be 
detected when GCC is built and run?  I have a long list of things I'd 
expect to be included in multi-target support, from a previous analysis I 
did, that are not included in this patch and that I don't see how this 
avoids, so I'd like the high-level overview.

How have you validated this patch?  Have you for example built a large 
body of code with two single-target compilers and for both targets of a 
two-target compiler and verified that the code is identical for both ways 
of building for a given target?

Ideally this would go in the installation, user and internals manuals, as 
applicable.  The information in install.texi in this patch seems extremely 
limited.  Random README files for bits of internals are strongly 
deprecated; put the information in README-multi-target in the internals 
manual instead.  It needs to be clear from the internals manual what 
maintainers need to do with START_TARGET_SPECIFIC, EXTRA_TARGET, 
EXTRA_TARGETS_DECL, etc., when making changes to the compiler: the 
circumstances in which they need to include or change such code so as not 
to break the invariants of multi-target configurations working and 
generating identical code when you select any one of the multiple targets 
to that generated by a single-target configuration for that target.  (If 
there isn't a clear and simple explanation with the present version of the 
code, that would indicate that more work is needed before it is ready to 
go in.)

> +Usually, the compiler is only built for a sigle target tarchitecture or

"single", "architecture"

> +E.g. to build a cross compiler with the main target ppc-elf and the

Use "@:" or comma after "E.g.".

> +all selected targets must agree in how they define @var{BITS_PER_UNIT}.

You mean @code{BITS_PER_UNIT}.

The conversion of OVERRIDE_OPTIONS to a hook definitely needs to be 
submitted first as a separate patch (though the initial conversion would 
have the hook take (void)).

Patches should not add their own ChangeLog files (ChangeLog.multi-target 
in this case).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Steven Bosscher]

On Sat, Jun 26, 2010 at 12:18 PM, Joern Rennecke <amylaar@spamcop.net> wrote:
> I have also confirmed that an i686-pc-linux-gnu X ppc-elf+spu-elf compiler
> can be built using configure options:
>  --target=ppc-elf --with-extra-target-list=spu-elf
> and building the makefile target 'all-gcc', and that the resulting cc1
> can compile a simple function for both architectures:
>
> [amylaar@laria gcc]$ cat foo.c
> int foo ()
> {
>  return 42;
> }
> [amylaar@laria gcc]$ ./cc1 -quiet -O2 foo.c
> [amylaar@laria gcc]$ cat foo.s
>        .file   "foo.c"
>        .section        ".text"
>        .align 2
>        .globl foo
>        .type   foo, @function
> foo:
>        li 3,42
>        blr
>        .size   foo, .-foo
>        .ident  "GCC: (GNU) 4.6.0 20100625 (experimental)"
> [amylaar@laria gcc]$ cat foop.c
> int foo () __attribute__ ((target_arch ("spu-elf")));
>
> int foo ()
> {
>  return 42;
> }
> [amylaar@laria gcc]$ ./cc1 -quiet -O2 foop.c
> [amylaar@laria gcc]$ cat foop.s
>        .file   "foop.c"
>        .arch   "spu-elf"
> .text
>        .align  2
>        .align  3
>        .global foo
>        .type   foo, @function
> foo:
>        il      $3,42
>        bi      $lr
>        .size   foo, .-foo
>        .ident  "GCC: (GNU) 4.6.0 20100625 (experimental)"

Also in a single file? i.e.

int foo_default_arch (void);
int foo_default_arch (void)
{
 return 42;
}

int foo_spu (void) __attribute__ ((target_arch ("spu-elf")));
int foo_spu (void)
{
 return 42;
}


> gcc/gcc:
>        * multi-target.h, README-multi-target, any-lang.c: New files.
>        * addresses.h: Include "multi-target.h" and add START_TARGET_SPECIFIC
>        and END_TARGET_SPECIFC markers.
                              ^ typo

Even though I really like the idea of supporting multi-target, this
approach looks a bit fragile to me. That is in part just a lack of
documentation, but I also think that this design just breaks easily.


Re. documentation: I didn't find any place where it is described what
should be wrapped in START_TARGET_SPECIFIC / END_TARGET_SPECIFIC. This
is important because if people don't know what to mark then new
target-specific things end up not put in the target-specific name
space. It is not obvious to me what is to be marked.

For example:
* why are create_mem_ref, addr_for_mem_ref, and maybe_fold_tmr wrapped
in START/END_TARGET_SPECIFIC in tree-flow.h, but tree_create_mem_ref
and get_address_description not?
* why is the whole df-problems.c file and a few other files target
specific? (Looks to me like these markers should be put fine-grained,
not on whole files)


Re. design breaks easily: There is no obvious way to check that all
functions that should be wrapped, are in fact wrapped. I suppose you
didn't find out what to wrap by trial and error. Do you have some
checking scripts that just list all functions that should be marked?

ISTM  it would be more robust to have a per-function-prototype marker
to signal that a function should be wrapped, and split off those
functions to separate files or a separate section in a file, instead
of wrapping the code in START/END_TARGET_SPECIFIC.

Otherwise, this patch is just waiting to be broken by ye random next
large patch (mem-ref2 merge, say) because nothing will catch missing
START/END_TARGET_SPECIFIC, reviewing is hard (you can't tell in a
patch whether a chunk is inside START/END_TARGET_SPECIFIC), and that's
all assuming people know what should be wrapped in the first place
(documentation).

Ciao!
Steven

Re: RFA: Fix other/44566

[amylaar]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> Could you give a general description of the approach you have taken, the
> limitations of it and the extent to which those limitations will be
> detected when GCC is built and run?

As stated in README-multi-target, the basic approach is to put code that
is target-specific dur to its nature and/or dependency on target macros or
types into separate namespaces.

The main target is in the default namespace, extra target use namespaces
derived from the non-canonicalized configuration name given to configure.

I have used START_TARGET_SPECIFIC and END_TARGET_SPECIFIC to denote the
start and end of extra target namespaces, because that makes it much easier
to search for the markers, it's compacter than #ifdef __cplusplus ...,
and it also should make it easier if we ever wanted to change the
underlying mechanism.

Because the delineation between target-specific and target-independent code
is at the moment not that clear drawn, it is sometimes tricky to place the
markers right.  Getting it wrong often results in errors from missing
declarations or link errors, but when you make a variable modification
target specific that should be done to global state, or vice versa, you can
get internal consistency problems that usually lead to crashes or ICEs.

> I have a long list of things I'd
> expect to be included in multi-target support, from a previous analysis I
> did, that are not included in this patch and that I don't see how this
> avoids,

You mentioned target macros in the PR; target macros are not a problem if
the are only used in code that is enclosed between  
START_TARGET_SPECIFIC / END_TARGET_SPECIFIC.  Of course that code has  
to work with the rest of the
code in a sane manner, so I had to insert manipulation of the target vector
pointer, direct access to the target vector array, and dispatching on  
depending
on the current target in a number of places.

> so I'd like the high-level overview.

I made some notes before, but then forgot to include them in the patch
posting; some were obsolete anyway, as I had factored out some code
which came with its own explanations, but here is the remainder:

I have added some comments to existing #else / #endif to denote what
ends there; without, it just gets too confusing where the
TARGET_SPECIFIC / EXTRA_TARGET markers are placed.

The patch has some shortcomings which I intentionally left in place
lest it becomes too large and tangled in controversial decisions to
ever be reviewed.  These shortcomings should only affect the presentation
of the source code and compilers that are actually built using a multi
target configuration.  I believe these isuses can be better addressed
once the basic infrastructure is in place.

  In order to get declarations into the right scope for TARGET_SEPCIFIC /
  EXTRA_TARGET code, I have moved around a few lines, but in general I have
  preferred to put in more markers - that results in smaller diffs and is
  thus easier to merge forward and review.  I think it would be useful to
  group things -including function definitions - better together in the
  future, but that could only be done in small chunks - like one file at
  a time.  Trying to do that in a global patch is not feasible.
  A few header files (e.g. rtl.h, basic-block.h) alleged that functions
  live in source files where they are not; in these cases, I've moved
  things arounf a bit more.  You can verify that not much changed in
  basic-block.h by piping the unpatched and patched versions through
  sort and comparing the results.

  CUMMULATIVE_ARGS should really be not in the target hooks interface.
  So far I have skirted around the issue by placing less of tm.c inside
  TARGET_SPECIFIC blocks that what really should be - basically all but
  general system/gcc includes is target specific.
  This involves some extra work for each port that is supposed to work
  as an extra (not main) target.
  It also means that multi-target configurations end up lying to the
  compiler about the types of the pointers that are passed to the hooks.
  I think we should ultimately change the interface how ports handle
  CUMULATIVE_ARGS.  Ironically, target macros used in default hook
  definitions are OK, but target-provided hooks would need to change.
  However, we need to have a consensus first what the new interface
  should be.

  Related to the CUMULATIVE_ARGS issue is the general issue of target-variant
  types and how they are handled by gengtype.
  E.g. CUMULATIVE_TYPES, struct machine_function.
  (Target-defined types that only coincidentally share names are not a
   problem, they can live in different name spaces; the problem arises
   when general gcc types reference or include a target-variant type.)
  I used some preprocessing and namespace kludges to produce target-types and
  target-gtypes.h, which allow multi-target comilers to be built, but
  multi-target configuration present a program to the build compiler
  that is not strictly conforming, and the garbage collection is not safe
  if target-variant types point to garbage-collection sensitive data,
  as the different targets can get mixed up.

  I have stripped the higher level optimizations out of the patch.  They
  made up only some 10% of the size (of the unsplitted patch, some 600+ KB),
  but added considerable complexity,
  and suffered from a somewhat experimental design and bit-rot.
  To keep these patches up-to-date would not only have required tracking
  the build system and the C++ saga, but also the gimple optimizer
  infrastructure, dataflow machinery, rtl optimizers, autoparallelization
  and autovectorization.
  I have left some basic support infrastructure in place that is not
  immediately useful with these high-level optimizers, but which I believe
  is sound in design and can be put back into use when the high level
  optimizers are added back, or new ones are created to fill their place.
  You can have a look at the old high-level optimizers in the milepost
  branches.

> How have you validated this patch?  Have you for example built a large
> body of code with two single-target compilers and for both targets of a
> two-target compiler and verified that the code is identical for both ways
> of building for a given target?

No, that is not even intended to work.  The idea is that you have a main
target, and code compiled for this should be the same as if it was the  
only target, unless specific optimizations are enabled that shift code  
from one
target to the other.
Because the main target governs things like type sizes, the other targets
are in general not even expected to work as if they were the sole target.

> Ideally this would go in the installation, user and internals manuals, as
> applicable.  The information in install.texi in this patch seems extremely
> limited.  Random README files for bits of internals are strongly
> deprecated; put the information in README-multi-target in the internals
> manual instead.  It needs to be clear from the internals manual what
> maintainers need to do with START_TARGET_SPECIFIC, EXTRA_TARGET,
> EXTRA_TARGETS_DECL, etc., when making changes to the compiler: the
> circumstances in which they need to include or change such code so as not
> to break the invariants of multi-target configurations working and
> generating identical code when you select any one of the multiple targets
> to that generated by a single-target configuration for that target.

I agree that such detailed documentation would be good to have (except that
as stated above, not all the targets are really expected to work the same
as a single-target configuration).
However, that is significant extra work, and I doubt that I'll find time to
do that, because I have to look for work that someone is willing to pay for
in the meantime.

I don't think we are at the stage we really have to worry about detailed
installation and user manual yet, anyway.  For the multi-target work to
be relevant to end users, we need high-level optimizers, so at the moment,
what is important is documentation that allows the multi-target
functionality to be improved.

>  (If
> there isn't a clear and simple explanation with the present version of the
> code, that would indicate that more work is needed before it is ready to
> go in.)

As noted above, globally rearranging all the GCC code to get a better
delineation between target-specific and target independent code, and thus
also reduce the number of START_TARGET_SPECIFIC / END_TARGET_SPECIFIC
markers, would make the patch impossible to maintain or review.  That
has to be done piecemeal as follow-on work.

> The conversion of OVERRIDE_OPTIONS to a hook definitely needs to be
> submitted first as a separate patch (though the initial conversion would
> have the hook take (void)).

Ok, I'll be working on a patch for that shortly.

Thanks for the documentation review.  Do you think the stuff from
README-multi-target fits in internals.texi?
Or should it go in a new doc/*.texi file?

> Patches should not add their own ChangeLog files (ChangeLog.multi-target
> in this case).

Sorry, I forgot to edit that out; it has its place in the branch, but not
as a patch to mainline.

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sat, 26 Jun 2010, amylaar@spamcop.net wrote:

> In order to get declarations into the right scope for TARGET_SEPCIFIC /
> EXTRA_TARGET code, I have moved around a few lines, but in general I have
> preferred to put in more markers - that results in smaller diffs and is
> thus easier to merge forward and review.  I think it would be useful to
> group things -including function definitions - better together in the
> future, but that could only be done in small chunks - like one file at
> a time.  Trying to do that in a global patch is not feasible.

Such rearrangements should come *first* as preparatory patches (many small 
patches rather than one large patch).

> > How have you validated this patch?  Have you for example built a large
> > body of code with two single-target compilers and for both targets of a
> > two-target compiler and verified that the code is identical for both ways
> > of building for a given target?
> 
> No, that is not even intended to work.  The idea is that you have a main
> target, and code compiled for this should be the same as if it was the only
> target, unless specific optimizations are enabled that shift code from one
> target to the other.
> Because the main target governs things like type sizes, the other targets
> are in general not even expected to work as if they were the sole target.

The closer they work to how they work as the sole target, the less likely 
multi-target configurations are to break when people change one of the 
targets without testing multi-target.  So you should clearly delineate and 
document the ways in which multi-target configurations will behave 
differently from one of the targets as sole target.  For example, you 
might enumerate every target macro used in parts of the compiler that are 
built only once, and list them in groups with each groups marked with the 
aspect of compiler behavior that will differ in multi-target 
configurations (type sizes, say).

That exercise of listing target macros (including undocumented target 
macros) will also identify cases where such a behavior difference is 
inappropriate.  For example, tree-inline.c uses MOVE_RATIO and 
MOVE_MAX_PIECES.  I see no good reason for the secondary target to use 
those settings from the main target.  So you should convert those macros 
to hooks (as separate, preparatory patches).  (Making tree-inline.c 
target-specific would be a worse approach; it's much better to have a 
simple rule about which files are target-specific, e.g. only RTL passes, 
than to have some files randomly needing to be target-specific because of 
target macro uses hidden in the middle of the file.  Certainly, if 
anything not obviously fitting a general rule is marked target-specific, 
there should be comments explaining why, just like the comments on various 
#include directives explaining why some files include headers that it 
seems they shouldn't need to include.)

I expect there are many other such cases; you should show how you have 
searched for and identified which files are using target macros so that 
reviewers can effectively verify that all files that should be marked 
target-specific are so marked.

You also need to explain how the assembler and linker fit in your model, 
given that the GNU assembler is rather further from multi-target than GCC 
at present (working entirely with macros rather than any sort of vector of 
functions) and that ELF doesn't have the concept of a single object file 
with code for multiple architectures.

> > Ideally this would go in the installation, user and internals manuals, as
> > applicable.  The information in install.texi in this patch seems extremely
> > limited.  Random README files for bits of internals are strongly
> > deprecated; put the information in README-multi-target in the internals
> > manual instead.  It needs to be clear from the internals manual what
> > maintainers need to do with START_TARGET_SPECIFIC, EXTRA_TARGET,
> > EXTRA_TARGETS_DECL, etc., when making changes to the compiler: the
> > circumstances in which they need to include or change such code so as not
> > to break the invariants of multi-target configurations working and
> > generating identical code when you select any one of the multiple targets
> > to that generated by a single-target configuration for that target.
> 
> I agree that such detailed documentation would be good to have (except that
> as stated above, not all the targets are really expected to work the same
> as a single-target configuration).
> However, that is significant extra work, and I doubt that I'll find time to
> do that, because I have to look for work that someone is willing to pay for
> in the meantime.

If a feature is going to be problematic to maintain in future, that 
generally indicates it should not be accepted on trunk.  That is, 
sufficient documentation is needed for other people to keep the feature 
working as they make unrelated changes, and the feature needs to be 
designed and implemented in a way that makes this no more complicated than 
necessary.

> As noted above, globally rearranging all the GCC code to get a better
> delineation between target-specific and target independent code, and thus
> also reduce the number of START_TARGET_SPECIFIC / END_TARGET_SPECIFIC
> markers, would make the patch impossible to maintain or review.  That
> has to be done piecemeal as follow-on work.

No, as preparatory work, not follow-on.  A patch where whole files are 
either all target-specific or have nothing target-specific is far more 
reviewable, and creates a far more maintainable source tree, than one with 
markers scattered in the middle of files.

The natural model I think of is the unified Cell debugger support in GDB.  
The feature is similar - support for debugging programs where some call 
frames are on one architecture and some on another.  The patch submissions 
included *many* patch series each of which had *many* patches in it.  It's 
not that it's good in itself to have large numbers of patches - it's that 
a lot of preparation is needed to get the source tree in a state such that 
such a feature fits in in a clean and maintainable way, and such 
preparation naturally splits up into a large number of pieces each of 
which is an improvement to the source tree in its own right.  I think the 
unified Cell debugger patch submissions are an excellent example to try to 
emulate.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting Steven Bosscher <stevenb.gcc@gmail.com>:

> Also in a single file? i.e.

Yes, I've put this into 2f.c and compiled with -O2:

         .file   "2f.c"
         .section        ".text"
         .align 2
         .globl foo_default_arch
         .type   foo_default_arch, @function
foo_default_arch:
         li 3,42
         blr
         .size   foo_default_arch, .-foo_default_arch
         .arch   "spu-elf"
.text
         .align  2
         .align  3
         .global foo_spu
         .type   foo_spu, @function
foo_spu:
         il      $3,42
         bi      $lr
         .size   foo_spu, .-foo_spu
         .ident  "GCC: (GNU) 4.6.0 20100625 (experimental)"

Note, there is a target hooks for the architecture switch of the output,
so if you want, you can put the output in separate files instead.

> Re. documentation: I didn't find any place where it is described what
> should be wrapped in START_TARGET_SPECIFIC / END_TARGET_SPECIFIC. This
> is important because if people don't know what to mark then new
> target-specific things end up not put in the target-specific name
> space. It is not obvious to me what is to be marked.

OK, we can do a bit of Q&A to get some material for our documentation.

> For example:
> * why are create_mem_ref, addr_for_mem_ref, and maybe_fold_tmr wrapped
> in START/END_TARGET_SPECIFIC in tree-flow.h, but tree_create_mem_ref
> and get_address_description not?

As a general rule, if a function uses a target specific variable or macro,
or another function which is itself target specific, it too is target
specific.

create_mem_ref uses sizetype and create_mem_ref_raw.  We could get the
right sizetype from the target vector, but create_mem_ref_raw uses
valid_mem_ref_p, which is intrinsically target specific.

addr_for_mem_ref uses LAST_VIRTUAL_REGISTER, thus it is target specific.

create_mem_ref_raw uses create_mem_ref_raw, thus it is target specific.

tree_create_mem_ref is a dispatching function, thus providing an interface
for target-independent code to call the appropriate create_mem_ref function.
There is nothing target specific about get_address_description.

> * why is the whole df-problems.c file and a few other files target
> specific? (Looks to me like these markers should be put fine-grained,
> not on whole files)
>
>
> Re. design breaks easily: There is no obvious way to check that all
> functions that should be wrapped, are in fact wrapped.

This is why I like to make entire files target specific.  If the general
theme of the file is to handle rtl, every function is suspect to be
target specific.  Even if not now, a small change might make it so
tomorrow.

> I suppose you
> didn't find out what to wrap by trial and error. Do you have some
> checking scripts that just list all functions that should be marked?

I've started out with making everything dealing with rtl target specific,
then searched for incidence of rtl handling functions and macros in the tree
files.  TDEP-files was sort of a to-do list in the milepost branches.
Some more tricky files I went over function-by-function.
Some functions had to be pulled back into target-independent territory with
target vector references or dispatchers to keep the bulk of the tree
optimizers target independent.

> ISTM  it would be more robust to have a per-function-prototype marker
> to signal that a function should be wrapped,

I don't see why; but I'd prefer the target specific prototypes to be  
all grouped together.

> and split off those
> functions to separate files or a separate section in a file, instead
> of wrapping the code in START/END_TARGET_SPECIFIC.

Agreed, but as I mentioned in my previous post, the resultant changes are
too big to be merged or reviewed together.
Once the basic infrastructure is in, we can move things around with patches
that have the sole objective of grouping target-specific and  
target-independent stuff.

> Otherwise, this patch is just waiting to be broken by ye random next
> large patch (mem-ref2 merge, say) because nothing will catch missing
> START/END_TARGET_SPECIFIC, reviewing is hard (you can't tell in a
> patch whether a chunk is inside START/END_TARGET_SPECIFIC), and that's
> all assuming people know what should be wrapped in the first place
> (documentation).

Actually, be rather erring on putting stuff into the TARGET_SPECIFIC bucket
if they conceptually look like they might, I've seen little or no problems
with existing functions changing their status; but what I've seem are
textual merge problems, functions switching files, functions deleted and
added, that require manual attention during merge.

Re: RFA: Fix other/44566

[Joseph S. Myers]

As I noted, some time ago I did an analysis of what would actually need to 
change for a proper multi-target toolchain, and of the issues that would 
arise.  Here are comments based on that analysis.

I should point out that the model I was thinking of in that analysis (of 
the features supported by the multi-target toolchain) was somewhat 
different from yours; the two overlap in what changes are needed, but each 
model involves significant changes not needed by the other model.  Thus, 
for several issues I identify, in your model they may well be less of an 
issue because they relate to things you explicitly do not expect to work.  
Nevertheless, I think they are relevant to understanding what the 
limitations of any proposed approach are.

The model I has was that all the targets supported by the compiler are 
essentially equal.  It would be possible to run the compiler - or other 
components such as the assembler - with a --target= option to select a 
particular target, and the results would be identical to those of a 
single-target compiler for that target.  Some method would be added for 
configure options to be specified to apply to a single target only.  (Your 
patch appears to discard all configure options when running sub-configures 
for secondary targets, which doesn't seem right either.)  Thus each target 
could have its own search paths for libraries and headers, for example, 
and its own set of multilibs.  Runtime libraries would be configured (for 
each multilib, for each target) with the appropriate --host for that 
target, and when the testsuite is run DejaGnu would be told it was run for 
one of the targets involved, so hardcoded target triplet names in those 
places would not need to change.  Can the testsuite be run at all for 
secondary targets with your implementation?

My model did not include the possibility of a single compilation using 
more than one target in a single source file.

We estimated in CodeSourcery that this would take on the order of two 
person-years to implement.

I did not think of your approach of using namespaces and compiling some 
source files many times, which avoids some of the changes I thought 
necessary but still requires a reliable way to identify target macros and 
where they are used so you can tell what files need to be built multiple 
times.  I was supposing each file to be built once, and there to be no 
tm.h at all included in most files, except maybe for a few files built 
once per target that use headers such as tm.h solely to define 
initializers for a target vector initialized in those files.

(When I refer to target macros, macros in config.in that are defined by 
configure, and might be defined differently for different targets, are 
subject to exactly the same issues of needing to depend on the target and 
needing to identify which files use them.)

Changes I expected to be needed
-------------------------------

* Completing the move of every last target macro into the target 
structure.  The intermediate conversion step of functions in targhooks.c 
that call target macros is not enough.

* Likewise in some way for all the target macros specific to a given CPU 
target (e.g. those local to config/rs6000).

* All target macros used in the driver gcc.c (etc.) must move into a 
separate target structure (that doesn't presently exist) in the driver 
program.  (This covers many spec strings, for example.)  Other programs 
such as collect2 also need their own target structures.

* libgcc must stop using target macros from tm.h; it can use macros 
predefined by the compiler instead, but every target macro use in libgcc 
needs tracking down and removing.  As I was envisaging the gcc/ directory 
being configured only once, properly completing the toplevel libgcc 
transition is indicated.

* Likewise for other runtime libraries - at least libobjc.

* Generated data and functions used in the driver, such as multilib data 
and command-line option information, must move into the target structure.

* Generated data and functions used in the compiler proper must move into 
the target structure.  This includes the gen_* functions for individual 
instructions, for which there are very many calls hardcoded throughout the 
compiler: some sort of renaming is needed to allow multiple targets which 
get different generated function implementations to be used.

* There must be configure syntax to make all the configure options depend 
on the particular target for which the compiler is run, instead of being 
global.

* Similar issues arise elsewhere.  The assembler has no beginnings of a 
target structure like that in GCC.  The linker, BFD and GDB are much 
better in this regard but BFD still has quite a few #ifdef cases (I 
haven't checked whether they relate to code built multiple times or 
whether they are actually harmful to proper all-target support).

Differences with your approach
------------------------------

With your approach, you define some specific areas where secondary targets 
will not behave like primary targets.  Thus header search paths do not 
need defining for secondary targets, since headers and predefined macros 
will always be based on the primary target, and secondary targets are not 
expected to get their own type sizes.  You would also appear to have the 
driver's behavior based on the primary target only.

I would however expect functions built for secondary targets still to need 
libgcc for some purposes, so you would need to work out how it is to be 
built, where it is to be installed and how it is to be found at runtime 
(for each secondary target multilib) ... with additional complications 
relating to functions for different targets being combined in a single 
object.

I noted the issues of target macros used within a config/$arch directory 
to control compilation of code within that directory.  Do you treat every 
source file in such a directory as target-specific, so they are built 
multiple times if there are multiple targets for that architecture?  (For 
example, Power targets that do not include e500.h have e500 support 
disabled at compile time.  If configured with target powerpc-linux and 
extra target powerpc-eabispe, would two copies of the config/rs6000 files 
be built, and two copies of all the generated insn-* files, the copies for 
powerpc-linux having e500 support disabled and those for powerpc-eabispe 
having it present?)

By building files multiple times you avoid dealing with *some* target 
macros - but my claim is that it needs to be a well-defined set of files 
that are built multiple times, and all macros used outside that set of 
files either need to be defined not to matter for clear reasons, or need 
to be converted to hooks.

I noted issues with the intermediate conversion of macros to hooks via 
default definitions in targhooks.c that call a macro for unconverted 
targets.  Your patch places a lot of conditionals / target-specific 
markers in targhooks.c.  It might be significantly simplified if you 
completed the transition for all the partly-transitioned macros....

I mentioned command-line options.  This is an area where I would think 
your approach makes things much *more* complicated since you need to deal 
with option settings on a per-target basis (different targets change the 
settings of target-independent optimization flags in different ways, for 
example, even if you don't actually allow command-line options to be 
marked as applying to an individual target or allow "target" attributes to 
be used in conjunction with your per-architecture functions.

As you will have seen I've been working on option handling machinery 
lately with a view to changes to how multilib selection works.  Your patch 
seems to insert a lot of conditionals in option-handling code (and 
inserting a lot of conditionals in *any* file generally seems a bad idea).  
Part of the model I envisage implementing is that option-handling hooks 
will set not global variables but elements of an options structure and 
that the compiler can contain multiple such structures.  I'd think that 
multi-target option handling should be a lot easier after those changes 
are in place, since you would have separate structures (including both 
target-independent and target-dependent options) for each target and hooks 
could set target-independent options in a target-specific copy of the 
structure.  (Properly, you'd then change all references to global options 
to explicitly look at current-function options instead; the quick hack 
would be to swap options structures when changing target architecture.)

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> I should point out that the model I was thinking of in that analysis (of
> the features supported by the multi-target toolchain) was somewhat
> different from yours; the two overlap in what changes are needed, but each
> model involves significant changes not needed by the other model.  Thus,
> for several issues I identify, in your model they may well be less of an
> issue because they relate to things you explicitly do not expect to work.
> Nevertheless, I think they are relevant to understanding what the
> limitations of any proposed approach are.

I had a limited set of requirements, but I think my approach can be readily
extended to cover your requirements as well.

Instead of having just one main target and a set of secondary targets, we'd
have a set of targets, some or all of which would be eligible as main
targets, and some or none of which would be eligible as secondary targets -
and not every secondary target would be necessarily compatible with
all the main targets.

All the !EXTRA_TARGET code (outside of the Makefile) would probably have
to be hookized or dispatched to or something similar.

> The model I has was that all the targets supported by the compiler are
> essentially equal.  It would be possible to run the compiler - or other
> components such as the assembler - with a --target= option to select a
> particular target, and the results would be identical to those of a
> single-target compiler for that target.

Yes, that's also a useful goal.  Of course, during compilation of a specific
module, you would have one target which is in charge - in your model, it
would be the only one active.

One thing you haven't mentioned that I'd need to do to accomodate this
would be to remove the special significance of targetm_array[0] - instead
there would have to be a variable that is an index in targetm_array to
indicate the currently active main target.

> Some method would be added for
> configure options to be specified to apply to a single target only.

Sounds sensible.

> (Your
> patch appears to discard all configure options when running sub-configures
> for secondary targets, which doesn't seem right either.)

Well, I just had no need for options passed to sub-configures.  So although
the generated Makefile has the gcc_config_arguments variable, I ended up
not making any use of it.
It's just a matter of defining a syntax and implementing it.

>  Thus each target
> could have its own search paths for libraries and headers, for example,
> and its own set of multilibs.  Runtime libraries would be configured (for
> each multilib, for each target) with the appropriate --host for that
> target, and when the testsuite is run DejaGnu would be told it was run for
> one of the targets involved, so hardcoded target triplet names in those
> places would not need to change.

Sounds sensible.  I think that'll be mostly a matter of adding some more
stanzas to EXTRA_TARGET_RULES.

> Can the testsuite be run at all for
> secondary targets with your implementation?

No, I didn't add a command line option to switch the target.  And although
basic libraries (basically, libgcc) for all targets were planned I didn't get
around to implement it, and besides, program startup was supposed to be
strictly controlled from the main target.
The original work done for Milepost targeted the ARCompact / mxp
architectures; IIRC the mxp had something like 32K words of instruction
memory, so you wouldn't want to put too much there anyway.

> (When I refer to target macros, macros in config.in that are defined by
> configure, and might be defined differently for different targets, are
> subject to exactly the same issues of needing to depend on the target and
> needing to identify which files use them.)

Well, they already depend on the target, as configure gets run in the various
EXTRA_TARGETS subdirectories.  But you do have to make sure they are only
used in places that are bracketed bewteen START_TARGET_SPECIFIC and
END_TARGET_SPECIFIC.
And I agree that getting these regions to file granularity is likely to
make maintenance easier.

We can use grep to get a list of all tm.h and config.in defines, and feed
that into another grep to find affected files - but than it'll get  
tedious, deciding which macros we can ignore because we don't really  
require any
of the targets that use them in a multi-target configuration (document these
targets), and fix up all the remaining files.

> Changes I expected to be needed
> -------------------------------
>
> * Completing the move of every last target macro into the target
> structure.  The intermediate conversion step of functions in targhooks.c
> that call target macros is not enough.

By having targhooks.c compiled multiple times in different namespaces,
keeping the old macros generally suits is fine; we can change the way
the target macros are used without having to patch every single hook in
for every single target.

> * Likewise in some way for all the target macros specific to a given CPU
> target (e.g. those local to config/rs6000).
>
> * All target macros used in the driver gcc.c (etc.) must move into a
> separate target structure (that doesn't presently exist) in the driver
> program.  (This covers many spec strings, for example.)  Other programs
> such as collect2 also need their own target structures.

We could also do the multile-namespaces trick with gcc.c, although it
will probably be cleaner to factor out the target dependencies; we can just
move them to a new file a' la targhooks.c , to convert target macros to
driver target hooks / driver target vector data items, without having to
patch every target; of course, targets can then start to define the driver
target vector components instead of the old target macros if we see any
benefit in that.

> * libgcc must stop using target macros from tm.h;

I don't see why this is necessary, as long as you make sure each target's
libgcc sees the right tm.h .  I would think that will come automatic from
the build directory tree structure.

> * Generated data and functions used in the compiler proper must move into
> the target structure.  This includes the gen_* functions for individual
> instructions, for which there are very many calls hardcoded throughout the
> compiler: some sort of renaming is needed to allow multiple targets which
> get different generated function implementations to be used.

I've put the optab table in the target vector.
Any code that uses directly gen_* rtx functions needs to be marked target
specific, or weaned from the direct gen_* use.

> * There must be configure syntax to make all the configure options depend
> on the particular target for which the compiler is run, instead of being
> global.

Yes, for a set of equal targets, that is needed.  It was a complete non-issue
for ARCompact/mxp because the mxp compiler is really not very useful on its
own because of the severe limitations on the code and data sizes, and both
targets were maintained by me anyway.  They lived in different config
directories because they were just way too different (CISCy RISC vs. VLIW,
small general-purpose register file with hand-tuned register classes
vs. large vector-only register file with vector CC0-like flags and  
vector accumulator and thousands of auto-generated register classes).

Once a configure syntax is agreed, a few lines of lispy GNU make stanzas
will probably take care of this.

> * Similar issues arise elsewhere.  The assembler has no beginnings of a
> target structure like that in GCC.

That was also a non-issue for ARCompact/mxp because an assembler
existed which supported both targets.  Well, sort of, the support wasn't
quite what we wanted it to be, but it was a unified assembler from the
start.

I did anticipate that some target combinations might have problems with that,
which is why the documentation for the hook to switch output targets says
that it may switch output files.
So if your could write your Makefiles so that it expects multiple *.s files
which have to be compiled with different assemblers, that'll work.
Or if the Makefile rewriting pain is too much, you might have an assembler
wrapper script that takes a single multi-target assembly file, separate
it into single-target *.s files, run the individual assemblers, packages the
results up again into combined 'object' files, and do a similar trick with
the linker, possibly also involving objcopy to get the secondary target
partially linked files in appropriate sections of the executable.

Of course, if you can find time/manpower to clean up / rewrite gas to
handle this all nicely internally, more power to you.

> Differences with your approach
> ------------------------------
>
> With your approach, you define some specific areas where secondary targets
> will not behave like primary targets.  Thus header search paths do not
> need defining for secondary targets, since headers and predefined macros
> will always be based on the primary target, and secondary targets are not
> expected to get their own type sizes.  You would also appear to have the
> driver's behavior based on the primary target only.

Yet, but that's only the implementation, not a constraint dictated by the
basic design, and it's a direct result of the requirements.
We wanted (and succeeded at making) machine-learning decide when to
move a loop from the main target to a secondary target so that its
auto-vectorization on the secondary target would save more time than the
overhead of moving the data and control around cost.

Changing the driver to support a set of multiple alternative main targets
can follow the same pattern as libbackend.a of using a target vector and
target hooks file.

> I would however expect functions built for secondary targets still to need
> libgcc for some purposes, so you would need to work out how it is to be
> built, where it is to be installed and how it is to be found at runtime
> (for each secondary target multilib) ... with additional complications
> relating to functions for different targets being combined in a single
> object.

It depends on what transformations you allow if libgcc is actually needed.
For the milepost project we avoided the issue by only shifting code to the
mxp that didn't need libgcc.  Well, a lot of libgcc functions, when considered
with their dependencies, wouldn't have fit on the hardware anyway.

But in general, I'd agree that it would be desirable that libgcc can be build
for every target.
And of course it is a requirement for a target that is supposed to be able
to become a main target, like in your model all are.

> I noted the issues of target macros used within a config/$arch directory
> to control compilation of code within that directory.  Do you treat every
> source file in such a directory as target-specific, so they are built
> multiple times if there are multiple targets for that architecture?  (For
> example, Power targets that do not include e500.h have e500 support
> disabled at compile time.  If configured with target powerpc-linux and
> extra target powerpc-eabispe, would two copies of the config/rs6000 files
> be built, and two copies of all the generated insn-* files, the copies for
> powerpc-linux having e500 support disabled and those for powerpc-eabispe
> having it present?)

Yes.

In general, you'd be expected to have one config directory supply only
one target, for efficiency reasons if for nothing else.

But in principle, you can combine target triplets that map to the same
config dir, and they will map to different namespaces, according to the
target triples (or pairs, or quadruplets) that you supply to configure.
The main target gets the global namespace, while extra targets get a
namespace named after the configure target name.

E.g. if you configure with --target=powerpc64-hurd
--with-extra-targets-list='powerpc-linux powerpc-eabispe',
powerpc64-hurd code will live in the global namespace,  powerpc-linux will
live in the powerpc_linux namespace, and powerpc-eabispe code will live in
the powerpc_eabispe namespace.

FWIW I didn't use the canonicalized names because I though that would make
mangled identifiers unnecessarily long.  But of course you could also play
games with this by making a target appear multiple times with different
non-canonical names.  And the name is also part of the target vector, so
you could make them behave slightly differently, e.g. have different
settings in OPTIMIZATION_OPTIONS.  The name of the target for target-specific
code is this_targetm.name, whereas the dynammically chosen target is
targetm_pnt->name.  In most files targetm.name is the same as the latter,
but in tm.c it is the same as the former.

> By building files multiple times you avoid dealing with *some* target
> macros - but my claim is that it needs to be a well-defined set of files
> that are built multiple times, and all macros used outside that set of
> files either need to be defined not to matter for clear reasons, or need
> to be converted to hooks.

Yes, but that hook conversion can take the form of creating a new file that
is built multiple times and uses the existing amcros to provide the hooks
for each target.

> I noted issues with the intermediate conversion of macros to hooks via
> default definitions in targhooks.c that call a macro for unconverted
> targets.  Your patch places a lot of conditionals / target-specific
> markers in targhooks.c.  It might be significantly simplified if you
> completed the transition for all the partly-transitioned macros....

Well, we could just separate the target-specific and the non-target specific
hooks to reduce the number of these markers.  Or we could change non-target
specific hooks to pro-forma target specific hooks - there are a number that
just return a constant.

> I mentioned command-line options.  This is an area where I would think
> your approach makes things much *more* complicated since you need to deal
> with option settings on a per-target basis (different targets change the
> settings of target-independent optimization flags in different ways, for
> example, even if you don't actually allow command-line options to be
> marked as applying to an individual target or allow "target" attributes to
> be used in conjunction with your per-architecture functions.

Yes, part of the problem is that it's so hard to define it properly.
E.g. a user might want to have delay branch scheduling for one target
but not the other.  OTOH (s)he might not even think of the possibility
and get exasperated when -f{no-,}delayed-branch won't work on all
functions as expected.
Conceptionally it would be simpler if all options are passed to all targets,
but all options variables are target-specific, so that one target doesn't see
the other's change.
However, that wouldn't make semantic sense.  E.g. the SH defaults to
flag_finite_math_only (because the hard FP comparison instruction set was
designed so that it's expensive to do -fno-flag_finite_math_only), while
most other targets default to -fflag_finite_math_only.
If your main target is an i386 and you have sh4 as a secondary target, you
wouldn't want optimizations to randomly disregard the finer points of
NaNs in comparisons.
So at least the flags that potentially change program behaviour must be
global, since they represent the user's choices in the context of the
selected main target.

> As you will have seen I've been working on option handling machinery
> lately with a view to changes to how multilib selection works.  Your patch
> seems to insert a lot of conditionals in option-handling code (and
> inserting a lot of conditionals in *any* file generally seems a bad idea).

On the plus side, the changes to override_options are only needed on targets
that are expected to function as seconadry targets.  I mainly used the SH as
secondary target in early tests because it was easy for me to hack; I'm not
sure if anyone would really want to use it this way.
Targets that would be natural targets as secondary targets are mxp in
ARCompact/mxp, spu in Cell, and GPUs in CPU/GPU combinations.

> Part of the model I envisage implementing is that option-handling hooks
> will set not global variables but elements of an options structure and
> that the compiler can contain multiple such structures.  I'd think that
> multi-target option handling should be a lot easier after those changes
> are in place, since you would have separate structures (including both
> target-independent and target-dependent options) for each target and hooks
> could set target-independent options in a target-specific copy of the
> structure.

This still doesn't solve the problem that the program should have one set
of semantics, irrespective of what optimizers decide to do.

What would be helpful would be if your project could help us to get a better
grasp on what options are actually tuning code generation for a single
target (even though nominally target-independent), which ones  
represent a speed/size/compile time/debug precision tradeoff choice  
that the user wants
to make on all targets, and which ones actually influence the semantics of
the program.

>  (Properly, you'd then change all references to global options
> to explicitly look at current-function options instead; the quick hack
> would be to swap options structures when changing target architecture.)

No, then you'd swap semantics.  That might be acceptable if the user
selected the target on the command line, but not if a function was moved
to a different target by the optimizer.

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting Joern Rennecke <amylaar@spamcop.net>:

> I had a limited set of requirements, but I think my approach can be readily
> extended to cover your requirements as well.

P.S.: I was thinking here of places like toplev.c and backend.c, not of
tree files that make the odd excursion into rtl-land.

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sat, 26 Jun 2010, Joern Rennecke wrote:

> > Changes I expected to be needed
> > -------------------------------
> > 
> > * Completing the move of every last target macro into the target
> > structure.  The intermediate conversion step of functions in targhooks.c
> > that call target macros is not enough.
> 
> By having targhooks.c compiled multiple times in different namespaces,
> keeping the old macros generally suits is fine; we can change the way
> the target macros are used without having to patch every single hook in
> for every single target.

The patch seemed to be making changes all over targhooks.c rather than 
just compiling the one file several times.  Perhaps the file needs to be 
split up.

> > I mentioned command-line options.  This is an area where I would think
> > your approach makes things much *more* complicated since you need to deal
> > with option settings on a per-target basis (different targets change the
> > settings of target-independent optimization flags in different ways, for
> > example, even if you don't actually allow command-line options to be
> > marked as applying to an individual target or allow "target" attributes to
> > be used in conjunction with your per-architecture functions.
> 
> Yes, part of the problem is that it's so hard to define it properly.
> E.g. a user might want to have delay branch scheduling for one target
> but not the other.  OTOH (s)he might not even think of the possibility
> and get exasperated when -f{no-,}delayed-branch won't work on all
> functions as expected.
> Conceptionally it would be simpler if all options are passed to all targets,
> but all options variables are target-specific, so that one target doesn't see
> the other's change.

I'd sort of imagine a -Wt,$target_name,option or similar syntax to specify 
an option for one target only.

> However, that wouldn't make semantic sense.  E.g. the SH defaults to
> flag_finite_math_only (because the hard FP comparison instruction set was
> designed so that it's expensive to do -fno-flag_finite_math_only), while
> most other targets default to -fflag_finite_math_only.
> If your main target is an i386 and you have sh4 as a secondary target, you
> wouldn't want optimizations to randomly disregard the finer points of
> NaNs in comparisons.
> So at least the flags that potentially change program behaviour must be
> global, since they represent the user's choices in the context of the
> selected main target.

Flags affecting semantics like that should be used to affect the GIMPLE 
generated rather than global flags being used later in the optimizers.  
That is, the front end would generate GIMPLE for one target marked with 
one set of semantics and the other target marked with the other semantics, 
and if a function is migrated later from one target to the other it would 
keep the original semantics.

> > (Properly, you'd then change all references to global options
> > to explicitly look at current-function options instead; the quick hack
> > would be to swap options structures when changing target architecture.)
> 
> No, then you'd swap semantics.  That might be acceptable if the user
> selected the target on the command line, but not if a function was moved
> to a different target by the optimizer.

In general the options set by default in target-dependent ways are not 
semantic ones, and as I noted above the semantics ought to be represented 
in individual GIMPLE and RTL notes (or a bit easier, as per-function 
flags) rather than global.  You could split up the structure (have several 
per-function pointers) so that semantic parts are kept when migrating 
target and non-semantic ones go with the target.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> On Sat, 26 Jun 2010, Joern Rennecke wrote:
>> So at least the flags that potentially change program behaviour must be
>> global, since they represent the user's choices in the context of the
>> selected main target.
>
> Flags affecting semantics like that should be used to affect the GIMPLE
> generated rather than global flags being used later in the optimizers.

But still, the place where flag_finite_math_only is manipulated by the sh
is OVERRIDE_OPTIONS (or targetm.target_option_override if my hookizing
patch gets approved), so a change to that logic is needed if the SH
is to be allowed as a secondary target.

And besides, the SH can't use GIMPLE codes in its instruction conditions.
Some instruction patterns are predicated on !TARGET_IEEE, which in turn is
influenced by the eventual setting of flag_finite_math_only inside
sh_override_options.

So, unless you want have rtl codes to encode possible presence of NaNs,
and hack places like cse.c and combine.c to propagate this into different
kinds of comparison / scc codes, and hack all targets to digest this, we
are stuck with global state (might be specific to a target / function, but
would apply to the entirety of a function at least) to control such things.

Re: RFA: Fix other/44566

[Richard Guenther]

On Sun, Jun 27, 2010 at 12:15 AM, Joern Rennecke <amylaar@spamcop.net> wrote:
> Quoting "Joseph S. Myers" <joseph@codesourcery.com>:
>
>> On Sat, 26 Jun 2010, Joern Rennecke wrote:
>>>
>>> So at least the flags that potentially change program behaviour must be
>>> global, since they represent the user's choices in the context of the
>>> selected main target.
>>
>> Flags affecting semantics like that should be used to affect the GIMPLE
>> generated rather than global flags being used later in the optimizers.
>
> But still, the place where flag_finite_math_only is manipulated by the sh
> is OVERRIDE_OPTIONS (or targetm.target_option_override if my hookizing
> patch gets approved), so a change to that logic is needed if the SH
> is to be allowed as a secondary target.
>
> And besides, the SH can't use GIMPLE codes in its instruction conditions.
> Some instruction patterns are predicated on !TARGET_IEEE, which in turn is
> influenced by the eventual setting of flag_finite_math_only inside
> sh_override_options.
>
> So, unless you want have rtl codes to encode possible presence of NaNs,
> and hack places like cse.c and combine.c to propagate this into different
> kinds of comparison / scc codes, and hack all targets to digest this, we
> are stuck with global state (might be specific to a target / function, but
> would apply to the entirety of a function at least) to control such things.

It's sensible to track semantic flags per function and disallow inlining.
This is what we are slowly moving towards (for example with the
different EH personalities or non-call exception settings with LTO).

Richard.

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sat, 26 Jun 2010, Joern Rennecke wrote:

> Quoting "Joseph S. Myers" <joseph@codesourcery.com>:
> 
> > On Sat, 26 Jun 2010, Joern Rennecke wrote:
> > > So at least the flags that potentially change program behaviour must be
> > > global, since they represent the user's choices in the context of the
> > > selected main target.
> > 
> > Flags affecting semantics like that should be used to affect the GIMPLE
> > generated rather than global flags being used later in the optimizers.
> 
> But still, the place where flag_finite_math_only is manipulated by the sh
> is OVERRIDE_OPTIONS (or targetm.target_option_override if my hookizing
> patch gets approved), so a change to that logic is needed if the SH
> is to be allowed as a secondary target.

The hook would, given my changes, modify an options structure to which a 
pointer is passed.  Instead of changing the individual hooks, if you don't 
want secondary targets changing semantic flags the target-independent code 
should save and restore those flags in the relevant structure before/after 
calling the hook for those targets (or some equivalent behavior).  Indeed 
you don't need my changes in order to save and restore some known set of 
semantic flags....

> So, unless you want have rtl codes to encode possible presence of NaNs,
> and hack places like cse.c and combine.c to propagate this into different
> kinds of comparison / scc codes, and hack all targets to digest this, we

That's the natural conclusion for proper representation of such things as 
the FENV_ACCESS pragma, yes - track the information on individual 
operations through both GIMPLE and RTL.  Per-function representation is 
simply an easier intermediate approach.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> The hook would, given my changes, modify an options structure to which a
> pointer is passed.  Instead of changing the individual hooks, if you don't
> want secondary targets changing semantic flags the target-independent code
> should save and restore those flags in the relevant structure before/after
> calling the hook for those targets (or some equivalent behavior).  Indeed
> you don't need my changes in order to save and restore some known set of
> semantic flags....

That'll work to avoid the sillyness of a secondary target nuking flag_pic,
but it wouldn't get the desired result with the current sh_override_options
code.  Although flag_finite_math_only would be restored, the SH port would
end up with the wrong setting for TARGET_IEEE.  If the main target decides
to leave flag_finite_math in its default cleared state, an SH secondary
target should respond by setting MASK_IEEE in its target_flags.

Re: RFA: Fix other/44566

[Mark Mitchell]

Joseph S. Myers wrote:

> The model I has was that all the targets supported by the compiler are 
> essentially equal.  It would be possible to run the compiler - or other 
> components such as the assembler - with a --target= option to select a 
> particular target, and the results would be identical to those of a 
> single-target compiler for that target.

That seems to me to be a far desirable model.  It's not like building
two compilers is that hard; if we can't achieve a single compiler that
can really target two targets equally well, then why not just build two
compilers?

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: RFA: Fix other/44566

[Steven Bosscher]

On Sun, Jun 27, 2010 at 6:28 PM, Mark Mitchell <mark@codesourcery.com> wrote:
> Joseph S. Myers wrote:
>
>> The model I has was that all the targets supported by the compiler are
>> essentially equal.  It would be possible to run the compiler - or other
>> components such as the assembler - with a --target= option to select a
>> particular target, and the results would be identical to those of a
>> single-target compiler for that target.
>
> That seems to me to be a far desirable model.  It's not like building
> two compilers is that hard; if we can't achieve a single compiler that
> can really target two targets equally well, then why not just build two
> compilers?

The only reason would be, AFAIU, that you can't compile one source
file to code for two targets. This seems to be the direction hardware
is taking, with many co-processors for dedicated tasks. See Cell BE,
and various efforts to make the GPU do some of the work. If things
continue in that direction, then eventually people will probably
expect the compiler to decide what code should be executed where, and
to compile bits of code in one file to execute on different
architectures.

Ciao!
Steven

Re: RFA: Fix other/44566

[Mark Mitchell]

Steven Bosscher wrote:

> The only reason would be, AFAIU, that you can't compile one source
> file to code for two targets. This seems to be the direction hardware
> is taking, with many co-processors for dedicated tasks. 

That's a good point, and I certainly agree that this is indeed a popular
hardware architecture direction.  I also agree that you cannot in all
cases separate the code into multiple files, not just for reasons of
code structure, but because you may need the compiler to see both sets
of code at once.  But, the hardware achitecture trend seems a strong
argument for doing the (large) project that Joseph estimated previously,
to build a robust multi-target compiler, not an argument for doing
something rather more fragile.

All the work people are doing to help turn things into hooks, and such,
is of course a step in the right direction; part of what we need for
this is to avoid having target macros affecting the front- and
middle-end compiler components.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting Mark Mitchell <mark@codesourcery.com>:
> All the work people are doing to help turn things into hooks, and such,
> is of course a step in the right direction; part of what we need for
> this is to avoid having target macros affecting the front- and
> middle-end compiler components.

As my work shows, having the hooks coded in the individual targets is not
required - using the existing macros to provide hooks works just fine.
Moreover, a rush to convert target macros into hooks with ill-conceived
parameters that become part of the interface for target ports actually
makes it harder to make a multi-target compiler, because then you actually
have to fix the parameters for every target that implements the hook.

Instead, what we should strive for is to use the existing macros to define
hooks in a few smallish files, to isolate the bulk of the code from target
dependencies.
If some code will run significantly slower if it has to use hooks instead of
macros, leaving it as target-dependent is also a viable choice, if we can
separate it properly from the target-independent code.  E.g. if we find that
the register allocator works better riddled with target macros,
than compiling it once per target is OK.

Simple macros like FIRST_PSEUDO_REGISTER can make it much easier for the
build compiler to optimize rtl passes and the utility functions they use
(e.g. unrolling works better with constants), so in the absence of
evidence to the contrary, I would suspect every rtl pass to suffer
performance degradation if it was fully hookized.

And rtl is always target-dependent anyway, so if we want compile flow to
diverge to go to different targets, it is natural to put this point at the
tree->rtl conversion.

Re: RFA: Fix other/44566

[Steven Bosscher]

On Sun, Jun 27, 2010 at 8:04 PM, Joern Rennecke <amylaar@spamcop.net> wrote:
> Quoting Mark Mitchell <mark@codesourcery.com>:
>>
>> All the work people are doing to help turn things into hooks, and such,
>> is of course a step in the right direction; part of what we need for
>> this is to avoid having target macros affecting the front- and
>> middle-end compiler components.
>
> As my work shows, having the hooks coded in the individual targets is not
> required - using the existing macros to provide hooks works just fine.

This is for the most part also how the macros-to-hooks conversion has
worked in the past: first make a hook that just encapsulates the
macro, then gradually hookize the targets one at a time.


> And rtl is always target-dependent anyway, so if we want compile flow to
> diverge to go to different targets, it is natural to put this point at the
> tree->rtl conversion.

I would say gimple->rtl, but yes, that is the natural point.

Ciao!
Steven

Re: RFA: Fix other/44566

[Mark Mitchell]

Joern Rennecke wrote:

> As my work shows, having the hooks coded in the individual targets is
> not required - using the existing macros to provide hooks works just
> fine.

I'm not concerned with whether we auto-generate hooks from macros or
code them directly.  That's a possibly important convenience issue for
target maintainers.  As long as we abstract that away from the
middle-end so that it can be target-independent, we win.

> If some code will run significantly slower if it has to use hooks 
> instead of macros, leaving it as target-dependent is also a viable
> choice, if we can separate it properly from the target-independent
> code.

I agree.  I think we have to measure to find out what matters and how
much it matters.  If going to a completely hooked world costs us 0.01%,
that's worth it.  If it costs 100%, it's not.  I'm not sure what the
right price to pay for this abstraction might be, but I think a
single-digit percentage increase in compile-time (in total) is about as
much as I'd like to see.

We may also be able to organize things so that only a multi-target
compiler pays the price.  Richard Sandiford's recent patch has this
clever idea of using pointers only when necessary.  The same thing might
work here; the hook is an inline function visible to the middle end if
single-target, and a pointer if multi-target.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sun, 27 Jun 2010, Joern Rennecke wrote:

> Quoting Mark Mitchell <mark@codesourcery.com>:
> > All the work people are doing to help turn things into hooks, and such,
> > is of course a step in the right direction; part of what we need for
> > this is to avoid having target macros affecting the front- and
> > middle-end compiler components.
> 
> As my work shows, having the hooks coded in the individual targets is not
> required - using the existing macros to provide hooks works just fine.

It is fragile, because you need a way to ensure that no patch causes a use 
of a macro to be introduced in a file built once-only; if the target macro 
doesn't exist at all, there is no such risk.  Furthermore, one long-term 
aim of modularization is to be able to build front ends and back ends as 
plug-ins separately from the core compiler, which certainly requires 
completing the hook conversion rather than relying on building parts of 
the compiler twice.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> On Sun, 27 Jun 2010, Joern Rennecke wrote:
>> As my work shows, having the hooks coded in the individual targets is not
>> required - using the existing macros to provide hooks works just fine.
>
> It is fragile, because you need a way to ensure that no patch causes a use
> of a macro to be introduced in a file built once-only;

We can fix that: we can poison the macro for these files.  Or to  
express it more robustly: poison them for all files that don't need  
access to this
class of target macros.  I.e. the compiler backend rtl files / targethooks.c
are the only ones to see compiler backend macros, the gcc driver hook file
is the only one to see the driver macros etc.

> Furthermore, one long-term
> aim of modularization is to be able to build front ends and back ends as
> plug-ins separately from the core compiler, which certainly requires
> completing the hook conversion rather than relying on building parts of
> the compiler twice.

No, it doesn't.  I would be a matter of basically packaging up what gets
build in an extra target directory as a plugin, and allow this to be inserted
in a target vector array that either is dynamically allocated, or has free
slots available.  (The latter needs less indirections, so a few dozen wasted
bytes and a limit on the number of simultanously plugged-in backends might
be a price worth paying.)

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sun, 27 Jun 2010, Joern Rennecke wrote:

> Quoting "Joseph S. Myers" <joseph@codesourcery.com>:
> 
> > On Sun, 27 Jun 2010, Joern Rennecke wrote:
> > > As my work shows, having the hooks coded in the individual targets is not
> > > required - using the existing macros to provide hooks works just fine.
> > 
> > It is fragile, because you need a way to ensure that no patch causes a use
> > of a macro to be introduced in a file built once-only;
> 
> We can fix that: we can poison the macro for these files.  Or to express it
> more robustly: poison them for all files that don't need access to this
> class of target macros.  I.e. the compiler backend rtl files / targethooks.c
> are the only ones to see compiler backend macros, the gcc driver hook file
> is the only one to see the driver macros etc.

Yes, there are workarounds.  I by far prefer the cleaner and simpler 
approach:

* The target vectors define everything about a target; there is no tm.h 
file at all, and every former target macro is poisoned for the whole 
compiler.  Nor are the macros used to initialize the target vector defined 
in .h files as some are at present (to ease the transition from macros to 
hooks, for those hooks that depend a lot on the OS as well as the target 
architecture); for target architecture vectors, there is exactly one 
definition of each hook, as a function that might if necessary contain 
further conditionals (if, not #if) internally.  I explicitly admit the 
possibility that there are separate target vectors for things depending 
heavily on both architecture and OS (e.g. specs in the driver), with 
arch-OS pairs providing .c files in such cases.

* We eat our own dogfood and make sure LTO/WHOPR can devirtualize target 
vector references so that clean abstractions and avoidance of macros do 
not make a single-target compiler less efficient.

I'm not asking you to implement this.  I am saying that the transition 
that has been under way for some time from target macros to hooks, with 
the explicit aim that each hook is defined directly as a function by each 
relevant target and the macros (and associated implementations of hooks in 
targhooks.c using those macros) go away completely, is a sound one and 
should continue, and we should not let the existence of multi-target 
approches that do not depend on this transition obstruct the continuation 
of this transition.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> On Sun, 27 Jun 2010, Joern Rennecke wrote:
>
>> Quoting "Joseph S. Myers" <joseph@codesourcery.com>:
>>
>> > On Sun, 27 Jun 2010, Joern Rennecke wrote:
>> > > As my work shows, having the hooks coded in the individual   
>> targets is not
>> > > required - using the existing macros to provide hooks works just fine.
>> >
>> > It is fragile, because you need a way to ensure that no patch causes a use
>> > of a macro to be introduced in a file built once-only;
>>
>> We can fix that: we can poison the macro for these files.  Or to express it
>> more robustly: poison them for all files that don't need access to this
>> class of target macros.  I.e. the compiler backend rtl files / targethooks.c
>> are the only ones to see compiler backend macros, the gcc driver hook file
>> is the only one to see the driver macros etc.
>
> Yes, there are workarounds.

I don't think it is merely a workaround; I think it is a proper
implementation.
Defining macros in config.in is an efficient way to specify target
dependencies.  Requiring each port maintainer to replace 5 lines of
shell script with several pages of code is not a good use of developer's
time.  Likewise, making developers jump through unnecessary hooks to
create a new port is not desirable.

> I by far prefer the cleaner and simpler
> approach:
>
> * The target vectors define everything about a target; there is no tm.h
> file at all, and every former target macro is poisoned for the whole
> compiler.  Nor are the macros used to initialize the target vector defined
> in .h files as some are at present (to ease the transition from macros to
> hooks, for those hooks that depend a lot on the OS as well as the target
> architecture); for target architecture vectors, there is exactly one
> definition of each hook, as a function that might if necessary contain
> further conditionals (if, not #if) internally.  I explicitly admit the
> possibility that there are separate target vectors for things depending
> heavily on both architecture and OS (e.g. specs in the driver), with
> arch-OS pairs providing .c files in such cases.
>
> * We eat our own dogfood and make sure LTO/WHOPR can devirtualize target
> vector references so that clean abstractions and avoidance of macros do
> not make a single-target compiler less efficient.

How long will single-target compilers be the main focus of GCC?  It looks
like desktops are heading are heading into a CPU/GPU future, and
embedded systems are already commonly heterogeneous systems.  My approach
allows to avoid hookization performance penalties for single as for
multi-targeted compilers, with the compiler technology we have available now.
And nothing would stop you from evolving this towards your scheme for
files that you can show can work without significant performance
penalty fully-hookized, compiled once.

You might want to see a stop to including tm.h in virtually every file,
but I don't see that anything is gained by abolishing tm.h altogether;
would you also argue against tm.c or the freedom of the port maintainer
to split tm.c into several files and include a common header file?
Well, if the target has a common header file, what is wrong if well-defined
set of files include it?  The E.g. the hookization for insn-emit.c is the
optab table, so why would you stop it from including tm.h?

> I'm not asking you to implement this.

Do you have an ETA for all that to fall into place?  Will GCC still be
relevant then?

> I am saying that the transition
> that has been under way for some time from target macros to hooks, with
> the explicit aim that each hook is defined directly as a function by each
> relevant target and the macros (and associated implementations of hooks in
> targhooks.c using those macros) go away completely, is a sound one and
> should continue, and we should not let the existence of multi-target
> approches that do not depend on this transition obstruct the continuation
> of this transition.

But for the fully hookized / LTO / WHOPR future to make sense for a  
multi-target compiler, you also have to make sure that you are not  
lying to the compiler about the types that the hooks take.
So getting the hooks definitions to make sense when CUMULATIVE_ARGS,
enum reg_class, and HARD_REG_SET do not each refer to a unique type
is required no matter what path we take.

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Sun, 27 Jun 2010, Joern Rennecke wrote:

> I don't think it is merely a workaround; I think it is a proper
> implementation.

I think a proper implementation has the totality of the definition of the 
interface for targets to define in just one place, i.e. the definition of 
the target structure and its associated documentation.  Targets providing 
headers that target-independent code includes involves a hole in this 
interface definition that makes it hard to tell quite what the interface 
is (enumerating target macros is hard when any identifier could be a 
macro).  Clarity and clear definition of internal interfaces is good in 
itself, and avoiding macros where possible generally helps make interfaces 
clearer and easier to analyze.

Certainly I'd consider that the function-like target macros, and others 
that expand to code rather than constants, are the highest priority to 
convert to hooks (and also less likely to have much performance impact; 
the targets with more complicated expansions hopefully already use 
function calls in the definitions of their macros).

This does not prevent tricks to build a file multiple times if that is 
really felt to be necessary (you could build it with your namespaces and 
with targetm defined to be a const target vector for a particular target, 
so that LTO/WHOPR can separately optimize the functions for each target).

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting "Joseph S. Myers" <joseph@codesourcery.com>:

> I think a proper implementation has the totality of the definition of the
> interface for targets to define in just one place, i.e. the definition of
> the target structure and its associated documentation.  Targets providing
> headers that target-independent code includes involves a hole in this
> interface definition that makes it hard to tell quite what the interface
> is (enumerating target macros is hard when any identifier could be a
> macro).

You have a point there.

That is true for the actual target-provided include file(s).

However, defines from tm_defines could be screened for a list of interface
names, so that any other defines are only seen in tm.c .

Requiring every interface macro to be mentioned in a relevant .texi file is
probably a good idea; we should be able to grep @defmacx* from there.

> Clarity and clear definition of internal interfaces is good in
> itself, and avoiding macros where possible generally helps make interfaces
> clearer and easier to analyze.
>
> Certainly I'd consider that the function-like target macros, and others
> that expand to code rather than constants, are the highest priority to
> convert to hooks (and also less likely to have much performance impact;
> the targets with more complicated expansions hopefully already use
> function calls in the definitions of their macros).

Well, some expand to very little code, or on some targets, none at all.
So just because you have a function-like macro that expands to code does
not mean that it'll be cheap in terms of compiler performance to hookize
it.

> This does not prevent tricks to build a file multiple times if that is
> really felt to be necessary (you could build it with your namespaces and
> with targetm defined to be a const target vector for a particular target,
> so that LTO/WHOPR can separately optimize the functions for each target).

Yes, that sounds like a plan that can work.
But that means you'll have to wait for LTO/WHOPR to be mature enough before
you can remove performance-critical target macros.
And I'd also like to know what that does to our bootstrap speed.

Alternatively, we could impose rules on how interface macros can be defined
in tm.h, and sanitize the defines for all but tm.c so that only interface
macros are visible.  And possibly helper macros in an agreed part of the
implementation namespace.
Admittedly that would be more of a hack, but it should be a hack that can
be put together over a long weekend - and it'll add very little to bootstrap
times.

Re: RFA: Fix other/44566

[Mark Mitchell]

Joern Rennecke wrote:

> And I'd also like to know what that does to our bootstrap speed.

What follows is a minor parenthetical in the broader discussion that you
and Joseph are having...

I know this may be perceived as the words of someone who vary rarely
bootstraps the compiler anymore, but I believe that bootstrap speed per
se is nearly irrelevant.  Compile-time is not irrelevant; we care what
the user experience is like.  But to the extent bootstrap time is
independent of compile-time, I don't think we should care.  (OK, I think
we should care about making it 10x greater, but not about making it 25%
greater.)

The reason is that I think that (a) bootstraps are fast enough that it
doesn't make a whole lot of difference, and (b) building the run-time
libraries and running tests takes much more time than the bootstrap per
se.  And, as for the bootstrap per se, I suspect that configure/link
overhead will dominate soon, if it doesn't already; configure and link
steps are (currently) serial, so cannot take advantage of massively
parallel systems in the way that the rest of the build process can.

So, concretely, carrying LTO/WHOPR overhead during the compiler build
itself to me seems OK if that gets us a better compiler and user's can't
tell the difference.  In fact, we probably want to turn on LTO/WHOPR by
default once we feel it's sufficiently reliable since presumably we'll
get better code that way, no matter how the compiler is structured.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting Mark Mitchell <mark@codesourcery.com>:

> I know this may be perceived as the words of someone who vary rarely
> bootstraps the compiler anymore, but I believe that bootstrap speed per
> se is nearly irrelevant.  Compile-time is not irrelevant; we care what
> the user experience is like.  But to the extent bootstrap time is
> independent of compile-time, I don't think we should care.  (OK, I think
> we should care about making it 10x greater, but not about making it 25%
> greater.)

If that's 25% bootstrap time increase to get rid of the
FIRST_PSEUDO_REGISTER macro, I'll have to disagree.

> The reason is that I think that (a) bootstraps are fast enough that it
> doesn't make a whole lot of difference, and (b) building the run-time
> libraries and running tests takes much more time than the bootstrap per
> se.

building run-time libraries is part of the bootstrap.  And building the
stage1 libraries suffers from a slow compiler which didn't get the benefit
of WHOPR/LTO optimizations.

C++ conformance issues often pop up when building stage2.  So if the
compiler we built in stage1 is slow, that means building stage1 libraries
and the stage2 compiler is slow; the stage1-built compiler is an important
factor in the turn around time.

> And, as for the bootstrap per se, I suspect that configure/link
> overhead will dominate soon, if it doesn't already; configure and link
> steps are (currently) serial, so cannot take advantage of massively
> parallel systems in the way that the rest of the build process can.

I have often more than one bootstrap going on, so my build scripts generally
use make "-j5 -l 7" to get an even load distribution across four cores;
I could happily use a 48-core machine for my bootstraps, if it wasn't that
horrendously expensive to buy.  Oh, and if bootstrap bloat caused it to
be fully loaded, it'd likely draw a lot of power, too.

> So, concretely, carrying LTO/WHOPR overhead during the compiler build
> itself to me seems OK if that gets us a better compiler and user's can't
> tell the difference.  In fact, we probably want to turn on LTO/WHOPR by
> default once we feel it's sufficiently reliable since presumably we'll
> get better code that way, no matter how the compiler is structured.

The worst performance hit is probably when using stage1 to build the  
stage1 libraries and the stage2 compiler, as a compiler built with -O0  
is used
to do LTO/WHOPR.

This will be worse if this -O0 built compiler has a lot of target hooks
in performance critical paths.
I'm not saying that it will definitely be unacceptable.  But I'm saying that
we don't know if the performance hit will be acceptable till we can measure
it.

Re: RFA: Fix other/44566

[Mark Mitchell]

Joern Rennecke wrote:

> I'm not saying that it will definitely be unacceptable.  But I'm saying
> that we don't know if the performance hit will be acceptable till we can measure
> it.

I'll agree with that.  We do have to measure to know what the damage is,
and we shouldn't pre-judge the outcome.  My guess is that if we really
have a bootstrap-time problem, we have a compile-time problem, and
that's much more of a concern to me, because it's about users, not just
about developers.  But, as you say, we just have to see.

-- 
Mark Mitchell
CodeSourcery
mark@codesourcery.com
(650) 331-3385 x713

Re: RFA: Fix other/44566

[Jan Hubicka]

>> So, concretely, carrying LTO/WHOPR overhead during the compiler build
>> itself to me seems OK if that gets us a better compiler and user's can't
>> tell the difference.  In fact, we probably want to turn on LTO/WHOPR by
>> default once we feel it's sufficiently reliable since presumably we'll
>> get better code that way, no matter how the compiler is structured.
>
> The worst performance hit is probably when using stage1 to build the  
> stage1 libraries and the stage2 compiler, as a compiler built with -O0  
> is used
> to do LTO/WHOPR.

I've setup WHOPR bootstrap and I use it daily.  It is sort of in useable stage.
Main observation is that we build backend _really_ many times.  Our LTO
code is still doing one build to produce .o and then compile again, so it
is automatically about twice sloer.

Since we produce several binaries (one for each frontend, that is 6 binaries)
and since we compile each twice to get in the checksum needed for PCH, we get
backend build 13 times for stage2 and 13times for stage3.  So we speak about
building backend 27 times instead of 3 that is considerable build time hit.

Reality is not that bad as it sounds when you have enough of parallelizm.
24 CPU machine needs slightly over a minute per binary and there is room
for parallelizm left, so the 6 binaries build in about 4 minutes in stage3.
Stage2 is about 3 times slower because of -O0 compiler.

I think we should shoot to make WHOPR bootstrap officially working at the
next release.  Immediately we still need to solve many things:

1) Build machinery needs updating. Alexandre recently added bootstrap-lto,
   but it does not do the trick since it just adds -flto to BOOT flags.
   We need -flto -fwhole-program and GOLD plugin flags (without GOLD we
   still drop all LTO from libbckend.a making it useless)
2) We need incremental WHOPR to avoid the second rebuild because of
   checksum
3) Ada currently fails because of debug output ICE
4) Plugin tests fails for me with -fwhole-program (probably because exported
   functions are not properly marked)

Also it would be nice to see noticeable imrpvoements on compiler performance
for WHOPR bootstrap. I did some benchmarking but fixed a lot of things afterwards
so will need to re-test where we stand.  

Honza

Re: RFA: Fix other/44566

[Joseph S. Myers]

On Mon, 28 Jun 2010, Jan Hubicka wrote:

> Since we produce several binaries (one for each frontend, that is 6 binaries)
> and since we compile each twice to get in the checksum needed for PCH, we get
> backend build 13 times for stage2 and 13times for stage3.  So we speak about
> building backend 27 times instead of 3 that is considerable build time hit.

A multi-front-end binary (that uses a --front-end= option in argv[1] to 
decide which front end it is) shouldn't be hard to build, to save build 
time and reduce the size of multi-language binary distributions.  You'd 
lose devirtualization of langhooks - but since some langhooks are modified 
at runtime anyway, I don't know if devirtualization of langhooks actually 
works, and I expect devirtualization of target hooks to be much more 
significant for performance.

To build a multi-front-end binary you'd need to:

* Finish eliminating calls to "convert" from non-front-end code.  Either 
change them to fold_convert, or if front-end conversions are needed then 
make it a langhook.  I think this is the last front-end function called 
directly from non-front-end code with no hook at all.

* Eliminate cases where langhooks-def.h defines a default name for a hook 
and multiple front ends define functions with that name: change front ends 
to use front-end-specific names for those hooks and remove the defaults.  
This is cases such as LANG_HOOKS_PUSHDECL defined to pushdecl.

* Make each front end use its own namespace for non-static functions 
internal to that front end.  This can be done with C++ namespaces, or 
simply by naming conventions such as c_* for non-static functions in the C 
front end.

* Where c-family code calls functions that both C and C++ define with the 
same name but different implementations, convert those to some form of 
hook.

* Where C code can call objc_* functions for C, getting the stub-objc.c 
implementations for C and real ones for ObjC, make either the C code or 
the real ObjC implementations include appropriate conditionals checking 
whether the compiler is in ObjC mode and acting like the stub versions if 
not.  (That is, eliminate stub-objc.c and just make ObjC a mode of the C 
front end that is selected at runtime.)

* Do something about ObjC++ (remove it, arrange for objc-act.c to need 
building only once like the c-family code or give functions different 
names for ObjC++ from those for ObjC); if it is kept, make it a runtime 
mode of the C++ front end.

* Check for any issues with any target's c_target_objs and cxx_target_objs 
being linked together.

* Actually link everything together and implement runtime language 
selection.

-- 
Joseph S. Myers
joseph@codesourcery.com

Re: RFA: Fix other/44566

[Jan Hubicka]

> On Mon, 28 Jun 2010, Jan Hubicka wrote:
> 
> > Since we produce several binaries (one for each frontend, that is 6 binaries)
> > and since we compile each twice to get in the checksum needed for PCH, we get
> > backend build 13 times for stage2 and 13times for stage3.  So we speak about
> > building backend 27 times instead of 3 that is considerable build time hit.
> 
> A multi-front-end binary (that uses a --front-end= option in argv[1] to 
> decide which front end it is) shouldn't be hard to build, to save build 
> time and reduce the size of multi-language binary distributions.  You'd 
> lose devirtualization of langhooks - but since some langhooks are modified 
> at runtime anyway, I don't know if devirtualization of langhooks actually 
> works, and I expect devirtualization of target hooks to be much more 
> significant for performance.

There is also alternative of getting incremental whopr to work cross 
binaries - i.e. if frontend changes nothing in a partition, it can be
used to build other binary.
This is tricky, we probably would need server handling the known partitions
to allow parallelizm that is something I definitly do not plan for next release.

I did some testing how much the partitions changes and at moment they do,
there are still macros and things that get compiled into it. Especially
concerning EH.

We do not devirtualize calls thruough static structures unless they are
declared const.

Having multi frontend binary (at least as an option) seems like nice goal,
especially because it imply cleanups and interface improvements.
From performance POV I am however affraid that increasing compiler binary
would lead to even longer startup time.  I am not sure how large percentage
of the backend binary frontends would occupy.  Perhaps not too large.

Honza

Re: RFA: Fix other/44566

[Jan Hubicka]

> > On Mon, 28 Jun 2010, Jan Hubicka wrote:
> > 
> > > Since we produce several binaries (one for each frontend, that is 6 binaries)
> > > and since we compile each twice to get in the checksum needed for PCH, we get
> > > backend build 13 times for stage2 and 13times for stage3.  So we speak about
> > > building backend 27 times instead of 3 that is considerable build time hit.
> > 
> > A multi-front-end binary (that uses a --front-end= option in argv[1] to 
> > decide which front end it is) shouldn't be hard to build, to save build 
> > time and reduce the size of multi-language binary distributions.  You'd 
> > lose devirtualization of langhooks - but since some langhooks are modified 
> > at runtime anyway, I don't know if devirtualization of langhooks actually 
> > works, and I expect devirtualization of target hooks to be much more 
> > significant for performance.
> 
> There is also alternative of getting incremental whopr to work cross 
> binaries - i.e. if frontend changes nothing in a partition, it can be
> used to build other binary.
> This is tricky, we probably would need server handling the known partitions
> to allow parallelizm that is something I definitly do not plan for next release.
> 
> I did some testing how much the partitions changes and at moment they do,
> there are still macros and things that get compiled into it. Especially
> concerning EH.
> 
> We do not devirtualize calls thruough static structures unless they are
> declared const.

Also we do devirtualize them just late, only turning indirect call to direct
call that has very little runtime effect on modern hardware.  Hopefully
ipa-prop infrastructure can be extended easilly to handle devirtualization
before inlining.

Honza

Re: RFA: Fix other/44566

[Andi Kleen]

Jan Hubicka <hubicka@ucw.cz> writes:
>
> I've setup WHOPR bootstrap and I use it daily.  It is sort of in useable stage.
> Main observation is that we build backend _really_ many times.  Our LTO
> code is still doing one build to produce .o and then compile again, so it
> is automatically about twice sloer.

I also noticed that LTO always compiles twice for other projects. I
understand this is needed for compatibility with old Makefiles that do
not invoke the linking step correctly. But it still seems like quite a
waste for updated Makefiles. Would it be possible to turn the first code
generation step off and only rely on the link to code generation?

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: RFA: Fix other/44566

[Jan Hubicka]

> Jan Hubicka <hubicka@ucw.cz> writes:
> >
> > I've setup WHOPR bootstrap and I use it daily.  It is sort of in useable stage.
> > Main observation is that we build backend _really_ many times.  Our LTO
> > code is still doing one build to produce .o and then compile again, so it
> > is automatically about twice sloer.
> 
> I also noticed that LTO always compiles twice for other projects. I
> understand this is needed for compatibility with old Makefiles that do
> not invoke the linking step correctly. But it still seems like quite a
> waste for updated Makefiles. Would it be possible to turn the first code
> generation step off and only rely on the link to code generation?

Yes, this is something I would like to see changed (probably for default,
I think users rarely will really use the fat object files).  The major
problem here is how linker works - it links based on symbols used/exported
by the object files and then it calls plugin.  So if you only output the
bytecode, it will not link anything.

We need to discuss how to deal with this.  It is not even correct, since optimized
code in .o and one produced by LTO might be different and might need different
symbols.  When the second code will use more symbols, linking fails.

I can just walk callgraph/varpool and references and output empty symbol
definitions and array of useses on what the program needs at time it is streamed
out, but that will result in .o files that when linked normal way would produce
broken program....

Honza
> 
> -Andi
> 
> -- 
> ak@linux.intel.com -- Speaking for myself only.

Re: RFA: Fix other/44566

[Joern Rennecke]

Quoting Jan Hubicka <hubicka@ucw.cz>:

> I can just walk callgraph/varpool and references and output empty symbol
> definitions and array of useses on what the program needs at time it  
>  is streamed
> out, but that will result in .o files that when linked normal way   
> would produce
> broken program....

Could you put these in a non-loadable section and use extra name mangling to
shift the symbols into the implementation namespace?