* [committed] Do finalize-multifile in parallel
@ 2021-03-27 12:57 Tom de Vries
0 siblings, 0 replies; only message in thread
From: Tom de Vries @ 2021-03-27 12:57 UTC (permalink / raw)
To: dwz, jakub, mark
Hi,
Run the finalize-multifile phase in parallel.
Using an experiment, we can see the effect:
...
$ for j in $(seq 1 2); do \
rm -f 1 2 3; \
cp debug/cc1 1; cp 1 2; \
echo "j: $j"; \
../measure/time.sh ./dwz -lnone -m 3 1 2 -j$j; \
done
j: 1
maxmem: 1295068
real: 26.38
user: 25.38
system: 0.77
j: 2
maxmem: 1297724
real: 23.55
user: 25.81
system: 0.88
...
Committed to trunk.
Thanks,
- Tom
Do finalize-multifile in parallel
2021-03-27 Tom de Vries <tdevries@suse.de>
* dwz.c (dwz_files_1): Run finalize-multifile phase in parallel, if
max_forks > 0.
---
dwz.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 62 insertions(+), 14 deletions(-)
diff --git a/dwz.c b/dwz.c
index 3cc0a26..eeae750 100644
--- a/dwz.c
+++ b/dwz.c
@@ -16608,21 +16608,69 @@ dwz_files_1 (int nr_files, char *files[], bool hardlink,
goto cleanup;
}
- for (i = 0; i < nr_files; i++)
+ if (max_forks > 1)
{
- dw_cu_ref cu;
- file = files[i];
- if (stats_p)
- init_stats (file);
- multifile_mode = MULTIFILE_MODE_FI;
- /* Don't process again files that couldn't
- be processed successfully. Also skip hard links. */
- if (resa[i].res == -1 || resa[i].res == -2
- || resa[i].skip_multifile)
- continue;
- for (cu = alt_first_cu; cu; cu = cu->cu_next)
- alt_clear_dups (cu->cu_die);
- ret |= dwz (file, NULL, &resa[i]);
+ pid_t pids[nr_files];
+ int nr_forks = 0;
+ for (i = 0; i < nr_files; i++)
+ pids[i] = 0;
+ for (i = 0; i < nr_files; i++)
+ {
+ file = files[i];
+ struct file_result *res = &resa[i];
+ multifile_mode = MULTIFILE_MODE_FI;
+ /* Don't process again files that couldn't
+ be processed successfully. Also skip hard links. */
+ if (resa[i].res == -1 || resa[i].res == -2
+ || resa[i].skip_multifile)
+ continue;
+
+ if (nr_forks == max_forks)
+ {
+ int thisret = wait_child_exit (-1, pids, i, res);
+ if (thisret == 1)
+ ret = 1;
+ nr_forks--;
+ }
+
+ pid_t fork_res = fork ();
+ assert (fork_res != -1);
+ if (fork_res == 0)
+ {
+ int thisret = dwz (file, NULL, res);
+ return encode_child_exit_status (thisret, res);
+ }
+ else
+ {
+ pids[i] = fork_res;
+ nr_forks++;
+ }
+ }
+ if (nr_forks > 0)
+ {
+ int thisret = wait_children_exit (pids, nr_files, resa);
+ if (thisret == 1)
+ ret = 1;
+ }
+ }
+ else
+ {
+ for (i = 0; i < nr_files; i++)
+ {
+ dw_cu_ref cu;
+ file = files[i];
+ if (stats_p)
+ init_stats (file);
+ multifile_mode = MULTIFILE_MODE_FI;
+ /* Don't process again files that couldn't
+ be processed successfully. Also skip hard links. */
+ if (resa[i].res == -1 || resa[i].res == -2
+ || resa[i].skip_multifile)
+ continue;
+ for (cu = alt_first_cu; cu; cu = cu->cu_next)
+ alt_clear_dups (cu->cu_die);
+ ret |= dwz (file, NULL, &resa[i]);
+ }
}
if (hardlink)
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2021-03-27 12:57 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-27 12:57 [committed] Do finalize-multifile in parallel Tom de Vries
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).