[Bug runtime/10516] New: on rawhide, getting a warning about bad percpu allocation

[Bug runtime/10516] New: on rawhide, getting a warning about bad percpu allocation

[dsmith at redhat dot com]

When the testsuite is run on a rawhide system (kernel
2.6.31-0.125.rc5.git2.fc12.x86_64), the following warning is seen:

----
Aug 13 09:45:30 dhcp-148 kernel: WARNING: at mm/percpu.c:786
pcpu_alloc+0x62/0x460() (Tainted: G        W )
Aug 13 09:45:30 dhcp-148 kernel: Hardware name: 
Aug 13 09:45:30 dhcp-148 kernel: illegal size (72720) or align (8) for percpu
allocation
Aug 13 09:45:30 dhcp-148 kernel: Modules linked in:
stap_06dc35dbd9c286e66fb95316e6b847b0_712048 onoffprobe uprobes autofs4 nfs
lockd fscache nfs_acl auth_rpcgss sunrpc ip6t_REJECT nf_conntrack_ipv6
ip6table_filter ip6_tables ipv6 dm_multipath uinput i2c_piix4 8139too 8139cp mii
virtio_balloon floppy i2c_core virtio_pci virtio_ring virtio ata_generic
pata_acpi [last unloaded: stap_b5f2d0fe68a55a18a7db86e374c3f43f_824]
Aug 13 09:45:30 dhcp-148 kernel: Pid: 3369, comm: stapio Tainted: G        W 
2.6.31-0.125.rc5.git2.0.hsv.1.fc12.x86_64 #1
Aug 13 09:45:30 dhcp-148 kernel: Call Trace:
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81064d5b>] warn_slowpath_common+0x95/0xc3
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81064e16>] warn_slowpath_fmt+0x50/0x66
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff8113b70d>] pcpu_alloc+0x62/0x460
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81009000>] ? _stext+0x0/0x1032
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff8113bb6a>] __alloc_percpu+0x23/0x39
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffffa02c5139>]
_stp_ctl_write_cmd+0x1a6/0x7c2 [stap_06dc35dbd9c286e66fb95316e6b847b0_712048]
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff8122e43a>] ? file_has_perm+0x92/0xb1
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff8122e839>] ?
selinux_file_permission+0x69/0x86
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81223f80>] ?
security_file_permission+0x29/0x3f
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81141f1d>] vfs_write+0xbd/0x12e
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81143187>] ? fget_light+0x63/0xfd
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff814feeea>] kretprobe_trampoline+0x0/0x60
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff814fa317>] ? thread_return+0x4e/0xd3
Aug 13 09:45:30 dhcp-148 kernel: [<ffffffff81012f42>] system_call_fastpath+0x16/0x1b
Aug 13 09:45:30 dhcp-148 kernel: ---[ end trace 39a02225aeaa1558 ]---
----

It appears that the request size is too large.

-- 
           Summary: on rawhide, getting a warning about bad percpu
                    allocation
           Product: systemtap
           Version: unspecified
            Status: NEW
          Severity: normal
          Priority: P2
         Component: runtime
        AssignedTo: systemtap at sources dot redhat dot com
        ReportedBy: dsmith at redhat dot com


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug runtime/10516] on rawhide, getting a warning about bad percpu allocation

[wenji dot huang at oracle dot com]

------- Additional Comments From wenji dot huang at oracle dot com  2009-08-18 01:25 -------
Reproduced on x86_64 2.6.31-rc6, but works fine on x86 2.6.31-rc6.

$ stap -v systemtap.printf/memory1.stp
Pass 1: parsed user script and 59 library script(s) in 310usr/60sys/369real ms.
Pass 2: analyzed script: 2 probe(s), 6 function(s), 14 embed(s), 0 global(s) in
780usr/380sys/1165real ms.
Pass 3: translated to C into
"/tmp/stapp5DGZt/stap_87fc0586871670d76847178631c22a98_21602.c" in
800usr/100sys/895real ms.
Pass 4: compiled C into "stap_87fc0586871670d76847178631c22a98_21602.ko" in
8370usr/6570sys/14450real ms.
Pass 5: starting run.
ERROR: percpu context (size 103912) allocation failed
Pass 5: run completed in 10usr/100sys/141real ms.

This is a VMware box with two cpus. Dmesg said,

[258068.721067] WARNING: at mm/percpu.c:786 pcpu_alloc+0x55/0x3d8()
[258068.722067] Hardware name: VMware Virtual Platform
[258068.723067] illegal size (103912) or align (8) for percpu allocation
[258068.724067] Modules linked in: stap_87fc0586871670d76847178631c22a98_21602
[last unloaded: stap_87fc0586871670d76847178631c22a98_21602]
[258068.725067] Pid: 19529, comm: stapio Tainted: G        W  2.6.31-rc6 #10
[258068.726067] Call Trace:
[258068.727067]  [<ffffffff810cfa77>] ? pcpu_alloc+0x55/0x3d8
[258068.727067]  [<ffffffff8103f80a>] warn_slowpath_common+0x7c/0x94
[258068.728067]  [<ffffffff8103f8a1>] warn_slowpath_fmt+0x69/0x6b
[258068.729067]  [<ffffffff810be6e4>] ? alloc_vmap_area+0x1f2/0x20a
[258068.730067]  [<ffffffff810be796>] ? __get_vm_area_node+0x9a/0x1ba
[258068.731067]  [<ffffffff810b3b09>] ? pmd_alloc+0x2e/0x39
[258068.731067]  [<ffffffff810b7417>] ? apply_to_page_range+0x1e1/0x25b
[258068.732067]  [<ffffffff810cfa77>] pcpu_alloc+0x55/0x3d8
[258068.733067]  [<ffffffff81027d7d>] ? vmalloc_sync_all+0xe7/0x10f
[258068.734067]  [<ffffffff810cfe1f>] __alloc_percpu+0x10/0x15
[258068.735067]  [<ffffffffa003cf2a>] _stp_ctl_write_cmd+0x229/0x539
[stap_87fc0586871670d76847178631c22a98_21602]
[258068.735067]  [<ffffffff8103249c>] ? __dequeue_entity+0x33/0x38
[258068.735067]  [<ffffffff810324c9>] ? set_next_entity+0x28/0x78
[258068.735067]  [<ffffffff8100ae80>] ? __switch_to+0x134/0x1c4
[258068.735067]  [<ffffffff81094811>] ? trace_hardirqs_on+0x9/0x20
[258068.736067]  [<ffffffff81035b80>] ? finish_task_switch+0x56/0x9f
[258068.738067]  [<ffffffff8119adb0>] ? security_file_permission+0x16/0x18
[258068.739067]  [<ffffffff810d2246>] vfs_write+0xb0/0x10a
[258068.741067]  [<ffffffff810d2364>] sys_write+0x4c/0x72
[258068.742067]  [<ffffffff8100bc9b>] system_call_fastpath+0x16/0x1b
[258068.743067] ---[ end trace ccde141178315c39 ]--- 

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug runtime/10516] on rawhide, getting a warning about bad percpu allocation

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-08-18 02:33 -------
According to gdb, probe_1812_locals are 10KB big, and get multiplied by
10 for MAXNESTING, which puts the total about 100KB.  That's too big.

We should teach the translator to set MAXNESTING to smaller figures for
nonrecursive probes.  Similarly, probe locals should not be included
in the nesting-sensitive data (locals[]).

Plus we don't REALLY have to use the percpu alloc stuff if it's that
limited.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug runtime/10516] on rawhide, getting a warning about bad percpu allocation

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-08-18 16:27 -------
commit 18e5ffd improves this

-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug runtime/10516] on rawhide, getting a warning about bad percpu allocation

[fche at redhat dot com]

------- Additional Comments From fche at redhat dot com  2009-08-18 19:56 -------
Commit a7ed0d3 improves this further.
The cumulative effect should be a much smaller
"+NN+"  "+ctx+"  value printk'd out at module
startup.


-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |WAITING


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.

[Bug runtime/10516] on rawhide, getting a warning about bad percpu allocation

[dsmith at redhat dot com]

------- Additional Comments From dsmith at redhat dot com  2009-11-30 18:32 -------
The changes have fixed these particular problems, but the problem can still
occur on random scripts.  Bug #11034 will hopefully solve the bigger problem by
avoiding the percpu allocation.

-- 
           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|WAITING                     |RESOLVED
         Resolution|                            |FIXED


http://sourceware.org/bugzilla/show_bug.cgi?id=10516

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.