[PATCH] powerpc/kprobes: Pass ppc_inst as a pointer to emulate

I remember running some kprobe tests before submitting the patch, how did I miss that ?
Is there anything special to do to activate the use of optprobes and/or to hit this bug ?

Post by Naveen N. Rao
81 24 00 00 lwz r9,0(r4)
struct ppc_inst, which is the instruction to be emulated. On ppc64,
structures are passed in registers when passed by value. However, per
the ppc32 ABI, structures are always passed to functions as pointers.

Yes, and that was one of the reasons I was reluctant to that new 'struct ppc_inst', as it means
stack frames, copies, etc ... whereas we could have just changed it from 'unsigned int' to 'unsigned
long'.
Ok, now we have it, so let's use it, but use it correctly, see my clean-up series
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=244719&state=* (v2 on the way).

Post by Naveen N. Rao
This isn't being adhered to when setting up the call to emulate_step()
in the optprobe trampoline. Fix the same.
Fixes: eacf4c0202654a ("powerpc: Enable OPTPROBES on PPC32")
---
arch/powerpc/kernel/optprobes.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/optprobes.c b/arch/powerpc/kernel/optprobes.c
index cdf87086fa33a0..2bc53fa48a1b33 100644
--- a/arch/powerpc/kernel/optprobes.c
+++ b/arch/powerpc/kernel/optprobes.c
@@ -281,8 +281,12 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p)
/*
* 3. load instruction to be emulated into relevant register, and
*/
- temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
- patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+ if (IS_ENABLED(CONFIG_PPC64)) {
+ temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
+ patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+ } else {
+ patch_imm_load_insns((unsigned long)p->ainsn.insn, 4, buff + TMPL_INSN_IDX);
+ }

It means commit https://github.com/linuxppc/linux/commit/693557ebf407a85ea400a0b501bb97687d8f4856
was not necessary and may be reverted.

Christophe

Naveen N. Rao

2021-05-20 10:54:28 UTC

I remember running some kprobe tests before submitting the patch, how did I miss that ?
Is there anything special to do to activate the use of optprobes and/or to hit this bug ?

Yeah, I was surprised when I hit this. One of the requirements we have
for optprobes on powerpc is that the instruction should be a compute
instruction (no load/store -- emulate_update_regs() should be enough)
with the exception of conditional branches. It's possible that you ended
up probing an instruction that couldn't be optimized.

An easy way to confirm if a probe has been optimized is to look at
kprobes/list in debugfs, and to watch out for [OPTIMIZED] flag there.

Post by Naveen N. Rao
diff --git a/arch/powerpc/kernel/optprobes.c
b/arch/powerpc/kernel/optprobes.c
index cdf87086fa33a0..2bc53fa48a1b33 100644
--- a/arch/powerpc/kernel/optprobes.c
+++ b/arch/powerpc/kernel/optprobes.c
@@ -281,8 +281,12 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe *p)
/*
* 3. load instruction to be emulated into relevant register, and
*/
- temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
- patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+ if (IS_ENABLED(CONFIG_PPC64)) {
+ temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
+ patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+ } else {
+ patch_imm_load_insns((unsigned long)p->ainsn.insn, 4, buff + TMPL_INSN_IDX);
+ }

It means commit https://github.com/linuxppc/linux/commit/693557ebf407a85ea400a0b501bb97687d8f4856
was not necessary and may be reverted.

Indeed, I will send a revert for it.

- Naveen

Christophe Leroy

2021-05-20 12:55:14 UTC

I remember running some kprobe tests before submitting the patch, how did I miss that ?
Is there anything special to do to activate the use of optprobes and/or to hit this bug ?

Yeah, I was surprised when I hit this. One of the requirements we have for optprobes on powerpc is
that the instruction should be a compute instruction (no load/store -- emulate_update_regs() should
be enough) with the exception of conditional branches. It's possible that you ended up probing an
instruction that couldn't be optimized.
An easy way to confirm if a probe has been optimized is to look at kprobes/list in debugfs, and to
watch out for [OPTIMIZED] flag there.

diff --git a/arch/powerpc/kernel/optprobes.c b/arch/powerpc/kernel/optprobes.c
index cdf87086fa33a0..2bc53fa48a1b33 100644
--- a/arch/powerpc/kernel/optprobes.c
+++ b/arch/powerpc/kernel/optprobes.c
@@ -281,8 +281,12 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe
*p)
      /*
       * 3. load instruction to be emulated into relevant register, and
       */
-    temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
-    patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+    if (IS_ENABLED(CONFIG_PPC64)) {
+        temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
+        patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+    } else {
+        patch_imm_load_insns((unsigned long)p->ainsn.insn, 4, buff + TMPL_INSN_IDX);
+    }

It means commit https://github.com/linuxppc/linux/commit/693557ebf407a85ea400a0b501bb97687d8f4856
was not necessary and may be reverted.

Indeed, I will send a revert for it.

I'm not completely sure it is worth reverting, on an other hand it is pointless anyway to have
something to convert to a u64 something that cannot be more than 32 bits on a PPC32, so now that we
have ppc_inst_as_ulong() it is as good I think.

Christophe

Naveen N. Rao

2021-05-21 07:01:54 UTC

Post by Naveen N. Rao
diff --git a/arch/powerpc/kernel/optprobes.c
b/arch/powerpc/kernel/optprobes.c
index cdf87086fa33a0..2bc53fa48a1b33 100644
--- a/arch/powerpc/kernel/optprobes.c
+++ b/arch/powerpc/kernel/optprobes.c
@@ -281,8 +281,12 @@ int arch_prepare_optimized_kprobe(struct optimized_kprobe *op, struct kprobe
*p)
      /*
       * 3. load instruction to be emulated into relevant register, and
       */
-    temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
-    patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+    if (IS_ENABLED(CONFIG_PPC64)) {
+        temp = ppc_inst_read((struct ppc_inst *)p->ainsn.insn);
+        patch_imm_load_insns(ppc_inst_as_ulong(temp), 4, buff + TMPL_INSN_IDX);
+    } else {
+        patch_imm_load_insns((unsigned long)p->ainsn.insn, 4, buff + TMPL_INSN_IDX);
+    }

It means commit https://github.com/linuxppc/linux/commit/693557ebf407a85ea400a0b501bb97687d8f4856
was not necessary and may be reverted.

Indeed, I will send a revert for it.

Sure. If necessary, the revert can go in separately as part of your
other cleanup series.

- Naveen

Naveen N. Rao

2021-06-07 11:34:27 UTC

Post by Naveen N. Rao
BUG: Unable to handle kernel data access on read at 0x7c0802a6
Faulting instruction address: 0xc002e9f0
Oops: Kernel access of bad area, sig: 11 [#1]
BE PAGE_SIZE=4K PowerPC 44x Platform
CPU: 0 PID: 89 Comm: sh Not tainted 5.13.0-rc1-01824-g3a81c0495fdb #7
NIP: c002e9f0 LR: c0011858 CTR: 00008a47
REGS: c292fd50 TRAP: 0300 Not tainted (5.13.0-rc1-01824-g3a81c0495fdb)
MSR: 00009000 <EE,ME> CR: 24002002 XER: 20000000
DEAR: 7c0802a6 ESR: 00000000
<snip>
NIP [c002e9f0] emulate_step+0x28/0x324
LR [c0011858] optinsn_slot+0x128/0x10000
opt_pre_handler+0x7c/0xb4 (unreliable)
optinsn_slot+0x128/0x10000
ret_from_syscall+0x0/0x28
81 24 00 00 lwz r9,0(r4)
struct ppc_inst, which is the instruction to be emulated. On ppc64,
structures are passed in registers when passed by value. However, per
the ppc32 ABI, structures are always passed to functions as pointers.
This isn't being adhered to when setting up the call to emulate_step()
in the optprobe trampoline. Fix the same.
Fixes: eacf4c0202654a ("powerpc: Enable OPTPROBES on PPC32")
---
arch/powerpc/kernel/optprobes.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

Christophe,
Can you confirm if this patch works for you? It would be good if this
can go in v5.13.

Thanks,
Naveen

Christophe Leroy

2021-06-07 14:31:04 UTC

    BUG: Unable to handle kernel data access on read at 0x7c0802a6
    Faulting instruction address: 0xc002e9f0
    Oops: Kernel access of bad area, sig: 11 [#1]
    BE PAGE_SIZE=4K PowerPC 44x Platform
    CPU: 0 PID: 89 Comm: sh Not tainted 5.13.0-rc1-01824-g3a81c0495fdb #7
    NIP: c002e9f0 LR: c0011858 CTR: 00008a47
    REGS: c292fd50 TRAP: 0300   Not tainted (5.13.0-rc1-01824-g3a81c0495fdb)
    MSR: 00009000 <EE,ME> CR: 24002002 XER: 20000000
    DEAR: 7c0802a6 ESR: 00000000
    <snip>
    NIP [c002e9f0] emulate_step+0x28/0x324
    LR [c0011858] optinsn_slot+0x128/0x10000
     opt_pre_handler+0x7c/0xb4 (unreliable)
     optinsn_slot+0x128/0x10000
     ret_from_syscall+0x0/0x28
    81 24 00 00     lwz     r9,0(r4)
struct ppc_inst, which is the instruction to be emulated. On ppc64,
structures are passed in registers when passed by value. However, per
the ppc32 ABI, structures are always passed to functions as pointers.
This isn't being adhered to when setting up the call to emulate_step()
in the optprobe trampoline. Fix the same.
Fixes: eacf4c0202654a ("powerpc: Enable OPTPROBES on PPC32")
---
arch/powerpc/kernel/optprobes.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

Christophe,
Can you confirm if this patch works for you? It would be good if this can go in v5.13.

I'm trying to use kprobes, but I must be missing something. I have tried to follow the exemple in
kernel's documentation:

# echo 'p:myprobe do_sys_open dfd=%r3' > /sys/kernel/debug/tracing/kprobe_events

# echo 1 > /sys/kernel/debug/tracing/events/kprobes/myprobe/enable

# cat /sys/kernel/debug/kprobes/list

c00122e4 k kretprobe_trampoline+0x0 [OPTIMIZED]
c018a1b4 k do_sys_open+0x0 [OPTIMIZED]

# cat /sys/kernel/debug/tracing/tracing_on

1

# cat /sys/kernel/debug/tracing/trace

# tracer: nop
#
# entries-in-buffer/entries-written: 0/0 #P:1
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |

So it looks like I get no event. I can't believe that do_sys_open() is never hit.

This is without your patch, so it should Oops ?

Then it looks like something is locked up somewhere, because I can't do anything else:

# echo 'p:myprobe2 do_sys_openat2 dfd=%r3' >/sys/kernel/debug/tracing/kprobe_events

-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy

# echo '-:myprobe' > /sys/kernel/debug/tracing/kprobe_events

-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy

# echo > /sys/kernel/debug/tracing/kprobe_events

-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy

Christophe

Christophe Leroy

2021-06-07 17:36:13 UTC

Christophe,
Can you confirm if this patch works for you? It would be good if this can go in v5.13.

I'm trying to use kprobes, but I must be missing something. I have tried to follow the exemple in
# echo 'p:myprobe do_sys_open dfd=%r3' > /sys/kernel/debug/tracing/kprobe_events
# echo 1 > /sys/kernel/debug/tracing/events/kprobes/myprobe/enable
# cat /sys/kernel/debug/kprobes/list
c00122e4 k kretprobe_trampoline+0x0    [OPTIMIZED]
c018a1b4 k do_sys_open+0x0    [OPTIMIZED]
# cat /sys/kernel/debug/tracing/tracing_on
1
# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 0/0   #P:1
#
#                                _-----=> irqs-off
#                               / _----=> need-resched
#                              | / _---=> hardirq/softirq
#                              || / _--=> preempt-depth
#                              ||| /     delay
#           TASK-PID     CPU# ||||   TIMESTAMP FUNCTION
#              | |         |   ||||      |         |
So it looks like I get no event. I can't believe that do_sys_open() is never hit.
This is without your patch, so it should Oops ?
# echo 'p:myprobe2 do_sys_openat2 dfd=%r3' >/sys/kernel/debug/tracing/kprobe_events
-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy
# echo '-:myprobe' > /sys/kernel/debug/tracing/kprobe_events
-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy
# echo > /sys/kernel/debug/tracing/kprobe_events
-sh: can't create /sys/kernel/debug/tracing/kprobe_events: Device or resource busy

Ok, did a new test. Seems like do_sys_open() is really never called.
I set the test at do_sys_openat2 , it was not optimised and was working.
I set the test at do_sys_openat2+0x10 , it was optimised and crashed.
Now I'm going to test the patch.

When I set an event, is that normal that it removes the previous one ? Then we can have only one
event at a time ? And then when that event is enabled we get 'Device or resource busy' when trying
to add a new one ?

Christophe

Christophe Leroy

2021-06-08 04:58:32 UTC