From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-4323.proton.ch (mail-4323.proton.ch [185.70.43.23]) by sourceware.org (Postfix) with ESMTPS id 38B9C3856254 for ; Fri, 13 May 2022 11:41:24 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 38B9C3856254 Date: Fri, 13 May 2022 11:41:13 +0000 To: GDB mailing list From: Jan Vrany Reply-To: Jan Vrany Subject: Python API to support JIT Message-ID: Feedback-ID: 40767693:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, SPF_HELO_NONE, SPF_PASS, TXREP, T_SCC_BODY_TEXT_LINE, WEIRD_PORT autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org X-BeenThere: gdb@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Gdb mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2022 11:41:27 -0000 Hi there, I just want to give you a heads-up on an experiment I'm (slowly) working= =C2=A0 on: a set of Python APIs to support JIT code. As of now, we have extensive python API to have custom pretty printers, frame decorators and frame unwinders - all these are essential= =C2=A0 when one wants good support for managed runtimes (such as JVM). JITed code= =C2=A0 often uses custom calling convention so custom unwinders are needed. Manage= d=C2=A0 heap objects are quite often just untyped pointers to managed memory so one need custom pretty printer to see "into" those objects. There's a "JIT reader API" which allows one to load custom code into GDB wh= ich then tells GDB that this range of memory belongs to function named "foo" an= d even tell which PC corresponds to what line. However, this JIT reader API it a C API so it requires one to implement tha= t logic in C. This means that you have half the support in written in C, other half= in Python. And I find it far more difficult to debug C-implemented JIT reader than any= Python script (such as pretty printer or unwinder). So I decided to try to close this gap and extended Python API which can "te= ach" GDB about dynamically compiled code [1]. As an example, I implemented an unwind= er and JIT-code registration code for OpenJ9 on RISC-V [2], [3]. As of now, it can do little more than (already existing) JIT reader API. He= re's how code registration looks in Python (keep in mind all this is just a prototype and= demo, so I'm cutting corners wherever I can :-) --- def registerCompiled(self): """ Register method in GDB. """ # First. build GDB line table (don't confuse this with # Java line number table) lineTable =3D [] pc =3D self.startPC while pc < self.endPC: line =3D self.lineNumberTable[self.bytecodeTable[pc]] lineTable.append(gdb.LineTableEntry(line, pc)) pc =3D pc + 4 self._objfile =3D gdb.Objfile(self.name) self._symtab =3D gdb.Symtab(self._objfile, self.className + '.java'= ) self._symtab.set_linetable(lineTable) self._block =3D self._symtab.add_block(self.name, self.startPC, sel= f.endPC) --- This is called from a breakpoint on JIT compiler function that finalizes th= e compilation: --- class MethodInfoRegistrar(gdb.Breakpoint): def __init__(self): super().__init__("TR::CompilationInfoPerThreadBase::logCompilationS= uccess", internal=3DFalse) def stop(self): metaData =3D gdb.newest_frame().read_var('metaData') compiler =3D gdb.newest_frame().read_var('compiler') methodInfo =3D MethodInfo(metaData, compiler) methodInfo.registerCompiled() return True # Stop --- With this in place, I can use standard gdb commands to set breakpoints, sho= w backtrace, source or disassembly: (gdb) b 'MinimalTest.jit2jitFactorial(I)I' Breakpoint 2 at 0x3f5b636028: file MinimalTest.java, line 115. (gdb) c Continuing. + (cold) MinimalTest.jit2jitFactorial(I)I @ 0000003F5B636024-0000003F5B6360= B8 OrdinaryMethod - Q_SZ=3D0 Q_SZI=3D0 QW=3D2 j9m=3D0000003FF054FFE0 bcsz= =3D16 sync compThreadID=3D0 CpuLoad=3D101%(25%avg) JvmCpu=3D101% [Switching to Thread 356032.356034] Thread 2 "main" hit Breakpoint 2, 0x0000003f5b636028 in MinimalTest.jit2jit= Factorial(I)I () at MinimalTest.java:115 115=09=09=09if (i =3D=3D 0) { (gdb) bt 5 #0 0x0000003f5b636028 in MinimalTest.jit2jitFactorial(I)I () at MinimalTes= t.java:115 #1 0x0000003ff7cf88e2 in c_cInterpreter () at /home/jv/Projects/J9/openj9-openjdk-jdk11-devscripts/openj9-openjdk-= jdk11/build/linux-riscv64-normal-server-slowdebug/vm/runtime/vm/riscvcinter= p.s:266 #2 0x0000003ff7beee6a in sidecarInvokeReflectMethodImpl (currentThread=3D0= x3ff00b0400, methodRef=3D0x3ff00ff3c0, recevierRef=3D0x3ff00ff3b8, argsRef=3D0x3ff00ff3b0) at /home/jv/Projects/J9/openj9-openjdk-jdk11-devscripts/openj9-openjdk-= jdk11/openj9/runtime/vm/callin.cpp:1209 #3 0x0000003ff7bef500 in sidecarInvokeReflectMethod (currentThread=3D0x3ff= 00b0400, methodRef=3D0x3ff00ff3c0, recevierRef=3D0x3ff00ff3b8, argsRef=3D0x3ff00ff3b0) at /home/jv/Projects/J9/openj9-openjdk-jdk11-devscripts/openj9-openjdk-= jdk11/openj9/runtime/vm/callin.cpp:1353 #4 0x0000003ff67519e0 in JVM_InvokeMethod_Impl (env=3D0x3ff00b0400, method= =3D0x3ff00ff3c0, obj=3D0x3ff00ff3b8, args=3D0x3ff00ff3b0) at /home/jv/Projects/J9/openj9-openjdk-jdk11-devscripts/openj9-openjdk-= jdk11/openj9/runtime/sunvmi/sunvmi.c:363 (More stack frames follow...) (gdb) disassemble /s 'MinimalTest.jit2jitFactorial(I)I' Dump of assembler code for function MinimalTest.jit2jitFactorial(I)I: MinimalTest.java: 115=09=09=09if (i =3D=3D 0) { 0x0000003f5b636024 <+0>:=09ld=09a0,8(s11) =3D> 0x0000003f5b636028 <+4>:=09lw=09a1,0(s11) 0x0000003f5b63602c <+8>:=09sd=09ra,-8(s11) 0x0000003f5b636030 <+12>:=09addi=09s11,s11,-48 0x0000003f5b636034 <+16>:=09ld=09t3,80(s10) 0x0000003f5b636038 <+20>:=09blt=09s11,t3,0x3f5b6360b4 0x0000003f5b63603c <+24>:=09sd=09s0,16(s11) 0x0000003f5b636040 <+28>:=09sd=09s1,24(s11) 0x0000003f5b636044 <+32>:=09sd=09a0,56(s11) 0x0000003f5b636048 <+36>:=09sw=09a1,48(s11) 0x0000003f5b63604c <+40>:=09lw=09s1,48(s11) 0x0000003f5b636050 <+44>:=09sext.w=09s0,zero 0x0000003f5b636054 <+48>:=09bne=09s1,s0,0x3f5b636070 116=09=09=09=09return 1; 0x0000003f5b636058 <+52>:=09addiw=09a0,zero,1 0x0000003f5b63605c <+56>:=09ld=09s0,16(s11) 0x0000003f5b636060 <+60>:=09ld=09s1,24(s11) 0x0000003f5b636064 <+64>:=09addi=09s11,s11,48 0x0000003f5b636068 <+68>:=09ld=09ra,-8(s11) 0x0000003f5b63606c <+72>:=09ret 117=09=09=09} else { 118=09=09=09=09return jit2jitFactorial(i - 1) * i; 0x0000003f5b636070 <+76>:=09ld=09s0,56(s11) 0x0000003f5b636074 <+80>:=09mv=09a0,s0 0x0000003f5b636078 <+84>:=09lw=09s1,48(s11) 0x0000003f5b63607c <+88>:=09addiw=09a1,s1,-1 0x0000003f5b636080 <+92>:=09ld=09s0,0(s0) 0x0000003f5b636084 <+96>:=09lui=09t3,0x0 0x0000003f5b636088 <+100>:=09addi=09t3,t3,-192 0x0000003f5b63608c <+104>:=09add=09t3,s0,t3 0x0000003f5b636090 <+108>:=09ld=09t3,0(t3) 0x0000003f5b636094 <+112>:=09jalr=09t3 0x0000003f5b636098 <+116>:=09mv=09s0,a0 0x0000003f5b63609c <+120>:=09mulw=09a0,s1,s0 0x0000003f5b6360a0 <+124>:=09ld=09s0,16(s11) 0x0000003f5b6360a4 <+128>:=09ld=09s1,24(s11) 0x0000003f5b6360a8 <+132>:=09addi=09s11,s11,48 0x0000003f5b6360ac <+136>:=09ld=09ra,-8(s11) 0x0000003f5b6360b0 <+140>:=09ret 0x0000003f5b6360b4 <+144>:=09ebreak End of assembler dump. GDB/MI commands work as well so one can use GDB/MI frontend to debug JITed = code. As I said, this is still very much work in progress with many rough edges. If anyone's interested, have questions or suggestions or working on the same problem, please let me know! I'd welcome any feedback! Best, Jan [1]: https://sourceware.org/git/?p=3Dbinutils-gdb.git;a=3Dshortlog;h=3Drefs= /heads/users/jv/wip/feature-py-jit-api [2]: https://github.com/janvrany/openj9-gdb [3]: https://github.com/janvrany/omr-gdb