Repository navigation
Expand file tree
/
Copy pathindex.json
More file actions
1 lines (1 loc) · 221 KB
/
Copy pathindex.json
File metadata and controls
1 lines (1 loc) · 221 KB
1
[{"authors":null,"categories":[],"content":"LIEF (Library to Instrument Executable Formats) is an Apache-2.0-licensed, cross-platform library to parse, inspect, modify, and build ELF, PE, Mach-O, and more through one consistent C++, Python, or Rust API. A subset of the API is also exposed in C. Key facts Latest release: LIEF 1.0.0 (2026-07-12) — https://github.com/lief-project/LIEF/releases/tag/1.0.0 Executable formats: ELF, PE, Mach-O, COFF, and more (see the documentation for the full list); debug formats: DWARF and PDB LIEF Extended adds an assembler, a disassembler, a DWARF editor, and PDB support on the same object model Languages: C++ core with Python, Rust, and C APIs Platforms: Linux, Windows, macOS, Android, and iOS License: Apache-2.0 API documentation: https://lief.re/doc/latest/ Downloads: https://lief.re/download/ Source code: https://github.com/lief-project/LIEF Python package: https://pypi.org/project/lief/ Rust crate: https://crates.io/crates/lief LIEF Extended: https://extended.lief.re/ Documentation for AI assistants Read the AI documentation guide at https://lief.re/doc/latest/ai.md and start with the discovery map at https://lief.re/doc/latest/llms.txt to find guides and API references. In the latest documentation, replace .html with .md to read resolved Markdown with code examples, every language tab, API signatures, and absolute citation links. Documentation content index: https://lief.re/doc/latest/index.json Section and API symbol retrieval: https://lief.re/doc/latest/index.chunks.json Complete documentation corpus for local indexing: https://lief.re/doc/latest/llms-full.txt (large API references may exceed an assistant’s context window). The latest channel tracks development; stable documents the latest release at https://lief.re/doc/stable/. Match API usage to the installed LIEF version, platform, architecture, and enabled features. One library, every layer Move from raw executable file formats to a high-level representation, make changes, then write a valid binary back to disk. Unified abstraction Work with symbols, sections, relocations, and entry points through shared concepts across executable formats. Parse and inspect Open binaries from files or memory and explore their structure. Modify and rebuild Add sections, change symbols, patch metadata, and serialize the result. Example: binary inspection in a few lines The same concepts stay the same whether you work in Python, C++, or Rust. 1import lief 2 3# Parse ELF, PE, or Mach-O with one entry point 4binary = lief.parse(\"/usr/bin/ssh\") 5 6print(binary.format) 7print(hex(binary.entrypoint)) 8 9for section in binary.sections: 10 print(section.name, section.size) 11 12# Modify and write a new binary 13binary.header.entrypoint = 0x401000 14binary.write(\"ssh.patched\") LIEF Extended Advanced assembly, disassembly, DWARF, and PDB tooling build on the same LIEF model for enhanced binary reverse-engineering workflows. Assembler: assemble x86-64, ARM64, and RISC-V instructions directly from the API. Disassembler: decode machine code and connect instructions to executable metadata. DWARF editor: inspect and transform rich debug information with structured APIs. PDB support: work with Microsoft program databases alongside the binary they describe. Learn more: https://extended.lief.re/ and https://lief.re/doc/latest/extended/intro.html Latest articles LIEF v1.0.0: 2026-07-13 — LIEF 1.0.0: a new cross-platform Runtime API, faster Rust bindings, stable-ABI and free-threaded Python wheels LIEF v0.17.0: 2025-09-14 — LIEF 0.17.0: Binary Ninja and Ghidra plugins, contextual assembly patching, a refactored PE module with TLS, import, and export editing, lief-patchelf, and COFF support. LIEF patchelf: 2025-07-13 — lief-patchelf: a LIEF-based reimplementation of NixOS patchelf to change the interpreter, add dependencies, and edit RPATH/RUNPATH, with prebuilt binaries. All articles: https://lief.re/blog/ Support the project Sponsorship gives LIEF maintainers time to fix bugs, keep up with new toolchains and OS releases, and work on new features. Sponsor LIEF: https://github.com/sponsors/lief-project/ Discuss a feature: contact@lief.re Community chat: https://discord.gg/jGQtyAYChJ","date":-62135596800,"description":"LIEF is a cross-platform C++, Python, and Rust library to parse, analyze, modify, assemble, disassemble, and rewrite ELF, PE, Mach-O, and other executable formats.","expirydate":-62135596800,"kind":"home","lang":"en","lastmod":1794253191,"markdown":"https://lief.re/index.md","objectID":"3976528693a0108357f4928017600865","permalink":"https://lief.re/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/","section":"","summary":"LIEF is a cross-platform C++, Python, and Rust library to parse, analyze, modify, assemble, disassemble, and rewrite ELF, PE, Mach-O, and other executable formats.","tags":[],"title":"LIEF: Library to Instrument Executable Formats","type":"page","wordcount":526},{"authors":null,"categories":[],"content":"Install LIEF from a package manager, download a prebuilt SDK archive, or build it from source. The current release is LIEF 1.0.0 (2026-07-12); release notes: https://github.com/lief-project/LIEF/releases/tag/1.0.0 Quick install Python: pip install lief — stable-ABI wheels on PyPI (https://pypi.org/project/lief/), including free-threaded variants where supported. Rust: cargo add lief — the lief crate on crates.io (https://crates.io/crates/lief). From source: clone https://github.com/lief-project/LIEF and follow the compilation guide at https://lief.re/doc/latest/compilation.html. Prebuilt packages Official SDK archives and language packages for supported systems and architectures. Platform Version Package URL Linux 1.0.0 SDK x86-64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-x86_64.tar.gz Linux 1.0.0 SDK x86-64 (Musl) https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-musl-x86_64.tar.gz Linux 1.0.0 SDK AArch64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-aarch64.tar.gz Linux 1.0.0 SDK RISC-V 64-bit https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-riscv64.tar.gz Linux 1.0.0 SDK i686 (Musl) https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-musl-i686.tar.gz Linux 1.0.0 SDK AArch64 (Musl) https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-musl-aarch64.tar.gz Linux 1.0.0 SDK RISC-V 64-bit (Musl) https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Linux-musl-riscv64.tar.gz Linux 1.0.0 Python Packages https://pypi.org/project/lief/ Android 1.0.0 SDK ARM https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Android-armv7-a.tar.gz Android 1.0.0 SDK AArch64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Android-aarch64.tar.gz Android 1.0.0 SDK x86-64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Android-x86_64.tar.gz macOS 1.0.0 SDK x86-64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Darwin-x86_64.tar.gz macOS 1.0.0 SDK AArch64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-Darwin-arm64.tar.gz macOS 1.0.0 Python Packages https://pypi.org/project/lief/ iOS 1.0.0 SDK AArch64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-iOS-arm64.tar.gz Windows 1.0.0 SDK x86-64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-win64.zip Windows 1.0.0 SDK x86 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-win32.zip Windows 1.0.0 SDK ARM64 https://github.com/lief-project/LIEF/releases/download/1.0.0/LIEF-1.0.0-windows-arm64.zip Windows 1.0.0 Python packages https://pypi.org/project/lief/ Nightly main SDK https://lief.s3-website.fr-par.scw.cloud/latest/sdk/index.html Nightly main Python Packages https://lief.s3-website.fr-par.scw.cloud/latest/lief/index.html Package channels GitHub releases (SDK archives and release notes): https://github.com/lief-project/LIEF/releases PyPI (Python wheels): https://pypi.org/project/lief/ crates.io (Rust crate): https://crates.io/crates/lief LIEF Extended (assembler, disassembler, DWARF, PDB): https://extended.lief.re/","date":-62135596800,"description":"Download LIEF SDKs and packages for Linux, Windows, macOS, Android, and iOS, or install the Python and Rust bindings.","expirydate":-62135596800,"kind":"section","lang":"en","lastmod":1794253191,"markdown":"https://lief.re/download/index.md","objectID":"ea2ec00a7eb2a149cca0f3dfbe8a00b0","permalink":"https://lief.re/download/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/download/","section":"download","summary":"Download LIEF SDKs and packages for Linux, Windows, macOS, Android, and iOS, or install the Python and Rust bindings.","tags":[],"title":"Download","type":"download","wordcount":217},{"authors":null,"categories":[],"content":"LIEF is a cross-platform library to parse, modify, and abstract ELF, PE, and Mach-O binaries. It handles the low-level details of each format so you can focus on your own project. Project facts More than ten years of continuous development More than 1,000,000 installs every month Four language APIs: C++, Python, Rust, and C Apache-2.0 license Maintainer: Romain Thomas Project history 2017 — First public release: LIEF became open source with initial ELF, PE, and Mach-O parsing and modification support. 2019 — Builders mature: format-aware rewriting expanded with stronger Mach-O modifications and builder APIs. 2021 — Security metadata: PE Authenticode parsing and verification brought signatures into the unified object model. 2022 — ELF rewriting at scale: a redesigned ELF builder improved performance, memory behavior, and control over generated binaries. 2024 — Rust and debug formats: Rust bindings evolved while DWARF and PDB support opened new analysis and tooling workflows. 2026 — LIEF 1.0: a milestone focused on stability, usability, security, and a new cross-platform Runtime API. Community Contribute on GitHub: https://github.com/lief-project/LIEF Join the Discord server: https://discord.gg/jGQtyAYChJ Sponsor LIEF: https://github.com/sponsors/lief-project/ Contact the maintainers: contact@lief.re How LIEF works: https://lief.re/doc/latest/intro.html","date":-62135596800,"description":"Learn how LIEF became a cross-platform executable-format toolkit, meet its maintainers, and explore the open-source project's milestones.","expirydate":-62135596800,"kind":"section","lang":"en","lastmod":1794253191,"markdown":"https://lief.re/about/index.md","objectID":"a69934980f795699f7cc4d8a50a604b2","permalink":"https://lief.re/about/","publishdate":"0001-01-01T00:00:00Z","relpermalink":"/about/","section":"about","summary":"Learn how LIEF became a cross-platform executable-format toolkit, meet its maintainers, and explore the open-source project's milestones.","tags":[],"title":"About","type":"about","wordcount":187},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"I’m really happy to announce the release of LIEF 1.0.0. Compared to previous versions, this release represents an important milestones for this project: stability, usability, and security. LIEF started almost ten years ago. While the journey was driven by adding new features, it also involved correcting a handful of early design decisions that turned out to be wrong and have been corrected, release after release. I won’t pretend this version is bug-free or that it does not need further improvements, but it now rests on solid foundations, and the project has been adopted across a range of industries with positive feedback. A special thanks goes to Quansight, which generously sponsored this project. Many thanks as well to Holepunch for their feature-based sponsorship, and to Calif for reviewing LIEF against frontier models (aka Mythos \u0026 GPT‑5‑Cyber). Bindings: Rust \u0026 Python The LIEF Python wheels are now built against the stable ABI and now offer a free-threaded variant. Building against the stable ABI lets a single wheel serve multiple interpreter versions, while the free-threaded variant lets you use LIEF with the GIL disabled. This variant is available for Python 3.14 and 3.15 onward. The C++ core is now thread-safe with respect to its few static variables, so it behaves correctly under free-threading. On the Rust side, I refactored the bindings to drop the unmaintained autocxx dependency. They are now built directly on top of cxx, without any extra wrapper. This change dramatically reduces compilation and iteration time. The crates also moved from api/rust/cargo/ to api/rust/crates/, and the minimum supported Rust version is now 1.85.0. Clang Lifetime Annotations I1 have started annotating the LIEF API with Clang lifetime annotations [[clang::lifetimebound]] to strengthen the compile-time verification of the codebase. These annotations enable the compiler to catch issues like this one: 1std::string lifetime_examples() { 2 Section* text = nullptr; 3 { 4 std::unique_ptr\u003cBinary\u003e elf = Parser::parse(\"/bin/ls\"); 5 text = elf-\u003eget_section(\".text\"); 6 } 7 return text-\u003ename(); 8} When compiled under strict lifetime-analysis flags, Clang raises the following error: example.cpp:50:12: error: local variable 'elf' does not live long enough [clang-diagnostic-lifetime-safety-use-after-scope] 50 | text = elf-\u003eget_section(\".text\"); | ^~~ example.cpp:51:3: note: local variable 'elf' is destroyed here 51 | } | ^ example.cpp:50:12: note: expression aliases the storage of local variable 'elf' 50 | text = elf-\u003eget_section(\".text\"); | ^~~~~ example.cpp:50:12: note: result of call to 'get_section' aliases the storage of local variable 'elf' 50 | text = elf-\u003eget_section(\".text\"); | ^~~~~~~~~~~~~~~~~~~~~~~~~ example.cpp:52:10: note: later used here 52 | return text-\u003ename(); | ^~~~ Executable File Format Improvements Mach-O support has been extended to include load commands introduced in recent versions of dyld: LC_LAZY_LOAD_DYLIB_INFO, LC_FUNCTION_VARIANTS, and LC_FUNCTION_VARIANT_FIXUPS. LIEF can now also create a FAT (universal) binary from several thin Mach-O binaries, select a specific architecture out of an existing FAT binary, and write big-endian Mach-O files. The ELF rewriter has been reworked to reduce the memory footprint of modified binaries: a rewritten ELF is now smaller than in previous releases, and LIEF can safely modify a binary it has already modified. Finally, thanks thanks to a security review by Calif, the various parsers are now safer and keep tighter control over how much memory they allocate when facing malformed inputs. Supported Platforms This release introduces pre-compiled packages for several new architectures and platforms: Python wheels are now available for Linux RISC-V 64-bit (including musl-based libc) SDK and Rust pre-compiled packages are provided for: Android x86-64, ARM64 \u0026 arm-v7a Linux riscv64gc (musl), riscv64a23 (glibc) Runtime API One of the most exciting additions in this release is the new Runtime API. The motivation came from a recurring need: parsing ELF, Mach-O and PE binaries directly from memory. LIEF already had everything required to parse a binary from a raw pointer thanks to its BinaryStream abstraction, but it lacked a friendly, high-level bridge to reach it. The Runtime API is that bridge: 1import lief 2 3for module in lief.runtime.modules(): 4 print(module) 5 binary = module.parse_from_memory() These runtime features go well beyond parsing an executable from memory. They provide cross-platform, cross-language access to inspect and manipulate the process in which LIEF is loaded. To make this concrete, here is a Python example that JIT-compiles and executes a Windows ARM64 “Hello World” using the new Runtime API. First, we allocate a chunk of memory to hold the assembled code: 1import lief 2 3chunk = lief.runtime.Memory.mmap( 4 lief.runtime.Process.page_size, 5 lief.runtime.Memory.ANONYMOUS | lief.runtime.Memory.PRIVATE, 6 lief.runtime.Memory.READ | lief.runtime.Memory.WRITE | lief.runtime.Memory.EXEC, 7) Then, we assemble our code straight into the chunk with assemble(...). This function is backed by LIEF’s assembly engine: 1lief.runtime.assemble( 2 chunk.addr, 3 r\"\"\" 4 .text 5 .global win_arm64_hello 6 .align 2 7 8 win_arm64_hello: 9 stp x29, x30, [sp, -64]! 10 mov x29, sp 11 stp x19, x20, [sp, 16] 12 stp x21, x22, [sp, 32] 13 // ------------------------------- 14 // Hello World code goes here 15 // ------------------------------- 16 mov x0, xzr 17 ldp x21, x22, [sp, 32] 18 ldp x19, x20, [sp, 16] 19 ldp x29, x30, [sp], 64 20 ret 21 \"\"\", 22) While we could implement the “Hello World” as pure, low-level shellcode issuing raw syscalls, we can also leverage the Contextual Assembly Patching feature to resolve symbols like GetStdHandle and WriteFile on the fly: 1class Config(lief.assembly.AssemblerConfig): 2 def resolve_symbol(self, name: str) -\u003e int | None: 3 kernel32 = lief.runtime.windows.dlopen(\"kernel32.dll\") 4 match name: 5 case \"GetStdHandle\": 6 return lief.to_int(kernel32.dlsym(\"GetStdHandle\")) 7 case \"WriteFile\": 8 return lief.to_int(kernel32.dlsym(\"WriteFile\")) 9 case _: 10 return None 11 12config = Config() 13 14lief.runtime.assemble( 15 chunk.addr, 16 r\"\"\" 17 .text 18 .global win_arm64_hello 19 .align 2 20 21 win_arm64_hello: 22 stp x29, x30, [sp, -64]! 23 mov x29, sp 24 stp x19, x20, [sp, 16] 25 stp x21, x22, [sp, 32] 26 // ------------------------------- 27 ldr x19, =GetStdHandle 28 ldr x20, =WriteFile 29 // ------------------------------- 30 mov x0, xzr 31 ldp x21, x22, [sp, 32] 32 ldp x19, x20, [sp, 16] 33 ldp x29, x30, [sp], 64 34 ret 35 \"\"\", config, 36) The same mechanism can also resolve data symbols. Let’s expose our message buffer and its length through the configuration, then actually call the two functions to print the string: 1msg = b\"Hello World\\n\" 2ctype_msg = ctypes.create_string_buffer(msg) 3 4class Config(lief.assembly.AssemblerConfig): 5 def resolve_symbol(self, name: str) -\u003e int | None: 6 kernel32 = lief.runtime.windows.dlopen(\"kernel32.dll\") 7 match name: 8 case \"GetStdHandle\": 9 return lief.to_int(kernel32.dlsym(\"GetStdHandle\")) 10 case \"WriteFile\": 11 return lief.to_int(kernel32.dlsym(\"WriteFile\")) 12 case \"var_msg\": 13 return ctypes.addressof(ctype_msg) 14 case \"var_msg_len\": 15 return len(msg) 16 case _: 17 return None 18 19config = Config() 20 21lief.runtime.assemble( 22 chunk.addr, 23 r\"\"\" 24 .text 25 .global win_arm64_hello 26 .align 2 27 28 win_arm64_hello: 29 stp x29, x30, [sp, -64]! 30 mov x29, sp 31 stp x19, x20, [sp, 16] 32 stp x21, x22, [sp, 32] 33 // ------------------------------- 34 ldr x19, =GetStdHandle 35 ldr x20, =WriteFile 36 ldr x21, =var_msg 37 ldr w22, =var_msg_len 38 39 // GetStdHandle(STD_OUTPUT_HANDLE=-11) -\u003e x0 40 mov w0, -11 41 blr x19 42 43 // WriteFile(x0, msg, len, \u0026written, NULL) 44 mov x1, x21 45 mov w2, w22 46 add x3, sp, 48 47 mov x4, xzr 48 blr x20 49 // ------------------------------- 50 mov x0, xzr 51 ldp x21, x22, [sp, 32] 52 ldp x19, x20, [sp, 16] 53 ldp x29, x30, [sp], 64 54 ret 55 \"\"\", config, 56) Once the code is assembled in memory, we flush the instruction cache, flip the region to read-execute, and call it like a regular function: 1lief.runtime.assemble(chunk.addr, \"...\", config) 2 3chunk.cache_flush() 4chunk.make_rx() 5 6void_void_func = ctypes.CFUNCTYPE(None) # void(*)() 7hello_jit = void_void_func(chunk.addr) 8 9hello_jit() # prints \"Hello World\" The same example is available for the Rust bindings and the C++ API, and you’ll find the Linux, Android, and macOS equivalents in the runtime documentation. The Runtime API also works the other way around: you can disassemble live code directly from memory. Here is a Rust example that disassembles a function and flags every instruction that touches the RISC-V stack pointer (x2): 1use lief::assembly::{Instructions, riscv::{Operands, Reg}}; 2 3fn say_hello() { 4 println!(\"Hello World\"); 5} 6 7fn main() { 8 for inst in lief::runtime::disassemble((say_hello as usize).try_into().unwrap()) { 9 println!(\"{inst}\"); 10 11 let Instructions::RiscV(riscv) = \u0026inst else { 12 continue; 13 }; 14 15 let uses_sp = riscv 16 .operands() 17 .any(|op| matches!(op, Operands::Mem(mem) if mem.base() == Reg::X2)); 18 19 if uses_sp { 20 println!(\"{inst} is a memory operation that uses x2 (sp)\"); 21 } 22 } 23} Runtime API You can explore the rest of the Runtime API in the runtime documentation. Because these features go beyond executable formats, they are only activated in the extended version of LIEF. However, you can also compile LIEF from source with the Runtime API enabled. LIEF Extended C++ Standard LIEF Extended and its components are now built using C++23. The supported platforms are: Linux: Ubuntu 24.04, Fedora 40, RHEL 10 / CentOS 10, Debian 13: ARM64, x86-64 (RISC-V soon) macOS 15.0+ (Sequoia): ARM64 \u0026 x86-64 Windows 11+: ARM64 \u0026 x86-64 Android API 30+: ARM64 \u0026 x86-64 (including Python wheels) This change does not affect the LIEF core library, which remains compatible with C++11 and only requires a C++17 compiler to build. DWARF \u0026 PDB -\u003e C/C++ LIEF Extended can now generate C/C++ declarations from DWARF and PDB debug information. For example, consider this libdexprotector.so binary, enriched with DWARF debug info recovered via reverse engineering. In Binary Ninja, it looks like this: 0000abc0 } .data (PROGBITS) section ended {0xa150-0xabc0} .bss (NOBITS) section started {0xabc0-0xabe0} 0000abc0 char prctl_data[0x6] = \"\\x00\\x00\\x00\\x00\\x00\", 0 0000abc6 00 00 .. 0000abc8 // pointer to the r_debug structure defined in the linker(64) 0000abc8 struct r_debug_t* linker64_r_debug = nullptr 0000abd0 int32_t err = 0x0 0000abd4 00 00 00 00 .... 0000abd8 // Pointer to the dynamically unpacked JNI_OnLoad 0000abd8 jint (* JNI_OnLoad_ptr)(JavaVM* vm, void* reserved) = nullptr Calling to_decl() on a DWARF function, variable, or type generates its C/C++ declaration: 1import lief 2 3elf = lief.ELF.parse(\"libdexprotector.so.dwarf\") 4dwarf = elf.debug_info 5 6linker64_r_debug = dwarf.find_variable(\"linker64_r_debug\") 7print(linker64_r_debug.to_decl()) Which outputs: 1/* 2 * pointer to the r_debug structure defined in the linker(64) 3 * Addr: 0xabc8 4 * size: 0x0008 5 */ 6static struct r_debug_t *linker64_r_debug; C/C++ Declaration Note that the Binary Ninja comment, the address, and the sizeof of the variable are also generated. You can control the output format using the configuration options accepted by to_decl. to_decl also works with types: 1import lief 2 3elf = lief.ELF.parse(\"libdexprotector.so.dwarf\") 4dwarf = elf.debug_info 5 6linker64_r_debug = dwarf.find_variable(\"linker64_r_debug\") 7print(linker64_r_debug.to_decl()) 8 9ptr_type: lief.dwarf.types.Pointer = linker64_r_debug.type 10print(ptr_type.underlying_type.to_decl()) Which outputs: 1struct r_debug_t { 2 int r_version; 3 char __padding1__[4]; 4 struct link_map *r_map; 5 Elf64_Addr r_brk; 6 enum { 7 RT_CONSISTENT = 0U, 8 RT_ADD = 1U, 9 RT_DELETE = 2U 10 } r_state; 11 char __padding4__[4]; 12 Elf64_Addr r_ldbase; 13} You can also configure the output to include field offsets: 1import lief 2 3elf = lief.ELF.parse(\"libdexprotector.so.dwarf\") 4dwarf = elf.debug_info 5 6linker64_r_debug = dwarf.find_variable(\"linker64_r_debug\") 7print(linker64_r_debug.to_decl()) 8 9ptr_type: lief.dwarf.types.Pointer = linker64_r_debug.type 10- print(ptr_type.underlying_type.to_decl()) 11+ opt = lief.DeclOpt() 12+ opt.show_field_offsets = True 13+ print(ptr_type.underlying_type.to_decl(opt)) 1struct r_debug_t { 2 /* 0x00 */ int r_version; 3 /* 0x04 */ char __padding1__[4]; 4 /* 0x08 */ struct link_map *r_map; 5 /* 0x10 */ Elf64_Addr r_brk; 6 /* 0x18 */ enum { 7 RT_CONSISTENT = 0U, 8 RT_ADD = 1U, 9 RT_DELETE = 2U 10 } r_state; 11 /* 0x1c */ char __padding4__[4]; 12 /* 0x20 */ Elf64_Addr r_ldbase; 13} A function containing basic block comments like this: is translated as follows: 1/* 2 * Address: 0x063c 3 */ 4r_debug_t *dp_derive_key(key_t *key) { 5 /* 6 * Stack addr: -0x00d0 7 * size: 0x0080 8 */ 9 struct_4 var_d0; 10 /* Start: 0x000988 */ { 11 /* Start: 0x00098c */ { 12 //This block derives the key based on the assembly of r_debug.r_brk. 13 // 14 //Frida hooks this function such as the regular \"ret\" is transformed by a trampoline 15 } /* End: 0x000990 */ 16 } /* End: 0x0009a4 */ 17} The same feature works with PDB files: 1pdb = lief.pdb.load(\"ntdll.pdb/6192BFDB9F04442995FFCB0BE95172E14/ntdll.pdb\") 2peb = pdb.find_type(\"_PEB\") 3opt = lief.DeclOpt() 4opt.show_field_offsets = True 5print(peb.to_decl(opt)) 1 2struct _PEB { 3 /* 0x00 */ unsigned char InheritedAddressSpace; 4 /* 0x01 */ unsigned char ReadImageFileExecOptions; 5 /* 0x02 */ unsigned char BeingDebugged; 6 /* 0x03 */ unsigned char BitField; 7 /* 0x03 */ unsigned char ImageUsesLargePages : 1; 8 /* 0x03 */ unsigned char IsProtectedProcess : 1; 9 /* 0x03 */ unsigned char IsLegacyProcess : 1; 10 /* 0x03 */ unsigned char IsImageDynamicallyRelocated : 1; 11 /* 0x03 */ unsigned char SkipPatchingUser32Forwarders : 1; 12 /* 0x03 */ unsigned char SpareBits : 3; 13 /* 0x08 */ void Mutant; 14 /* 0x10 */ void ImageBaseAddress; 15 /* 0x18 */ struct _PEB_LDR_DATA *Ldr; 16 /* 0x20 */ struct _RTL_USER_PROCESS_PARAMETERS *ProcessParameters; 17 /* 0x28 */ void SubSystemData; 18 /* 0x30 */ void ProcessHeap; 19 [...] 20 /* 0x378 */ unsigned long TracingFlags; 21 /* 0x378 */ unsigned long HeapTracingEnabled : 1; 22 /* 0x378 */ unsigned long CritSecTracingEnabled : 1; 23 /* 0x378 */ unsigned long SpareTracingBits : 30; 24} RISC-V, MIPS, eBPF \u0026 PowerPC The disassembler can now expose the high-level semantics of instruction operands for RISC-V, MIPS, eBPF and PowerPC. Each operand is surfaced as a Register, Immediate, Memory or PCRelative object: 1import lief 2 3elf = lief.parse(\"hello.bpf.o\") 4 5section = elf.get_section(\"tp/syscalls/sys_enter_write\") 6instructions = list(elf.disassemble_from_bytes(bytes(section.content))) 7 8for inst in instructions: 9 print(inst) 10 for idx, op in enumerate(inst.operands): 11 match op: 12 case lief.assembly.ebpf.operands.Register(value=reg): 13 print(f\" op[{idx}]: Register -\u003e {reg}\") 14 15 case lief.assembly.ebpf.operands.Immediate(value=imm): 16 print(f\" op[{idx}]: Immediate -\u003e {imm}\") 17 18 case lief.assembly.ebpf.operands.Memory(): 19 print(f\" op[{idx}]: Memory -\u003e {op}\") 20 21 case lief.assembly.ebpf.operands.PCRelative(value=target): 22 print(f\" op[{idx}]: PCRelative -\u003e {target}\") Last Word The first internal version of LIEF was released in July 2016. Ten years later, the project is still alive and keeps growing, supported by a large and active community of users and contributors. Thank you to everyone who has been a part of this journey. The detailed changelog is available here: Changelog The initial pass over the codebase to add these annotations was bootstrapped with the help of AI, then reviewed and adjusted by hand. ↩︎","date":1783900800,"description":"LIEF 1.0.0: a new cross-platform Runtime API, faster Rust bindings, stable-ABI and free-threaded Python wheels","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1783900800,"markdown":"https://lief.re/blog/2026-07-13-lief-1-0-0/index.md","objectID":"ae0afe6c908cb08b565cf73119dfe53e","permalink":"https://lief.re/blog/2026-07-13-lief-1-0-0/","publishdate":"2026-07-13T00:00:00Z","relpermalink":"/blog/2026-07-13-lief-1-0-0/","section":"blog","summary":"After nearly ten years, LIEF hits 1.0.0: a milestone release built around stability, usability and security. This release also introduces a brand-new cross-platform Runtime API for inspecting and manipulating live processes.","tags":["release","Runtime API","Rust","Python","LIEF Extended","DWARF","PDB"],"title":"LIEF v1.0.0","type":"blog","wordcount":2323},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"This new version of LIEF introduces several improvements and features that expand the scope of LIEF’s use cases. Reverse Engineering Plugins Reverse engineering frameworks like Binary Ninja and Ghidra provide excellent support for analyzing instructions and functions. However, they might lack an in-depth analysis of all structures associated with executable formats. For example, the latest version of Ghidra (11.4.2) and Binary Ninja (5.1.8104) are not able to accurately process Windows ARM64EC binaries, which combine ARM64 code with x86_64. ARM64EC binaries use specific structures such as IMAGE_ARM64EC_METADATA, that are not (yet) recognized by most of the reverse engineering frameworks: DAT_1400111f8 XREF[1]: 140011be8(*) 1400111f8 02 ?? 02h 1400111f9 00 ?? 00h 1400111fa 00 ?? 00h 1400111fb 00 ?? 00h 1400111fc 00 ?? 00h 1400111fd 1e ?? 1Eh 1400111fe 01 ?? 01h 1400111ff 00 ?? 00h 140011200 02 ?? 02h 140011201 00 ?? 00h 140011202 00 ?? 00h 140011203 00 ?? 00h 140011204 b4 ?? B4h 140011205 1c ?? 1Ch 140011206 01 ?? 01h 140011207 00 ?? 00h 140011208 00 ?? 00h 140011209 90 ?? 90h 14001120a 01 ?? 01h 14001120b 00 ?? 00h 14001120c 00 ?? 00h 14001120d 10 ?? 10h 14001120e 01 ?? 01h 14001120f 00 ?? 00h 140011210 08 ?? 08h 140011211 10 ?? 10h 140011212 01 ?? 01h 140011213 00 ?? 00h 140011214 18 ?? 18h IMAGE_ARM64EC_METADATA_V2_1400111f8 XREF[1]: 140011be8(*) 1400111f8 02 00 00 00 00 1e 01 00 02 00 00 00 b4 IMAGE_ARM64EC_METADATA_V2 1c 01 00 00 90 01 00 00 10 01 00 08 10 01 00 18 10 01 00 10 10 01 00 20 10 01 1400111f8 02 00 00 00 00 1e 01 00 02 00 00 00 b4 IMAGE_ARM64EC_METADATA V1 1c 01 00 00 90 01 00 00 10 01 00 08 10 01 00 18 10 01 00 10 10 01 00 20 10 01 1400111f8 02 00 00 00 ddw 2h Version 1400111fc 00 1e 01 00 ibo32 IMAGE_ARM64EC_METADATA_CODE_RANGE_ARRAY_140011e00 CodeMap 140011200 02 00 00 00 ddw 2h CodeMapCount 140011204 b4 1c 01 00 ibo32 IMAGE_ARM64EC_CODE_RANGE_ENTRY_POINT_ARRAY_140011cb4 CodeRangesToEntryPoints 140011208 00 90 01 00 ibo32 IMAGE_ARM64EC_METADATA_REDIRECTION_ARRAY_140019000 RedirectionMetadata 14001120c 00 10 01 00 ibo32 DAT_140011000 __os_arm64x_dispatch_call_no_redirect 140011210 08 10 01 00 ibo32 DAT_140011008 __os_arm64x_dispatch_ret 140011214 18 10 01 00 ibo32 DAT_140011018 __os_arm64x_dispatch_call 140011218 10 10 01 00 ibo32 PTR_DAT_140011010 __os_arm64x_dispatch_icall 14001121c 20 10 01 00 ibo32 PTR_DAT_140011020 __os_arm64x_dispatch_icall_cfg 140011220 00 00 00 00 ibo32 NaP AlternateEntryPoint 140011224 00 50 01 00 ibo32 PTR_DAT_140015000 AuxiliaryIAT 140011228 01 00 00 00 ddw 1h CodeRangesToEntryPointsCount 14001122c 01 00 00 00 ddw 1h RedirectionMetadataCount 140011230 28 10 01 00 ibo32 DAT_140011028 GetX64InformationFunctionPointer 140011234 30 10 01 00 ibo32 DAT_140011030 SetX64InformationFunctionPointer 140011238 00 70 01 00 ibo32 IMAGE_ARM64_RUNTIME_FUNCTION_UNPACKED_ENTRY_140017000 ExtraRFETable 14001123c 38 0d 00 00 ddw D38h ExtraRFETableSize 140011240 38 10 01 00 ibo32 DAT_140011038 __os_arm64x_dispatch_fptr 140011244 48 3e 01 00 ibo32 PTR_DAT_140013e48 AuxiliaryIATCopy 140011248 00 00 00 00 ibo32 NaP AuxDelayloadIAT 14001124c 00 00 00 00 ibo32 NaP AuxDelayloadIATCopy 140011250 00 00 00 00 ddw 0h ReservedBitField This IMAGE_ARM64EC_METADATA structure contains the ExtraRFE attribute, which is used to reference an exception table that is specific to the ARM64EC: .rdata section started {0x140010000-0x1400152d8} 140012710 0b 00 20 08 81 e4 e3 e3 14 00 40 40 0a 00 c0 02 .. .......@@.... 140012720 e1 81 e6 e6 e6 e6 e7 66 89 fc e4 81 e7 4e 88 e7 .......f.....N.. 140012730 4c 86 e7 4a 84 e7 48 82 e7 66 89 fc e3 e3 e4 e3 L..J..H..f...... 140012740 06 00 40 08 05 00 80 00 01 e4 e4 00 0b 00 40 10 ..@...........@. .rdata section started {0x140010000-0x1400152d8} 140012710 struct LIEF_IMAGE_ARM64_RUNTIME_FUNCTION_ENTRY_XDATA __arm64_runtime_function_entry_xdata = 140012710 { 140012710 uint32_t HeaderData = 0x820000b 140012714 } 140012714 uint8_t __arm64_unwind_code[0x4] = 140012714 { 140012714 [0x0] = 0x81 140012715 [0x1] = 0xe4 140012716 [0x2] = 0xe3 140012717 [0x3] = 0xe3 140012718 } 140012718 struct LIEF_IMAGE_ARM64_RUNTIME_FUNCTION_ENTRY_XDATA __arm64_runtime_function_entry_xdata = 140012718 { 140012718 uint32_t HeaderData = 0x40400014 14001271c } 14001271c uint32_t __arm64_epilog_scopes[0x1] = 14001271c { 14001271c [0x0] = 0x02c0000a 140012720 } 140012720 uint8_t __arm64_unwind_code[0x20] = 140012720 { 140012720 [0x00] = 0xe1 140012721 [0x01] = 0x81 140012722 [0x02] = 0xe6 140012723 [0x03] = 0xe6 140012724 [0x04] = 0xe6 140012725 [0x05] = 0xe6 [....] 140012740 } As for the x86_64 architecture, this table can be used to increase the coverage of functions recognized by BinaryNinja: The source code of these plugins is located in the main LIEF repository under the following directories: plugins/binaryninja plugins/ghidra This new release of LIEF introduces official and maintained support for both Binary Ninja and Ghidra, enhancing the analysis capabilities and type definitions for these frameworks. Contextual Assembly Patching Note The feature is only available in LIEF Extended When patching assembly code, we frequently need to refer to external data or functions within our assembly listing. For instance, we might want to naturally write: 1elf = lief.ELF.parse(\"libdexprotector.so\") 2elf.assemble( 3 elf.get_function_address(\"libdp_init\"), 4 \"\"\" 5 adrp x4, g_protections_conf 6 add x4, x4, :lo12:g_protections_conf 7 strb wzr, [x4, 0xe] // Disable debug check. 8 \"\"\" 9) In this code snippet, we assume that the address of g_protections_conf is known. With the latest release, the assembler engine has introduced support for dynamically resolving symbols referenced in an assembly listing. This functionality works by providing an additional configuration parameter: 1class Config(lief.assembly.AssemblerConfig): 2 def __init__(self, target: lief.Binary): 3 super().__init__() 4 self._target = target 5 6 def resolve_symbol(self, name: str) -\u003e int | None: 7 dwarf_info: lief.dwarf.DebugInfo = self._target.debug_info 8 if var := dwarf_info.find_variable(name): 9 print(f\"'{name}' is located at address: 0x{var.address:016x}\") 10 return var.address 11 return super().resolve_symbol(name) 12 13elf = lief.ELF.parse(\"libdexprotector.so\") 14 15config = Config(elf) 16elf.assemble( 17 elf.get_function_address(\"libdp_init\"), 18 \"\"\" 19 adrp x4, g_protections_conf 20 add x4, x4, :lo12:g_protections_conf 21 strb wzr, [x4, 0xe] // Disable debug check. 22 \"\"\", config 23) The logic of the resolve_symbol function depends on DWARF information that is generated by BinaryNinja (see: BinaryNinja - DWARF Plugin) After patching, we can see the final results, which show that the adrp instructions have been correctly generated to access the g_protections_conf.debug_check variable: uint64_t libdp_init(struct _JNIEnv* env) 000504e0 fd7bbba9 stp fp, lr, [sp, #-0x50]! {__saved_fp} {__saved_lr} 000504e4 fc6701a9 stp x28, x25, [sp, #0x10] {__saved_x28} {__saved_x25} 000504e8 f85f02a9 stp x24, x23, [sp, #0x20] {__saved_x24} {__saved_x23} 000504ec f65703a9 stp x22, x21, [sp, #0x30] {__saved_x22} {__saved_x21} 000504f0 f44f04a9 stp x20, x19, [sp, #0x40] {__saved_x20} {__saved_x19} 000504f4 fd030091 mov fp, sp {__saved_fp} 000504f8 ff030ed1 sub sp, sp, #0x380 000504fc f3030091 mov x19, sp {rlim} 00050500 57d03bd5 mrs x23, tpidr_el0 00050504 e81640f9 ldr x8, [x23, #0x28] 00050508 f40300aa mov x20, x0 0005050c a8831ff8 stur x8, [fp, #-0x8 {var_58}] 00050510 e996ff97 bl dp_check_android_version 00050514 f603002a mov w22, w0 00050518 60100035 cbnz w0, 0x50724 0005051c 80008052 mov w0, #0x4 00050520 e1031f2a mov w1, wzr {0x0} 00050524 e2031f2a mov w2, wzr {0x0} 00050528 e3031f2a mov w3, wzr {0x0} 0005052c e4031f2a mov w4, wzr {0x0} 00050530 2811ff97 bl dp_sys_prctl 00050534 c8fdffb0 adrp x8, data_9000 {\"supersu.rc#/sbin/daemonsu#/sbin/su#/sbin/supersu/supersu_is_here...\"} uint64_t libdp_init(struct _JNIEnv* env) 000504e0 44010090 adrp x4, 0x78000 000504e4 84802f91 add x4, x4, #0xbe0 {g_protections_conf} 000504e8 9f380039 strb wzr, [x4, #0xe] {0x0} {g_protections_conf.debug_check} 000504ec f65703a9 stp x22, x21, [sp, #0x30] {__saved_x22} {__saved_x21} 000504f0 f44f04a9 stp x20, x19, [sp, #0x40] {__saved_x20} {__saved_x19} 000504f4 fd030091 mov fp, sp {arg_0} 000504f8 ff030ed1 sub sp, sp, #0x380 000504fc f3030091 mov x19, sp {rlim} 00050500 57d03bd5 mrs x23, tpidr_el0 00050504 e81640f9 ldr x8, [x23, #0x28] 00050508 f40300aa mov x20, x0 0005050c a8831ff8 stur x8, [fp, #-0x8 {var_8}] 00050510 e996ff97 bl dp_check_android_version 00050514 f603002a mov w22, w0 00050518 60100035 cbnz w0, 0x50724 0005051c 80008052 mov w0, #0x4 00050520 e1031f2a mov w1, wzr {0x0} 00050524 e2031f2a mov w2, wzr {0x0} 00050528 e3031f2a mov w3, wzr {0x0} 0005052c e4031f2a mov w4, wzr {0x0} 00050530 2811ff97 bl dp_sys_prctl 00050534 c8fdffb0 adrp x8, data_9000 {\"supersu.rc#/sbin/daemonsu#/sbin/su#/sbin/supersu/supersu_is_here...\"} For more details about the API, you can check: https://lief.re/doc/latest/extended/assembler/index.html#contextual-assembly-patching PE Refactoring LIEF’s PE module has been significantly refactored, including improvements to the parser, builder, and documentation. These improvements bring this format to a level of maturity comparable to other formats (ELF/Mach-O). The most notable enhancements involve the ability to modify TLS, as well as manage imports and exports. It also provides support for ARM64EC and ARM64X binaries. For more details, please refer to: this blog post: https://lief.re/blog/2025-02-16-arm64ec-pe-support/ this dedicated changelog: https://lief.re/doc/latest/changelog/pe-0-17-0.html#pe-0170-changelog lief-patchelf For this release, I started to bootstrap LIEF-based tools (mostly CLI) that aim to provide specific functionalities using LIEF. The first tool of this bootstrap is lief-patchelf, which provides a drop-in replacement for the well-known NixOS/patchelf. You can find insight about this tool in this blog post: https://lief.re/blog/2025-07-13-patchelf/ COFF Format The COFF format is now supported by LIEF, but it does not support modifications yet. The API is really similar to the other formats and available in C++/Rust/Python: 1import lief 2coff: lief.COFF.Binary = lief.COFF.parse(r\"C:\\Users\\romain\\test.obj\") 3 4# Access symbols and aux info 5for symbol in coff.symbols: 6 print(symbol.name) 7 for aux in symbol.auxiliary_symbols: 8 assert str(aux) == \"\"\" 9 AuxiliaryCLRToken { 10 Aux Type: 1 11 Reserved: 1 12 Symbol index: 10 13 Symbol: ??0CppInlineNamespaceAttribute@?A0xb81de522@vc.cppcli.attributes@@$$FQE$AAM@PE$[...] 14 Rgb reserved: 15 +---------------------------------------------------------------------+ 16 | 00 00 00 00 00 00 00 00 00 00 00 00 | ............ | 17 +---------------------------------------------------------------------+ 18 } 19 \"\"\" 20 21# Disassembler support 22for inst in coff.disassemble(\"?foo@@YAHHH@Z\") 23 print(inst) The documentation for this format is here: https://lief.re/doc/latest/formats/coff/index.html Final Word Version 0.17.0 is the latest release in the 0.X.Y series. With significant improvements to the PE format and the global scale at which LIEF is used, I believe the project is now ready for its upcoming 1.0 version. The complete changelog is here: https://lief.re/doc/latest/changelog.html Happy LIEF, Romain","date":1757808e3,"description":"LIEF 0.17.0: Binary Ninja and Ghidra plugins, contextual assembly patching, a refactored PE module with TLS, import, and export editing, lief-patchelf, and COFF support.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1757808e3,"markdown":"https://lief.re/blog/2025-09-14-lief-0-17-0/index.md","objectID":"ae571a184f835e217060fb629f9119ba","permalink":"https://lief.re/blog/2025-09-14-lief-0-17-0/","publishdate":"2025-09-14T00:00:00Z","relpermalink":"/blog/2025-09-14-lief-0-17-0/","section":"blog","summary":"LIEF 0.17.0 ships reverse-engineering plugins for Binary Ninja and Ghidra, contextual assembly patching, a refactored PE module that can edit TLS, imports, and exports (including ARM64EC and ARM64X), lief-patchelf, and a COFF parser with symbol and disassembler support.","tags":["release","PE","COFF","assembler","Binary Ninja","Ghidra","patchelf"],"title":"LIEF v0.17.0","type":"blog","wordcount":1563},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Introduction patchelf is a utility developed by the NixOS community to facilitate modifications of some parts of an ELF binary. The features offered by patchelf include, but are not limited to: 1# Change the interpreter path 2$ patchelf --set-interpreter /lib/my-ld-linux.so.2 my-program 3 4# Add a new dependency 5$ patchelf --add-needed libfoo.so.1 my-program 6 7# Change RUN/RPATH values 8$ patchelf --set-rpath /opt/my-libs/lib:/other-libs my-program In addition to its use in NixOS, many projects leverage patchelf to perform post-compilation modifications. For example, pypa/auditwheel uses patchelf to change the rpath value and/or change the names of DT_NEEDED libraries. LIEF \u0026 Patchelf Unlike LIEF, patchelf is not a library, and the current version (0.18.0) can only be accessed via the command line. The source code of patchelf is relatively small (around 3,000 lines of code) and is focused on specific modifications such as renaming/removing DT_NEEDED entries and changing/removing/adding DT_RPATH/DT_RUNPATH. When it comes to design decisions for modifying ELF binaries, there is a significant difference between patchelf and LIEF: Section vs Segments Patchelf relies on the section layout to modify and rewrite ELF binaries, while LIEF uses segments representation. This distinction makes patchelf less reliable for ELF binaries that have a non-standard section layout, whereas the representation used by LIEF corresponds to how binaries are loaded, making it less prone to errors during modifications. LIEF-based patchelf LIEF already had (almost) all the internal features to provide the same functionalities as patchelf. However, LIEF does not offer a comprehensive command-line tool like patchelf does. To create a drop-in replacement for patchelf using the LIEF backend, we had to ensure compatibility with the current state of patchelf, including: A command-line interface Completion for zsh (at least) A man-page Thanks to the recent LIEF’s Rust bindings, we can leverage the Rust ecosystem to simplify the development of this command-line tool: The crate clap-rs/clap provides all the features to create a command-line interface with a man-page and completion for the main shells. Cargo and its dependencies management are convenient for generating an executable. The source code is available here: lief-project/LIEF/tools/lief-patchelf and it is worth mentioning that this LIEF-based implementation has the exact same command line interface as the original patchelf: 1$ lief-patchelf --help 2Patchelf based on LIEF 3 4Usage: lief-patchelf [OPTIONS] [filenames]... 5 6Arguments: 7 [filenames]... 8 9Options: 10 --set-interpreter \u003cINTERPRETER\u003e 11 Change the dynamic loader ('ELF interpreter') of executable given to INTERPRETER. 12 13 --page-size \u003cSIZE\u003e 14 Uses the given page size instead of the default 15 16 --print-interpreter 17 Prints the ELF interpreter of the executable. (e.g. `/lib64/ld-linux-x86-64.so.2`) 18 19 --print-os-abi 20 Prints the OS ABI of the executable (`EI_OSABI` field of an ELF file). Moreover, it passes1 all the original patchelf’s test suite except one test related to the IA-64 architecture support: PASS: set-interpreter-same.sh PASS: no-rpath-alpha.sh PASS: no-rpath-amd64.sh PASS: no-rpath-armel.sh PASS: no-rpath-armhf.sh PASS: no-rpath-hurd-i386.sh PASS: no-rpath-i386.sh FAIL: no-rpath-ia64.sh PASS: no-rpath-kfreebsd-amd64.sh PASS: no-rpath-kfreebsd-i386.sh PASS: no-rpath-mips.sh PASS: no-rpath-mipsel.sh PASS: no-rpath-powerpc.sh PASS: no-rpath-s390.sh PASS: no-rpath-sh4.sh PASS: no-rpath-sparc.sh ============================================================================ Testsuite summary for patchelf 0.18.0 (based on LIEF 0.17.0) ============================================================================ # TOTAL: 58 # PASS: 55 # SKIP: 2 # XFAIL: 0 # FAIL: 1 # XPASS: 0 # ERROR: 0 Download If you want to test this new LIEF-based implementation, you can download the compiled version here: lief-tools-aarch64-apple-darwin.zip lief-tools-aarch64-pc-windows-msvc.zip lief-tools-aarch64-unknown-linux-gnu.zip lief-tools-aarch64-unknown-linux-musl.zip lief-tools-i686-unknown-linux-musl.zip lief-tools-x86_64-apple-darwin.zip lief-tools-x86_64-pc-windows-msvc.zip lief-tools-x86_64-unknown-linux-gnu.zip lief-tools-x86_64-unknown-linux-musl.zip Since LIEF performs in-depth modifications and can raise different error messages, I had to make minor changes on some checks: patchelf-test-lief.diff ↩︎","date":1752364800,"description":"lief-patchelf: a LIEF-based reimplementation of NixOS patchelf to change the interpreter, add dependencies, and edit RPATH/RUNPATH, with prebuilt binaries.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1752364800,"markdown":"https://lief.re/blog/2025-07-13-patchelf/index.md","objectID":"3235a5c3953be4ddc6d3438ed9ba555a","permalink":"https://lief.re/blog/2025-07-13-patchelf/","publishdate":"2025-07-13T00:00:00Z","relpermalink":"/blog/2025-07-13-patchelf/","section":"blog","summary":"lief-patchelf reimplements NixOS patchelf on top of LIEF: change the ELF interpreter, add or replace dependencies, and edit RPATH and RUNPATH, with prebuilt binaries for Linux, macOS, and Windows and results from the upstream test suite.","tags":["ELF","patchelf","tooling","Linux"],"title":"LIEF patchelf","type":"blog","wordcount":565},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Introduction When reverse engineering binaries, we could want, at some point, to share the reverse-engineered information with others. The DWARF format, originally designed to hold debug information associated with the original source code, is also well-suited for storing reverse-engineered information such as structures and function names. This blog post introduces a new API in LIEF extended to create DWARF files. It also introduces two plugins for Ghidra and BinaryNinja to export binary analysis into DWARF. Creating DWARF with LIEF (extended) LIEF extended now provides a comprehensive API to create DWARF files. This API is available in Python, Rust, and C++ and it looks like this: 1import lief 2 3elf = lief.ELF.parse(\"./libd5A7BCF0524B8.so\") 4 5editor: lief.dwarf.Editor = lief.dwarf.Editor.from_binary(elf) 6unit: lief.dwarf.editor.CompilationUnit = editor.create_compilation_unit() 7unit.set_producer(\"Generated by LIEF (LLVM backend)\") 8 9func: lief.dwarf.editor.Function = unit.create_function(\"vm_set_register\") 10func.set_address(0x1400023) 11 12editor.write(\"libd5A7BCF0524B8.dwarf\") Under the hood, LIEF uses the LLVM’s DWARF backend to create and generate the final DWARF. In contrast to LLVM’s low-level API, LIEF provides an abstraction that simplifies the implementation details of the DWARF format. For instance, if we want to create a DWARF for a function that contains a stack variable at the offset (on the stack) 8, we can use the following API: 1func: lief.dwarf.editor.Function = unit.create_function(\"vm_set_register\") 2 3var: lief.dwarf.editor.Variable = func.create_stack_variable(\"my_stack_variable\") 4var.set_stack_offset(8) This code generates the following DWARF: 10x0000000c: DW_TAG_compile_unit 2 DW_AT_producer (\"Generated by LIEF (LLVM backend)\") 3 40x00000011: DW_TAG_subprogram 5 DW_AT_name (\"vm_set_register\") 6 DW_AT_entry_pc (0x0000000001400023) 7 80x0000001e: DW_TAG_variable 9 DW_AT_name (\"my_stack_variable\") 10 DW_AT_location (DW_OP_fbreg -8) Defining the DW_AT_location for a stack variable is not as simple as it sounds. It requires defining some kind of DWARF expression and if you are curious about the actual implementation, you can check this GitHub Gist. Summary LIEF exposes a high-level API to create DWARF based on the LLVM's low-level API DWARF and Reverse Engineering Reverse engineering tools typically use their own format to store information about analyzed binaries such as *.idb and *.bndb. Most of these tools are not compatible with each other, except Binary Ninja which has a support for loading IDB (Migrating from IDA). For Ghidra, importing IDA database is a non-goal (c.f. issue #2921) One alternative is to export binary information using BinExport or quokka, but many tools lack support for importing the exported data. In contrast, Binary Ninja, Ghidra, and IDA all have built-in support for loading DWARF files and external DWARF files. The DWARF format is primarily designed to hold information about the original source code, and the purpose of reverse engineering is to recover the semantic of the source code information from the binary. Therefore, we could use the DWARF as a reverse-engineering shared format to export types, functions, and variables from reverse-engineered binaries. DWARF is compatible with PE binaries, even though it is not the default format for storing debug information on Windows. PE / DWARF If you compile a Windows executable with clang[-cl] and with the flags -g -gdwarf-5, the final PE will contains DWARF information along with an external .pdb. Currently, Binary Ninja is the only tool with a built-in plugin that can generate a DWARF file from a BinaryView representation. However, it lacks the ability to export stack-based variables, which can be crucial information. The next section introduces two plugins for Ghidra and BinaryNinja to generate DWARF from these tools. BinaryNinja \u0026 Ghidra Plugins To provide some background on this feature, I initially developed the BinaryNinja’s DWARF exporter plugin for my own needs before Vector35 team released an official plugin in BinaryNinja 3.5. I use this plugin in my reverse engineering workflow to symbolize QBDI traces from DWARF information: I statically reverse-engineer the binary I generate a DWARF file I trace the binary with QBDI that uses the DWARF file to symbolize: Stack accesses (hence the need to stack variables in the DWARF) Function calls and their parameters Static variables accesses goto 1. For instance, I used this process to reverse engineer the DroidGuard VM a few years ago. I’ll take this blog post as an opportunity to share the DWARF associated with my reverse engineering of the VM libd5A7BCF0524B8.so: Original Binary: libd5A7BCF0524B8.so Binary with generated DWARF: libd5A7BCF0524B8.so.debug As mentioned earlier, this functionality is integrated into BinaryNinja since version 3.5, so I’ll focus more on the Ghidra plugin. For more information about the Binary Ninja integration, visit LIEF - Binary Ninja. The Ghidra plugin allows us to export Ghidra’s Program information into a DWARF file. This can be done from the Project Manager interface by selecting the DWARF format in the export section: You can also use this plugin from the CodeBrowser tool, by left-clicking on the LIEF menu and selecting Export as DWARF: The plugin is primarily written in Java (using the JNI) and you can also generate a DWARF file from a headless Java script: 1import lief.ghidra.core.dwarf.export.Manager; 2import lief.ghidra.core.NativeBridge; 3 4public class LiefDwarfExportScript extends GhidraScript { 5 @Override 6 protected void run() throws Exception { 7 NativeBridge.init(); 8 Manager manager = new Manager(currentProgram); 9 File output = new File(\"/home/romain/output.dwarf\"); 10 manager.export(output); 11 } 12} IDA Support I do not plan to support IDA for this functionality. However, if there is strong demand for it, feel free to reach out to me. You can also create your own IDA script using the Python or C++ API. Last Word This DWARF export functionality is still in early development, so I cannot guarantee it is free of bugs. The current version also does not export comments, but I plan to support this feature in the future. The source code for the plugins is here: plugins/binaryninja plugins/ghidra Thank you for using LIEF. Romain.","date":1748304e3,"description":"Create DWARF debug information with the LIEF Extended DWARF editor and share reverse-engineered types and functions between Binary Ninja and Ghidra plugins.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1748304e3,"markdown":"https://lief.re/blog/2025-05-27-dwarf-editor/index.md","objectID":"ffe7c6734dda5c31937e60011f2f22bc","permalink":"https://lief.re/blog/2025-05-27-dwarf-editor/","publishdate":"2025-05-27T00:00:00Z","relpermalink":"/blog/2025-05-27-dwarf-editor/","section":"blog","summary":"DWARF as a shared reverse-engineering format: the LIEF Extended DWARF editor creates debug information from scratch, and companion Binary Ninja and Ghidra plugins export and import reverse-engineered functions and structures.","tags":["DWARF","LIEF Extended","reverse-engineering","Binary Ninja","Ghidra"],"title":"DWARF as a Shared Reverse Engineering Format","type":"blog","wordcount":925},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"I'm pleased to share that LIEF's PE support has been significantly improved on both aspects: parsing and writing. Parsing Improvements One of the significant updates in LIEF is the enhanced processing of the IMAGE_LOAD_CONFIG_DIRECTORY which now includes a detailed parsing of the underlying structures it references. For instance, the Dynamic Value Relocation Table1 is now fully supported and can be accessed through the Python, C++, and Rust APIs: 1import lief 2 3pe = lief.PE.parse(\"win11_arm64x_Windows.Media.Protection.PlayReady.dll\") 4 5dyn_reloc_table = pe.load_configuration.dynamic_relocations 6 7for r in dyn_reloc_table: 8 print(r) 1Dynamic Value Relocation Table (version: 1) 2Symbol VA: 0x0000000000000006 (RELOCATION_ARM64X) 3Fixup RVAs (ARM64X) 4 [0000] RVA 0x0000010c, 2 bytes, target value 64:86 5 [0001] RVA 0x00000130, 4 bytes, target value d0:bc:57:00 6 [0002] RVA 0x00000190, 4 bytes, target value 30:22:d8:00 7 [0003] RVA 0x00000194, 4 bytes, target value 30:01:00:00 8 ... All the structures involved in these dynamic relocations can be accessed through an API, including support for special relocations: IMAGE_DYNAMIC_RELOCATION_ARM64X IMAGE_DYNAMIC_RELOCATION_FUNCTION_OVERRIDE IMAGE_DYNAMIC_RELOCATION_ARM64_KERNEL_IMPORT_CALL_TRANSFER IMAGE_DYNAMIC_RELOCATION_GUARD_IMPORT_CONTROL_TRANSFER With the introduction of this new dynamic relocation support, LIEF can now offer a relocated view of the ARM64EC binary that is embedded within an ARM64X binary. To achieve this, simply parse the ARM64X PE binary with all options enabled. 1let pe = lief::pe::Binary::parse_with_config( 2 \"win11_arm64x_Windows.Media.Protection.PlayReady.dll\", 3 lief::pe::ParserConfig::with_all_options(), 4).unwrap(); 5 6if let Some(nested_arm64ec) = pe.nested_pe_binary() { 7 println!(\"{:?}\", nested_arm64ec.header()); 8} or in Python: 1pe = lief.PE.parse(\"win11_arm64x_Windows.Media.Protection.PlayReady.dll\", 2 lief.PE.ParserConfig.all) 3 4if nested_arm64ec := pe.nested_pe_binary: 5 print(nested_arm64ec.header) 1Signature: 50 45 00 00 2Machine: AMD64 3Number of sections: 19 4Pointer to symbol table: 0x0 5Number of symbols: 0 6Size of optional header: 0xf0 7Characteristics: EXECUTABLE_IMAGE, LARGE_ADDRESS_AWARE, DLL 8Timtestamp: 1966221527 Parser Config To maintain the performance of LIEF v0.17.0 in line with previous versions, the parsing of nested ARM64EC binaries must be explicitly enabled during the lief.PE.parse() operation (the default is false). LIEF provides helper functions such as: lief.PE.Binary.is_arm64x() and lief.PE.Binary.is_arm64ec() to determine whether a given PE is ARM64 emulation compatible (EC) or if it’s a fat ARM64X. These helper functions rely on the CHPE metadata (Compiled Hybrid Portable Executable) which is accessible through LIEF for both: ARM64/ARM64EC binaries and legacy x86 binaries executed via WoW64: 1import lief 2 3# ARM64 CHPE 4pe = lief.PE.parse(\"arm64x_ImagingEngine.dll\") 5 6chpe: lief.PE.CHPEMetadataARM64 = pe.load_configuration.chpe_metadata 7print(chpe_metadata.auxiliary_delay_import) 8print(chpe_metadata.code_ranges) 9 10# x86/WoW64 CHPE 11pe = lief.PE.parse(\"Windows.Media.dll\") 12 13chpe: lief.PE.CHPEMetadataX86 = pe.load_configuration.chpe_metadata 14print(chpe_metadata.compiler_iat_pointer) 15print(chpe_metadata.wowa64_dispatch_ret_function_pointer) LIEF can now also process in-depth exception information for x86-64 and ARM64 binaries as well as COFF strings and COFF symbols that are still used by some toolchains. Writing Improvements As mentioned at the beginning of this blog post, LIEF’s PE modification engine, also named Builder, has been completely refactored to enhance its reliability and consistency when modifying PE binaries. To support this update, the documentation includes a dedicated section detailing the types of modifications supported by LIEF and their limitations: Imports Modification Resources Modification TLS Modification Debug Modification Exports Modification For instance, PE import modification has been completely redesigned to avoid using trampolines. Likewise, LIEF can now commit changes made to the export table. This enables the creation of new exports that can be used to expose reverse-engineered functions or to deceive disassemblers, as mentioned in The Poor Man’s Obfuscator BinaryNinja ARM64EC Support The ARM64EC ABI2, created by Microsoft, was developed to ease interoperability between ARM64 and x86-64 architectures, particularly for parts of the code that require emulation (EC = Emulation Compatible). The specifications of this ABI have a somewhat counterintuitive aspect: the targeted architecture indicated in the PE header is AMD64, while the majority of the functions are compiled for ARM64. Darek Mihocka, one of the authors of the ARM64EC ABI, details some of these design choices in a great blog post3 that also provides a historical background about the support of new architectures in Windows. Because the PE header specifies the x86-64 architecture, which does not accurately reflect most of the functions compiled in the binary, many disassemblers become confused when trying to analyze such a binary: To address this incomplete support, I developed a BinaryNinja plugin designed to identify additional functions and accurately determine the architecture for which each function has been compiled. For example, after using this plugin, we obtain a feature map in BinaryNinja showing that a majority of the functions have been correctly recognized: The plugin is written in Rust using both LIEF \u0026 BinaryNinja Rust bindings. This also serves as an opportunity to evaluate the LIEF Rust API and its integration into other projects. First, we can get a LIEF PE instance from a BinaryView with: 1fn enhance_with_lief(bv: \u0026BinaryView) { 2 let filename = bv.file().filename(); 3 let pe = lief::pe::Binary::parse_with_config( 4 filename.as_str(), 5 // This is required to get access to the exceptions info that are used later 6 lief::pe::ParserConfig::with_all_options(), 7 ).unwrap(); 8} Memory Footprint Memory-wise, parsing the PE loaded by BinaryNinja with LIEF is not ideal since the same binary exists in memory twice: once for the BinaryNinja instance and once for the LIEF instance. I think that the BinaryNinja team will enhance the support for ARM64EC binaries at some point, so in the meantime this memory overhead is acceptable. Once we have this instance, we can use the exceptions table as a source of function addresses allowing BinaryNinja to begin disassembling the binary from these addresses: 1let pe: \u0026lief::pe::Binary; 2let bv: \u0026BinaryView; 3 4let windows_arm64 = Platform::by_name(\"windows-aarch64\").unwrap().to_owned(); 5let windows_x64 = Platform::by_name(\"windows-x86_64\").unwrap().to_owned(); 6 7let imagebase = pe.optional_header().imagebase(); 8 9for exception in pe.exceptions() { 10 match exception { 11 lief::pe::RuntimeExceptionFunction::X86_64(x64) =\u003e { 12 let addr: u64 = imagebase + (x64.rva_start() as u64); 13 bv.add_auto_function(\u0026windows_x64, addr); 14 } 15 16 lief::pe::RuntimeExceptionFunction::AArch64(arm64) =\u003e { 17 let addr: u64 = imagebase + (arm64.rva_start() as u64); 18 bv.add_auto_function(\u0026windows_arm64, addr); 19 } 20 } 21} 22 23bv.update_analysis(); This code leads to the feature map shown in the screenshot above. Although the algorithm is straightforward, it hides an important aspect of the ARM64EC ABI: The exceptions table referenced by the data directory at the index: IMAGE_DIRECTORY_ENTRY_EXCEPTION points to the X86_64 table. To access the ARM64 exceptions table we need to refer to the CHPE metadata: 1let loadconfig = pe.load_configuration().unwrap(); 2 3if let Some(lief::pe::CHPEMetadata::ARM64(arm64)) = loadconfig.chpe_metadata() { 4 let arm64_exception_table_start = arm64.extra_rfe_table(); 5 let arm64_exception_table_size = arm64.extra_rfe_table_size(); 6 println!(\"{:#x} ({} bytes)\", arm64_exception_table_start, arm64_exception_table_size); 7} LIEF abstracts the processing of CHPEMetadata and provides an iterator that outputs one of the following exceptions object: lief::pe::exception_x64::RuntimeFunction lief::pe::exception_aarch64::RuntimeFunction You can find more details about the exception support by LIEF in the dedicated PE changelog: https://lief.re/doc/latest/changelog/pe-0-17-0.html#pe-0170-changelog The source code for the BinaryNinja plugin is available on GitHub: lief-project/lief-binaryninja-arm64ec. It contains additional features, but the code should be read through the lens of a proof of concept. https://denuvosoftwaresolutions.github.io/DVRT/dvrt.html ↩︎ https://learn.microsoft.com/en-us/windows/arm/arm64ec ↩︎ http://www.emulators.com/docs/abc_arm64ec_explained.htm ↩︎","date":1739664e3,"description":"PE improvements in LIEF: ARM64EC and WoW64 CHPE metadata parsing, a rebuilt PE builder for imports, resources, and TLS, and LIEF-backed ARM64EC support in Binary Ninja.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1739664e3,"markdown":"https://lief.re/blog/2025-02-16-arm64ec-pe-support/index.md","objectID":"5738c1eb345e9e2fababc7f5769e2032","permalink":"https://lief.re/blog/2025-02-16-arm64ec-pe-support/","publishdate":"2025-02-16T00:00:00Z","relpermalink":"/blog/2025-02-16-arm64ec-pe-support/","section":"blog","summary":"LIEF's PE support gains CHPE metadata parsing for ARM64EC and x86/WoW64 binaries, a fully refactored builder with documented import, resource, and TLS modifications, and LIEF-backed ARM64EC support for Binary Ninja.","tags":["PE","ARM64EC","Windows","Binary Ninja"],"title":"PE Support Enhancements","type":"blog","wordcount":1099},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Documentation Documentation is an important aspect of LIEF and since the beginning of the project, I have spent a decent amount of time keeping comprehensive and intuitive documentation. Usually, I don’t include documentation updates in the changelog but in this case, I thought it could be worth sharing this experience. LIEF is written in C++ with bindings for Python and Rust. Originally, the documentation was driven by languages API (isolated from each other) and generated by Sphinx with the Breathe extension to reference the C++ Doxygen domain. Recently, Rust landed in the arena. Compared to Python and C++, the Rust language embeds a built-in documentation engine to process and generate in-code documentation into HTML pages. Given the new Rust bindings and the Rust built-in documentation engine, two questions emerged: Do we want to add (yet) another API page for Rust? How do we reference Rust API in Sphinx? For the first point, I moved from a language-driven documentation structure to a functionality-driven structure. This changes how the documentation is consumed. Instead of looking for a language’s format-specific API, you first choose the task, such as ELF processing or Dyld shared cache parsing. You can then access the language API you need. So instead of adding another Rust API reference page, the Rust API has been transparently integrated with the new layout. The second point has been a bit more tricky to approach. With a reverse engineering background, I really value the cross-reference feature provided by Sphinx: 1blah blah blah :py:class:`lief.ELF.Binary` another blah: :cpp:class:`LIEF::ELF::Binary` Python is a built-in domain supported by Sphinx and Breathe extension is doing the bridge between C++ Doxygen XML files and Sphinx. For Rust, there are some attempts to create a bridge but I decided to take another path. I created a Rust-sphinx domain that cross-references to the official or nightly Rust documentation. Basically with this custom domain, the following cross-references redirect to the official or nightly documentation: 1:rust:module:`lief::assembly` 2:rust:enum:`lief::assembly::Instructions` Is translated into: 1https://lief-rs.s3.fr-par.scw.cloud/doc/latest/lief/assembly/index.html 2https://lief-rs.s3.fr-par.scw.cloud/doc/latest/lief/assembly/enum.Instructions.html By doing so, we can leverage Sphinx’s cross-reference functionalities while still keeping the built-in Rust documentation. In addition to this Rust-specific domain, I created a .. lief-api:: directive that can pack similar cross-language API into a single block. For instance, this directive: 1.. lief-api:: lief.Binary.disassemble() 2 3 :rust:method:`lief::generic::Binary::disassemble [trait]` 4 :rust:method:`lief::generic::Binary::disassemble_symbol [trait]` 5 :rust:method:`lief::generic::Binary::disassemble_address [trait]` 6 :rust:method:`lief::generic::Binary::disassemble_slice [trait]` 7 :cpp:func:`LIEF::Binary::disassemble` 8 :py:meth:`lief.Binary.disassemble` 9 :py:meth:`lief.Binary.disassemble_from_bytes` Is rendered as: This allows us to refer the API for different languages without being too verbose and impacting readability. Combined with Sphinx substitution, we can write: This is an example that cross-reference |lief-disassemble| .. |lief-disassemble| lief-api:: lief.Binary.disassemble() :rust:method:`lief::generic::Binary::disassemble [trait]` :rust:method:`lief::generic::Binary::disassemble_symbol [trait]` :rust:method:`lief::generic::Binary::disassemble_address [trait]` :rust:method:`lief::generic::Binary::disassemble_slice [trait]` :cpp:func:`LIEF::Binary::disassemble` :py:meth:`lief.Binary.disassemble` :py:meth:`lief.Binary.disassemble_from_bytes` You can go checking out this page https://lief.re/doc/latest/formats/pe/index.html to see a concrete rendering of these changes. Extended Features Public Release The extended version is now publicly available at this address: https://extended.lief.re Assembler \u0026 Disassembler Adding (or not adding) a disassembler in LIEF has been a long-standing question and with the extended version, I found a fair trade-off: LIEF core focuses on executable formats, free from any extra features that might have a significant impact on the build complexity or library size. On the other hand, LIEF extended provides additional functionalities that require a more complex build pipeline and increase the binary size. Among these extended functionalities, there are a disassembler and an assembler based on the LLVM’s MC layer. The disassembling API is provided at different levels: LIEF::Binary 1import lief 2 3pe = lief.PE.parse(\"cmd.exe\") 4for inst in pe.disassemble(0x400000): 5 print(inst) 6 7 # Instruction semantic 8 print(inst.is_syscall) 9 print(inst.is_memory_access) 10 print(inst.is_call) 11 12 # Instruction operands (for AArch64 and x86-64) 13 if isinstance(inst, lief.assembly.aarch64.Instruction): 14 for idx, operand in enumerate(inst.operands): 15 match operand: 16 case lief.assembly.aarch64.operands.Register(): 17 print(f\"OP[{idx}] -- REG: {operand.value}\") 18 case lief.assembly.aarch64.operands.Memory(): 19 print(f\"OP[{idx}] -- MEM: {operand.base} {operand.offset}\") 20 case lief.assembly.aarch64.operands.PCRelative(): 21 print(f\"OP[{idx}] -- PCR: {operand.value}\") 22 case lief.assembly.aarch64.operands.Immediate(): 23 print(f\"OP[{idx}] -- IMM: {operand.value}\") LIEF::dwarf::Function 1import lief 2 3elf = lief.ELF.parse(\"my-dbg.elf\") 4dwarf: lief.dwarf.DebugInfo = elf.debug_info 5func: lief.dwarf.Function = dwarf.find_function(\"my_debug_function\") 6 7for inst in func.instructions: 8 print(inst) LIEF::dsc::DyldSharedCache 1import lief 2 3cache = lief.dsc.load(\"ios-18/\") 4for inst in cache.disassemble(0x1886f4a44): 5 print(inst) In terms of implementation, the disassembler wraps a lazy iterator that evaluates/disassembles an instruction only when the iterator is processed. It means that you don’t pay any overhead until you access the iterator’s value: 1# O(0) 2inst = macho.disassemble(0x400000) 3 4inst = macho.disassemble(0x400000) 5# O(10) 6for _ in range(10): 7 next(inst) The .end() sentinel of the iterator is based on two properties: Either a range is specified (e.g. macho.disassemble(0x400000, /*size*/0x1000)) and the iterator past the end of the range. The instruction can’t be disassembled. This kind of sentinel allows us to use this API: macho.disassemble(0x400000) which will disassemble (lazily) instructions at the address 0x400000 until it fails. C++ \u0026 Rust \u0026 Python The disassembler/assembler API is uniformly available in Rust, C++, and Python. Capstone? Nyxstone? As stated in the documentation the major design difference with Capstone is that LIEF uses a mainstream version of LLVM with limited patches1 on the MC layer (the current version is based on LLVM 19.1.2). The design difference with Nyxstone is that LLVM is hidden from the public API which means that it does not require to have an LLVM version pre-install on the system. Moreover, it exposes opcodes and control-flow/semantic information about the instructions. On the other hand, LIEF does not provide a standalone API to disassemble arbitrary instructions. The disassembler engine is bound to the object from which the API is exposed. Assembler In association with a disassembler, LIEF exposes a (basic) assembly API that allows generating and patching instructions: 1import lief 2 3elf = lief.ELF.parse(\"my-android-obfuscated.so\") 4text = elf.get_section(\".text\") 5# Disassembler 6syscall = [inst for inst in elf.disassemble(bytes(text)) if inst.is_syscall] 7 8# Assembler 9for syscall_inst in syscall: 10 new_bytes = elf.assemble(syscall_inst.address, \"nop;\") # Assemble AND patch 11 print(new_bytes.hex(\", \")) Warning In this current version, the assembler is working pretty well for x86/x86_64 and AArch64 but might break on other architectures. In addition, llvm::MCFixup are not supported. This can be used to patch LIEF’s binary object directly at the assembly level. I plan to provide the assembly engine with LIEF Binary context. If the binary defines a function such as call_me() that is exported or present in the debug information, users would be able to call it at the assembly level: 1fn patch_with_context(macho: \u0026mut lief::macho::Binary) { 2 macho.assemble(0x140000090, r#\" 3 adrp x0, call_me; 4 add x0, x0, :lo12:call_me; 5 mov x1, 0x90; 6 str x1, [x0]; 7 \"#r); 8} And LIEF would handle the relocation/resolution process to instruct LLVM about the location and the definition of call_me. C++ \u0026 Rust \u0026 Python The disassembler/assembler API is seamlessly available in Rust, C++, and Python :) Dyld Shared Cache Initial support for processing Apple’s Dyld shared cache with LIEF has been released along with an API to deoptimize in-cache Dylib. The API looks like this: 1import lief 2 3cache = lief.dsc.load(\"ios-18.1/\") 4for dylib in cache.libraries: 5 print(f\"0x{dylib.address:016x} {dylib.path}\") 6 # Extract the dylib as a regular lief.MachO.Binary 7 macho: lief.MachO.Binary = dylib.get() Warning Please note that the deoptimization feature is not working well on all the shared cache libraries. This support is going to be improved over time. One could also use this API to diff two shared caches: 1use lief; 2let ios_17 = lief::dsc::load_from_path(\"ios-17.7.1\"); 3let ios_18 = lief::dsc::load_from_path(\"ios-18.1.1\"); 4 5let libraries_17: HashSet\u003cString\u003e = ios_17.libraries() 6 .map(|lib| lib.path()) 7 .collect(); 8 9let libraries_18: HashSet\u003cString\u003e = ios_18.libraries() 10 .map(|lib| lib.path()) 11 .collect(); 12 13println!(\"{:?}\", libraries_17.symmetric_difference(\u0026libraries_18)) Rust Rust bindings got their first mutable functions which are listed in the changelog. These mutable functions are limited but they allow us to make basic modifications like adding a library or patching assembly code: 1fn add_library(elf: \u0026mut lief::elf::Binary) { 2 elf.add_library(\"libtest.so\"); 3 elf.write(\"patched.elf\"); 4} 1fn patch_asm(elf: \u0026mut lief::macho::Binary) { 2 macho.assemble(0x100004090, r#\" 3 mov x0, x16; 4 br x0; 5 \"#); 6 macho.write(\"patched.macho\"); 7} In addition, the support for the x86_64-unknown-linux-musl target triple is now available and the minimal GLIBC version for x86_64-unknown-linux-gnu has been lowered to 2.28. It means that Linux Rust bindings can now run on Debian 10, Ubuntu 19.10, … while before it required Debian 11 or Ubuntu 20.04. The new x86_64-unknown-linux-musl triple can be used to generate full static without any dependencies to the libstdc++, libc, .... For instance, given this code: 1use lief; 2use lief::generic::Section; 3 4fn main() { 5 let path = std::env::args().last().unwrap(); 6 let mut file = std::fs::File::open(path).expect(\"Can't open the file\"); 7 8 if let Some(lief::Binary::PE(pe)) = lief::Binary::from(\u0026mut file) { 9 for section in pe.sections() { 10 println!( 11 \"{:20}: [0x{:016x}-0x{:016x}]\", 12 section.name(), 13 section.virtual_address(), 14 section.virtual_address() + section.virtual_size() as u64 15 ); 16 } 17 } 18} We can generate a dependencies-free executable by running: 1$ cargo build [--release] --target x86_64-unknown-linux-musl 1$ ldd target/x86_64-unknown-linux-musl/release/reader 2 statically linked 1$ target/x86_64-unknown-linux-musl/release/reader steam.exe 2.text : [0x0000000000001000-0x00000000002cbe53] 3.rdata : [0x00000000002cc000-0x00000000003a7fa2] 4.data : [0x00000000003a8000-0x000000000043ada0] 5.rsrc : [0x000000000043b000-0x0000000000471b8c] 6.reloc : [0x0000000000472000-0x0000000000490c74] Python Bindings LIEF is now using nanobind v2.4.0 which improves the support for typing. Among these typing improvements, C++ enums flags are now properly inheriting from enum.Flag which results in a better interface with Python code. Typing stub files (*.pyi) are now also generated with nanobind’s built-in stubgen.py instead for mypy. Final Words Additional changes are listed in the detailed changelog. Many thanks to dornstetter and kohnakagawa for their feedback about the dyld shared cache feature. Thank you also to Konstantin Vinogradov and dctoralves for their sponsorship. All the patches have been PR-submitted to the LLVM. You can check LIEF \u0026 LLVM for the details ↩︎","date":1733788800,"description":"LIEF 0.16.0: rebuilt documentation, an assembler/disassembler in LIEF Extended, dyld shared cache extraction, first mutable Rust APIs, and nanobind 2.4 bindings.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1733788800,"markdown":"https://lief.re/blog/2024-12-10-lief-0-16-0/index.md","objectID":"1494404171927e7e7a85cc70eb089b74","permalink":"https://lief.re/blog/2024-12-10-lief-0-16-0/","publishdate":"2024-12-10T00:00:00Z","relpermalink":"/blog/2024-12-10-lief-0-16-0/","section":"blog","summary":"LIEF 0.16.0 rebuilds the documentation, adds an assembler and disassembler with instruction semantics to LIEF Extended, extracts dylibs from the dyld shared cache, introduces the first mutable Rust APIs, and moves the Python bindings to nanobind 2.4.","tags":["release","LIEF Extended","assembler","disassembler","dyld-shared-cache","Rust","Python"],"title":"LIEF v0.16.0","type":"blog","wordcount":1580},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"While this new release adds new functionalities and addresses different bugs, It is worth mentioning that it is the first release to officially expose Rust binding! In addition, an extended version was also released to provide additional functionalities not strictly related to the executable formats. Rust bindings As discussed in these blog posts: LIEF Rust bindings updates Rust bindings for LIEF LIEF is now available in Rust for the following architectures: aarch64-unknown-linux-gnu x86_64-apple-darwin x86_64-pc-windows-msvc (MT/MD runtimes) x86_64-unknown-linux-gnu aarch64-apple-ios aarch64-apple-darwin I published the release on crates.io so you should be able to start using LIEF in Rust with: 1[package] 2name = \"lief-demo\" 3version = \"0.0.1\" 4edition = \"2021\" 5 6[dependencies] 7lief = \"0.15.0\" LIEF Extended LIEF is now providing additional features thanks to an extended version. Among those features, it provides support for DWARF and PDB debug formats as well as Objective-C metadata. Objective-C This support is a kind of spin-off of iCDump which is now completely integrated into LIEF. Compared to the original iCDump project, it fixes the issue with the new chained relocations (c.f. issue#4) format and can be used on all the platforms supported by LIEF (including Windows) in C++/Rust/Python: Rust: 1let macho: lief::macho::Binary; 2 3if let Some(metadata) = macho.objc_metadata() { 4 println!(\"Objective-C metadata found\"); 5 for class in metadata.classes() { 6 println!(\"name={}\", class.name()); 7 for method in class.methods() { 8 println!(\" method.name={}\", method.name()); 9 } 10 } 11} Python: 1import lief 2macho: lief.MachO.Binary = ... 3metadata: lief.objc.Metadata = macho.objc_metadata 4 5if metadata is not None: 6 print(\"Objective-C metadata found\") 7 8 for clazz in metadata.classes: 9 print(f\"name={clazz.name}\") 10 for meth in clazz.methods: 11 print(f\" method.name={meth.name}\") 12 13 # Generate a header like \"class-dump\" 14 print(metadata.to_decl()) DWARF \u0026 PDB Supporting debug formats like DWARF or PDB has been a long-standing discussion (c.f. issue #17). The main reasons to avoid supporting these formats from scratch were: The maintenance effort There already exists libraries to process these debug formats: pyelftools for DWARF LLVM (DWARF \u0026 PDB) gimli (DWARF) On the other hand, I understand the need to process debug information from a LIEF binary object when it is present. The existing projects expose powerful low-level APIs that match the debug format specifications. However, they do not provide1 an abstraction over the complexity of these specifications. Developers and reverse engineers work with concepts such as compilation units, functions, global variables, and stack variables. Before accessing this information from a DWARF or PDB file, however, you need to understand a PDB DBI stream or know that a function’s address in DWARF can be determined by either DW_AT_entry_pc or DW_AT_low_pc. The idea behind the support of the DWARF and PDB formats in LIEF is to: bridge concepts that make sense to the developers/reverse engineers with their concrete specifications in DWARF/PDB Have a (documented) C++ API and bindings for Python/Rust. This LIEF bridge is based on LLVM which did the heavy job of supporting DWARF \u0026 PDB within a single framework. The DWARF \u0026 PDB support in LIEF leverages the LLVM API to abstract concepts as listed above. For instance, you can iterate through every public symbol in the ntoskrnl.pdb PDB through: 1import lief 2 3ntoskrnl: lief.pdb.DebugInfo = lief.pdb.load(\"./ntoskrnl.pdb\") 4 5for sym in ntoskrnl.public_symbols: 6 print(f\"{sym.demangled_name}: 0x{sym.RVA:06x}\") If the PDB embeds extended information about the compilation units we can do (in Rust): 1let pdb = lief::pdb::load(\"peacecannary.pdb\"); 2for cu in pdb.compilation_units() { 3 for func in cu.functions() { 4 if func.name().starts_with(\"peacecannary::CObfuscator\") { 5 println!(\"{}: {} (0x{:04x})\", cu.module_name(), func.name(), func.rva()); 6 } 7 } 8} The API for the DWARF format is pretty similar: 1import lief 2 3elf: lief.ELF.Binary = ... 4# If the binary embeds DWARF debug info in the ELF: 5dwarf: lief.dwarf.DebugInfo = elf.debug_info 6# Otherwise: 7dwarf: lief.dwarf.DebugInfo = lief.dwarf.load(\"my_dwarf.dwarf\") 8 9for cu in dwarf.compilation_units: 10 print(f\"Produced by: {cu.producer} in {cu.compilation_dir}\") 11 12 for func in cu.functions: 13 print(f\"0x{func.address:04x}: {func.name} ({func.size} bytes)\") 14 15 for var in cu.variables: 16 if var.is_constexpr: 17 continue 18 # Look for global variables only 19 if var.address is not None and var.address \u003e 0: 20 print(f\"0x{var.address:04x}: {var.linkage_name} ({var.size} bytes)\") For more details about the API, you can take a look at these dedicated sections: DWARF PDB Other Updates Mach-O AI LIEF is now powered by AI supporting Apple *.hwx files which are some kind of Mach-O file for the Apple Neural Engine (ANE). These *.hwx start with a new magic identifier: 0xbeefface and embed custom LC_ command like the command 0x40 LC Command 0x40 I could be interested in adding the support of this private command in LIEF so if anyone already reversed or has some info about the layout of this command, feel free to reach out. To support unknown or non-public LC commands in LIEF, I created an artificial LIEF::MachO::UnknownCommand which is a placeholder for any Mach-O commands that are not recognized by LIEF. For instance, we can inspect the private 0x40 command as follows: 1import lief 2target = lief.MachO.parse(\"personsemantics-u8-v4.H16.espresso.hwx\").at(0) 3lc_0x40: lief.MachO.UnknownCommand = macho.commands[18].command 4 5print(lc_0x40.original_command) # Outputs 0x40/61 6print(bytes(lc_0x40.data)) # Print the raw content of the command These .hwx files have been involved in the Dopamine jailbreak and you can also find a BlackHat presentation about the Apple Neural Engine: Apple Neural Engine Internal. PE Authenticode LIEF can inspect and verify the PE Authenticode and with this release, we can even do that in Rust! 1use lief::pe; 2 3let mut file = std::fs::File::open(path).expect(\"Can't open the file\"); 4if let Some(lief::Binary::PE(pe)) = lief::Binary::from(\u0026mut file) { 5 let result = pe.verify_signature(pe::signature::VerificationChecks::DEFAULT); 6 if result.is_ok() { 7 println!(\"Valid signature!\"); 8 } else { 9 println!(\"Signature not valid: {}\", result); 10 } 11 return ExitCode::SUCCESS; 12} This new release also adds support for the MS-CounterSignature attribute (OID: 1.3.6.1.4.1.311.3.3.1) and some other attributes like Ms-ManifestBinaryID (OID: 1.3.6.1.4.1.311.10.3.28) ELF No breaking updates for the ELF format. LIEF is now able to parse and modify binaries compiled with the new DT_RELR and DT_ANDROID_REL_ relocations. I also added the helper: LIEF::ELF::Binary::get_relocated_dynamic_array which allows us to get a relocated view of the DT_INIT_ARRAY/DT_FINI_ARRAY. This can be useful when – for instance – the init array values are null because of relocations: 1import lief 2 3elf: lief.ELF.Binary = ... 4 5# Return: [0, 0, 0, 0, ...] 6elf.get(lief.ELF.DynamicEntry.TAG.INIT_ARRAY).array 7 8# Return relocated values: [0x96db10, 0x9b9c14, 0xe7f660, 0xe7f70c, ...] 9elf.get_relocated_dynamic_array(lief.ELF.DynamicEntry.TAG.INIT_ARRAY) Enums Since the beginning of LIEF, all the enums used by the different formats were located in a single header file (e.g. LIEF/PE/enums.hpp or lief.PE.{enums, ...} in Python). Some of them were clashing with system headers that were also #define some of these enums. To work around this issue, we had a dirty hack based on LIEF/{ELF.PE,MachO}/undef.h that undefines these values before being included. In LIEF 0.15.0 the scope of the enums has been redefined so that we should no longer need the undef.h. For instance the standalone enum LIEF::ELF::ELF_SECTION_TYPES (or lief.ELF.SECTION_TYPES) has been re-scoped in the LIEF::ELF::Section class: 1// \u003cLIEF/ELF/Section.hpp\u003e 2class LIEF_API Section : public LIEF::Section { 3 enum class TYPE : uint64_t { 4 SHT_NULL = 0, /**\u003c No associated section (inactive entry). */ 5 PROGBITS = 1, /**\u003c Program-defined contents. */ 6 ... 7 }; 8}; This means that instead of using LIEF::ELF::ELF_SECTION_TYPES::SHT_PROGBITS or lief.ELF.SECTION_TYPES.SHT_PROGBITS you should now use: 1- LIEF::ELF::ELF_SECTION_TYPES::SHT_PROGBITS 2+ LIEF::ELF::Section::TYPE::PROGBITS 3 4- lief.ELF.SECTION_TYPES.SHT_PROGBITS 5+ lief.ELF.Section.TYPE.PROGBITS The list of the enums affected by this change is listed in the changelog. Performances PE Parser I received some feedback about performance issues in the latest release (0.14.x) compared to former releases. This regression affects Mach-O and PE binaries and I’m happy to say that this v0.15.0 release should be faster on ELF, PE, and Mach-O compared to previous releases. The PE regression comes from the LIEF::PE::OptionalHeader::computed_checksum introduced in LIEF 0.12.0 and discussed in this issue: #660. As of LIEF 0.12.0, this computed_checksum was computed during the parsing phase, and on large binaries, this computation might have a significant impact on the performances. In LIEF 0.15.0, the OptionalHeader’s checksum can be re-computed over the LIEF::PE::Binary object: 1import lief 2 3pe: lief.PE.Binary = ... 4computed_checksum = pe.compute_checksum() Thus, avoiding the computation during the parsing phase and moving to an “on-demand” API. Mach-O Parser On the other hand, the Mach-O regression was pretty tricky to identify (c.f. issue #1069). The root cause of the regression was these lines: 1// https://github.com/lief-project/LIEF/blob/0.14.1/src/MachO/BinaryParser.cpp#L285-L290 2for (LARGE_LOOP) { 3 if (!is_printable(name)) { 4 ... 5 } 6} with is_printable implemented as follows: 1bool is_printable(const std::string\u0026 str) { 2 return std::all_of(std::begin(str), std::end(str), 3 [] (char c) { return std::isprint\u003cchar\u003e(c, std::locale(\"C\")); }); 4} Then, while processing large Mach-O binaries with LIEF we can observe: On Linux: No regression On macOS: REGRESSION On Windows: REGRESSION It turned out that std::locale(\"C\") is cached by the STL on Linux but not on macOS \u0026 Windows. This means that we were invoking std::locale(\"C\") for each character of each string (which has a cost). One solution is to store std::locale(\"C\") in a static variable as it is done – under the hood – in the Linux STL. 1bool is_printable(const std::string\u0026 str) { 2 return std::all_of(std::begin(str), std::end(str), 3- [] (char c) { return std::isprint\u003cchar\u003e(c, std::locale(\"C\")); }); 4+ [] (char c) { 5+ static std::locale LC(\"C\"); 6+ return std::isprint\u003cchar\u003e(c, LC); 7+ }); 8} This actual fix is slightly different though: 7c3f63194. Python Wheels LIEF Python wheels are now available for musl-based systems. This support is motivated by the fact that Python Docker images tagged with the suffix the -alpine are using Alpine system which is based on musl libc. Thus, we can now use Docker’s python-alpine as image base to install LIEF: 1FROM python:3.13.0b3-alpine 2 3RUN pip install --no-cache-dir lief==0.15.0 Note that the LIEF Python wheel for Alpine weighs about 2.5 megabytes compressed and 7 megabytes decompressed. Final Words This new Rust-oriented release is a major milestone for LIEF. While the library is widely used among Python community with ~16,000 daily downloads on PyPI, I’m eager to see new use cases or issues brought by the Rust community. As a reminder, there is a Discord channel where you can drop your questions, and remarks (that are not issues 😉). Thank you also to arttson and lexika979, for their sponsorship. Which makes sense since this is not the purpose of these projects ↩︎","date":172152e4,"description":"LIEF 0.15.0 introduces official Rust bindings and LIEF Extended, with Objective-C, DWARF, PDB, parser performance, and Python wheel updates.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":172152e4,"markdown":"https://lief.re/blog/2024-07-21-lief-0.15-0/index.md","objectID":"fe5e82b354107e244f5842d160669736","permalink":"https://lief.re/blog/2024-07-21-lief-0.15-0/","publishdate":"2024-07-21T00:00:00Z","relpermalink":"/blog/2024-07-21-lief-0.15-0/","section":"blog","summary":"LIEF 0.15.0 introduces official Rust bindings and LIEF Extended, with Objective-C, DWARF, PDB, parser performance, and Python wheel updates.","tags":["release","Rust","LIEF Extended","Objective-C","DWARF","PDB","Python"],"title":"LIEF v0.15.0","type":"blog","wordcount":1680},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"The rust bindings for LIEF are getting more and more production-ready for the next official release of LIEF (v0.15.0). This blog post exposes the recent updates on these bindings and some use cases. Documentation The Rust documentation for the current bindings is now almost complete such as most of the functions and structures are documented: One can access the nightly doc at this address: https://lief-rs.s3.fr-par.scw.cloud/doc/latest/lief/index.html New Architectures Supported As mentioned in the previous blog post, the Rust bindings work with a “pre-compilation” step. Since the previous blog post, I added the support of iOS (i.e. aarch64-apple-ios) and for Linux ARM64 (i.e. aarch64-unknown-linux-gnu) which gives us this support in LIEF compared to the Rust Platform Support Rust Tier 1 Support Triplet Support Comment aarch64-unknown-linux-gnu ✅ i686-pc-windows-gnu ❌ i686-pc-windows-msvc 🤷 Could be supported if needed i686-unknown-linux-gnu 🤷 Could be supported if needed x86_64-apple-darwin ✅ x86_64-pc-windows-gnu ❌ x86_64-pc-windows-msvc ✅ x86_64-unknown-linux-gnu ✅ Rust Tier 2 Support Triplet Support Comment aarch64-apple-ios ✅ aarch64-apple-ios-sim 🤷 Could be supported if needed aarch64-linux-android ⏱️ Planned aarch64-apple-darwin ✅ x86_64-unknown-linux-musl ⏱️ Planned The support for some triplets like i686-pc-windows-msvc will be done on an as-needed basis so feel free to reach out or to open an issue/discussion on GitHub if you need this support. Uses Cases 1// This code checks the PE Authenticode 2 3let path = std::env::args().last().unwrap(); 4let mut file = std::fs::File::open(path).expect(\"Can't open the file\"); 5 6if let Some(lief::Binary::PE(pe)) = lief::Binary::from(\u0026mut file) { 7 let result = pe.verify_signature(pe::signature::VerificationChecks::DEFAULT); 8 if result.is_ok() { 9 println!(\"Valid signature!\"); 10 } else { 11 println!(\"Signature not valid: {}\", result); 12 } 13 return ExitCode::SUCCESS; 14} 15ExitCode::FAILURE 1// This code list all the libraries needed by an ELF binary as well as 2// the versioning of the symbols. 3// Example of output: 4// Dependencies: 5// - libclang-cpp.so.17 6// - libLLVM-17.so 7// - libstdc++.so.6 8// - libc.so.6 9// Versions: 10// From libc.so.6 11// - GLIBC_ABI_DT_RELR 12// - GLIBC_2.14 13// - GLIBC_2.34 14// - GLIBC_2.32 15// From libstdc++.so.6 16// - GLIBCXX_3.4.29 17// - GLIBCXX_3.4.30 18// From libLLVM-17.so 19// - LLVM_17 20 21let mut args = std::env::args(); 22if args.len() != 2 { 23 println!(\"Usage: {} \u003cbinary\u003e\", args.next().unwrap()); 24 return ExitCode::FAILURE; 25} 26 27let path = std::env::args().last().unwrap(); 28let mut file = std::fs::File::open(\u0026path).expect(\"Can't open the file\"); 29if let Some(lief::Binary::ELF(elf)) = lief::Binary::from(\u0026mut file) { 30 println!(\"Dependencies:\"); 31 for entry in elf.dynamic_entries() { 32 if let dynamic::Entries::Library(lib) = entry { 33 println!(\" - {}\", lib.name()); 34 } 35 } 36 println!(\"Versions:\"); 37 for version in elf.symbols_version_requirement() { 38 println!(\" From {}\", version.name()); 39 for aux in version.auxiliary_symbols() { 40 println!(\" - {}\", aux.name()); 41 } 42 } 43 44 return ExitCode::SUCCESS; 45} 46println!(\"Can't process {}\", path); 47ExitCode::FAILURE 1// Inspecting the PE rich header 2 3let path = std::env::args().last().unwrap(); 4let mut file = std::fs::File::open(\u0026path).expect(\"Can't open the file\"); 5 6if let Some(lief::Binary::PE(pe)) = lief::Binary::from(\u0026mut file) { 7 let rich_header = pe.rich_header().unwrap_or_else(|| { 8 println!(\"Rich header not found!\"); 9 process::exit(0); 10 }); 11 12 println!(\"Rich header key: 0x{:x}\", rich_header.key()); 13 for entry in rich_header.entries() { 14 println!(\"id: 0x{:04x} build_id: 0x{:04x} count: #{}\", 15 entry.id(), entry.build_id(), entry.count()); 16 } 17 18 return ExitCode::SUCCESS; 19} 20println!(\"Can't process {}\", path); 21ExitCode::FAILURE 1// Dumping which section of an iOS app is encrypted 2 3let path = std::env::args().last().unwrap(); 4let mut file = std::fs::File::open(\u0026path).expect(\"Can't open the file\"); 5 6if let Some(lief::Binary::MachO(fat)) = lief::Binary::from(\u0026mut file) { 7 for macho in fat.iter() { 8 for cmd in macho.commands() { 9 if let lief::macho::Commands::EncryptionInfo(info) = cmd { 10 println!(\"Encrypted area: 0x{:08x} - 0x{:08x} (id: {})\", 11 info.crypt_offset(), info.crypt_offset() + info.crypt_size(), 12 info.crypt_id() 13 ) 14 } 15 } 16 } 17 return ExitCode::SUCCESS; 18}","date":1718496e3,"description":"LIEF Rust bindings ahead of 0.15.0: new documentation, more supported architectures, Rust tier 1 and tier 2 coverage, and use cases such as Authenticode checks.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1718496e3,"markdown":"https://lief.re/blog/2024-06-16-rust-update/index.md","objectID":"b42f778cde8701ed59cfa743776df5a0","permalink":"https://lief.re/blog/2024-06-16-rust-update/","publishdate":"2024-06-16T00:00:00Z","relpermalink":"/blog/2024-06-16-rust-update/","section":"blog","summary":"The Rust bindings get closer to production for LIEF 0.15.0: dedicated documentation, support for additional architectures across Rust tier 1 and tier 2 targets, and worked use cases such as verifying a PE Authenticode signature.","tags":["Rust","bindings"],"title":"LIEF Rust bindings updates","type":"blog","wordcount":591},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"LIEF Rust bindings are now available. This blog post introduces these bindings and the technical challenges behind this journey. tl;dr 1[package] 2name = \"lief-demo\" 3version = \"0.0.1\" 4edition = \"2021\" 5 6[dependencies] 7lief = { git = \"https://github.com/lief-project/LIEF\", branch = \"main\"} 1use lief::Binary; 2 3fn main() { 4 let mut file = File::open(path).expect(\"Can't open the file\"); 5 6 match Binary::from(\u0026mut file) { 7 Some(Binary::ELF(elf)) =\u003e { 8 for section in elf.sections() { 9 println!(\"{}: 0x{:x}\", section.name(), section.virtual_address()); 10 } 11 }, 12 Some(Binary::PE(pe)) =\u003e { 13 // ... 14 }, 15 Some(Binary::MachO(macho)) =\u003e { 16 // ... 17 }, 18 None =\u003e { 19 // Parsing error 20 } 21 } 22} Nightly documentation is available here: https://lief-rs.s3.fr-par.scw.cloud/doc/latest/lief/index.html and the package will be published on https://crates.io/crates/lief for the 0.15.0 release. Introduction It has been a long journey to have Rust bindings for LIEF, and I’m happy to announce that these bindings are starting to be ready for public release. I’ll take this blog post as an opportunity to share the different challenges that led me to the current design of the bindings. I’m not a Rust guru, so feel free to share your feedback or suggestions! Idiomatic bindings First off, I’m attached to have bindings that are idiomatic in the language they target. Reaching the current state of the Rust API took me most of the time during the development. The Rust language introduces new concepts that do not exactly match what we can find in object-oriented languages. You can get an idea of the Rust API with these examples. Iterate over ELF sections 1use lief::Binary; 2use lief::generic::Section; // for the \"abstract\" traits 3 4let path = std::env::args().last().unwrap(); 5if let Some(Binary::ELF(elf)) = Binary::parse(path.as_str()) { 6 for section in elf.sections() { 7 println!(\"{}\", section.name()); 8 } 9} Get PE PDB path 1use lief::Binary; 2use lief::pe::debug::Entries::CodeViewPDB; 3 4if let Some(Binary::PE(pe)) = Binary::parse(path.as_str()) { 5 for entry in pe.debug() { 6 if let CodeViewPDB(pdb_view) = entry { 7 println!(\"{}\", pdb_view.filename()); 8 } 9 } 10} Access Mach-O Dyld Info 1use lief::Binary; 2use lief::macho::commands::Commands; 3use lief::macho::binding_info::BindingInfo; 4 5if let Some(Binary::MachO(fat)) = Binary::parse(path.as_str()) { 6 for macho in fat.iter() { 7 8 // First version, iterate over the commands 9 for cmd in macho.commands() { 10 // Alternative to `if let` pattern 11 match cmd { 12 Commands::DyldInfo(dyld_info) =\u003e { 13 for binding in dyld_info.bindings() { 14 if let BindingInfo::Chained(chained) = binding { 15 println!(\"Library: 0x{:x}\", chained.address()); 16 } 17 } 18 } 19 _ =\u003e {} 20 } 21 } 22 23 // Second version, using the helper 24 if let Some(dyld_info) = macho.dyld_info() { 25 for binding in dyld_info.bindings() { 26 if let BindingInfo::Chained(chained) = binding { 27 println!(\"Library: 0x{:x}\", chained.address()); 28 } 29 } 30 } 31 } 32} Given this idiomatic goal, there were some challenges in exposing C++ code to Rust. Polymorphism \u0026 Inheritance How to idiomatically bind this C++ code in Rust? 1class Base { 2 virtual std::string get_name() { 3 return \"Base\"; 4 } 5}; 6 7class Derived : public Base { 8 virtual std::string get_name() { 9 return \"Derived\"; 10 } 11}; 12 13class OtherDerived : public Base { 14 virtual std::string get_name() { 15 return \"OtherDerived\"; 16 } 17}; For the inheritance relationship, the idea is to leverage Rust’s enum structure in which, all the leaves of the inheritance tree are an entry of the enum: 1pub enum Inheritance { 2 Derived(Derived), 3 OtherDerived(OtherDerived), 4} Secondly, all these objects inherit and share the get_name() virtual function. To provide this shared property in Rust, we can leverage a Rust trait that would make get_name available for the structures that implement this trait: 1pub trait AsBase { 2 fn get_name(\u0026self) -\u003e String; 3} 4 5impl AsBase for Derived { 6 fn get_name(\u0026self) -\u003e String { 7 ... 8 } 9} 10 11impl AsBase for OtherDerived { 12 fn get_name(\u0026self) -\u003e String { 13 ... 14 } 15} One can also simplify the definition of the trait such as the derived objects only have to provide the FFI reference to the base class: 1pub trait AsBase { 2- fn get_name(\u0026self) -\u003e String; 3+ fn as_base(\u0026self) -\u003e ffi::BaseImpl; 4+ 5+ fn get_name(\u0026self) -\u003e String { 6+ self.as_base().get_name().to_string() 7+ } 8} 9 10impl AsBase for Derived { 11- fn get_name(\u0026self) -\u003e String { 12+ fn as_base(\u0026self) -\u003e ffi::BaseImpl { 13 ... 14 } 15} 16 17impl AsBase for OtherDerived { 18 fn get_name(\u0026self) -\u003e String { 19 ... 20 } 21} LIEF’s Rust bindings highly rely on these patterns to expose classes with polymorphism and inheritance properties. Lifetime In C++, we don’t have the concept of a lifetime for an object. For instance, it’s perfectly fine to write this code: 1int main() { 2 LIEF::PE::Binary* pe = nullptr; 3 { 4 std::unique_ptr\u003cLIEF::PE::Binary\u003e pe_unique = LIEF::PE::Parser::parse(\"...\"); 5 pe = pe_unique.get(); 6 } 7 printf(\"%s\\n\", pe-\u003eget_section(\".text\").name()); // Use-after-free 8 return 0; 9} Nevertheless, the pe pointer used in printf is no longer valid because of the scope of the std::unique_ptr. In Python, nanobind and pybind11 provide helpers to define the lifetime of an object according to its parent or its scope: 1nb::class\u003cLIEF::PE::Binary\u003e(m, \"Binary\") 2 .def_prop_ro(\"sections\", 3 nb::overload_cast\u003c\u003e(\u0026Binary::sections), 4 nb::keep_alive\u003c0, 1\u003e()) With nb::keep_alive, we indicate that the lifetime of the PE section iterator must be at least as long as the lifetime of the PE Binary instance. In Rust, we could express this lifetime with something like: 1pub struct Iterator\u003c'a\u003e { 2 pub it: ffi::Impl, 3} 4 5impl\u003c'a\u003e Iterator\u003c'a\u003e { 6 pub fn new(it: ffi::Impl) -\u003e Self { 7 Self { 8 it, 9 } 10 } 11} 12 13 14impl Binary { 15 pub fn get_iterator(\u0026'a self) { 16 Iterator::new(self.get_ffi_impl()) 17 } 18} But this code is not correct since the lifetime \u003c'a\u003e of the Iterator structure is not bound to an attribute in the structure. For technical-ffi reasons, we can’t bind this lifetime to ffi::Impl. One solution consists of using PhantomData to provide the lifetime semantic: 1pub struct Iterator\u003c'a\u003e { 2 pub it: ffi::Impl, 3 _owner: PhantomData\u003c\u0026'a ffi::PE_Binary\u003e, 4} Safety First! LIEF is developed in what we could say, an “unsafe” language (i.e. C++). On the other hand, Rust provides strong guarantees about memory, concurrency, … Even though LIEF’s core can’t provide the safety guarantees that Rust is giving, I tried to provide some guarantees about the bindings. Coverage 65% of the functions exposed by the Rust binding are covered by the test suite and you can access the coverage report here: https://lief-rs.s3.fr-par.scw.cloud/coverage/index.html (nightly generated). Coverage By covered I mean: \"the function that bridges from C++ to Rust is executed in the test suite\". ASAN Regarding memory safety, Rust allows packages to be compiled with ASAN through compiler options: export RUSTFLAGS=\"-Z sanitizer=address -Clink-args=-fsanitize=address\" export TARGET_CXXFLAGS=\"-fsanitize=address -fno-omit-frame-pointer -O1\" ... Thus, we also leverage this option to compile both: LIEF core and the Rust binding with ASAN. Given the fact that 65% of the functions and 70% of the lines are test-covered, running these tests with ASAN gives us some confidence about the fact that the bindings do not introduce leaks or memory issues. I don’t pretend that the code is free of bugs but at least these mechanisms are in place in the development cycle of the project. Compilation The bindings rely on autocxx to automatically generate rust FFI code from existing C++ include file. Autocxx is powerful but it can fail to process complex headers like LIEF/ELF/Binary.hpp. Thus, I had to create some kind of wrapper over the existing LIEF/*.hpp header files such as autocxx can process them. These wrappers are available in the directory api/rust/include/ The time to generate the Rust FFI code for the different C++ headers is significant: about ~50s with the current bindings. This generation time can be problematic for the end user especially if LIEF is indirectly imported from other dependencies. On the other hand, for fixed versions of LIEF, cxxgen and, autocxx, the code generated by cxxgen and autocxx is always the same. Thus, we can pregenerate and precompile these files to save time during the pure-rust compilation step. Docker All the different steps mentioned in the previous parts: pre-compilation, ASAN, and code coverage are CI-compiled and fully Dockerized. It might also be worth mentioning that the pre-compiled FFI artifacts are also compiled and cross-compiled with Docker. Yes, cross-compiled. Cross-Compilation \u0026 CI Digression Feel free to skip this part which is not strictly related to LIEF \u0026 Rust. LIEF uses GitHub Actions for CI. From my experience, macOS and Windows runners are less available than Linux runners (i.e. you wait more for these runners). In addition, if you use these runners for a private repository (which is not the case for LIEF), you have a pool of 2000 minutes for the CI of the private repo. Depending on the runner you are using, these minutes are counted with a multiplier1: Operating system Minute multiplier Linux 1 Windows 2 OSX 10 1 minute spent on a macOS runner is equivalent to 10 minutes spent on a Linux runner. Hence, if your private project is exclusively using the macOS runner, you don’t have 2000 minutes (~33h) but 200 minutes (~3h). And then, after this pool of 2000 minutes, 1 minute on a macOS 6 vCPU is priced at 0.16$ while the same minute on a Linux 8 vCPU is priced at 0.032$. Given those facts, cross-compiling for macOS and Windows can be interesting. LLVM provides all the facilities to perform this cross-compilation2 and since we are only generating static libraries, we don’t even need the libraries for these platforms. So yes, LIEF core and the Rust FFI library are cross-compiled for Windows(MT/MD CRT) and OSX(aarch64, x86_64) with a Docker container running on Linux :) The Windows and OSX runners are only used for testing that the cross-compilation worked well (i.e. ld64 can link.exe can link the cross-compiled libraries) and that the test suite is also working. Long story short, we save resources and CI minutes by cross-compiling for Windows and OSX in a Docker running on a Linux runner. As a side effect, we also get fully reproducible builds. The whole pipeline (LIEF core compilation, ASAN, coverage, S3 upload) takes less than 15 minutes (with cache optimizations). Other Projects LIEF Rust bindings might not be suitable for all the projects. Especially, if you are looking for a pure-safety-rust library or a #![no_std] context, please consider using these alternatives which are the standards libraries in Rust: Goblin: https://github.com/m4b/goblin gimli-rs - object: https://github.com/gimli-rs/object Acknowledgment Thank you to Erynian for the initial introduction of autocxx when I was working at Quarkslab 😉 https://docs.github.com/en/billing/managing-billing-for-github-actions/about-billing-for-github-actions#minute-multipliers ↩︎ Including the generation of an ad-hoc signature for the Apple Silicon binaries (c.f ld/MachO/SyntheticSections.cpp) ↩︎","date":1714262400,"description":"An introduction to LIEF's Rust bindings, their idiomatic API design, memory-safety model, cross-compilation support, and engineering tradeoffs.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1714262400,"markdown":"https://lief.re/blog/2024-04-28-rust/index.md","objectID":"2e26408ffc682262592f43592c2ea6f3","permalink":"https://lief.re/blog/2024-04-28-rust/","publishdate":"2024-04-28T00:00:00Z","relpermalink":"/blog/2024-04-28-rust/","section":"blog","summary":"An introduction to LIEF's Rust bindings, their idiomatic API design, memory-safety model, cross-compilation support, and engineering tradeoffs.","tags":["Rust","bindings","memory-safety"],"title":"Rust bindings for LIEF","type":"blog","wordcount":1768},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"LIEF v0.14.0 is out, here is an overview of the main changes! What’s new? Python Bindings LIEF v0.14.0 comes with some internal enhancements for the bindings. First, LIEF now uses nanobind instead of Pybind11. This change is motivated by the fact that nanobind reduces the compilation time while also improving the overall performances of the bindings1. The typing stubs (.pyi) are almost complete. This means that almost all the functions and classes have accurate typing information that is not object or Any. Finally, setuptools has been replaced by scikit-build-core as it provides a cleaner API to generate native wheels. ELF LIEF’s ELF module now supports the GNU properties notes and exposes a friendly API to access the underlying properties information. For instance, one can check if AArch64’s PAC is used by an ELF binary using the following API: 1import lief 2 3elf = lief.ELF.parse(\"aarch64-binary.elf\") 4prop: lief.ELF.NoteGnuProperty = elf.get(lief.ELF.Note.TYPE.GNU_PROPERTY_TYPE_0) 5aarch64_feat: lief.ELF.AArch64Feature = prop.find(lief.ELF.NoteGnuProperty.Property.TYPE.AARCH64_FEATURES) 6 7if lief.ELF.AArch64Feature.FEATURE.PAC in aarch64_feat.features: 8 print(\"PAC is supported!\") In addition, the ELF parser can be tweaked to disable parsing some specific parts of an ELF file. For instance, one can skip parsing the relocations as follows: 1import lief 2config = lief.ELF.ParserConfig() 3config.parse_relocations = False 4 5# ELF object without relocations information 6elf = lief.ELF.parse(\"some-binary.elf\", config) PE As of now, one of the major design issues in LIEF is the enum API. Indeed, when I started to develop LIEF, I wanted to have class and enum names as close to their names mentioned in official documentation. But it turned out that those names are – sometimes – already #define in system headers. It means that including a system header which already defines one of these names causes a compilation error: 1#include \u003cum/winnt.h\u003e // #define IMAGE_FILE_MACHINE_AM33 0x01d3 2#include \u003cLIEF/PE/enums.hpp\u003e // /!\\ Compilation error on IMAGE_FILE_MACHINE_AM33 The current (hacky) workaround for this issue is a undef.h file which #undef the names that create conflict between system definition and LIEF (c.f. LIEF/PE/undef.h). Yes, it’s a hack and the current ongoing work to address this issue is a complete refactoring of the enums API which starts with a re-scoping. Currently, all the enums are defined in a single header file and some of them are used by only one class. For instance, the enum LIEF::PE::SIG_ATTRIBUTE_TYPES, has been re-scoped in the LIEF::PE::Attribute: 1// Before (v0.13.x): LIEF/PE/enums.hpp 2enum class SIG_ATTRIBUTE_TYPES { 3 UNKNOWN = 0, 4 CONTENT_TYPE, 5 ... 6}; 7 8// Now (v0.14.0): LIEF/PE/signature/Attribute.hpp 9class LIEF_API Attribute : public Object { 10 public: 11 enum class TYPE { 12 UNKNOWN = 0, 13 CONTENT_TYPE, 14 ... 15 }; 16} As of LIEF v0.14.0, the PE format is mostly impacted by this refactoring and the other formats should be progressively updated accordingly. On Going Work As a reminder, LIEF is exclusively developed on my spare time, so some functionalities might take time to be completed and integrated Rust Bindings This is still ongoing and the bindings are almost completed for ELF, PE, and Mach-O. I still need to create the bindings for the enums and figure out a way to reduce the compilation time but it keeps moving! DWARF \u0026 PDB LIEF will welcome DWARF and PDB debug information support through an external extension. This module will provide a comprehensive API to iterate over DWARF \u0026 PDB information. Final Word Since LIEF 0.13.2, this new version introduces 274 new commits, with 35 292 additions and 39 392 deletions thanks to 15 contributors! The complete changelog is also available. Thank you also to F., antipatico, and MobSF for their sponsoring. https://nanobind.readthedocs.io/en/latest/why.html ↩︎","date":1705795200,"description":"LIEF 0.14.0 release highlights: faster Python bindings, updates to ELF and PE, and ongoing work on Rust, DWARF, and PDB support.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1705795200,"markdown":"https://lief.re/blog/2024-01-20-lief-0-14-0/index.md","objectID":"5217fc7e9b230746183413124f032c6b","permalink":"https://lief.re/blog/2024-01-20-lief-0-14-0/","publishdate":"2024-01-21T00:00:00Z","relpermalink":"/blog/2024-01-20-lief-0-14-0/","section":"blog","summary":"LIEF 0.14.0 release highlights: faster Python bindings, updates to ELF and PE, and ongoing work on Rust, DWARF, and PDB support.","tags":["release","Python","ELF","PE","Rust","DWARF","PDB"],"title":"LIEF v0.14.0","type":"blog","wordcount":585},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"LIEF v0.13.0 is eventually out! The full changelog is available here, but here is a summary of the main changes. Mach-O in Memory Parser LIEF is now able to parse a Mach-O file from an in-memory pointer: 1uintptr_t mhdr = ...; // Absolute address to an in-memory Mach-O file 2auto macho = LIEF::MachO::Parser::parse_from_memory(mhdr); This feature can be handy on iOS to access the in-memory content of the binary. Firstly because some parts of the application code are encrypted (thanks to the LC_ENCRYPTION_INFO commands). Secondly, it can be used on protected code that fills the __data segment with clear (original) strings (c.f. Gotta Catch ‘Em All: Frida \u0026 jailbreak detection). This feature could also be used in pair with _dyld_get_image_header, once we are injected into the targeted process: 1size_t count = _dyld_image_count(); 2for (size_t i = 0; i \u003c count; ++i) { 3 llvm::StringRef Name = _dyld_get_image_name(i); 4 if (Name.contains(\"MyApp.app/Target\")) { 5 auto* mhdr = _dyld_get_image_header(i); 6 auto macho = MachO::Parser::parse_from_memory((uintptr_t)mhdr); 7 } 8} It might also work to access the dyld shared cache libraries but this aspect does not have been heavily tested. Framed ELF Sections The ELF format is – by far – the most tricky format especially when it comes dealing with sections and segments. These ELF structures are two different ways of slicing the binary data: Sections are used by the compiler and the linker Segments are used by the system loader. LIEF implements a mechanism to deal with this dual representation so that if the user updates the .text section, the changes are also committed in the associated segment (if any). It turns out that in some scenarios1, we might want to NOT commit the changes we are doing on the sections. The lief.ELF.Section.as_frame() function can be used to make the section “frame only”. All the attributes of the section will be committed in the final ELF binary but LIEF won’t consider this section to write the content in the binary. One can use – for instance – this function to corrupt sections attributes: 1elf = lief.parse(\"/bin/ls\") 2text = elf.get_section(\".text\").as_frame() 3text.offset = 0xffffff 4elf.write(\"ls.modified\") As the .text section is set as “framed”, its 0xffffff offset is not considered for changing its content. Thus, this code does not update anything: 1text.content = ... Since the ELF loader only relies on segment, this modification does not affect the execution of the modified binary. Internal Changes As announced in the previous v0.12.0 release, the LIEF codebase is now free from exceptions and RTTI. This means that the core library can be compiled in an -fno-exceptions context. The Python build process is now compliant with the PEP 621 pyproject.toml requirement. You can find this file, along with config-default.toml, in the api/python directory. The config-default.toml file can be used to tweak the compilation of the bindings. This is essentially an interface over the LIEF CMake option. Lastly, the setup.py file used for compiling the binding has moved from the root directory to api/python. On top of that, the Python bindings also generate stub interfaces (.pyi) which are handy for type checking and code completion (cf. issues/650). Final Words Some features like the Rust bindings are not released yet they still require some ongoing work. I also really do hope to be able to work on enhancing the PE format modification in the next release but this will be balanced with the spare time I have to work on it :) Thank you for using LIEF! https://passthesalt.ubicast.tv/videos/the-poor-mans-obfuscator/ ↩︎","date":1680998400,"description":"LIEF 0.13.0: in-memory Mach-O parsing, framed ELF sections, an exception-free and RTTI-free core, PEP 621 Python packaging, and generated .pyi type stubs.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1680998400,"markdown":"https://lief.re/blog/2023-04-09-lief-0-13-0/index.md","objectID":"9acc4ef447b5cbcde4f60bd3fdaa686c","permalink":"https://lief.re/blog/2023-04-09-lief-0-13-0/","publishdate":"2023-04-09T00:00:00Z","relpermalink":"/blog/2023-04-09-lief-0-13-0/","section":"blog","summary":"LIEF 0.13.0 adds an in-memory Mach-O parser and framed ELF sections, ships a core free of exceptions and RTTI that compiles with exceptions disabled, moves the Python bindings to PEP 621 packaging, and generates .pyi stubs for type checking.","tags":["release","Mach-O","ELF","Python"],"title":"LIEF v0.13.0","type":"blog","wordcount":576},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"tl;dr The next release of LIEF (v0.13.0) is fixing several Mach-O layout issues when adding new sections/segments. I also added the support for the two new load commands: LC_DYLD_CHAINED_FIXUPS LC_DYLD_EXPORTS_TRIE The support of LIEF for modifying Mach-O binaries was mostly limited to adding new load commands and thus, extending the load commands table. The tutorial #11 explains the technical details to extend the load commands table which consists in shifting the content right after the load commands table and patching the relocations accordingly. Nevertheless, the Mach-O binaries generated by LIEF after the modifications were somehow inconsistent regarding codesign. As a consequence, the binaries generated by LIEF could not be signed and executed on iOS or – more recently – an Apple M1. LIEF is now able to generate Mach-O-modified files that can be signed and that follow a strict layout, enforced by dyld and codesign. To better understand what was wrong, let’s consider the following script in which we add two new segments: 1import lief 2 3target = lief.parse(\"mbedtls_selftest_arm64.bin\") 4 5segment = lief.MachO.SegmentCommand(\"__NEW\", [0] * 0x123) 6target.add(segment) 7 8segment = lief.MachO.SegmentCommand(\"__NEW\", [0] * 0x456) 9target.add(segment) 10 11target.write(\"test.out\") Under the hood, LIEF was relocating the binary to add two new LC_SEGMENT commands and was allocating space at the end of the file to store the content of the new segments. In particular, the new segments data were located after the content of the __LINKEDIT segment which breaks the layout required by codesign. The following figure depicts the layout of a Mach-O file from the original layout to the layout generated by LIEF v0.13.0. In LIEF v0.13.0 we fixed this inconsistency to make sure that the content of the new segments are located before the content of the __LINKEDIT segment. We can perform this change without breaking the binary as __LINKEDIT is a kind of self-contained blob of data1. codesign requires the __LINKEDIT segment at the end of the file because the signature is appended at the end of the file. Otherwise, codesign would have to perform the similar relocation process done by LIEF. __LINKEDIT The __LINKEDIT segment plays an important role in the layout of the Mach-O format and its execution. This segment is used to store information about the exports, the symbols, the relocations, the signature, and more broadly, information used by the dyld loader to load the binary. This segment has a known layout which is described in the following figure: This layout is very strict and its content must follow the same order as mentioned in the previous figure. In addition, there are sanity checks that ensure all the __LINKEDIT’s chunks are contiguous within the __LINKEDIT content. If the layout is wrong, the executable could run but it won’t likely pass the codesign checks. This strict layout can be seen – at first sight – as a major hurdle for modifying Mach-O files but since the __LINKEDIT segment is located at the end of the file, we can extend it or shrink it quite easily. LIEF v0.13.0 is able to regenerate the content of this segment from the LIEF objects stored in the LIEF::MachO::Binary object Completely regenerating the __LINKEDIT segment enables to perform advanced modifications like creating exports and adding or removing symbols as it is discussed in the next sections. LC_DYLD_CHAINED_FIXUPS \u0026 LC_DYLD_EXPORTS_TRIE Compared to the ELF and PE formats, the relocations and the exported functions of Mach-O binaries are not wrapped by a table of entries In the Mach-O format, the relocations are encoded either: By a bytecode located in the LC_DYLD_INFO command By a chained fixups located in the LC_DYLD_CHAINED_FIXUPS On the other hand, the exports are encoded in a Trie located either In the LC_DYLD_INFO command In the LC_DYLD_EXPORTS_TRIE LC_DYLD_CHAINED_FIXUPS appeared more recently compared to the LC_DYLD_INFO command for which the differences are described in the blog post: How iOS 15 makes your app launch faster. The LC_DYLD_EXPORTS_TRIE has the same structure as LC_DYLD_INFO[Export Trie] but the export information has been moved in this dedicated load command. Converting a Mach-O Binary into a Library Converting a binary into a library can be useful to harness a fuzzed binary or to instrument/debug a specific function in a controlled environment (like an unknown cryptography function or a whiteboxed function) In the tutorial #8, we described the process to perform this transformation on an ELF binary and the transformation for a Mach-O binary is a bit more straightforward. Let’s consider the following code: 1#include \u003cstdint.h\u003e 2#include \u003cstdio.h\u003e 3#include \u003cstdlib.h\u003e 4 5static int X = 1; 6 7int compute() { 8 return X++; 9} 10 11int main(int argc, const char** argv) { 12 for (size_t i = 0; i \u003c argc; ++i) { 13 printf(\"compute(): %d\\n\", compute()); 14 } 15 return 0; 16} It can be compiled with: 1romain@Mac-M1 % clang -O3 -fvisibility=hidden -Wl,-x -o bin2lib.bin bin2lib.c Which produces this executable: bin2lib.bin To convert this binary into a library, we first need to change its type in the Mach-O’s header: 1import lief 2bin2lib = lief.parse(\"bin2lib.bin\") 3 4bin2lib.header.file_type = lief.MachO.FILE_TYPES.DYLIB 5 6bin2lib.write(\"bin2lib.dyld\") It’s should be technically enough, but dyld_info raises some concerns: 1romain@Mac-M1 % dyld_info ./bin2lib.dylib 2dyld_info: './bin2lib.dylib' in './bin2lib.dylib' MH_DYLIB is missing LC_ID_DYLIB This can be confirmed by looking at the source code of dyld. To fix this error, we just have to create a new LC_ID_DYLIB command: 1import lief 2bin2lib = lief.parse(\"bin2lib.bin\") 3 4bin2lib.header.file_type = lief.MachO.FILE_TYPES.DYLIB 5+ bin2lib.add(lief.MachO.DylibCommand.id_dylib(\"bin2lib.dylib\", 0, 1, 2)) 6 7bin2lib.write(\"bin2lib.dyld\") Which enables to dlopen bin2lib.dyld 1import ctypes 2handler = ctypes.cdll.LoadLibrary(\"bin2lib.dyld\") 3# \u003cCDLL './bin2lib.dyld', handle 208270460 at 0x107d277f0\u003e Adding Symbols Thanks to the improvements on the __LINKEDIT segment, we can now create new exports. If we consider the stripped function int compute() from the binary in the previous section, we can create a new export as follows: address = 0x100003f18 original.add_exported_function(address, \"_compute\") Code Injection Another use case of these improvements is the capability to inject code in Mach-O file and to re-sign the modified binary. Code signing is not required for x86-64 binaries but it becomes mandatory when targeting the arm64 architecture. Let’s consider the library _heapq.cpython-39-darwin.so which is one of the first libraries dynamically loaded by the Python interpreter. The injection consists in: Creating new segments in the library _heapq.cpython-39-darwin.so that will embed our shellcode Changing the address of one of the exported functions to redirect the execution to the shellcode’s entrypoint. By running the python interpreter with the environment variable DYLD_PRINT_APIS=1 we can observe the following output: 1romain@Mac-M1 ~ % DYLD_PRINT_APIS=1 python3 -c \"import io\" 2dyld[76439]: _dyld_is_memory_immutable(0x1b3f8cea0, 26) =\u003e 1 3dyld[76439]: dlopen(\"/opt/homebrew/Cellar/python@3.9/3.9.5/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload/_heapq.cpython-39-darwin.so\", 0x00000002) 4dyld[76439]: dlopen(_heapq.cpython-39-darwin.so) =\u003e 0x208f35800 5dyld[76439]: dlsym(0x208f35800, \"PyInit__heapq\") 6dyld[76439]: dlsym(\"PyInit__heapq\") =\u003e 0x104bcb824 It suggests that PyInit__heapq is a suitable function for redirecting the execution to the shellcode’s entrypoint. To create the shellcode, we can use gdelugre/shell-factory developed by a former colleague and which provides no less than a C++ STL-like to create shellcode. Thanks to this project, we can create the following shellcode: 1volatile uintptr_t ORIGINAL_EP = 0xdeadc0de; 2volatile uintptr_t IMAGEBASE = 0x00c0de; 3using PyInit__heapq_t = void(*)(); 4 5inline uintptr_t imagebase() { 6 /* 7 * The value of IMAGEBASE is set by the injector. 8 * After the patch, it contains the relative virtual address of \u0026IMAGEBASE 9 * in the final binary. 10 */ 11 return reinterpret_cast\u003cuintptr_t\u003e(\u0026IMAGEBASE) - IMAGEBASE; 12} 13 14SHELLCODE_ENTRY 15{ 16 uintptr_t base = imagebase(); 17 Pico::printf(\"LIEF says hello!\\n\"); 18 Pico::printf(\"Time to jump on the real function: %p\\n\", ORIGINAL_EP); 19 auto PyInit__heapq = reinterpret_cast\u003cPyInit__heapq_t\u003e(base + ORIGINAL_EP); 20 return PyInit__heapq(); 21} Pico::printf The attentive reader may have noticed the Pico::printf(\"[...] %p\") which is correctly supported by shell-factory (see: include/pico/format.h) The compiled shellcode can be downloaded here: lief_demo_darwin_arm64.bin. To inject the shellcode in _heapq.cpython-39-darwin.so, we first need to copy the shellcode’s segments in the library: 1shellcode = lief.parse(\"lief_demo_darwin_arm64.bin\") 2heapq = lief.parse(\"_heapq.cpython-39-darwin.so\") 3 4for segment in shellcode.segments: 5 seg_name = segment.name.replace(\"__\", \"\") 6 seg = lief.MachO.SegmentCommand(f\"__L{new_seg_name}\", list(segment.content)) 7 8 heapq.add(new_seg) Then, we have to patch the Mach-O exports trie to change the address of PyInit__heapq to the shellcode’s entrypoint: 1shellcode_rva_entry = ... 2for exp in heapq.dyld_info.exports: 3 if exp.symbol.name != \"_PyInit__heapq\": 4 continue 5 6 original = exp.address 7 exp.address = shellcode_rva_entry 8 return original Finally, we can rewrite the library: 1heapq.write(\"_heapq.cpython-39-darwin.so.patched\") and sign it: 1romain@Mac-M1 ~ % codesign -f --verbose -s - _heapq.cpython-39-darwin.so.patched Now when running the Python interpreter, we can observe the execution of the shellcode: 1romain@Mac-M1 ~ % python3 2LIEF says hello! 3Time to jump on the real function: 0x15f8 4Python 3.9.5 (default, May 3 2021, 19:12:05) 5[Clang 12.0.5 (clang-1205.0.22.9)] on darwin 6Type \"help\", \"copyright\", \"credits\" or \"license\" for more information. 7\u003e\u003e\u003e Injection The script that contains the complete logic of the transformation is available here and, _heapq.cpython-39-darwin.so.patched can be downloaded here. Surprisingly, we open the patched version of the library (_heapq.cpython-39-darwin.so.patched) in IDA and we jump on the symbol _PyInit__heapq, it actually displays this function: IDA Version 7.7.211224, January 18, 2022 Which is the original function and not the function associated with the shellcode whilst the patched library prints LIEF says hello [...] On the other hand, if we get the address of _PyInit__heapq with LIEF: 1import lief 2patched = lief.parse(\"./_heapq.cpython-39-darwin.so.patched\") 3symbol = patched.get_symbol(\"_PyInit__heapq\") 4print(hex(symbol.export_info.address)) The result is: _PyInit__heapq: 0xf824 Jumping on this address gives a better output (once manually disassembled): We recognize the shellcode’s entrypoint function . What’s happened in IDA since this is the function located at 0xf824 which is executed and thus, resolved by dyld and not IDA? IDA is confused because Mach-O’s symbols can be stored in two different commands: LC_DYLD_INFO.export_trie / LC_DYLD_EXPORTS_TRIE LC_SYMTAB LC_DYLD_INFO.export_trie / LC_DYLD_EXPORTS_TRIE are used to store the exported symbols while LC_SYMTAB stores symbols for other purposes. The important point is that the same symbol can be duplicated in these two commands with different addresses. IDA gives the priority to the LC_SYMTAB over the exports trie while the Mach-O loader uses the exports trie. The following figure illustrates why it can be confusing: Actually, I intentionally took a shortcut in the LIEF script that resolves the address of _PyInit__heapq and we can programmatically access these two addresses as follows: 1import lief 2patched = lief.parse(\"./_heapq.cpython-39-darwin.so.patched\") 3symbol = patched.get_symbol(\"_PyInit__heapq\") 4+ print(hex(symbol.value)) 5print(hex(symbol.export_info.address)) 6 7+ # 0x15f8 address from the LC_SYMTAB 8 # 0xf824 address from the export trie We can observe a similar issue with BinaryNinja, Ghidra and, to a lesser extent, Radare2 BinaryNinja Version 3.0 Ghidra Version 10.1.2 - Jan 26, 2022 Radare2 Version: 5.6.6 - Mar 22, 2022 1$ r2 _heapq.cpython-39-darwin.so.patched 2[0x00000000]\u003e aaa 3... 4[0x00000000]\u003e ia 5 6[Imports] 7nth vaddr bind type lib name 8――――――――――――――――――――――――――――――――― 90 0x000021ec NONE FUNC PyErr_SetString 101 0x00000000 NONE FUNC PyExc_IndexError 112 0x00000000 NONE FUNC PyExc_RuntimeError 123 0x00000000 NONE FUNC PyExc_TypeError 134 0x000021f8 NONE FUNC PyList_Append 145 0x00002204 NONE FUNC PyList_SetSlice 156 0x00002210 NONE FUNC PyModuleDef_Init 167 0x0000221c NONE FUNC PyModule_AddObject 178 0x00002228 NONE FUNC PyObject_RichCompareBool 189 0x00002234 NONE FUNC PyUnicode_FromString 1910 0x00002240 NONE FUNC _PyArg_CheckPositional 2011 0x0000224c NONE FUNC _Py_Dealloc 2112 0x00000000 NONE FUNC _Py_NoneStruct 2213 0x00000000 NONE FUNC dyld_stub_binder 23 24[Exports] 25 26nth paddr vaddr bind type size lib name 27――――――――――――――――――――――――――――――――――――――――――――――――――― 280 0x000015f8 0x000015f8 GLOBAL FUNC 0 _PyInit__heapq On the other hand, the afl command outputs a better result: 1[0x00000000]\u003e afl 20x000015f8 1 12 sym._PyInit__heapq 30x00001604 6 108 sym._heapq_exec 40x00002238 1 8 fcn.00002238 50x00002220 1 8 fcn.00002220 60x00002250 1 8 fcn.00002250 7... 80x0000f824 1 88 sym.imp._PyInit__heapq Demo Conclusion These changes strengthen LIEF to read and modify Mach-O binaries. It should enable to develop and create new reverse engineering and binary analysis techniques. For those who are interested in Mach-O (and ELF) tricks that could prevent static analysis tools from working correctly, I’ll present The Poor Man’s Obfuscator at Pass The Salt in July 2022 :) In the general case, we can’t insert content between two arbitrary segments as it could break the binary. For instance, if the __TEXT segment references variables in the __DATA segment with relative addressing, inserting some data between these two segments will likely break the relative addressing. ↩︎","date":1651968e3,"description":"Mach-O rewriting in LIEF: __LINKEDIT fixes, chained fixups and exports trie support, converting a binary into a dylib, adding exported symbols, and code injection.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1651968e3,"markdown":"https://lief.re/blog/2022-05-08-macho/index.md","objectID":"63f90daea71fec85d376b29358b1c8ea","permalink":"https://lief.re/blog/2022-05-08-macho/","publishdate":"2022-05-08T00:00:00Z","relpermalink":"/blog/2022-05-08-macho/","section":"blog","summary":"Mach-O support enhancements ahead of LIEF 0.13.0: __LINKEDIT fixes, LC_DYLD_CHAINED_FIXUPS and LC_DYLD_EXPORTS_TRIE support, converting an executable into a loadable library, adding exported symbols, and injecting code, checked against BinaryNinja, Ghidra, and Radare2.","tags":["Mach-O","macOS","iOS","code-injection"],"title":"Mach-O Support Enhancements","type":"blog","wordcount":1962},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"We are thrilled to announce that LIEF v0.12.0 is released! You can find the complete changelog here. LIEF v0.12.0: What’s New? LIEF v0.12.0 is a balanced mix of new features, internal refactoring, and performance improvement. New Features Regarding the new features, we added support for recomputing the PE’s rich header and the PE’s checksum. The PE’s rich header is a well-known-hidden1 feature that can be helpful to fingerprint a PE binary. LIEF enables – since the version v0.7.0 – to access this part of the PE file with the following API: 1import lief 2pe_file = lief.parse(\"hello.exe\") 3 4rich_header = pe_file.rich_header 5print(f\"XOR Key: {rich_header.key}\") 6for e in rich_header.entries: 7 print(f\"{e.id}: {e.build_id} {e.count}\") In LIEF v0.12.0, we added two functions: LIEF::PE::RichHeader::raw: To generate the rich header blob with or without an XOR key. LIEF::PE::RichHeader::hash: To generate the MD5/SHA-1/SHA-256/(…) of the rich header blob. For those who are looking for PE’s markers or tracking PE binaries, these two functions could be used to generate a characteristic of the binary, regardless of the xor-key: 1# [...] 2rich_header = pe_file.rich_header 3 4marker = bytes(rich_header.hash(lief.PE.ALGORITHMS.SHA_1)).hex() Still about the PE format, we added LIEF::PE::OptionalHeader::computed_checksum() which returns the recomputed value of the PE’s checksum (LIEF::PE::OptionalHeader::checksum()). For regular binaries, the verification of the OptionalHeader’s checksum is not enforced by Windows and the integrity checks are usually deferred to the PE’s Authenticode. Nonetheless, verifying the checksum() value with the output of computed_checksum() could help identify binaries that would have been modified after the compilation. Finally, we added the support for the PE’s delayed imports in LIEF and Luca Moro added the support of the LC_FILESET_ENTRY command in the Mach-O format. Refactoring \u0026 Performance Improvement We also refactored and enhanced LIEF’s internal codebase. Among those changes, we started to get rid of the C++ exceptions as described in this blog post: LIEF RTTI \u0026 Exceptions We also introduced a std::span like interface (based on tcbrindle/span) to avoid returning and potentially copying std::vector\u003cuint8_t\u003e. For instance, LIEF::Section::content now uses the span interface. In the Python API, functions and properties that bind a span-returning function now return a py::memoryview instead of the list of bytes. The original list of bytes can be recovered as follows: 1bin = lief.parse(\"/bin/ls\") 2section = bin.get_section(\".text\") 3 4if section is not None: 5 memory_view = section.content 6 list_of_bytes = list(memory_view) About the performances, we did a global refactoring of the ELF builder as described in this blog post: New ELF Builder. We also reduced the memory footprint of the ELF parser. For instance, in LIEF v0.11.5 a binary of 1.5G takes 3G or RAM2 while in LIEF v0.12.0, it takes quite the same memory as the file size. Eric Kilmer also did a nice and complete cleaning of the LIEF CMake integration In February 2022, tmp.0ut v2 has been released and @netspooky presented interesting tricks on the ELF format 3 4. We fixed the ELF parser to make sure we handle these tricks. What’s Next? We started to implement Rust bindings for LIEF thanks to cxx and google/autocxx. These bindings are in their early stages and we can’t confirm they will be present in the next release. In the current development stage, the API looks like this: 1let mut path: String = \"/bin/ls\"; 2 3match Binary::parse(\u0026path) { 4 Binary::ELF(elf) =\u003e { 5 println!(\"ELF binary\"); 6 for segment in elf.segments() { 7 println!(\"Address: {:x}\", segment.virtual_address); 8 } 9 }, 10 Binary::PE(pe) =\u003e { 11 println!(\"PE binary\"); 12 let text_section = pe.get_section(\".text\"); 13 text_section.name = \".foo\"; 14 text_section.file_offset = 0x123; 15 16 text_section.commit(); // Commit the changes 17 }, 18 Binary::MachO(macho) =\u003e { 19 println!(\"MachO binary\"); 20 for command in macho.commands() { 21 match command { 22 Commands::Dylib(dylib) =\u003e { 23 ... 24 }, 25 Commands::Main(main_cmd) =\u003e { 26 ... 27 }, 28 } 29 } 30 31 }, 32 Binary::Unknown(x) =\u003e { 33 println!(\"Unknown\"); 34 }, 35} We will also merge the (still private) branch that enables to parse Mach-O from memory as well as the global improvement of the Mach-O’s builder. Regarding LIEF’s experimentations and work in progress, here is a list of topics on which we are working or we would like to work: Topic Status Parsing ELF files from memory Not started yet Parsing DART/Flutter snapshots PoC Creating an ELF from scratch PoC Parsing PE’s private Authenticode: MS Counter Signature Not started yet Refactoring the PE’s builder Not started yet, priority undefined Supporting the Mach-O’s commands: LC_DYLD_CHAINED_FIXUPS / LC_DYLD_EXPORTS_TRIE Done, under testing Supporting the archive format (AR) Early stage If you are interested in supporting some of these topics, feel free to reach out. Enjoy! https://www.virusbulletin.com/virusbulletin/2020/01/vb2019-paper-rich-headers-leveraging-mysterious-artifact-pe-format/ ↩︎ More generally, we have a factor 2 in memory compared to the file size. ↩︎ https://tmpout.sh/2/3.html ↩︎ https://tmpout.sh/2/14.html ↩︎","date":1648339200,"description":"LIEF 0.12.0: PE rich header and checksum recomputation, span-based section content with memoryview in Python, and the start of the exception-free refactoring.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1648339200,"markdown":"https://lief.re/blog/2022-03-27-lief-v0-12-0/index.md","objectID":"78e4b1f7fb19b205fd17b361c0e8cc0e","permalink":"https://lief.re/blog/2022-03-27-lief-v0-12-0/","publishdate":"2022-03-27T00:00:00Z","relpermalink":"/blog/2022-03-27-lief-v0-12-0/","section":"blog","summary":"LIEF 0.12.0 adds RichHeader::raw and RichHeader::hash and PE checksum recomputation, introduces a std::span-like interface exposed as memoryview in Python, and begins removing C++ exceptions from the codebase.","tags":["release","PE","Python","C++"],"title":"LIEF v0.12.0","type":"blog","wordcount":779},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"try { When we started to develop LIEF, we choose to manage errors through the C++ exceptions as it is widely spread in Java. However, with a little hindsight it was not the best choice in the design of LIEF. First off, LIEF is a library and the API functions that throw exceptions are not compatible with library’s users that are not using exceptions (e.g with the -fno-exceptions flag). It is also considered as a bad practice quoting from C++ Coding Standards: C++ Coding Standards: Item 62 “Don't throw stones into your neighbor's garden: There is no ubiquitous binary standard for C++ exception handling.” For instance, the function LIEF::ELF::Binary::get_section(const std::string\u0026 name) threw an exception if the section were not found. To avoid raising the exception, the API exposes helpers that can be used to check – beforehand – that it will not take the exception path: 1if (bin.has_section(\".toto\")) { 2 auto\u0026 sec = bin.get_section(\".toto\"); // Ok no exception 3} 4 5// With exception: 6try { 7 bin.get_section(\".toto\"); 8} catch (const std::exception\u0026) { 9 // .toto does not exist :( 10} Actually, the has_\u003celement\u003e / get_\u003celement\u003e pattern hides another issue: the performances. Basically, has_section(...) iterates over the list of the sections to check if a section with the given name exists and get_section() iterates again on this list to access the section. The code performs twice the same iteration. This is not a big deal for the ELF sections as they are quite small but it can be problematic for large sequences like the symbols table. In LIEF v0.12.0 we changed the API of these functions to return a pointer on these objects instead of a reference. If the item can’t be found, it returns a nullptr. The API contract of these functions is changing from raising an exception into returning a nullptr. The documentation has been updated accordingly and the list of the functions which have changed are listed here As a result, the previous code can be re-written as follows: 1if (bin.has_section(\".toto\")) { 2 auto* sec = bin.get_section(\".toto\"); // Non nullptr instead of a reference 3} 4 5// Or: 6if (auto* sec = bin.get_section(\".toto\")) { 7 // ... 8} This kind of API change is doable and meaningful for functions that aim at returning an optional object but it is less meaningful to transform a function like: 1uint64_t Binary::virtual_address_to_offset(...) { ... } that returns an integer (while still potentially raising an exception). In LIEF 0.12.0, this kind of function still raises an exception but in the next version (LIEF v0.13.0) the returned value will be wrapped by Boost’s Leaf1 such as the returned type will become: 1// Future returned type 2result\u003cuint64_t\u003e Binary::virtual_address_to_offset(...) { ... } 3 4// To use it: 5auto res = bin.virtual_address_to_offset(); 6if (!res) { 7 // Error 8} else { 9 uint64_t val = res.value(); // or val = *res 10} In LIEF v0.12.0 only internal/private functions associated with the Parser/Builder module are using this mechanism and we plan to move to this mechanism in the public API2 in LIEF v0.13.0. Warning Boost LEAF is required in the public headers of LIEF. If you find conflicts, compilation issues, integration issues, or you think that it is a bad idea, please let us know before it becomes the default interface to manage errors. The RTTI LIEF also relies on the RTTI information which includes calling functions like typeid() or dynamic_cast\u003c\u003e(). For instance, to check if a Mach-O’s LoadCommand exists, the main MachO::Binary class calls at some point this helper: 1template\u003cclass T\u003e 2bool Binary::has_command() const { 3 static_assert(std::is_base_of\u003cLoadCommand, T\u003e::value, 4 \"Require inheritance from 'LoadCommand'\"); 5 6 const auto it_cmd = std::find_if( 7 std::begin(commands_), std::end(commands_), 8 [] (const LoadCommand* command) { 9 return typeid(T) == typeid(*command); 10 }); 11 12 return it_cmd != std::end(commands_); 13} This code generates extra data for the RTTI information of the LoadCommand objects which can be perfectly fine. Actually, this RTTI information is redundant as the type of a Mach-O’s LoadCommand is already stored in the class itself: 1class LoadCommand { 2 ... 3 private: 4 LOAD_COMMAND_TYPES command_; 5}; So instead of having this redundant RTTI information, we implemented an LLVM-like RTTI3 based on classof() and that uses the already present command_ attribute. In the end, the previous has_command() can be updated as follows: 1template\u003cclass T\u003e 2bool Binary::has_command() const { 3 static_assert(std::is_base_of\u003cLoadCommand, T\u003e::value, 4 \"Require inheritance from 'LoadCommand'\"); 5 const auto it_cmd = std::find_if( 6 std::begin(commands_), std::end(commands_), 7 [] (const LoadCommand* command) { 8 return T::classof(command); 9 }); 10 return it_cmd != std::end(commands_); 11} We applied this pattern for the LIEF’s object where typeid was present and as a result, we managed to completely remove this function as it was redundant with an existing attribute. } catch (const std::length_error\u0026) { We welcome feedback on these changes – whether positive or negative – as it impacts the public API. Thank you for reading! } See the section Error Handling of the documentation for more details. ↩︎ It still keeps the public headers compliant with C++11 ↩︎ https://llvm.org/docs/HowToSetUpLLVMStyleRTTI.html ↩︎","date":1644710400,"description":"Why LIEF is removing C++ exceptions and RTTI: error-handling costs, the has_/get_ pattern, and the move to result-based APIs and LLVM-style RTTI.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1644710400,"markdown":"https://lief.re/blog/2022-02-13-lief-rtti-exceptions/index.md","objectID":"6ba95e8c883ea8e087f070418b4e48c4","permalink":"https://lief.re/blog/2022-02-13-lief-rtti-exceptions/","publishdate":"2022-02-13T00:00:00Z","relpermalink":"/blog/2022-02-13-lief-rtti-exceptions/","section":"blog","summary":"How and why LIEF started removing C++ exceptions and RTTI from its codebase: the cost of exception-based error handling, the has_/get_ pattern, and the switch to result-based APIs and LLVM-style RTTI.","tags":["C++","internals","error-handling"],"title":"LIEF RTTI \u0026 Exceptions","type":"blog","wordcount":829},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"LIEF’s Modification Process Let’s start with a small recap of the LIEF modification process. To enable executable file formats modification, LIEF transforms the raw executable formats into an object representation. This object can be manipulated with an API that is mainly exposed through the following interfaces: C++ Python LIEF::ELF::Binary lief.ELF.Binary LIEF::PE::Binary lief.PE.Binary LIEF::MachO::Binary lief.MachO.Binary Then, the LIEF’s builders take the object representation and (try to) reconstruct an executable according to the user’s changes. Challenges in Modifying ELF Binaries Compared to the PE and Mach-O formats, the ELF format is far trickier to handle for both parsing and modifying. First off, there is a strong relationship between the segment’s virtual address and the file’s offset associated with its content. This relationship is ruled by the following property: $$\\text{\\textcolor{red}{file\\_offset}} \\equiv \\text{\\textcolor{blue}{virtual\\_address}} \\mod{\\textcolor{green}{\\text{page\\_size}}}$$ So basically, we can’t insert a segment at an arbitrary virtual address. The second difficulty is about the strings table optimization that is performed on the .dynstr section. To understand how this optimization works, let’s consider these two functions: 1int foo() { 2 return 1; 3} 4 5int call_foo() { 6 return foo(); 7} When these functions are compiled, the compiler generates two symbols for which the names of the symbols are referenced by the field st_name. Usually, this field points in the .dynstr section: 1struct Elf_Sym { 2 Elf_Word st_name; // Offset of the symbol's name in the .dynstr section 3 ... 4}; Naively, we could imagine that the .dynstr section contains these two symbols names, one next to the other: 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000140 0d 00 00 00 12 00 01 00 00 00 00 00 00 00 00 00 |................| 00000150 0b 00 00 00 00 00 00 00 0a 00 00 00 12 00 01 00 |................| 00000160 0b 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 |................| 00000170 00 74 6f 74 6f 2e 63 70 70 00 66 6f 6f 00 64 6f |.test.cpp.foo.do| 00000180 5f 66 6f 6f 00 00 00 00 10 00 00 00 00 00 00 00 |_foo............| 00000 With such a layout, Elf_Sym(\"foo\").st_name would point to the offset 0x17A while Elf_Sym(\"do_foo\").st_name would point to the offset 0x17E. But the real layout of the .dynstr is a bit smaller: 00000130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| 00000140 0d 00 00 00 12 00 01 00 00 00 00 00 00 00 00 00 |................| 00000150 0b 00 00 00 00 00 00 00 0a 00 00 00 12 00 01 00 |................| 00000160 0b 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 |................| 00000170 00 74 6f 74 6f 2e 63 70 70 00 64 6f 5f 66 6f 6f |.test.cpp.do_foo| 00000180 00 00 00 00 00 00 00 00 10 00 00 00 00 00 00 00 |................| 00000190 04 00 00 00 03 00 00 00 fc ff ff ff ff ff ff ff |................| As we can see, it only contains the do_foo string. Since foo is a suffix of do_foo, st_name can point to a different offset of the same string. In this layout Elf_Sym(\"foo\").st_name points to the offset 0x17C and Elf_Sym(\"foo\").st_name points to 0x17A. Consequently, instead of taking the space of len(call_foo) + 1 + len(foo) + 1, it only takes len(call_foo) + 1 The consequence of this optimization is that we can’t naively push back the symbols names in the .dynstr section. Instead, we have to sort the symbols names such as this optimization can take place. In addition to this strings optimization, ELF object files (.o) generated by Clang share the same section for the names of the sections and for the symbols’ names. 1$ readelf -hWS ./hello.o 2 3ELF Header: 4 Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 5 [...] 6 Number of section headers: 11 7 Section header string table index: 1 8 9Section Headers: 10 [Nr] Name Type Address Off Size ES Flg Lk Inf Al 11 [ 0] NULL 0000000000000000 000000 000000 00 0 0 0 12 [ 1] .strtab STRTAB 0000000000000000 000199 000078 00 0 0 1 13 [..] 14 [10] .symtab SYMTAB 0000000000000000 0000c0 000090 18 1 4 8 As we can notice, the Section header string table index of the ELF header indexes the .strtab which is also the section associated with the symbols’ names (cf. the link attribute of the .symtab). It results that we have to consider this kind of ELF file differently from regular libraries or executables. There are other nasty tricks like the management of the ELF constructors between Linux and Android but this will be covered in another blog post. The New ELF Builder For the historical context, I created LIEF during my internship at Quarkslab with the supervision of Serge-Sans-Paille and Adrien Guinet and the trust/boost from Fred Raynal. Even though I had the chance to get valuable feedback and review from them, I clearly made poor design decisions in LIEF and the implementation of the ELF builder is one of them. Basically, the implementation is recursive such as in the extreme cases the builder re-computes the same information several times. In the new implementation, we added a new stage in the build process that pre-computes the offsets of the new sections and the data that need to be relocated. This pre-computation enables to know exactly which parts of the ELF structures need to be relocated according to the user’s changes. This computation is managed by the Layout class which has two implementations depending on whether it is an ELF object or a library/executable. Compared to the previous ELF builder, this new implementation produces smaller files (with fewer ELF segments) as exposed in the following figure. This figure compares the number of segments between the former and the new implementation: In addition, it supports larger binaries faster as a consequence of the new linear implementation of the ELF builder :) To perform these benchmarks, we generated ELF binaries with the modifications described in the following script: 1import lief 2 3elf: lief.ELF.Binary = lief.parse(file_path.as_posix()) 4 5# Force relocating the .dynamic/.dynstr 6elf.add_library(\"a_very_long_name.so\") 7 8# For relocating the interpreter 9elf.interpreter = \"/a/very/longlonglong/interpreter-1.2.3.bin\" 10 11# Force relocating .dynsym / .gnu.hash table 12for i in range(10): 13 elf.add_exported_function(0xdeadc0de + i, f\"new_export_{i}\") 14 15# Add a segment 16segment = lief.ELF.Segment() 17segment.type = lief.ELF.SEGMENT_TYPES.LOAD 18segment.content = [0xcc] * 0x23 19 20elf.add(segment) 21 22elf.write(\"/tmp/bench.bin\") The raw results of the benchmark are also available here Final Words These new improvements introduce breaking changes in the ELF binaries generated by LIEF but: The final binary size should be smaller The building time should be much faster We tried to cover most of the cases in the tests suite but some corner cases with exotic compilers or linkers might break the final binaries. Since this improvement aims at being in the next release, feel free to drop an email or to open an issue if you find a bug with this new implementation. Since September I continue maintaining LIEF exclusively in my spare so issues and new features are addressed with more delay.","date":1642896e3,"description":"After spending months on refactoring the ELF builder, here are the improvements.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1642896e3,"markdown":"https://lief.re/blog/2022-01-23-new-elf-builder/index.md","objectID":"ace950ecf166c5acc57659728094c97a","permalink":"https://lief.re/blog/2022-01-23-new-elf-builder/","publishdate":"2022-01-23T00:00:00Z","relpermalink":"/blog/2022-01-23-new-elf-builder/","section":"blog","summary":"After spending months on refactoring the ELF builder, here are the improvements.","tags":["ELF","builder","internals","performance"],"title":"New ELF Builder","type":"blog","wordcount":1212},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Tl;DR This blog post is not, strictly speaking, related to LIEF but it aims at completing the previous blog about profiling code with Frida. In particular, it exposes the limits of our approach regarding the Microsoft/Itanium ABI. Long story short, the previous code does not work on Linux/OSX for virtual functions. The previous blog post tried to show a use case of Frida to profile C++ functions. In particular, it exposed what we called a trick to convert a C++ member function into a void*: 1template\u003ctypename Func\u003e 2inline void* cast_func(Func f) { 3 union { 4 Func func; 5 void* p; 6 }; 7 func = f; 8 return p; 9} First, and as noticed by Julien Jorge, writing a union’s field and accessing another field of this union is undefined behavior: It’s undefined behavior to read from the member of the union that wasn’t most recently written. Many compilers implement, as a non-standard language extension, the ability to read inactive members of a union. Thanks also to the feedback from Julien Jorge, there is another issue when converting a C++ member function into a raw pointer. Basically, a member function pointer is not the same kind of pointer as a regular C function. While the regular size of a C function pointer is the same as sizeof(void*), the size of a member function pointer is usually greater: 1struct Foo { 2 void bar() {} 3}; 4 5int main() { 6 printf(\"sizeof(\u0026Foo::bar): %d\\n\", sizeof(\u0026Foo::bar)); 7 return 0; 8} 1$ clang++ sizeof_member.cpp -o sizof_member 2$ ./sizeof_member 3sizeof(\u0026Foo::bar): 16 The layout of a member function pointer is ABI specific but according to LLVM’s source code we can distinguish two ABI that describe this layout: Itanium CXX ABI which is used on Linux, iOS, OSX, Android, … Microsoft Itanium ABI For the Itanium CXX ABI and according to the official documentation, non-virtual functions have the following structure: 1struct { 2 uintptr_t ptr; 3 ptrdiff_t adj; 4}; Where, ptr is the address of the function and adj is an offset applied on this in the case of multi-inheritance. So in our bad-coded casting function cast_func(), it works as expected for non-virtual functions since we access the first field ptr which is the function pointer. We can observe these two fields with the following piece of code 1: 1template\u003ctypename Func\u003e 2void print(Func f) { 3 union { 4 Func fcn; 5 struct { 6 uintptr_t ptr; 7 ptrdiff_t adj; 8 }; 9 }; 10 fcn = f; 11 printf(\"%016lx | %016lx\\n\", ptr, adj); 12} This outputs values such as: 1struct Foo { 2 void bar() {} 3}; 4 5int main() { 6 print(\u0026Foo::bar); 7 return 0; 8} $ ./show_fields 00005568e19021e0 | 0000000000000000 If bar() were a virtual function, the meaning of the ptr field would be different. Still according to the Itanium CXX ABI, the value of ptr in the case of a virtual function is 1 plus the offset of the function within the v-table. In particular, we can’t access the address of the function without this since the vtable is embedded in the layout of the object. 2 Microsoft ABI Regarding the Microsoft ABI, there is not as much documentation compared to the “Linux/OSX” ABI. LLVM supports this ABI as described in clang/lib/CodeGen/MicrosoftCXXABI.cpp but I was still curious to know how (without LLVM) the layout of a function member pointer looks like. One could look at c1xx.dll/c2.dll located in the Visual Studio directory but these libraries are not straightforward to reverse. Alternately, we can try to infer the layout from the assembly code output. First of all, the result of sizeof() applied to a function member pointer is 16. 16 being twice a pointer’s size on an 64-bits architecture, we can start following the Itanium ABI and confirm or infirm our choices: 1struct FuncMemPtr { 2 uintptr_t unknown1; 3 uintptr_t unknown2; 4}; Then we can unpack the fields of the function member pointer with the union trick: 1struct Base1 { 2 virtual void f() { } 3}; 4 5struct Base2 { 6 virtual void g() {} 7}; 8 9struct Derived2 : Base2, Base1 { 10 virtual void f() {} 11 virtual void g() {} 12 virtual h() {} 13}; 14 15template\u003ctypename Func\u003e 16void info(Func f) { 17 union { 18 Func fcn; 19 struct { 20 uintptr_t unknown1; 21 uintptr_t unknown2; 22 }; 23 }; 24 fcn = f; 25} 26 27int main() { 28 info(\u0026Derived2::h); 29 info(\u0026Derived2::f); 30 return 0; 31} The layout of the non-virtual function Derived2::h() seems to follow the same layout as the Itanium ABI where we find the function pointer in the first field. For the virtual function Derived2::f, we can notice a first memory write that fills the first field with a pointer to a thunk 3 function while the second field contains a constant which matches the value of this adjustor. For the second field (this adjustor), we can switch from \u0026Derived2::f to \u0026Derived2::g to confirm that it changes accordingly to the output of /d1reportAllClassLayout This leads to the following guessing: 1struct MsvcCXXFuncMember { 2 uintptr_t fnc_ptr; // That can be a thunk for virtual function 3 int adjustor; // int because of mov DWORD and not mov QWORD in this assembly output 4}; These two fields follow the LLVM implementation: 1struct { 2 // A pointer to the member function to call. If the member function is 3 // virtual, this will be a thunk that forwards to the appropriate vftable 4 // slot. 5 void *FunctionPointerOrVirtualThunk; 6 7 // An offset to add to the address of the vbtable pointer after 8 // (possibly) selecting the virtual base but before resolving and calling 9 // the function. 10 // Only needed if the class has any virtual bases or bases at a non-zero 11 // offset. 12 int NonVirtualBaseAdjustment; 13 14 // The offset of the vb-table pointer within the object. Only needed for 15 // incomplete types. 16 int VBPtrOffset; 17 18 // An offset within the vb-table that selects the virtual base containing 19 // the member. Loading from this offset produces a new offset that is 20 // added to the address of the vb-table pointer to produce the base. 21 int VirtualBaseAdjustmentOffset; 22}; From LLVM, we also learn that the full layout can contain up to four fields. We can trigger the third field with the following change: 1@@ -11,3 +11,3 @@ 2-struct Derived2 : Base2, Base1 { 3+struct Derived2 : Base2, virtual Base1 { 4 virtual void f() {} 5@@ -32 +32,2 @@ 6 } The fourth field is a bit more tricky to trigger and the following code comes from the LLVM test suite 4 1struct B1 { 2 void foo(); 3 int b; 4}; 5struct B2 { 6 int b2; 7 int v; 8 void foo(); 9}; 10 11struct UnspecWithVBPtr; 12int UnspecWithVBPtr::*forceUnspecWithVBPtr; 13struct UnspecWithVBPtr : B1, virtual B2 { 14 int u; 15 void foo(); 16}; We can notice that the result of sizeof() applied to UnspecWithVBPtr::foo is 24: sizeof(uintptr_t) + 3 * sizeof(int) + padding Conclusion The profiler described in the first blog post works as expected for non-virtual but does not work with virtual functions that follow the Itanium ABI. To work with virtual functions, we would need to pass an extra parameter to the object that implements the virtual functions. By assuming that the vtable is placed at the beginning of the object’s layout, we can support such functions with the following modifications: 1diff --git a/main.cpp b/main.cpp 2index d30a0c1..65d18eb 100644 3--- a/main.cpp 4+++ b/main.cpp 5 6+struct Foo { 7+ virtual void bar() { 8+ std::cout \u003c\u003c \"In bar\" \u003c\u003c std::endl; 9+ } 10+ uint8_t x = 1; 11+}; 12+ 13 14@@ -88,9 +95,10 @@ struct Profiler { 15 16- void setup() { 17- PROFILE(LIEF::ELF::Parser::init); 18- PROFILE(LIEF::ELF::Parser::parse_segments\u003cLIEF::ELF::ELF64\u003e); 19+ template\u003cclass T\u003e 20+ void setup(const T\u0026 obj) { 21+ const uintptr_t vtable = *reinterpret_cast\u003cconst uintptr_t*\u003e(\u0026obj); 22+ profile_func(\u0026Foo::bar, \"Foo:bar\", vtable); 23 } 24 25@@ -98,8 +106,13 @@ struct Profiler { 26 template\u003ctypename Func\u003e 27- void profile_func(Func func, std::string name) { 28+ void profile_func(Func func, std::string name, uintptr_t vtable = 0) { 29 void* addr = cast_func(func); 30+ 31+ if (vtable \u003e 0) { 32+ const uintptr_t voff = reinterpret_cast\u003cuintptr_t\u003e(addr) - 1; 33+ addr = *reinterpret_cast\u003cvoid**\u003e(vtable + voff); 34+ } 35 funcs[reinterpret_cast\u003cuintptr_t\u003e(addr)] = std::move(name); 36 gum_interceptor_begin_transaction (ctx_-\u003einterceptor); 37 gum_interceptor_attach (ctx_-\u003einterceptor, 38@@ -130,8 +143,9 @@ int main(int argc, const char** argv) { 39 return 1; 40 } 41 42+ Foo f; 43 Profiler\u0026 prof = Profiler::get(); 44- prof.setup(); 45- LIEF::ELF::Parser::parse(argv[1]); 46+ prof.setup(f); 47+ f.bar(); 48 return 0; 49 } The Microsoft C++ ABI is poorly documented but the LLVM project is a good reference for that. One might also be interested in this presentation (Bringing Clang and LLVM to Visual C++ users) that outlines the challenges for LLVM developers to support this ABI. Acknowledgment Thanks to Julien Jorge for proofreading this post and his valuable feedback. Which is still UB ↩︎ ARM is an exception. In the 32-bit ARM representation, the this adjustment stored in adj is left-shifted by one. The low bit of adj indicates whether ptr is a function pointer (including null) or the offset of a v-table entry. A virtual member function pointer sets ptr to the v-table entry offset as if by reinterpret_cast\u003cfnptr_t\u003e(uintfnptr_t(offset)). A null member function pointer sets ptr to a null function pointer and must ensure that the low bit of adj is clear; the upper bits of adj remain unspecified. ↩︎ A thunk function is generated by the compiler as a trampoline to the right virtual function. This trampoline can also be used to fix this pointer with the given adjustor. ↩︎ The layout of this code goes beyond my understanding ↩︎","date":161784e4,"description":"A practical follow-up on profiling C++ with Frida, including static-library hooks and the limitations introduced by different C++ ABIs.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":161784e4,"markdown":"https://lief.re/blog/2021-04-08-profiling-cpp-code-with-frida-part2/index.md","objectID":"a8347144c5c2855bdc158f2bb590c997","permalink":"https://lief.re/blog/2021-04-08-profiling-cpp-code-with-frida-part2/","publishdate":"2021-04-08T00:00:00Z","relpermalink":"/blog/2021-04-08-profiling-cpp-code-with-frida-part2/","section":"blog","summary":"A practical follow-up on profiling C++ with Frida, including static-library hooks and the limitations introduced by different C++ ABIs.","tags":["Frida","C++","profiling","ABI","Windows"],"title":"Profiling C++ code with Frida (2nd part)","type":"blog","wordcount":1603},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Frida is a well-known reverse engineering framework that enables (along with other functionalities) to hook functions on closed-source binaries. While hooking is generally used to get dynamic information about functions for which we don’t have the source code, this blog post introduces another use case to profile C/C++ code. Code Profiling LIEF starts to be quite mature but there are still some concerns regarding: The speed (especially when rebuilding large ELF binaries) The memory consumption Compilation time These limitations are “quite” acceptable on modern computers but when we target embedded systems like iPhone or Android devices, it starts to reach the limits. Since (spoiler) I started to implement a parser for the Dyld shared cache and for parsing in-memory Mach-O files, I faced some of these issues. To address these problems, we must identify where are the bottleneck and ideally, without modifying too much the source code. To profile memory consumption, valgrind --tool=massif does the job pretty well out of the box: we don’t need to pass extra compilation flags nor modifying the source code. Regarding the code execution, we can profile it with: Valgrind (or QBDI?) Inserting log functions in the source code Using compiler instrumentation: -finstrument-functions In the context of profiling LIEF, I’m mostly interested in profiling the code at the functions level: “How long does a function take to be executed?” Valgrind provides CPU cycles that are somehow correlated to the execution time but it requires an extra processing step to identify the function’s overhead. Moreover, since Valgrind instruments the code, it can take time to profile a large codebase. On the other hand, inserting log messages in the code is the easiest way to get the execution time of functions. I was not completely convinced with this solution since it adds log messages that are not always needed. Finally, Clang and GCC enable to instrument the source code through the -finstrument-functions compilation flag. This flag basically inserts the __cyg_profile_func_enter and __cyg_profile_func_exit functions at the beginning and at the end of the original functions. Frida works on compiled code and provides a mechanism (hook) to insert a callback before a given function and after the execution of the function. It is very similar to the -finstrument-functions, except that it is done post-compilation. To set up a hook, we only have to provide a pointer to the function that aims at being hooked. In the context of profiling execution time, the callback at the beginning of the function can initialize a std::chrono object and the callback at the end of the function can print the time spent since the initialization of the std::chrono. Let’s take a simple example to explain what Frida does. If we have the following function: 1void heavy_function() { 2 for (size_t i = 0; i \u003c 1000000; ++i) { 3 // Code that takes time ... 4 } 5} Frida enables (from a logical point of view) to have: 1void heavy_function() { 2 frida_on_enter(); 3 4 for (size_t i = 0; i \u003c 1000000; ++i) { 5 // Code that takes time ... 6 } 7 8 frida_on_leave(); 9} … without tweaking the compilation flags :) Frida Bootstrap Most documentation and blog posts about Frida use the JavaScript API. Frida also provides the frida-gum SDK 1, which exposes a C API over the hook engine. This SDK comes with the frida-gum-example.c file, which shows how to set up the hook engine. Regarding the API of our profiler, we would like to have : 1#include \u003cLIEF/ELF.hpp\u003e 2 3// Functions to profile 4profile(\u0026LIEF::ELF::Parser::parse_symbol_version); 5profile(\u0026LIEF::ELF::Parser::parse_segments\u003cLIEF::ELF::ELF64\u003e); 6 7LIEF::ELF::Parser::parse(\"./sample.bin\"); And an output like: 1$ ./run 2LIEF::ELF::Parser::parse_symbol_version() took 39ms 3LIEF::ELF::Parser::parse_segments() took 109ms I won’t go through all the details of the implementation of the profiler since the source code is on GitHub but the next section covers some tricky parts. Firstly, and as mentioned previous section, Frida takes a void* pointer on the function to hook. Therefore, we have to cast \u0026LIEF::ELF::Parser::parse_symbol_version into a void*. One might want to do reinterpret_cast\u003cvoid*\u003e() on the function pointer but it does not work. The trick here is to use a union to get the void*: 1template\u003ctypename Func\u003e 2inline void* cast_func(Func f) { 3 union { 4 Func func; 5 void* p; 6 }; 7 func = f; 8 return p; 9} Secondly, the example frida-gum-example.c uses an enum to identify the function being hooked: 1typedef enum _ExampleHookId ExampleHookId; 2enum _ExampleHookId 3{ 4 EXAMPLE_HOOK_OPEN, 5 EXAMPLE_HOOK_CLOSE 6}; 7... 8gum_interceptor_attach (interceptor, 9 GSIZE_TO_POINTER (gum_module_find_export_by_name (NULL, \"open\")), 10 listener, 11 GSIZE_TO_POINTER (EXAMPLE_HOOK_OPEN)); In our case, we don’t know beforehand which functions will be hooked or profiled by the user. Consequently, instead of using an enum we use the function’s absolute address and we register its name in a map: 1template\u003ctypename Func\u003e 2void profile_func(Func func, std::string name) { 3 void* addr = cast_func(func); 4 funcs[reinterpret_cast\u003cuintptr_t\u003e(addr)] = std::move(name); 5 gum_interceptor_begin_transaction(ctx_-\u003einterceptor); 6 gum_interceptor_attach(ctx_-\u003einterceptor, 7 /* Target */ reinterpret_cast\u003cgpointer\u003e(addr), 8 /* Param */ reinterpret_cast\u003cGumInvocationListener*\u003e(ctx_), 9 /* id */ reinterpret_cast\u003cgpointer\u003e(addr)); 10 gum_interceptor_end_transaction(ctx_-\u003einterceptor); 11} Last but not least, we might want to profile private or protected functions. To enable the access to the Profiler to protected/private members we can friend an opaque Profile structure: 1struct Profiler; 2 3namespace LIEF { 4class LIEF_API Parser : public LIEF::Parser { 5 public: 6 friend struct ::Profiler; 7 ... 8}; 9} Conclusion Through this blog post, we have shown that Frida also has some applications in the field of software engineering not only for reverse-engineering :) This approach can be quite convenient to isolate the profiling process from the compilation process. It also enables to quickly switch from a given SDK version to another as long as the profiled functions still exist. 1$ clang++ [-other-flags] LIEF-0.9.0/lib/libLIEF.a profile.cpp 2$ clang++ [-other-flags] LIEF-0.12.0/lib/libLIEF.a profile.cpp The source code used in this blog post is available on GitHub: lief-project/frida-profiler See frida-gum-devkit-14.2.13-linux-x86_64.tar.xz on https://github.com/frida/frida/releases ↩︎","date":1615334400,"description":"Profile C/C++ code with Frida: hook functions through the frida-gum C API and use LIEF to enumerate the functions to instrument, without recompiling the target.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1615334400,"markdown":"https://lief.re/blog/2021-03-10-profiling-cpp-code-with-frida/index.md","objectID":"fa80b9333467808feea8afb386bea4d5","permalink":"https://lief.re/blog/2021-03-10-profiling-cpp-code-with-frida/","publishdate":"2021-03-10T00:00:00Z","relpermalink":"/blog/2021-03-10-profiling-cpp-code-with-frida/","section":"blog","summary":"Frida is usually used to hook closed-source binaries; this post turns it into a C/C++ profiler by bootstrapping the frida-gum C API and using LIEF to enumerate the functions to instrument, without recompiling the target.","tags":["Frida","C++","profiling"],"title":"Profiling C++ code with Frida","type":"blog","wordcount":961},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Tl;DR LIEF v0.11.1 fixes some issues related to PE Authentihash computation. The new packages are available on PyPI and the SDKs can be downloaded on the official website. Enjoy! LIEF 0.11.0 missed handling some cases in the processing of the PE Authentihash. This new release addresses these issues and the following blog post explains the cases we did not handle. Section name PE section’s names are stored in a fixed char array (8 bytes) which means that a section’s name can contain trailing bytes after the null char: 1struct pe_section { 2 char name[8]; 3 uint32_t RVA; 4 // ... 5}; Before v0.11.1, LIEF didn’t take into account the trailing bytes and stopped to read the section’s name on the first null char: 1this-\u003ename_ = std::string(header-\u003ename, sizeof(header-\u003ename)).c_str(); This implementation has two drawbacks. First, we lose information since we don’t store the extra trailing bytes. Regular binaries have zero trailing bytes after the first null char but some of them might use this spot to hide data. Secondly, the full section name (i.e the whole 8 bytes) is used to compute the Authentihash. Therefore, if the first null char is followed by trailing bytes different from zero, the computed hash is inconsistent. Data directory According to the PE specifications 1 the last entry of the data directory table must contain a null entry (i.e. an entry with an RVA and size set to 0). It turns out that this requirement is not enforced by the loader. In the case of the binary (bc203f2b6a…) the last entry is set to 0x02b7bc68/0x01a7a0 (used for watermarking?). In the previous versions of LIEF we assumed that the last entry of the data directory table was always zero. Since the last entry is used to compute the Authentihash value, it led to a bad signature while it was effectively correct. This issue has been addressed in the commit 3c65ffe Return value of verify_signature() As noticed by Cedric Halbronn in the issue issues/532, the return value of LIEF::PE::Binary::verify_signature lacks information when the verification failed. The return value was either VERIFICATION_FLAGS.OK or VERIFICATION_FLAGS.BAD_SIGNATURE because of a fail-fast implementation of the verification flag. The function now returns flags as follows: VERIFICATION_FLAGS.BAD_DIGEST | VERIFICATION_FLAGS.BAD_SIGNATURE | VERIFICATION_FLAGS.CERT_EXPIRED Other issues One of the critical issues raised by imidoriya and fixed in the new version is the processing of the overlay data when the “data directory signature” is located in this area (c.f. 463bb0ec3…). This kind of layout triggers a memory error on this part of the processing Binary.cpp#L1174-L1187. It has been addressed in the commit 05103f5 Acknowledgment Thanks to Andrew Williams for providing the different samples that raised some of these errors! Thank you also to Cedric Halbronn and the CERT Gouvernemental of Luxembourg for their feedback about the API. https://docs.microsoft.com/en-us/windows/win32/debug/pe-format#optional-header-data-directories-image-only ↩︎","date":1613952e3,"description":"LIEF 0.11.1 fixes PE Authentihash computation: section name handling, data directory coverage, and the return value of verify_signature().","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1613952e3,"markdown":"https://lief.re/blog/2021-02-22-lief-0-11-1/index.md","objectID":"75766af7e151ed7790e370e93d7aed06","permalink":"https://lief.re/blog/2021-02-22-lief-0-11-1/","publishdate":"2021-02-22T00:00:00Z","relpermalink":"/blog/2021-02-22-lief-0-11-1/","section":"blog","summary":"A bug-fix release for PE Authenticode support: LIEF 0.11.1 corrects the Authentihash computation for section names and data directories and fixes the value returned by verify_signature().","tags":["release","PE","Authenticode"],"title":"LIEF - Release 0.11.1","type":"blog","wordcount":460},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Tl;DR LIEF v0.11.0 is out. The main changelog is available here and packages can be downloaded on the official website. Installation As for the previous versions, release packages are available on the GitHub release page and Python packages can be installed from PyPI: 1$ pip install [--user] lief==0.11.0 Release Highlight It has spent more than one year since the release of the version 0.10.1 but we are glad to announce that LIEF v0.11.0 is finally out! This new version does not introduce a lot of new features but rather small improvements in the different formats. One of the main changes in terms of new functionalities is the refactoring of the PE Authenticode. We fixed parsing issues and we implemented verification functions so that we can now verify a PE signed binary through: 1import lief 2pe = lief.parse(\"signed.exe\") 3assert pe.verify_signature() == lief.PE.Signature.VERIFICATION_FLAGS.OK We also improved the computation of imphash so that it can generate the same value as pefile (and therefore, Virus Total) 1pe = lief.parse(\"example.exe\") 2vt_imphash = lief.PE.get_imphash(pe, lief.PE.IMPHASH_MODE.PEFILE) 3lief_imphash = lief.PE.get_imphash(pe, lief.PE.IMPHASH_MODE.DEFAULT) Regarding the contributions, Janusz Lisiecki fixed a performance issue in the ELF builder that moved from N² computations to N log(N). His contribution raised a major weakness in LIEF: performances issue when re-building objects. We started to refactor the whole ELF builder to avoid recursive calls. Adrien Guinet updated the bin2lib tutorial to support recent versions of glibc, which introduced the DF_1_PIE flag. kohnakagawa and Clcanny also fixed various issues related to the ELF \u0026 PE formats. Ninja on Windows \u0026 CI We improved AppVeyor Windows CI to be more efficient on the compiler cache. It results in a decrease of 1-hour compilation time to ~20 minutes thanks to sccache and Ninja. If Ninja is installed on Windows, one can now use the --ninja flag when calling setup.py: 1$ python.exe .\\setup.py --ninja build install [--user] Using Ninja on Windows requires to invoke the vcvarsall.bat script beforehand. This script can be tricky to locate depending on the MSVC versions. Thankfully, setuptools provides the msvc.msvc14_get_vc_env() helper to get the environment variables that need to populate the calling script. We use it in LIEF’s setup.py as follows: 1... 2env = os.environ 3if platform.system() == \"Windows\": 4 from setuptools import msvc 5 if build_with_ninja: 6 arch = 'x64' if is64 else 'x86' 7 ninja_env = msvc.msvc14_get_vc_env(arch) 8 env.update(ninja_env) 9 else: 10 ... 11... Regarding the CI, we added Android and iOS SDK packages as well as Python wheels for Linux AArch64 (manylinux2014 compliant). The nightly builds are available on the gh-pages branch of the repository lief-project/packages: The sdk directory contains a shared and a static version of LIEF library for iOS, macOS, Android, Windows, Linux, … The lief directory contains the Python wheels for the supported platforms What’s next We have a few ideas of what would like to improve and introduce in the next releases of LIEF which includes: Refactoring the ELF builder to address performances issues (see also #482) Supporting OAT/VDEX/CDEX for Android 9, 10 and 11 Supporting Mach-O signature (as for PE Authenticode) Supporting Android packed relocations (in the parser and in the builder) Improving the C API to ease Rust bindings Supporting DART snapshot formats to ease reverse-engineering of Flutter applications. Spoiler: we can process all the clusters of a snapshot for a fixed version of the DART runtime. += Fixing issues Although LIEF’s plans mostly follow Quarkslab’s needs, the R\u0026D time we have, and the topics we enjoy working on, we are open to the development of private or public features as it has been done for improving PE Authenticode. Acknowledgment Thank you to CERT Gouvernemental of Luxembourg that sponsored new functionalities in this release. Thanks also to Quarkslab for the time allocated to make this release.","date":1611014400,"description":"LIEF 0.11.0: refactored PE Authenticode parsing with signature verification, pefile-compatible imphash, a faster ELF builder, and Ninja-based Windows CI.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1611014400,"markdown":"https://lief.re/blog/2021-01-19-lief-0-11-0/index.md","objectID":"5ccc6e7fc14008e43e5ad1fd84aa8b2d","permalink":"https://lief.re/blog/2021-01-19-lief-0-11-0/","publishdate":"2021-01-19T00:00:00Z","relpermalink":"/blog/2021-01-19-lief-0-11-0/","section":"blog","summary":"LIEF 0.11.0 refactors PE Authenticode parsing and adds verify_signature(), computes imphash the same way as pefile and VirusTotal, speeds up the ELF builder from quadratic to N log N, and moves the Windows CI to Ninja.","tags":["release","PE","Authenticode","ELF","Windows"],"title":"LIEF - Release 0.11.0","type":"blog","wordcount":621},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Installation Release packages are available on the GitHub page and Python package can be installed with: 1$ pip install [--user] lief==0.9.0 Release highlight Android Formats This new version of LIEF comes with support for Android formats related to the ART runtime: OAT, VDEX, DEX and ART. As the OAT format is a derivation of ELF, it made sense to add it in LIEF. Basically, this format is used by Android to wrap native code being the result of Dalvik bytecode optimization. Regarding VDEX, DEX, and ART, these formats have somehow a relation with OAT and therefore we also choose to add them. For more information about these Android formats and how to use them, a tutorial is available in the LIEF documentation: Android Formats. We can currently only parse these formats, but support for modification will be added incrementally. Some attacks rely on modifying the OAT format, as Collin Mulliner explains in “Inside Android’s SafetyNetAttestation: Attack and Defense” 1. Tencent’s Xuanwu Lab also discusses them in “How Samsung Secures Your Wallet \u0026 How To Break It” 2. In a future version, we plan to provide an API for adding native code to OAT. JSON serialization As one purpose of this project is to provide an API that can be easily integrated in other projects, we are glad to announce that JSON serialization is now available for all LIEF objects. It means that one can now access to format information through a JSON interface. Previous versions had a JSON support for ELF and PE formats, the v0.9 now supports all formats and all objects. Objects can be serialized with the lief.to_json function: 1import lief 2 3gcc = lief.parse(\"/usr/bin/gcc\") 4lief.to_json(gcc.header) 5 6{ 7 'entrypoint': 4209824, 8 'file_type': 'EXECUTABLE', 9 'header_size': 64, 10 'identity_class': 'CLASS64', 11 'identity_data': 'LSB' 12} 13 14libSystem = lief.parse(\"/usr/lib/libSystem.dylib\") 15lief.to_json(libSystem.commands[1]) 16 17{ 18 'command': 'SEGMENT', 19 'command_offset': 492, 20 'command_size': 464, 21 'content_hash': 18446744072658165641, 22 'data_hash': 1841536728, 23 'file_offset': 8192, 24 'file_size': 4096, 25 'flags': 0, 26 'init_protection': 3, 27 'max_protection': 7, 28 'name': '__DATA', 29 'numberof_sections': 6, 30 'sections': ['__nl_symbol_ptr', 31 '__la_symbol_ptr', 32 '__mod_init_func', 33 '__const', 34 '__data', 35 '__common'], 36 'virtual_address': 8192, 37 'virtual_size': 4096 38} One can also disable the JSON module using a CMake configuration flag: 1$ cmake -DLIEF_ENABLE_JSON=off ... What’s next LIEF v0.9 still has a poor support for Mach-O modification and only supports modifications on header and some Load commands. One of the primitives to do more general modification on Mach-O format is the ability to add arbitrary Load commands. Some tools 3 4 exist to add commands, but they usually use padding between the load command table and the raw content or they remove / replace existing one. The main limitation with this technique is that the number of load command which can be added depends on the size of the padding. In LIEF, we took advantage of the fact that Mach-O are PIE to shift the content that follow the load command table. This enable us to inject more than one or two commands. To keep a consistent state of format (relocations, segment’s virtual address, …), the Mach-O builder of LIEF rebuilds the export-trie, regenerates binding opcode, rebase opcodes, … In our tests, we succeeded in adding arbitrary number of LC_DYLIB command in clang as well as adding 10 new sections in the __TEXT segment. We are currently working on stabilization of the instrumentation process, but it should be merged soon in then master branch. Stay tuned! We will be also be presenting about file formats instrumentation at Recon Montréal and Pass The Salt for a talk about file formats instrumentation. In this talk we will present techniques to perform code injection, hooking by using formats. Slide 58 of Inside SafetyNet Attestation Attacks and Defense. ↩︎ Slide 89 of How Samsung Secures Your Wallet And How To Break It. ↩︎ insert_dylib. ↩︎ optool ↩︎","date":1528675200,"description":"Major changes in LIEF 0.9, plus work-in-progress features planned for future releases.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1528675200,"markdown":"https://lief.re/blog/2018-06-11-lief-0-9-0/index.md","objectID":"6af3cb0a31132349390362d7e24f4601","permalink":"https://lief.re/blog/2018-06-11-lief-0-9-0/","publishdate":"2018-06-11T00:00:00Z","relpermalink":"/blog/2018-06-11-lief-0-9-0/","section":"blog","summary":"Major changes in LIEF 0.9, plus work-in-progress features planned for future releases.","tags":["release","Android","JSON"],"title":"LIEF - Release 0.9.0","type":"featured","wordcount":643},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Tl;DR LIEF v0.8.3 is out. The main changelog is available here and packages can be downloaded on the official website. Development process We attach a great importance to the automation of some development tasks like testing, distributing, packaging, etc. Here is a summary of these processes: Each commits is tested on Linux - x86-64 - Python{2.7, 3.5, 3.6} Windows - x86 / x86-64 - Python{2.7, 3.5, 3.6} OSX - x86-64 - Python{2.7, 3.5, 3.6} The test suite includes: Tests on the Python API Tests on the C API Tests on the parsers Tests on the builders If tests succeeds packages are automatically uploaded on the https://github.com/lief-project/packages repository. For tagged version, packages are uploaded on the GitHub release page: https://github.com/lief-project/LIEF/releases. Dockerlief To facilitate the compilation and the use of LIEF, we created the Dockerlief repo which includes various Dockerfiles as well as the dockerlief utility. dockerlief is basically a wrapper on docker build . Among Dockerfiles, we provide a Dockerfile to cross compile LIEF for Android (ARM, AARCH64, x86, x86-64) To cross compile LIEF for Android ARM, one can run: 1$ dockerlief build --api-level 21 --arm lief-android 2 3[INFO] - Location of the Dockerfiles: ~/dockerfiles 4[INFO] - Building Dockerfile: 'lief-android' 5[INFO] - Target architecture: armeabi-v7a 6[INFO] - Target API Level: 21 The SDK package LIEF-0.8.3-Android_API21_armeabi-v7a.tar.gz is automatically pulled from the Docker to the current directory. Integration of libFuzzer Fuzzing our own library is a good way to detect bugs, memory leak, unsanitized inputs … Thus, we integrated libFuzzer in the project. Fuzzing the LIEF ELF, PE, Mach-O parser is as simple as: 1#include \u003cLIEF/LIEF.hpp\u003e 2#include \u003cvector\u003e 3#include \u003cmemory\u003e 4 5extern \"C\" int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { 6 std::vector\u003cuint8_t\u003e raw = {data, data + size}; 7 try { 8 std::unique_ptr\u003cLIEF::Binary\u003e b{LIEF::Parser::parse(raw)}; 9 } catch (const LIEF::exception\u0026 e) { 10 std::cout \u003c\u003c e.what() \u003c\u003c std::endl; 11 } 12 return 0; 13} To launch the fuzzer, one can run the following commands: 1$ make fuzz-elf # Launch ELF Fuzzer 2$ make fuzz-pe # Launch PE Fuzzer 3$ make fuzz-macho # Launch MachO Fuzzer 4$ make fuzz # Launch ELF, PE and MachO Fuzzer ELF Play with ELF symbols - Part 2 In the tutorial #03 we demonstrated how to swap dynamic symbols between a binary and a library. In this part, we will see how we can rename these symbols. Changing symbol names is not a trivial modification, since modifying the string table of the PT_DYNAMIC segment has side effects: It requires to update the hash table (GNU Hash / SYSV). It usually requires to extend the DYNAMIC part of the ELF format. The previous version of LIEF already implements the rebuilding of the hash table but not the extending of the DYNAMIC part. With the v0.8.3 we can extend the DYNAMIC part. Therefore: We can add new entries in the .dynamic section We can change dynamic symbols names We can change DT_RUNPATH and DT_RPATH without length restriction We will rename all imported functions of gpg that are imported from libgcrypt.so.20 into a_very_long_name_of_function_XX and all exported functions of libgcrypt.so.20 into the same name (XX is the symbol index). 1 1import lief 2 3# Load targets 4gpg = lief.parse(\"/usr/bin/gpg\") 5libgcrypt = lief.parse(\"/usr/lib/libgcrypt.so.20\") 6 7# Change names 8for idx, lsym in enumerate(filter(lambda e : e.exported, libgcrypt.dynamic_symbols)): 9 new_name = 'a_very_long_name_of_function_{:d}'.format(idx) 10 print(\"New name for '{}': {}\".format(lsym.name, new_name)) 11 for bsym in filter(lambda e : e.name == lsym.name, gpg.dynamic_symbols): 12 bsym.name = new_name 13 lsym.name = new_name 14 15# Write back 16binary.write(gpg.name) 17libgcrypt.write(libgcrypt.name) By using readelf we can check that function names have been modified: 1$ readelf -s ./gpg|grep \"a_very_long_name\" 2 3 2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND a_very_long_name_of_funct@GCRYPT_1.6 (2) 4 3: 0000000000000000 0 FUNC GLOBAL DEFAULT UND a_very_long_name_of_funct@GCRYPT_1.6 (2) 5 11: 0000000000000000 0 FUNC GLOBAL DEFAULT UND a_very_long_name_of_funct@GCRYPT_1.6 (2) 6 13: 0000000000000000 0 FUNC GLOBAL DEFAULT UND a_very_long_name_of_funct@GCRYPT_1.6 (2) 7 ... 8 9$ readelf -s ./libgcrypt.so.20|grep \"a_very_long_name\" 10 11 88: 000000000000d050 6 FUNC GLOBAL DEFAULT 10 a_very_long_name_of_funct@@GCRYPT_1.6 12 89: 000000000000dcd0 69 FUNC GLOBAL DEFAULT 10 a_very_long_name_of_funct@@GCRYPT_1.6 13 90: 000000000000d310 34 FUNC GLOBAL DEFAULT 10 a_very_long_name_of_funct@@GCRYPT_1.6 14 91: 000000000000de70 81 FUNC GLOBAL DEFAULT 10 a_very_long_name_of_funct@@GCRYPT_1.6 15 ... Now if we run the new gpg binary, we get the following error: 1$ ./gpg --output bar.txt --symmetric ./foo.txt 2relocation error: ./gpg: symbol a_very_long_name_of_function_8, version GCRYPT_1.6 not defined in file libgcrypt.so.20 with link time reference Because the Linux loader tries to resolve the function a_very_long_name_of_function_8 against /usr/lib/libgcrypt.so.20 and that library doesn’t include the updated names we get the error. One way to fix this error is to set the environment variable LD_LIBRARY_PATH to the current directory: 1$ LD_LIBRARY_PATH=. ./gpg --output bar.txt --symmetric ./foo.txt 2$ xxd ./bar.txt|head -n1 3 400000000: 8c0d 0407 0302 c5af 9fba cab1 9545 ebd2 .............E.. 5 6$ LD_LIBRARY_PATH=. ./gpg --output foo_decrypted.txt --decrypt ./bar.txt 7$ xxd ./foo_decrypted.txt|head -n1 8 900000000: 4865 6c6c 6f20 576f 726c 640a Hello World. Another way to fix it is to add a new entry in .dynamic section. As mentioned at the beginning, we can now add new entries in the .dynamic so let’s add a DT_RUNPATH entry with the $ORIGIN value so that the Linux loader resolves the modified libgcrypt.so.20 instead of the system one: 1... 2# Add a DT_RUNPATH entry 3gpg += lief.ELF.DynamicEntryRunPath(\"$ORIGIN\") 4 5# Write back 6binary.write(gpg.name) 7libgcrypt.write(libgcrypt.name) And we don’t need the LD_LIBRARY_PATH anymore: 1$ readelf -d ./gpg|grep RUNPATH 2 30x000000000000001d (RUNPATH) Library runpath: [$ORIGIN] 4 5$ ./gpg --decrypt ./bar.txt 6 7gpg: AES encrypted data 8gpg: encrypted with 1 passphrase 9Hello World Hiding its symbols While IDA v7.0 has been released recently, among the changelog one can notice two changes: ELF: describe symbols using symtab from DYNAMIC section ELF: IDA now uses the PHT by default instead of the SHT to load segments from ELF files These changes are partially true. Let’s see what go wrong in IDA with the following snippet: 1id = lief.parse(\"/usr/bin/id\") 2dynsym = id.get_section(\".dynsym\") 3dynsym.entry_size = dynsym.size // 2 4id.write(\"id_test\") This snippet defines the size of one symbol as the entire size of .dynsym section divided by 2. The normal size of ELF symbols would be: 1\u003e\u003e\u003e print(int(lief.ELF.ELF32.SIZES.SYM)) # For 32-bits 216 3\u003e\u003e\u003e print(int(lief.ELF.ELF64.SIZES.SYM)) # For 64-bits 424 In the case of the 64-bits id binary, we set this size to 924. When opening id_test in IDA and forcing to use Segment for parsing and not Sections we get the following imports : Only one import is resolved and the others are **hidden**. Note that id_test is still executable: 1$ id_test 2uid=1000(romain) gid=1000(romain) ... By using readelf we can still retrieve the symbols and we have an error indicating that symbol size is corrupted. 1$ readelf -s id_test 2readelf: Error: Section 5 has invalid sh_entsize of 000000000000039c 3readelf: Error: (Using the expected size of 24 for the rest of this dump) 4 5Symbol table '.dynsym' contains 77 entries: 6 Num: Value Size Type Bind Vis Ndx Name 7 0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND 8 1: 0000000000000000 0 FUNC GLOBAL DEFAULT UND endgrent@GLIBC_2.2.5 (2) 9 2: 0000000000000000 0 FUNC GLOBAL DEFAULT UND __uflow@GLIBC_2.2.5 (2) 10 3: 0000000000000000 0 FUNC GLOBAL DEFAULT UND getenv@GLIBC_2.2.5 (2) 11 4: 0000000000000000 0 FUNC GLOBAL DEFAULT UND free@GLIBC_2.2.5 (2) 12 5: 0000000000000000 0 FUNC GLOBAL DEFAULT UND abort@GLIBC_2.2.5 (2) 13 ... In LIEF the (dynamic) symbol table address is computed through the DT_SYMTAB from the PT_DYNAMIC segment. To compute the number of dynamic symbols LIEF uses three heuristics: Based on hash tables (Gnu Hash / SYSV Hash) Based on relocations Based on sections Malwares start to use this kind of corruption as we will see in the next part. Rootnik Malware Rootnik is a malware targeting Android devices. It has been analyzed by Fortinet security researcher. A full analysis of the malware is available on the Fortinet blog. This part is focused on the ELF format analysis of one component: libshell. Actually there are two libraries libshella_2.10.3.1.so and libshellx_2.10.3.1.so. As they have the same purpose, we will use the x86 version. First if we look at the ELF sections of libshellx_2.10.3.1.so we can notice that the address, offset, and size of some sections like .text, .init_array, .dynstr, .dynsym are set to 0. This kind of modification is used to disturb tools that rely on sections to parse some ELF structures (like objdump, readelf, IDA …) 1$ readelf -S ./libshellx-2.10.3.1.so 2There are 21 section headers, starting at offset 0x2431c: 3 4Section Headers: 5 [Nr] Name Type Addr Off Size ES Flg Lk Inf Al 6 [ 0] NULL 00000000 000000 000000 00 0 0 0 7 [ 1] .dynsym DYNSYM 00000114 000114 000300 10 A 2 1 4 8 [ 2] .dynstr STRTAB 00000414 000414 0001e2 00 A 0 0 1 9 [ 3] .hash HASH 00000000 000000 000000 04 A 1 0 4 10 [ 4] .rel.dyn REL 00000000 000000 000000 08 A 1 0 4 11 [ 5] .rel.plt REL 00000000 000000 000000 08 AI 1 6 4 12 [ 6] .plt PROGBITS 00000000 000000 000000 04 AX 0 0 16 13 [ 7] .text PROGBITS 00000000 000000 000000 00 AX 0 0 16 14 [ 8] .code PROGBITS 00000000 000000 000000 00 AX 0 0 16 15 [ 9] .eh_frame PROGBITS 00000000 000000 000000 00 A 0 0 4 16 [10] .eh_frame_hdr PROGBITS 00000000 000000 000000 00 A 0 0 4 17 [11] .fini_array FINI_ARRAY 00000000 000000 000000 00 WA 0 0 4 18 [12] .init_array INIT_ARRAY 00000000 000000 000000 00 WA 0 0 4 19 [13] .dynamic DYNAMIC 0000ce50 00be50 0000f8 08 WA 2 0 4 20 [14] .got PROGBITS 00000000 000000 000000 00 WA 0 0 4 21 [15] .got.plt PROGBITS 00000000 000000 000000 00 WA 0 0 4 22 [16] .data PROGBITS 00000000 000000 000000 00 WA 0 0 16 23 [17] .bss NOBITS 0000d398 00c395 000000 00 WA 0 0 4 24 [18] .comment PROGBITS 00000000 00c395 000045 01 MS 0 0 1 25 [19] .note.gnu.gold-ve NOTE 00000000 00c3dc 00001c 00 0 0 4 26 [20] .shstrtab STRTAB 00000000 024268 0000b1 00 0 0 1 27Key to Flags: 28 W (write), A (alloc), X (execute), M (merge), S (strings), I (info), 29 L (link order), O (extra OS processing required), G (group), T (TLS), 30 C (compressed), x (unknown), o (OS specific), E (exclude), 31 p (processor specific) If we open the given library in IDA we have no exports, no imports, and no sections: Based on the segments and dynamic entries we can recover most of these information: .init_array address and size are available through the DT_INIT_ARRAY and DT_INIT_ARRAYSZ entries .dynstr address and size are available through the DT_STRTAB and DT_STRSZ .dynsym address is available through the DT_SYMTAB The script recover_shellx.py recovers the missing values, patch sections and rebuild a fixed library. Now if we open the new libshellx-2.10.3.1_FIXED.so we have access to imports / exports and some sections. The .init_array section contains 2 functions: tencent652524168491435794009 sub_60C0 The tencent652524168491435794009 function basically do a stack alignment and the sub_60C0 is one of the decryption routines 2. This function is obfuscated with graph flattening and looks like to O-LLVM graph flattening passe 3: Fortunately, only a few “relevant blocks” remain, and they are not obfuscated. The function sub_60C0 basically iterates over the program headers to find the encrypted one and decrypt it using a custom algorithm (based on shift, xor, etc). Triggering CVE-2017-1000249 The CVE-2017-1000249 is a stack based buffer overflow in the file utility. It affects the versions 5.29, 5.30 and 5.31. Basically the overflow occurs in the size of the note description. Using LIEF we can trigger the overflow as follows: 1target = lief.parse(\"/usr/bin/id\") 2note_build_id = target[lief.ELF.NOTE_TYPES.BUILD_ID] 3note_build_id.description = [0x41] * 30 4target.write(\"id_overflow\") 1$ file --version 2file-5.29 3magic file from /usr/share/file/misc/magic 4 5$ id_overflow 6uid=1000(romain) gid=1000(romain) ... 7 8$ file id_overflow 9*** buffer overflow detected ***: file terminated 10./id_overflow: [1] 3418 abort (core dumped) file ./id_overflow Here is the commit that introduced the bug: 9611f3 PE The Load Configuration directory is now parsed into the LoadConfiguration object. This structure evolves with the Windows versions and LIEF has been designed to support this evolution. You can take a look at LoadConfigurationV0, LoadConfigurationV6. One can find the different versions of this structure in the following directories: include/LIEF/PE/LoadConfigurations src/PE/LoadConfigurations The current version of LIEF is able to parse the structure up to Windows 10 build 15002 with the hotpatch table offset. Here are some examples of the LoadConfiguration API: 1\u003e\u003e\u003e target = lief.parse(\"PE64_x86-64_binary_WinApp.exe\") 2\u003e\u003e\u003e target.has_configuration 3True 4\u003e\u003e\u003e config = target.load_configuration 5\u003e\u003e\u003e config.version 6WIN_VERSION.WIN10_0_15002 7\u003e\u003e\u003e hex(config.guard_rf_failure_routine) 8'0x140001040' LIEF also provides an API to serialize any ELF or PE objects into JSON 4 For examples to transform LoadConfiguration object into Json: 1\u003e\u003e\u003e from lief import to_json 2\u003e\u003e\u003e to_json(config) 3'{\"characteristics\":248,\"code_integrity\":{\"catalog\":0,\"catalog_offset\":0 ... }}' # Not fully printed One can also serialize the whole Binary object: 1\u003e\u003e\u003e to_json(target) 2'{\"data_directories\":[{\"RVA\":0,\"size\":0,\"type\":\"EXPORT_TABLE\"},{\"RVA\":62584,\"section\" ...}}' # # Not fully printed Mach-O For Mach-O binary, dynamic executables embed the LC_DYLD_INFO command which is associated with the dyld_info_command structure. The structure is basically a list of offsets and sizes pointing to other data structures. From /usr/lib/mach-o/loader.h the structure looks like this: 1struct dyld_info_command { 2 uint32_t cmd; 3 uint32_t cmdsize; 4 uint32_t rebase_off; 5 uint32_t rebase_size; 6 uint32_t bind_off; 7 uint32_t bind_size; 8 uint32_t weak_bind_off; 9 uint32_t weak_bind_size; 10 uint32_t lazy_bind_off; 11 uint32_t lazy_bind_size; 12 uint32_t export_off; 13 uint32_t export_size; 14}; The dyld loader uses this structure to: Rebase the executable Bind symbols to addresses Retrieve exported functions (or symbols) Whereas in the ELF and PE format relocations are basically a table, Mach-O format uses byte streams to rebase the image and to bind symbols with addresses. For exports it uses a trie as subjacent structure. In the new version of LIEF, the Mach-O parser is able to handle these underlying structures to provide a user-friendly API: The export trie is represented by the ExportInfo object which is usually tied to a Symbol. The binding byte stream is represented trough the BindingInfo object. For the rebase byte stream, the parser create virtual relocations to model the rebasing process. These virtual relocations are represented by the RelocationDyld object and among other attributes it contains address, size and type 5. Here is an example using the Python API: 1\u003e\u003e\u003e id = lief.parse(\"/usr/bin/id\") 2\u003e\u003e\u003e print(id.relocations[0]) 3100002000 POINTER 64 DYLDINFO __DATA.__eh_frame dyld_stub_binder 4\u003e\u003e\u003e print(id.has_dyld_info) 5True 6\u003e\u003e\u003e dyldinfo = id.dyld_info 7\u003e\u003e\u003e print(dyldinfo.bindings[0]) 8Class: STANDARD 9Type: POINTER 10Address: 0x100002010 11Symbol: ___stderrp 12Segment: __DATA 13Library: /usr/lib/libSystem.B.dylib 14\u003e\u003e\u003e print(dyldinfo.exports[0]) 15Node Offset: 18 16Flags: 0 17Address: 0 18Symbol: __mh_execute_header Conclusion In this release we did a large improvement of the ELF builder. Mach-O and PE parts gain new objects and new functions. LIEF is now available on PyPI and can be added in the requirements of Python projects whatever the Python version and the target platform. Since the v0.7.0 LIEF has been presented at RMLL and the MISP project uses it for its PyMISP objects. Some may complain about the C API. They are right! Until the v1.0.0 we will provide a minimal C API. Once C++ API is stable we plan to provide full APIs for Python, C, Java, OCaml 6, etc. Next version should be focused on the Mach-O builder especially for adding sections and segments. We also plan to support PE .NET headers and fix some performances issues. For questions you can join the Gitter channel All Python examples are done with the 3.5 version ↩︎ See the blog post about O-LLVM analysis: https://blog.quarkslab.com/deobfuscation-recovering-an-ollvm-protected-program.html ↩︎ As mentioned in the Fortinet blog post, the library is packed. ↩︎ This feature is not yet available for MachO objects ↩︎ Due to the inheritance relationship and abstraction these attributes are located in the MachO::Relocation and LIEF::Relocation objects. ↩︎ https://github.com/aziem/LIEF-ocaml ↩︎","date":1509321600,"description":"LIEF 0.8.3: libFuzzer integration, Dockerlief, ELF symbol renaming and hiding, DT_RUNPATH injection, PE load configuration, and Mach-O dyld info parsing.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1509321600,"markdown":"https://lief.re/blog/2017-10-30-lief-0-8-3/index.md","objectID":"5390db8429858f2c07b028a24992ab0d","permalink":"https://lief.re/blog/2017-10-30-lief-0-8-3/","publishdate":"2017-10-30T00:00:00Z","relpermalink":"/blog/2017-10-30-lief-0-8-3/","section":"blog","summary":"LIEF 0.8.3 adds libFuzzer-based fuzzing and Dockerlief builds, then walks through ELF symbol renaming and hiding, DT_RUNPATH injection, unpacking the Rootnik malware, triggering CVE-2017-1000249, PE load configuration parsing, and Mach-O dyld info structures.","tags":["release","ELF","PE","Mach-O","fuzzing","malware-analysis"],"title":"Have fun with LIEF and Executable Formats!","type":"blog","wordcount":2583},{"authors":{"avatar":"img/avatar/romain.png","display_name":"Romain Thomas","family_name":"Thomas","given_name":"Romain","profile":"https://www.romainthomas.fr"},"categories":[],"content":"Tl;DR LIEF is a library to parse and manipulate ELF, PE and Mach-O formats. Source code is available on GitHub and use cases are here. Executable File Formats in a Nutshell When dealing with executable files, the first layer of information is the format in which the code is wrapped. We can see an executable file format as an envelope. It contains information so that the postman (i.e. Operating System) can handle and deliver (i.e. execute) it. The message wrapped by this envelope would be the machine code. There are mainly three mainstream formats, one per OS: Portable Executable (PE) for Windows systems Executable and Linkable Format (ELF) for UN*X systems (Linux, Android…). Mach-O for OS-X, iOS… Other executable file formats, such as COFF, exist but they are less relevant. Usually each format has a header which describes at least the target architecture, the program’s entry point and the type of the wrapped object (executable, library…) Then we have blocks of data that will be mapped by the OS’s loader. These blocks of data could hold machine code (.text), read-only data (.rodata) or other OS specific information. For PE there is only one kind of such block: Section. For ELF and Mach-O formats, a section has a different meaning. In these formats, sections are used by the linker at the compilation step, whereas segments (second type of block) are used by the OS’s loader at execution step. Thus, sections are not mandatory for ELF and Mach-O formats and can be removed without affecting the execution. Purpose of LIEF It turns out that many projects need to parse executable file formats but don’t use a standard library and re-implement their own parser (and the wheel). Moreover, these parsers are usually bound to one language. On Unix system one can find the objdump and objcopy utilities but they are limited to Unix and the API is not user-friendly. The purpose of LIEF is to fill this void: Providing a cross-platform library which can parse and modify (to a certain extent) ELF, PE, and Mach-O formats using a common abstraction Providing an API for different languages (Python, C++, C, and others) Abstract common features from the different formats (Section, header, entry point, symbols…) The following snippets show how to obtain information about an executable using different API of LIEF: 1import lief 2# ELF 3binary = lief.parse(\"/usr/bin/ls\") 4print(binary) 5 6# PE 7binary = lief.parse(\"C:\\\\Windows\\\\explorer.exe\") 8print(binary) 9 10# Mach-O 11binary = lief.parse(\"/usr/bin/ls\") 12print(binary) With the C++ API: 1#include \u003cLIEF/LIEF.hpp\u003e 2int main(int argc, const char** argv) { 3 LIEF::ELF::Binary* elf = LIEF::ELF::Parser::parse(\"/usr/bin/ls\"); 4 LIEF::PE::Binary* pe = LIEF::PE::Parser::parse(\"C:\\\\Windows\\\\explorer.exe\"); 5 LIEF::MachO::Binary* macho = LIEF::MachO::Parser::parse(\"/usr/bin/ls\"); 6 7 std::cout \u003c\u003c *elf \u003c\u003c std::endl; 8 std::cout \u003c\u003c *pe \u003c\u003c std::endl; 9 std::cout \u003c\u003c *macho \u003c\u003c std::endl; 10 11 delete elf; 12 delete pe; 13 delete macho; 14} And finally with the C API: 1#include \u003cLIEF/LIEF.h\u003e 2int main(int argc, const char** argv) { 3 4 Elf_Binary_t* elf_binary = elf_parse(\"/usr/bin/ls\"); 5 Pe_Binary_t* pe_binary = pe_parse(\"C:\\\\Windows\\\\explorer.exe\"); 6 Macho_Binary_t** macho_binaries = macho_parse(\"/usr/bin/ls\"); 7 8 Pe_Section_t** pe_sections = pe_binary-\u003esections; 9 Elf_Section_t** elf_sections = elf_binary-\u003esections; 10 Macho_Section_t** macho_sections = macho_binaries[0]-\u003esections; 11 12 for (size_t i = 0; pe_sections[i] != NULL; ++i) { 13 printf(\"%s\\n\", pe_sections[i]-\u003ename) 14 } 15 16 for (size_t i = 0; elf_sections[i] != NULL; ++i) { 17 printf(\"%s\\n\", elf_sections[i]-\u003ename) 18 } 19 20 for (size_t i = 0; macho_sections[i] != NULL; ++i) { 21 printf(\"%s\\n\", macho_sections[i]-\u003ename) 22 } 23 24 elf_binary_destroy(elf_binary); 25 pe_binary_destroy(pe_binary); 26 macho_binaries_destroy(macho_binaries); 27} LIEF supports FAT-MachO and one can iterate over binaries as follows: 1import lief 2binaries = lief.MachO.parse(\"/usr/lib/libc++abi.dylib\") 3for binary in binaries: 4 print(binary) Note The above script uses the lief.MachO.parse function instead of the lief.parse function because lief.parse returns a single lief.MachO.binary object whereas lief.MachO.parse returns a list of lief.MachO.binary (according to the FAT-MachO format). Along with standard format components like headers, sections, import table, load commands, symbols, etc. LIEF is also able to parse PE Authenticode: 1import lief 2driver = lief.parse(\"driver.sys\") 3 4for crt in driver.signature.certificates: 5 print(crt) 1Version: 3 2Serial Number: 61:07:02:dc:00:00:00:00:00:0b 3Signature Algorithm: SHA1_WITH_RSA_ENCRYPTION 4Valid from: 2005-9-15 21:55:41 5Valid to: 2016-3-15 22:5:41 6Issuer: DC=com, DC=microsoft, CN=Microsoft Root Certificate Authority 7Subject: C=US, ST=Washington, L=Redmond, O=Microsoft Corporation, CN=Microsoft Windows Verification PCA 8... Full API documentation is available here Python API C++ API C API Architecture In the LIEF architecture, each format implements at least the following classes: Parser: Parse the format and decompose it into a Binary class Binary: Modelize the format and provide an API to modify and explore it. Builder: Transform the binary object into a valid file. To factor common characteristics in formats we have an inheritance relationship between these characteristics. For symbols it gives the following diagram: It enables to write cross-format utility like nm. nm is a Unix utility to list symbols in an executable. The source code is available here: binutils With the given inheritance relationship one can write this utility for the three formats in a single script: 1import lief 2import sys 3 4def nm(binary): 5 for symbol in binary.symbols: 6 print(symbol) 7 8 return 0 9 10if __name__ == \"__main__\": 11 r = nm(sys.argv[1]) 12 sys.exit(r) Conclusion As LIEF is still a young project we hope to have feedback, ideas, suggestions, and pull requests. The source code is available here: https://github.com/lief-project (under Apache 2.0 license) and the associated website: http://lief.quarkslab.com If you are interested in use cases, you can take a look at these tutorials: Parse and manipulate formats Create a PE from scratch Play with ELF symbols Hooking Infecting the PLT/GOT The project will be presented at the Third French Japanese Meeting on Cybersecurity Contact lief [at] quarkslab [dot] com Gitter: lief-project Thanks Thanks to Serge Guelton and Adrien Guinet for their advice about the design and their code review. Thanks to Quarkslab for making this project open-source.","date":1492473600,"description":"How LIEF was open-sourced to provide a cross-platform API for parsing, inspecting, and modifying ELF, PE, and Mach-O executable formats.","expirydate":-62135596800,"kind":"page","lang":"en","lastmod":1492473600,"markdown":"https://lief.re/blog/2017-04-18-lief/index.md","objectID":"b983072d4a5bc4f7b532cb32c8a7d99f","permalink":"https://lief.re/blog/2017-04-18-lief/","publishdate":"2017-04-18T00:00:00Z","relpermalink":"/blog/2017-04-18-lief/","section":"blog","summary":"How LIEF was open-sourced to provide a cross-platform API for parsing, inspecting, and modifying ELF, PE, and Mach-O executable formats.","tags":["ELF","PE","Mach-O","architecture"],"title":"LIEF - Library to Instrument Executable Formats","type":"featured","wordcount":957}]