When Dragons Misplace Elves: Fixing Ghidra’s Broken ELF Export

Ishmael42 12 min read (2511 words)

Ghidra’s built-in ELF exporter just dumps memory blocks end to end, breaking on overlapping segments and silent gap-riddled reads. Here’s how to actually build one that works.

meme

What should have been around a hour of tinkering ended up taking one whole day, thanks to pyghidra’s weird APIs

“But there’s an exporter built-into ghidra already”

Yes, it exists. Problem is, it doesn’t work. All it does, is it blindly writes to a file the list of current “memory blocks”. These memory blocks are more similar to ELF sections. This means:

  1. Nested segments (PT_PHDR, etc) are written twice.
  2. Various offsets (eg: e_phoff, e_shoff) are messed up.
  3. The kernel can’t find the interpreter coz the .interp section is messed up

The code if you need to verify this yourself.

Building the (functional) exporter

Assembling an ELF file from what ghidra has in memory is a relatively simple task. For now, we will only focus on making a “functional” file - ie an ELF with only the bare minimum (elf header + program headers + segments).

This means that we don’t care about exporting the section table, shstr, etc. We’ll only focus on the bytes covered by the program headers.

  1. Grab the elf header from the first mapped memory block.
  2. Also grab the list of program headers
  3. For each program header, grab the file contents from the p_vaddr and place it at the offsets specified in p_offset.
  4. Write the output bytes into a file

Only if the ELF format was so simple…

Nested segments

Consider these program headers:

TEXT 
Program Headers:
Type Offset   VirtAddr PhysAddr FileSiz  MemSiz   Flg Align
PHDR 0x000040 0x000040 0x000040 0x000380 0x000380 R   0x8
LOAD 0x000000 0x000000 0x000000 0x000910 0x000910 R   0x1000

The first segment starts at offset 0x40 and has size of 0x0380, and the second segment starts at offset 0x00 and has a size 0x0910. The PHDR segment is fully nested inside of the LOAD segment.

So that means placing the segments is not just appending them one after the another, which is exactly what the ghidra exporter does.

To mitigate this, I had to create an OffsetWriter which places bytes at an offset into a bytearray

PY 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

class OffsetWriter:
    def __init__(self) -> None:
        self.contents: bytearray = bytearray()
        self.written_segments: list[tuple[int, int]] = []
        pass

    def write_at(self, off: int, byts: bytes):
        # check if this is in an already written segment
        for seg in self.written_segments:
            start, stop = seg
            within_bounds = off >= start and off + len(byts) <= stop
            if within_bounds:
                return

        self.written_segments.append((off, off + len(byts)))

        # and then, write to that segment.
        new_cap = len(byts) + off
        if new_cap > len(self.contents):
            self.contents.resize(new_cap)

        self.contents[off:new_cap] = byts

Ghidra’s weird memory read API

Ghidra provides the following function in it’s flatapi:

PYTHON 
1

flatapi.getBytes(addr, size)

So, doing flatapi.getBytes(toAddr(0x1000), 0x200) should return a bytes object of length 0x200. But, it instead returns a bytes object with 23.

This seems like a bug in ghidra, until you start looking at the implementation.

JAVA 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

	public int getBytes(Address addr, byte[] dest, int dIndex, int size)
			throws MemoryAccessException {
		int numRead = 0;
		long lastRead = 0;
		while (numRead < size) {
			try {
				addr = addr.addNoWrap(lastRead);
				MemoryBlock block = getBlock(addr);
				if (block == null) {
					break;
				}
				if (block.isInitialized() || block.isMapped()) {
					lastRead = block.getBytes(addr, dest, numRead + dIndex, size - numRead);
				}
				else {
					break;
				}
				numRead += lastRead;
			}
			catch (AddressOverflowException e) {
				break;
			}
		}
		if (numRead == 0 && size > 0) {
			throw new MemoryAccessException("Unable to read bytes at " + addr.toString(true));
		}
		return numRead;
	}

It reads data across multiple blocks in a loop, until the required amount of bytes is read. But it makes an assumption that memory blocks are continuous. A quick smoke-test proves that the premise is false.

Broken Assumption

In order to work-around this limitation, I ended up writing my own getBytes() in python which filled the holes with 00

PY 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

def read_segment_bytes(flatapi: FlatProgramAPI, addr: Address, size: int):
    mem = flatapi.currentProgram.getMemory()
    end_addr = addr.add(size - 1)

    want = AddressSet(addr, end_addr)
    have = mem.getLoadedAndInitializedAddressSet()
    readable = want.intersect(have)

    out = bytearray(size)
    ranges = readable.getAddressRanges()
    for rng in ranges:
        chunk = flatapi.getBytes(rng.getMinAddress(), int(rng.getLength()))
        offset = rng.getMinAddress().subtract(addr)
        out[offset : offset + len(chunk)] = bytes(chunk)

    return bytes(out)

The solution

This is the final script, that reconstructs elf header, segments and segment table:

PY — ELFLIB.PY 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377

# Elf header parsing module, DO NOT USE on untrusted inputs. This assumes a perfect
# header, and if it's fucked up it still parses shit. So don't cry later if
# it doesn't work for you.

from dataclasses import dataclass
from operator import attrgetter
import struct
from typing import ClassVar, Self, final, override

EI_CLASS = 4  #  /* File class byte index */
ELFCLASSNONE = 0  # /* Invalid class */
ELFCLASS32 = 1  # /* 32-bit objects */
ELFCLASS64 = 2  # /* 64-bit objects */
ELFCLASSNUM = 3  #

EI_DATA = 5  # /* Data encoding byte index */
ELFDATANONE = 0  # /* Invalid data encoding */
ELFDATA2LSB = 1  # /* 2's complement, little endian */
ELFDATA2MSB = 2  # /* 2's complement, big endian */
ELFDATANUM = 3  #

EI_VERSION = 6  # /* File version byte index */

EI_OSABI = 7  # /* OS ABI identification */
ELFOSABI_NONE = 0  # /* UNIX System V ABI */
ELFOSABI_SYSV = 0  # /* Alias.  */
ELFOSABI_HPUX = 1  # /* HP-UX */
ELFOSABI_NETBSD = 2  # /* NetBSD.  */
ELFOSABI_GNU = 3  # /* Object uses GNU ELF extensions.  */
ELFOSABI_LINUX = ELFOSABI_GNU  # /* Compatibility alias.  */
ELFOSABI_SOLARIS = 6  # /* Sun Solaris.  */
ELFOSABI_AIX = 7  # /* IBM AIX.  */
ELFOSABI_IRIX = 8  # /* SGI Irix.  */
ELFOSABI_FREEBSD = 9  # /* FreeBSD.  */
ELFOSABI_TRU64 = 10  # /* Compaq TRU64 UNIX.  */
ELFOSABI_MODESTO = 11  # /* Novell Modesto.  */
ELFOSABI_OPENBSD = 12  # /* OpenBSD.  */
ELFOSABI_ARM_AEABI = 64  # /* ARM EABI */
ELFOSABI_ARM = 97  # /* ARM */
ELFOSABI_STANDALONE = 255  # /* Standalone (embedded) application */


@dataclass(slots=True)
class _ElfEhdrBase:
    e_ident: bytes
    e_type: int
    e_machine: int
    e_version: int
    e_entry: int
    e_phoff: int
    e_shoff: int
    e_flags: int
    e_ehsize: int
    e_phentsize: int
    e_phnum: int
    e_shentsize: int
    e_shnum: int
    e_shstrndx: int

    _struct_le: ClassVar[struct.Struct]
    _struct_be: ClassVar[struct.Struct]

    @classmethod
    def from_bytes(cls, byts: bytes) -> Self:
        s = cls._struct_le if byts[EI_DATA] == ELFDATA2LSB else cls._struct_be
        return cls(*s.unpack(byts[: s.size]))  # pyright: ignore[reportAny]

    @classmethod
    def get_size(cls):
        return cls._struct_le.size

    def to_bytes(self, little_endian: bool) -> bytes:
        s = self._struct_le if little_endian else self._struct_be
        return s.pack(
            self.e_ident,
            self.e_type,
            self.e_machine,
            self.e_version,
            self.e_entry,
            self.e_phoff,
            self.e_shoff,
            self.e_flags,
            self.e_ehsize,
            self.e_phentsize,
            self.e_phnum,
            self.e_shentsize,
            self.e_shnum,
            self.e_shstrndx,
        )


@final
@dataclass(slots=True)
class Elf32_Ehdr(_ElfEhdrBase):
    _struct_le = struct.Struct("<16sHHIIIIIHHHHHH")
    _struct_be = struct.Struct(">16sHHIIIIIHHHHHH")


@final
@dataclass(slots=True)
class Elf64_Ehdr(_ElfEhdrBase):
    _struct_le = struct.Struct("<16sHHIQQQIHHHHHH")
    _struct_be = struct.Struct(">16sHHIQQQIHHHHHH")


PT_TYPE_NAMES = {
    0: "NULL",
    1: "LOAD",
    2: "DYNAMIC",
    3: "INTERP",
    4: "NOTE",
    5: "SHLIB",
    6: "PHDR",
    7: "TLS",
    0x6474E550: "GNU_EH_FRAME",
    0x6474E551: "GNU_STACK",
    0x6474E552: "GNU_RELRO",
    0x6474E553: "GNU_PROPERTY",
    0x6FFFFFFA: "SUNW_UNWIND",  # rare, but harmless to include
}


def phdr_type_name(p_type: int) -> str:
    name = PT_TYPE_NAMES.get(p_type)
    if name is not None:
        return name
    if 0x60000000 <= p_type <= 0x6FFFFFFF:
        return f"LOOS+0x{p_type - 0x60000000:x}"
    if 0x70000000 <= p_type <= 0x7FFFFFFF:
        return f"LOPROC+0x{p_type - 0x70000000:x}"
    return f"0x{p_type:x}"


def phdr_flags_str(p_flags: int) -> str:
    # readelf order is always RWE, blank if not set
    r = "R" if p_flags & 0x4 else " "
    w = "W" if p_flags & 0x2 else " "
    x = "E" if p_flags & 0x1 else " "
    return f"{r}{w}{x}"


def _phdr_str(self) -> str:
    type_name = phdr_type_name(self.p_type)
    flags = phdr_flags_str(self.p_flags)
    return (
        f"{type_name:<15}"
        f"0x{self.p_offset:06x} "
        f"0x{self.p_vaddr:016x} "
        f"0x{self.p_paddr:016x} "
        f"0x{self.p_filesz:06x} "
        f"0x{self.p_memsz:06x} "
        f"{flags:<3} "
        f"0x{self.p_align:x}"
    )


@final
@dataclass(slots=True)
class Elf32_Phdr:
    p_type: int
    p_offset: int
    p_vaddr: int
    p_paddr: int
    p_filesz: int
    p_memsz: int
    p_flags: int
    p_align: int

    _struct_le: ClassVar[struct.Struct] = struct.Struct("<IIIIIIII")
    _struct_be: ClassVar[struct.Struct] = struct.Struct(">IIIIIIII")

    def to_bytes(self, little_endian: bool) -> bytes:
        s = self._struct_le if little_endian else self._struct_be
        return s.pack(
            self.p_type,
            self.p_offset,
            self.p_vaddr,
            self.p_paddr,
            self.p_filesz,
            self.p_memsz,
            self.p_flags,
            self.p_align,
        )

    @classmethod
    def from_bytes(cls, byts: bytes) -> Self:
        s = cls._struct_le if byts[EI_DATA] == ELFDATA2LSB else cls._struct_be
        return cls(*s.unpack(byts[: s.size]))  # pyright: ignore[reportAny]

    @classmethod
    def get_size(cls) -> int:
        return cls._struct_le.size

    @override
    def __str__(self) -> str:
        return _phdr_str(self)


@final
@dataclass(slots=True)
class Elf64_Phdr:
    p_type: int
    p_flags: int
    p_offset: int
    p_vaddr: int
    p_paddr: int
    p_filesz: int
    p_memsz: int
    p_align: int

    _struct_le: ClassVar[struct.Struct] = struct.Struct("<IIQQQQQQ")
    _struct_be: ClassVar[struct.Struct] = struct.Struct(">IIQQQQQQ")

    def to_bytes(self, little_endian: bool) -> bytes:
        s = self._struct_le if little_endian else self._struct_be
        return s.pack(
            self.p_type,
            self.p_flags,
            self.p_offset,
            self.p_vaddr,
            self.p_paddr,
            self.p_filesz,
            self.p_memsz,
            self.p_align,
        )

    @classmethod
    def from_bytes(cls, byts: bytes) -> Self:
        s = cls._struct_le if byts[EI_DATA] == ELFDATA2LSB else cls._struct_be
        return cls(*s.unpack(byts[: s.size]))  # pyright: ignore[reportAny]

    @classmethod
    def get_size(cls) -> int:
        return cls._struct_le.size

    @override
    def __str__(self) -> str:
        return _phdr_str(self)


type ElfEhdr = Elf32_Ehdr | Elf64_Ehdr
type ElfPhdr = Elf32_Phdr | Elf64_Phdr


def parse_elf(data: bytes) -> tuple[ElfEhdr, list[ElfPhdr]]:
    ei_class = data[EI_CLASS]
    ei_data = data[EI_DATA]

    ehdr_cls: type[ElfEhdr] = Elf64_Ehdr if ei_class == ELFCLASS64 else Elf32_Ehdr
    phdr_cls: type[ElfPhdr] = Elf64_Phdr if ei_class == ELFCLASS64 else Elf32_Phdr

    ehdr = ehdr_cls.from_bytes(data)

    # NOTE: intentionally not using phdr_cls.from_bytes() here -- see bug note below.
    phdr_struct = phdr_cls._struct_le if ei_data == ELFDATA2LSB else phdr_cls._struct_be

    phdrs: list[ElfPhdr] = []
    for i in range(ehdr.e_phnum):
        off = ehdr.e_phoff + i * ehdr.e_phentsize
        chunk = data[off : off + phdr_struct.size]
        phdrs.append(phdr_cls(*phdr_struct.unpack(chunk)))

    return ehdr, phdrs


from ghidra.program.flatapi import FlatProgramAPI
from ghidra.program.model.address import Address, AddressRange
from ghidra.program.model.mem import MemoryAccessException


def read_bytes(flatapi: FlatProgramAPI, addr: Address, length: int) -> bytes:
    try:
        return bytes(flatapi.getBytes(addr, length))
    except MemoryAccessException as e:
        raise ValueError(f"failed to read {length} bytes at {addr}: {e}") from e


def parse_elf_from_program(
    flatapi: FlatProgramAPI,
) -> tuple["Elf32_Ehdr | Elf64_Ehdr", "list[Elf32_Phdr | Elf64_Phdr]"]:
    img_base = flatapi.getCurrentProgram().getImageBase()

    peek = read_bytes(flatapi, img_base, 20)  # e_ident + e_type + e_machine + e_version
    ei_class = peek[EI_CLASS]
    ei_data = peek[EI_DATA]

    ehdr_cls = Elf64_Ehdr if ei_class == ELFCLASS64 else Elf32_Ehdr
    phdr_cls = Elf64_Phdr if ei_class == ELFCLASS64 else Elf32_Phdr

    ehdr_bytes = read_bytes(flatapi, img_base, ehdr_cls.get_size())
    ehdr = ehdr_cls.from_bytes(ehdr_bytes)

    phdr_struct = phdr_cls._struct_le if ei_data == ELFDATA2LSB else phdr_cls._struct_be

    phdr_table_addr = img_base.add(ehdr.e_phoff)

    phdrs: list[Elf32_Phdr | Elf64_Phdr] = []
    for i in range(ehdr.e_phnum):
        addr = phdr_table_addr.add(i * ehdr.e_phentsize)
        chunk = read_bytes(flatapi, addr, phdr_struct.size)
        phdrs.append(phdr_cls(*phdr_struct.unpack(chunk)))  # pyright: ignore[reportAny]

    return ehdr, phdrs


class OffsetWriter:
    def __init__(self) -> None:
        self.contents: bytearray = bytearray()
        self.written_segments: list[tuple[int, int]] = []
        pass

    def write_at(self, off: int, byts: bytes):
        # check if this is in an already written segment
        for seg in self.written_segments:
            start, stop = seg
            within_bounds = off >= start and off + len(byts) <= stop
            if within_bounds:
                return

        self.written_segments.append((off, off + len(byts)))

        # and then, write to that segment.
        new_cap = len(byts) + off
        if new_cap > len(self.contents):
            self.contents.resize(new_cap)

        self.contents[off:new_cap] = byts


PT_LOAD = 1
tablehdr = "Type           Offset   VirtAddr           PhysAddr           FileSiz  MemSiz   Flg Align"


from ghidra.program.model.address import AddressSet


def read_segment_bytes(flatapi: FlatProgramAPI, addr: Address, size: int):
    mem = flatapi.currentProgram.getMemory()
    end_addr = addr.add(size - 1)

    want = AddressSet(addr, end_addr)
    have = mem.getLoadedAndInitializedAddressSet()
    readable = want.intersect(have)

    out = bytearray(size)
    ranges = readable.getAddressRanges()
    for rng in ranges:
        chunk = flatapi.getBytes(rng.getMinAddress(), int(rng.getLength()))
        offset = rng.getMinAddress().subtract(addr)
        out[offset : offset + len(chunk)] = bytes(chunk)

    return bytes(out)


def build_elf_file(
    flatapi: FlatProgramAPI,
    ehdr: "Elf32_Ehdr | Elf64_Ehdr",
    phdrs: "list[Elf32_Phdr | Elf64_Phdr]",
    out_path: str,
) -> None:
    file_entries = (phdr for phdr in phdrs if phdr.p_filesz > 0)
    in_file_order = sorted(file_entries, key=attrgetter("p_offset"))
    offwr = OffsetWriter()

    for phdr in in_file_order:
        print(phdr.p_type)
        if phdr.p_flags == 5:
            __import__("ipdb").set_trace()  # pyright: ignore[reportAny]

        addr = flatapi.toAddr(phdr.p_vaddr)  # pyright: ignore[reportUnknownMemberType]
        if ehdr.e_type == 3:
            addr = flatapi.getCurrentProgram().imageBase.add(phdr.p_vaddr)

        byts = read_segment_bytes(flatapi, addr, phdr.p_memsz)
        offwr.write_at(phdr.p_offset, byts)

    _ = open(out_path, "wb").write(bytes(offwr.contents))
PY — RECREATEELF.PY 
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

from typing import TYPE_CHECKING
import pyghidra


if TYPE_CHECKING:
    from ghidra.ghidra_builtins import *

_ = pyghidra.start()

from ghidra.program.flatapi import FlatProgramAPI
from elflib import build_elf_file, parse_elf_from_program

proj = pyghidra.open_project("/home/hknhmr/ghidraproj", "new", create=False)
loader = pyghidra.program_loader().project(proj)

# loader = loader.source("/home/hknhmr/ghidra_scripts/babyrace_level2.1")  # pyright: ignore[reportUnknownMemberType]
# with loader.load() as load_res:  # pyright: ignore[reportUnknownVariableType]
#     load_res.save(pyghidra.task_monitor())  # pyright: ignore[reportUnknownMemberType]

program, _ = pyghidra.consume_program(proj, "/home/hknhmr/ctf/pwn.college/system-security/race-conditions/level-3/babyrace_level2.1")
flatapi = FlatProgramAPI(program)


ehdr, phdrs = parse_elf_from_program(flatapi)

build_elf_file(flatapi, ehdr, phdrs, "out.elf")

Conclusion

Both bugs came down to Ghidra assuming things ELF doesn’t guarantee: that segments don’t overlap, and that it’s representation of memory blocks is contiguous. The fixes were small - an offset-aware writer, and a gap-aware reader

  • but finding them meant stepping through MemoryMapDB.java instead of trusting the docs.

The script above only rebuilds the ELF header, program headers, and segment bytes - no section headers, no symtab. Good enough to get something the OS can load and ld-linux-x86-64.so.2 won’t complain about.