Show stacks for unpoison & poison

[ramosian-glider]

Originally reported on Google Code with ID 191

For normal malloc/free memory, a use-after-free results in the stack for the allocation,
the deallocation, and the final use.

For arena memory, a use-after-poison would ideally have the stack for the most recent
unpoison, the poison, and the final use.  In addition to the allocation for the arena
itself, I guess.

Reported by jruderman on 2013-06-04 19:55:33

[ramosian-glider]

This is doable, but would slow down (un)poisoning and make it more complex: even if
we're going to save a stack trace for each memory region poisoning, we'll have to store
it somewhere (there is no metadata or redzones for arbitrary user-provided memory).

Reported by [email protected] on 2013-06-05 08:59:51

• Labels added: Type-Enhancement
• Labels removed: Type-Defect

[ramosian-glider]

I think it would at least be very helpful to have an option (default off) to enable
this. We use poisoning/unpoisoning heavily at Mozilla to mark freed/allocated memory
in our own allocators.

Reported by decoder.oh on 2013-06-05 10:14:36

[ramosian-glider]

This actually *is* a bit tricky to implement.
In regular malloc-ed memory we have the redzone which we use to store all the metadata.
For the pool-alloced memory we don't have a redzone, so we will need to store that
data somewhere else. A lock-free resizable hash-table? 

Reported by konstantin.s.serebryany on 2013-06-05 10:19:22

[ramosian-glider]

Maybe we can implement (resizable, but with limited capacity) hash_table last_poisoning_stack[(addr,
size)].

That is, if we call __asan_poison_memory_region(addr, size) for the same (addr, size)
twice, the second call stack will overwrite the first one. Then, when we're trying
to report poisoning stack on failure, we scan the whole table and find the latest poisoning
touching the given address.

Reported by [email protected] on 2013-06-05 10:28:58

[kcc]

This is very tricky to implement, not working on this any time soon.

[kcc]

People keep asking...
https://code.google.com/p/chromium/issues/detail?id=577952
(Still not working on it, but let's keep this open)

[vonosmas]

Suppose the poisoned region is at least 16 bytes. Then we can do the following:

On poison:
(1) take the first 16 bytes in that region (user memory) and append it to a (circular) global buffer.
(2) overwrite these bytes with 4-byte magic, followed by 4-byte index into a global buffer, followed by 8-byte StackDepot ID of poisoning stack trace (can actually be 4 bytes IIRC).

On access to poisoned memory region:
(1) scan left until we find the magic, get stack depot ID, fetch poisoning stack trace.

On unpoison:
(1) restore the contents of user memory by copying 16 bytes back from global buffer.

Sadly, this doesn't work that nice for randomly overlapping poisoned/unpoisoned memory regions :(

[kcc]

Uggh. We can do simpler: have a bounded FIFO that will remember all ranges ever poisoned.
On u-a-p report just do a linear search.

[kcc]

Of course, any naive global circular buffer will not work as it will be a contention point

[morehouse]

No further comments on https://code.google.com/p/chromium/issues/detail?id=577952 for 2 years now. Without more demand for this, we probably won't implement.

[eugenis]

I think there is a lot of demand and I'm surprised we don't put more effort into this.
Any time I see a report of manually poisoned memory (like from an annotated custom allocator), I'm stumped. I don't know how our users are dealing with this.

[Ralith]

Currently wrestling with one such case. A backtrace indicating where the poison happened would be great!

[tenderlove]

Hi,

Just wanted to chime in here. I am using ASAN for debugging Ruby's garbage collector. We manually poison objects that get released back to the GC. It would be really great if I could get a stack trace at the time of poison on use-after-poison errors 🙏🏻

[choller]

We've just hit this issue again. With lots of custom allocators, the reports are significantly less actionable because there is no poison stack available. From our experience, regular uaf reports have a fix rate of > 90% without steps to reproduce. For poison reports, this is significantly lower. I'm looking at one such report again now and even with steps to reproduce, it takes significantly longer to debug.

It would be nice if this could be optionally turned on, esp. for debugging.

[rohit-nutanix]

Manual poisoning will be significantly more useful if we can get a poisoning backtrace on use-after-poison. To determine a use-after-poison in our code base, we had to resort to setting a hardware watchpoint in gdb in the shadow memory containing the poison marker to determine what was poisoning the memory - it turned out to be a thirdparty library which was poisoning it and the memory was not getting unpoisoned because we were using a custom allocator which is not intercepted by ASAN. A backtrace from ASAN would have saved us a lot of time.