Re: Cannot get stack trace for dangling raw_ptr

395 views
Skip to first unread message

Bartek Nowierski

unread,
Jun 4, 2024, 1:03:35 AM6/4/24
to Joe Mason, Arthur Sonzogni, Eric Sum, [email protected], memory-safety-dev

On Tue, Jun 4, 2024 at 4:42 AM 'Joe Mason' via memory-dev <[email protected]> wrote:
grt@ did a lot of work recently to speed up tests by skipping symbolization of stacks that would be thrown away anyway. (Mainly for CQ machines that spent a lot of time symbolizing stacks that nobody ever looked at.) I'm not sure if any of that work would affect local tests, but off the top of my head, it's possible that this dangling reference is happening during test teardown, and your manual CHECK's are in the test body, or some other timing-related difference.

To quickly debug, try changing the "PA_BASE_CHECK(cond)" to "if (!cond) { PA_LOG(ERROR) << base::debug::StackTrace(); return nullptr; }" which will explicitly print the stack and not screw up the test state by crashing. Hopefully completing the test will let it symbolize no matter when the crash happens.


On Mon, Jun 3, 2024 at 1:56 PM 'Eric Sum' via memory-dev <[email protected]> wrote:
Hi,

I'm running ash_unittests locally on my linux machine and am getting this crash:
[FATAL:raw_ptr_backup_ref_impl.h(228)] Check failed: IsPointeeAlive(address).
#00 0x563a8792cc3e  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x7b67c3e)
#01 0x563a86be4c43  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6e1fc43)
#02 0x563a82512eb2  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x274deb2)
#03 0x563a86e221d9  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x705d1d9)
#04 0x563a86e53985  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x708e985)
#05 0x563a86e52f12  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x708df12)
#06 0x563a86e54375  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x708f375)
#07 0x563a86ec2061  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x70fd061)
#08 0x563a86e54bcb  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x708fbcb)
#09 0x563a86e08647  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x7043647)
#10 0x563a86e0958d  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x704458d)
#11 0x563a86d61df7  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6f9cdf7)
#12 0x563a86d5f547  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6f9a547)
#13 0x563a8346f93a  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x36aa93a)
#14 0x563a862fc94e  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x653794e)
#15 0x563a862fd282  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6538282)
#16 0x563a86306956  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6541956)
#17 0x563a863065ef  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x65415ef)
#18 0x563a86efd2ea  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x71382ea)
#19 0x563a83388456  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x35c3456)
#20 0x563a86f0530b  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x714030b)
#21 0x563a86f05129  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x7140129)
#22 0x563a86d071c4  (/usr/local/google/home/esum/chromium/src/out/Default/ash_unittests+0x6f421c4)
#23 0x7f2ba6a196ca  (/usr/lib/x86_64-linux-gnu/libc.so.6+0x276ca)


Any idea why I'm unable to get a stack trace (or just any information for that matter that tells me which pointer is dangling)? Here are my gn args:

dcheck_always_on = true
is_component_build = false
is_chrome_branded = true
is_debug = false
is_official_build = false
optimize_webui = false
target_os = "chromeos"
use_remoteexec=true
chrome_enable_logging_by_default = true
symbol_level = 1
enable_backup_ref_ptr_feature_flag = true
enable_dangling_raw_ptr_checks = true
enable_dangling_raw_ptr_feature_flag = true
enable_backup_ref_ptr_instance_tracer = true
ffmpeg_branding = "ChromeOS"


Note if I intentionally add a CHECK failure somewhere else in the unit test, I get a stack trace, but I cannot here for some reason.

Thanks!
--
-Eric

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAJTMNZroap%2Bg5rEwd83Mfe852BzGrBpOff9Y8eUAVag6brKD3Q%40mail.gmail.com.

--
You received this message because you are subscribed to the Google Groups "memory-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/memory-dev/CAH%3DT95Q3O1_p9%2BhcewPUQgvD9oAtiiwY%3DpOLoMsagpWeOMgznQ%40mail.gmail.com.

Daniel Cheng

unread,
Jun 4, 2024, 4:11:26 AM6/4/24
to Eric Sum, Bartek Nowierski, Joe Mason, Arthur Sonzogni, [email protected], memory-safety-dev
The next time you see this, try piping things through tools/valgrind/asan/asan_symbolize.py, e.g.

./ash_unittests <flags here> |& tools/valgrind/asan/asan_symbolize.py

Daniel

On Mon, 3 Jun 2024 at 17:50, 'Eric Sum' via memory-dev <[email protected]> wrote:
Thanks for the response.

I tried for a couple hours, but couldn't get the base::debug::StackTrace() solution to work unfortunately. Getting that to compile is tough because of the circular include dependencies (stack_trace uses raw_ptr) and the circular BUILD.gn dependencies it introduces (raw_ptr and stack_trace are in different targets that would need to depend on each other).

I was ultimately able to identify the dangling pointer through the process of elimination, so I'm unblocked. And yes, the dangling reference is happening during teardown, and the CHECK I added was at some point during the main test body. Maybe that has something to do with it.

Either way, I appreciate you looking into this!

Arthur Sonzogni

unread,
Jun 4, 2024, 11:34:18 AM6/4/24
to Eric Sum, Daniel Cheng, Bartek Nowierski, Joe Mason, Arthur Sonzogni, [email protected], memory-safety-dev
IsPointeeAlive(...)  indicates we detected a UAF.

Did you try `is_debug = true`, I think this should work.
@dcheng instructions should also work.
Arthur @arthursonzogni


On Tue, Jun 4, 2024 at 4:32 AM Eric Sum <[email protected]> wrote:
Will do. Thanks for the tip.
--
-Eric

Joe Mason

unread,
Jun 4, 2024, 5:41:26 PM6/4/24
to Arthur Sonzogni, Eric Sum, Daniel Cheng, Bartek Nowierski, Arthur Sonzogni, [email protected], memory-safety-dev
Oh yeah, I should suggested adding "is_asan=true" to your args.gn. That would have cleared it up right away. Sorry I didn't think of that yesterday...
Reply all
Reply to author
Forward
0 new messages