[syzbot] [ocfs2?] possible deadlock in ocfs2_try_remove_refcount_tree

12 views
Skip to first unread message

syzbot

unread,
Aug 19, 2024, 11:48:26 AM8/19/24
Hello,

syzbot found the following issue on:

HEAD commit: 1fb918967b56 Merge tag 'for-6.11-rc3-tag' of git://git.ker..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=14b8bbd9980000
kernel config: https://syzkaller.appspot.com/x/.config?x=7229118d88b4a71b
dashboard link: https://syzkaller.appspot.com/bug?extid=1fed2de07d8e11a3ec1b
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1142e5ed980000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=14aa396b980000

Downloadable assets:
disk image (non-bootable): https://storage.googleapis.com/syzbot-assets/7bc7510fe41f/non_bootable_disk-1fb91896.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/cd6e8883313a/vmlinux-1fb91896.xz
kernel image: https://storage.googleapis.com/syzbot-assets/87b718d2d1df/bzImage-1fb91896.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/6b8f724adfab/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: [email protected]

Code: c4 28 c3 e8 01 1a 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 48 3d 01 f0 ff ff 73 01 <c3> 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f
======================================================
WARNING: possible circular locking dependency detected
6.11.0-rc3-syzkaller-00066-g1fb918967b56 #0 Not tainted
------------------------------------------------------
syz-executor622/5097 is trying to acquire lock:
ffff88803ec0bf60 (&oi->ip_alloc_sem){+.+.}-{3:3}, at: ocfs2_try_remove_refcount_tree+0xb6/0x330 fs/ocfs2/refcounttree.c:931

but task is already holding lock:
ffff88803ec0bff8 (&oi->ip_xattr_sem){++++}-{3:3}, at: ocfs2_try_remove_refcount_tree+0xa5/0x330 fs/ocfs2/refcounttree.c:930

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #4 (&oi->ip_xattr_sem){++++}-{3:3}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
ocfs2_init_acl+0x397/0x930 fs/ocfs2/acl.c:366
ocfs2_mknod+0x1c05/0x2b40 fs/ocfs2/namei.c:408
ocfs2_create+0x1ab/0x480 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3578 [inline]
open_last_lookups fs/namei.c:3647 [inline]
path_openat+0x1a9a/0x3470 fs/namei.c:3883
do_filp_open+0x235/0x490 fs/namei.c:3913
file_open_name fs/open.c:1361 [inline]
filp_open+0x261/0x2d0 fs/open.c:1381
do_coredump+0x2259/0x2a30 fs/coredump.c:708
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

-> #3 (jbd2_handle){.+.+}-{0:0}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
start_this_handle+0x1eb4/0x2110 fs/jbd2/transaction.c:448
jbd2__journal_start+0x2da/0x5d0 fs/jbd2/transaction.c:505
jbd2_journal_start+0x29/0x40 fs/jbd2/transaction.c:544
ocfs2_start_trans+0x3c9/0x700 fs/ocfs2/journal.c:352
ocfs2_mknod+0x150c/0x2b40 fs/ocfs2/namei.c:359
ocfs2_create+0x1ab/0x480 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3578 [inline]
open_last_lookups fs/namei.c:3647 [inline]
path_openat+0x1a9a/0x3470 fs/namei.c:3883
do_filp_open+0x235/0x490 fs/namei.c:3913
file_open_name fs/open.c:1361 [inline]
filp_open+0x261/0x2d0 fs/open.c:1381
do_coredump+0x2259/0x2a30 fs/coredump.c:708
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

-> #2 (&journal->j_trans_barrier){.+.+}-{3:3}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
ocfs2_start_trans+0x3be/0x700 fs/ocfs2/journal.c:350
ocfs2_mknod+0x150c/0x2b40 fs/ocfs2/namei.c:359
ocfs2_create+0x1ab/0x480 fs/ocfs2/namei.c:672
lookup_open fs/namei.c:3578 [inline]
open_last_lookups fs/namei.c:3647 [inline]
path_openat+0x1a9a/0x3470 fs/namei.c:3883
do_filp_open+0x235/0x490 fs/namei.c:3913
file_open_name fs/open.c:1361 [inline]
filp_open+0x261/0x2d0 fs/open.c:1381
do_coredump+0x2259/0x2a30 fs/coredump.c:708
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

-> #1 (sb_internal#2){.+.+}-{0:0}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
percpu_down_read include/linux/percpu-rwsem.h:51 [inline]
__sb_start_write include/linux/fs.h:1675 [inline]
sb_start_intwrite include/linux/fs.h:1858 [inline]
ocfs2_start_trans+0x2b9/0x700 fs/ocfs2/journal.c:348
ocfs2_orphan_for_truncate fs/ocfs2/file.c:396 [inline]
ocfs2_truncate_file+0x68c/0x1560 fs/ocfs2/file.c:500
ocfs2_setattr+0x1217/0x1f50 fs/ocfs2/file.c:1208
notify_change+0xb9d/0xe70 fs/attr.c:495
do_truncate+0x220/0x310 fs/open.c:65
do_coredump+0x2702/0x2a30 fs/coredump.c:744
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

-> #0 (&oi->ip_alloc_sem){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3133 [inline]
check_prevs_add kernel/locking/lockdep.c:3252 [inline]
validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
__lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_write+0x99/0x220 kernel/locking/rwsem.c:1579
ocfs2_try_remove_refcount_tree+0xb6/0x330 fs/ocfs2/refcounttree.c:931
ocfs2_truncate_file+0xe08/0x1560 fs/ocfs2/file.c:521
ocfs2_setattr+0x1217/0x1f50 fs/ocfs2/file.c:1208
notify_change+0xb9d/0xe70 fs/attr.c:495
do_truncate+0x220/0x310 fs/open.c:65
do_coredump+0x2702/0x2a30 fs/coredump.c:744
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

other info that might help us debug this:

Chain exists of:
&oi->ip_alloc_sem --> jbd2_handle --> &oi->ip_xattr_sem

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&oi->ip_xattr_sem);
lock(jbd2_handle);
lock(&oi->ip_xattr_sem);
lock(&oi->ip_alloc_sem);

*** DEADLOCK ***

2 locks held by syz-executor622/5097:
#0: ffff88803ec0c2c0 (&sb->s_type->i_mutex_key#17){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:799 [inline]
#0: ffff88803ec0c2c0 (&sb->s_type->i_mutex_key#17){+.+.}-{3:3}, at: do_truncate+0x20c/0x310 fs/open.c:63
#1: ffff88803ec0bff8 (&oi->ip_xattr_sem){++++}-{3:3}, at: ocfs2_try_remove_refcount_tree+0xa5/0x330 fs/ocfs2/refcounttree.c:930

stack backtrace:
CPU: 0 UID: 0 PID: 5097 Comm: syz-executor622 Not tainted 6.11.0-rc3-syzkaller-00066-g1fb918967b56 #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2186
check_prev_add kernel/locking/lockdep.c:3133 [inline]
check_prevs_add kernel/locking/lockdep.c:3252 [inline]
validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
__lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_write+0x99/0x220 kernel/locking/rwsem.c:1579
ocfs2_try_remove_refcount_tree+0xb6/0x330 fs/ocfs2/refcounttree.c:931
ocfs2_truncate_file+0xe08/0x1560 fs/ocfs2/file.c:521
ocfs2_setattr+0x1217/0x1f50 fs/ocfs2/file.c:1208
notify_change+0xb9d/0xe70 fs/attr.c:495
do_truncate+0x220/0x310 fs/open.c:65
do_coredump+0x2702/0x2a30 fs/coredump.c:744
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623
RIP: 0033:0x7f27cd76b711
Code: c4 28 c3 e8 01 1a 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 48 3d 01 f0 ff ff 73 01 <c3> 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f
RSP: 002b:ffffffffffffffe0 EFLAGS: 00010217
RAX: 0000000000000000 RBX: 0000000000000003 RCX: 00007f27cd76b709
RDX: 0000000000000000 RSI: ffffffffffffffe0 RDI: 0000000004008011
RBP: 65756e69746e6f63 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 633d73726f727265
R13: 00007ffd27cfc3fc R14: 00007ffd27cfc440 R15: 00007ffd27cfc430
</TASK>
OCFS2: ERROR (device loop0): int ocfs2_claim_suballoc_bits(struct ocfs2_alloc_context *, handle_t *, u32, u32, struct ocfs2_suballoc_result *): Chain allocator dinode 23 has 4294967295 used bits but only 16777215 total
On-disk corruption discovered. Please run fsck.ocfs2 once the filesystem is unmounted.
OCFS2: Returning error to the calling process.
(syz-executor622,5097,0):ocfs2_claim_suballoc_bits:2038 ERROR: status = -5
(syz-executor622,5097,0):__ocfs2_claim_clusters:2412 ERROR: status = -5
(syz-executor622,5097,0):__ocfs2_claim_clusters:2420 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_local_alloc_new_window:1197 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_local_alloc_new_window:1222 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_local_alloc_slide_window:1296 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_local_alloc_slide_window:1315 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_reserve_local_alloc_bits:672 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_reserve_local_alloc_bits:710 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_reserve_clusters_with_limit:1170 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_reserve_clusters_with_limit:1219 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_convert_inline_data_to_extents:7080 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_try_to_write_inline_data:1564 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_write_begin_nolock:1676 ERROR: status = -5
(syz-executor622,5097,0):ocfs2_write_begin:1906 ERROR: status = -5
syz-executor622 (5097) used greatest stack depth: 17712 bytes left
----------------
Code disassembly (best guess), 1 bytes skipped:
0: 28 c3 sub %al,%bl
2: e8 01 1a 00 00 call 0x1a08
7: 90 nop
8: 48 89 f8 mov %rdi,%rax
b: 48 89 f7 mov %rsi,%rdi
e: 48 89 d6 mov %rdx,%rsi
11: 48 89 ca mov %rcx,%rdx
14: 4d 89 c2 mov %r8,%r10
17: 4d 89 c8 mov %r9,%r8
1a: 4c 8b 4c 24 08 mov 0x8(%rsp),%r9
1f: 0f 05 syscall
21: 48 3d 01 f0 ff ff cmp $0xfffffffffffff001,%rax
27: 73 01 jae 0x2a
* 29: c3 ret <-- trapping instruction
2a: 48 c7 c1 b8 ff ff ff mov $0xffffffffffffffb8,%rcx
31: f7 d8 neg %eax
33: 64 89 01 mov %eax,%fs:(%rcx)
36: 48 83 c8 ff or $0xffffffffffffffff,%rax
3a: c3 ret
3b: 66 data16
3c: 2e cs
3d: 0f .byte 0xf
3e: 1f (bad)


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at [email protected].

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

Edward Adam Davis

unread,
Aug 21, 2024, 12:32:26 PM8/21/24
use trylock to replace

#syz test: upstream 1fb918967b56

diff --git a/fs/ocfs2/acl.c b/fs/ocfs2/acl.c
index 62464d194da3..f60e3178fe74 100644
--- a/fs/ocfs2/acl.c
+++ b/fs/ocfs2/acl.c
@@ -363,7 +363,8 @@ int ocfs2_init_acl(handle_t *handle,

if (!S_ISLNK(inode->i_mode)) {
if (osb->s_mount_opt & OCFS2_MOUNT_POSIX_ACL) {
- down_read(&OCFS2_I(dir)->ip_xattr_sem);
+ if (!down_read_trylock(&OCFS2_I(dir)->ip_xattr_sem))
+ return -EBUSY;
acl = ocfs2_get_acl_nolock(dir, ACL_TYPE_DEFAULT,
dir_bh);
up_read(&OCFS2_I(dir)->ip_xattr_sem);

syzbot

unread,
Aug 21, 2024, 12:48:04 PM8/21/24
Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
possible deadlock in ocfs2_get_system_file_inode

======================================================
WARNING: possible circular locking dependency detected
6.11.0-rc3-syzkaller-00066-g1fb918967b56-dirty #0 Not tainted
------------------------------------------------------
syz.0.18/5710 is trying to acquire lock:
ffff88804e4b95a8 (&osb->system_file_mutex){+.+.}-{3:3}, at: ocfs2_get_system_file_inode+0x18f/0x7b0 fs/ocfs2/sysfile.c:101

but task is already holding lock:
ffff888011b74da0 (&ocfs2_file_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_write_begin+0x1d1/0x3a0 fs/ocfs2/aops.c:1901

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&ocfs2_file_ip_alloc_sem_key){++++}-{3:3}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
down_read+0xb1/0xa40 kernel/locking/rwsem.c:1526
ocfs2_read_virt_blocks+0x2ca/0xa50 fs/ocfs2/extent_map.c:976
ocfs2_read_dir_block fs/ocfs2/dir.c:508 [inline]
ocfs2_find_entry_el fs/ocfs2/dir.c:715 [inline]
ocfs2_find_entry+0x43b/0x2780 fs/ocfs2/dir.c:1080
ocfs2_find_files_on_disk+0xff/0x360 fs/ocfs2/dir.c:1980
ocfs2_lookup_ino_from_name+0xb1/0x1e0 fs/ocfs2/dir.c:2002
_ocfs2_get_system_file_inode fs/ocfs2/sysfile.c:136 [inline]
ocfs2_get_system_file_inode+0x305/0x7b0 fs/ocfs2/sysfile.c:112
ocfs2_init_global_system_inodes+0x32c/0x730 fs/ocfs2/super.c:457
ocfs2_initialize_super fs/ocfs2/super.c:2250 [inline]
ocfs2_fill_super+0x3068/0x5880 fs/ocfs2/super.c:994
mount_bdev+0x20a/0x2d0 fs/super.c:1679
legacy_get_tree+0xee/0x190 fs/fs_context.c:662
vfs_get_tree+0x90/0x2a0 fs/super.c:1800
do_new_mount+0x2be/0xb40 fs/namespace.c:3472
do_mount fs/namespace.c:3812 [inline]
__do_sys_mount fs/namespace.c:4020 [inline]
__se_sys_mount+0x2d6/0x3c0 fs/namespace.c:3997
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x77/0x7f

-> #0 (&osb->system_file_mutex){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3133 [inline]
check_prevs_add kernel/locking/lockdep.c:3252 [inline]
validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
__lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
__mutex_lock_common kernel/locking/mutex.c:608 [inline]
__mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
ocfs2_get_system_file_inode+0x18f/0x7b0 fs/ocfs2/sysfile.c:101
ocfs2_reserve_local_alloc_bits+0x107/0x2870 fs/ocfs2/localalloc.c:627
ocfs2_reserve_clusters_with_limit+0x1b8/0xb60 fs/ocfs2/suballoc.c:1166
ocfs2_convert_inline_data_to_extents+0x29d/0x17f0 fs/ocfs2/alloc.c:7078
ocfs2_try_to_write_inline_data fs/ocfs2/aops.c:1562 [inline]
ocfs2_write_begin_nolock+0x1c79/0x4d30 fs/ocfs2/aops.c:1669
ocfs2_write_begin+0x205/0x3a0 fs/ocfs2/aops.c:1903
generic_perform_write+0x399/0x840 mm/filemap.c:4019
ocfs2_file_write_iter+0x17b4/0x1f60 fs/ocfs2/file.c:2455
__kernel_write_iter+0x40d/0x900 fs/read_write.c:523
__kernel_write+0x120/0x180 fs/read_write.c:543
__dump_emit+0x237/0x360 fs/coredump.c:816
elf_core_dump+0x35e0/0x4720 fs/binfmt_elf.c:2095
do_coredump+0x1b04/0x2a30 fs/coredump.c:767
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623

other info that might help us debug this:

Possible unsafe locking scenario:

CPU0 CPU1
---- ----
lock(&ocfs2_file_ip_alloc_sem_key);
lock(&osb->system_file_mutex);
lock(&ocfs2_file_ip_alloc_sem_key);
lock(&osb->system_file_mutex);

*** DEADLOCK ***

3 locks held by syz.0.18/5710:
#0: ffff88804e4fc420 (sb_writers#12){.+.+}-{0:0}, at: do_coredump+0x1adc/0x2a30 fs/coredump.c:766
#1: ffff888011b75100 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:799 [inline]
#1: ffff888011b75100 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: ocfs2_file_write_iter+0x46a/0x1f60 fs/ocfs2/file.c:2387
#2: ffff888011b74da0 (&ocfs2_file_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_write_begin+0x1d1/0x3a0 fs/ocfs2/aops.c:1901

stack backtrace:
CPU: 0 UID: 0 PID: 5710 Comm: syz.0.18 Not tainted 6.11.0-rc3-syzkaller-00066-g1fb918967b56-dirty #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:93 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:119
check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2186
check_prev_add kernel/locking/lockdep.c:3133 [inline]
check_prevs_add kernel/locking/lockdep.c:3252 [inline]
validate_chain+0x18e0/0x5900 kernel/locking/lockdep.c:3868
__lock_acquire+0x137a/0x2040 kernel/locking/lockdep.c:5142
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5759
__mutex_lock_common kernel/locking/mutex.c:608 [inline]
__mutex_lock+0x136/0xd70 kernel/locking/mutex.c:752
ocfs2_get_system_file_inode+0x18f/0x7b0 fs/ocfs2/sysfile.c:101
ocfs2_reserve_local_alloc_bits+0x107/0x2870 fs/ocfs2/localalloc.c:627
ocfs2_reserve_clusters_with_limit+0x1b8/0xb60 fs/ocfs2/suballoc.c:1166
ocfs2_convert_inline_data_to_extents+0x29d/0x17f0 fs/ocfs2/alloc.c:7078
ocfs2_try_to_write_inline_data fs/ocfs2/aops.c:1562 [inline]
ocfs2_write_begin_nolock+0x1c79/0x4d30 fs/ocfs2/aops.c:1669
ocfs2_write_begin+0x205/0x3a0 fs/ocfs2/aops.c:1903
generic_perform_write+0x399/0x840 mm/filemap.c:4019
ocfs2_file_write_iter+0x17b4/0x1f60 fs/ocfs2/file.c:2455
__kernel_write_iter+0x40d/0x900 fs/read_write.c:523
__kernel_write+0x120/0x180 fs/read_write.c:543
__dump_emit+0x237/0x360 fs/coredump.c:816
elf_core_dump+0x35e0/0x4720 fs/binfmt_elf.c:2095
do_coredump+0x1b04/0x2a30 fs/coredump.c:767
get_signal+0x13fa/0x1740 kernel/signal.c:2902
arch_do_signal_or_restart+0x96/0x860 arch/x86/kernel/signal.c:310
exit_to_user_mode_loop kernel/entry/common.c:111 [inline]
exit_to_user_mode_prepare include/linux/entry-common.h:328 [inline]
irqentry_exit_to_user_mode+0x79/0x280 kernel/entry/common.c:231
exc_page_fault+0x590/0x8c0 arch/x86/mm/fault.c:1542
asm_exc_page_fault+0x26/0x30 arch/x86/include/asm/idtentry.h:623
RIP: 0033:0x7f9a89b799c1
Code: 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 48 3d 01 f0 ff ff 73 01 <c3> 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f
RSP: 002b:ffffffffffffffe0 EFLAGS: 00010217
RAX: 0000000000000000 RBX: 00007f9a89d15f80 RCX: 00007f9a89b799b9
RDX: 0000000000000000 RSI: ffffffffffffffe0 RDI: 0000000004008011
RBP: 00007f9a89be78d8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f9a89d15f80 R15: 00007fff4ea50e98
</TASK>
OCFS2: ERROR (device loop0): int ocfs2_claim_suballoc_bits(struct ocfs2_alloc_context *, handle_t *, u32, u32, struct ocfs2_suballoc_result *): Chain allocator dinode 23 has 4294967295 used bits but only 16777215 total
On-disk corruption discovered. Please run fsck.ocfs2 once the filesystem is unmounted.
OCFS2: Returning error to the calling process.
(syz.0.18,5710,0):ocfs2_claim_suballoc_bits:2038 ERROR: status = -5
(syz.0.18,5710,0):__ocfs2_claim_clusters:2412 ERROR: status = -5
(syz.0.18,5710,0):__ocfs2_claim_clusters:2420 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_local_alloc_new_window:1197 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_local_alloc_new_window:1222 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_local_alloc_slide_window:1296 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_local_alloc_slide_window:1315 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_reserve_local_alloc_bits:672 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_reserve_local_alloc_bits:710 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_reserve_clusters_with_limit:1170 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_reserve_clusters_with_limit:1219 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_convert_inline_data_to_extents:7080 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_try_to_write_inline_data:1564 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_write_begin_nolock:1676 ERROR: status = -5
(syz.0.18,5710,0):ocfs2_write_begin:1906 ERROR: status = -5
ocfs2: Unmounting device (7,0) on (node local)


Tested on:

commit: 1fb91896 Merge tag 'for-6.11-rc3-tag' of git://git.ker..
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=129badf3980000
patch: https://syzkaller.appspot.com/x/patch.diff?x=11e96ea3980000

Diogo Jahchan Koike

unread,
Aug 22, 2024, 12:04:40 AM8/22/24
to [email protected], [email protected], Diogo Jahchan Koike
#syz test

Acquiring the locks in refcounttree should follow
the ip_alloc --> ip_xattr ordering, as done by multiple
code paths in ocfs2; otherwise, we risk an ABBA deadlock
(i.e in the start transaction path).

Signed-off-by: Diogo Jahchan Koike <[email protected]>
---
fs/ocfs2/refcounttree.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/fs/ocfs2/refcounttree.c b/fs/ocfs2/refcounttree.c
index 1f303b1adf1a..32c0249ff253 100644
--- a/fs/ocfs2/refcounttree.c
+++ b/fs/ocfs2/refcounttree.c
@@ -927,8 +927,8 @@ int ocfs2_try_remove_refcount_tree(struct inode *inode,
struct ocfs2_inode_info *oi = OCFS2_I(inode);
struct ocfs2_dinode *di = (struct ocfs2_dinode *)di_bh->b_data;

- down_write(&oi->ip_xattr_sem);
down_write(&oi->ip_alloc_sem);
+ down_write(&oi->ip_xattr_sem);

if (oi->ip_clusters)
goto out;
@@ -944,8 +944,8 @@ int ocfs2_try_remove_refcount_tree(struct inode *inode,
if (ret)
mlog_errno(ret);
out:
- up_write(&oi->ip_alloc_sem);
up_write(&oi->ip_xattr_sem);
+ up_write(&oi->ip_alloc_sem);
return 0;
}

--
2.39.2

syzbot

unread,
Aug 22, 2024, 12:20:04 AM8/22/24
Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
possible deadlock in ocfs2_get_system_file_inode

======================================================
WARNING: possible circular locking dependency detected
6.11.0-rc4-syzkaller-00019-gb311c1b497e5-dirty #0 Not tainted
------------------------------------------------------
syz.0.20/5691 is trying to acquire lock:
ffff8880424195a8 (&osb->system_file_mutex){+.+.}-{3:3}, at: ocfs2_get_system_file_inode+0x18f/0x7b0 fs/ocfs2/sysfile.c:101

but task is already holding lock:
ffff88804e85a2e0 (&ocfs2_file_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_write_begin+0x1d1/0x3a0 fs/ocfs2/aops.c:1901
3 locks held by syz.0.20/5691:
#0: ffff88804f948420 (sb_writers#12){.+.+}-{0:0}, at: do_coredump+0x1adc/0x2a30 fs/coredump.c:766
#1: ffff88804e85a640 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:799 [inline]
#1: ffff88804e85a640 (&sb->s_type->i_mutex_key#21){+.+.}-{3:3}, at: ocfs2_file_write_iter+0x46a/0x1f60 fs/ocfs2/file.c:2387
#2: ffff88804e85a2e0 (&ocfs2_file_ip_alloc_sem_key){++++}-{3:3}, at: ocfs2_write_begin+0x1d1/0x3a0 fs/ocfs2/aops.c:1901

stack backtrace:
CPU: 0 UID: 0 PID: 5691 Comm: syz.0.20 Not tainted 6.11.0-rc4-syzkaller-00019-gb311c1b497e5-dirty #0
RIP: 0033:0x7f58d97799c1
Code: 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 48 3d 01 f0 ff ff 73 01 <c3> 48 c7 c1 a8 ff ff ff f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f
RSP: 002b:ffffffffffffffe0 EFLAGS: 00010217
RAX: 0000000000000000 RBX: 00007f58d9915f80 RCX: 00007f58d97799b9
RDX: 0000000000000000 RSI: ffffffffffffffe0 RDI: 0000000004008011
RBP: 00007f58d97e78d8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
R13: 0000000000000000 R14: 00007f58d9915f80 R15: 00007ffc8b417b38
</TASK>
OCFS2: ERROR (device loop0): int ocfs2_claim_suballoc_bits(struct ocfs2_alloc_context *, handle_t *, u32, u32, struct ocfs2_suballoc_result *): Chain allocator dinode 23 has 4294967295 used bits but only 16777215 total
On-disk corruption discovered. Please run fsck.ocfs2 once the filesystem is unmounted.
OCFS2: Returning error to the calling process.
(syz.0.20,5691,0):ocfs2_claim_suballoc_bits:2038 ERROR: status = -5
(syz.0.20,5691,0):__ocfs2_claim_clusters:2412 ERROR: status = -5
(syz.0.20,5691,0):__ocfs2_claim_clusters:2420 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_local_alloc_new_window:1197 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_local_alloc_new_window:1222 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_local_alloc_slide_window:1296 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_local_alloc_slide_window:1315 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_reserve_local_alloc_bits:672 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_reserve_local_alloc_bits:710 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_reserve_clusters_with_limit:1170 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_reserve_clusters_with_limit:1219 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_convert_inline_data_to_extents:7080 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_try_to_write_inline_data:1564 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_write_begin_nolock:1676 ERROR: status = -5
(syz.0.20,5691,0):ocfs2_write_begin:1906 ERROR: status = -5
ocfs2: Unmounting device (7,0) on (node local)


Tested on:

commit: b311c1b4 Merge tag '6.11-rc4-server-fixes' of git://gi..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16dfa42b980000
kernel config: https://syzkaller.appspot.com/x/.config?x=df2f0ed7e30a639d
dashboard link: https://syzkaller.appspot.com/bug?extid=1fed2de07d8e11a3ec1b
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=178f93d5980000

syzbot

unread,
Sep 24, 2024, 1:04:04 AM9/24/24
syzbot has bisected this issue to:

commit a3c06ae158dd6fa8336157c31d9234689d068d02
Author: Parav Pandit <[email protected]>
Date: Tue Jan 5 10:32:03 2021 +0000

vdpa_sim_net: Add support for user supported devices

bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=11f6ca27980000
start commit: de5cb0dcb74c Merge branch 'address-masking'
git tree: upstream
final oops: https://syzkaller.appspot.com/x/report.txt?x=13f6ca27980000
console output: https://syzkaller.appspot.com/x/log.txt?x=15f6ca27980000
kernel config: https://syzkaller.appspot.com/x/.config?x=74ffdb3b3fad1a43
dashboard link: https://syzkaller.appspot.com/bug?extid=1fed2de07d8e11a3ec1b
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=12f23880580000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15851c27980000

Reported-by: [email protected]
Fixes: a3c06ae158dd ("vdpa_sim_net: Add support for user supported devices")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection
Reply all
Reply to author
Forward
0 new messages