Project

General

Profile

Bug #58128

"Lock order violation" from a windows client to a samba on FreeNAS with acl_tdb VFS module

Added by Max Vlasov almost 3 years ago. Updated almost 3 years ago.

Status:
Closed
Priority:
No priority
Assignee:
Andrew Walker
Category:
Services
Target version:
Seen in:
Severity:
New
Reason for Closing:
Not to be fixed
Reason for Blocked:
Needs QA:
No
Needs Doc:
No
Needs Merging:
No
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No
Tags:

Description

Basically a windows client should perform the following actions in order for samba to fail
(here we have an existing File1 and File2). The operation described invoked in the same program thread without pauses.
- Open File1 for reading
- Delete File2
- Open(Create)File2
- Close File2
- Try To Read File1. This operation will fail. At this moment the samba lock contains lines starting with a "Lock order violation"

...................

The code that reproduces this in a Windows client looks like this (pascal, but requires minor changes for other languages with direct access to windows api)

  AReadHandle := CreateFile(File1Path, GENERIC_READ,0, nil, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0);
  if AReadHandle > 0 then
  begin
    DeleteFile(File2Path);

    AWriteHandle := CreateFile(File2Path, GENERIC_READ or GENERIC_WRITE,0, nil, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
    if AWriteHandle > 0 then CloseHandle(AWriteHandle);

    if ReadFile(AReadHandle, Buf, SizeOf(Buf), ARead, nil) then
    begin
      // Ok on windows network shares, samba shares with acl_xattr
    end
    else
    begin
      // Failed on samba shares with acl_tdb VFS module
    end;
  end;
  if AReadHandle> 0 then CloseHandle(AReadHandle);

..................

This happens when the VFS module used for ACL is acl_tdb. If switched to acl_xattr, the issue is gone.

................

The log part at samba side

[2018/11/16 14:34:22.578862, 0] ../lib/dbwrap/dbwrap.c:166(dbwrap_check_lock_order)
Lock order violation: Trying /var/db/samba4/file_ntacls.tdb at 1 while /var/lock/locking.tdb at 1 is locked
[2018/11/16 14:34:22.578881, 0] ../lib/dbwrap/dbwrap.c:115(debug_lock_order)
lock order: 1:/var/lock/locking.tdb 2:<none> 3:<none>
[2018/11/16 14:34:22.578892, 0] ../source3/lib/util.c:804(smb_panic_s3)
PANIC (pid 42215): invalid lock_order
[2018/11/16 14:34:22.579775, 0] ../source3/lib/util.c:915(log_stack_trace)
BACKTRACE: 34 stack frames:
#0 0x80326f35e <log_stack_trace+0x1e> at /usr/local/lib/libsmbconf.so.0
#1 0x80326f158 <smb_panic_s3+0x98> at /usr/local/lib/libsmbconf.so.0
#2 0x800c9e597 <smb_panic+0x27> at /usr/local/lib/libsamba-util.so.0
#3 0x8075d63af <dbwrap_name+0x27f> at /usr/local/lib/samba/libdbwrap-samba4.so
#4 0x8075d5162 <dbwrap_fetch_locked+0x82> at /usr/local/lib/samba/libdbwrap-samba4.so
#5 0x8075d5114 <dbwrap_fetch_locked+0x34> at /usr/local/lib/samba/libdbwrap-samba4.so
#6 0x81aa1b291 <samba_init_module+0x2921> at /usr/local/lib/shared-modules/vfs/acl_tdb.so
#7 0x81aa1ab38 <samba_init_module+0x21c8> at /usr/local/lib/shared-modules/vfs/acl_tdb.so
#8 0x81aa19482 <samba_init_module+0xb12> at /usr/local/lib/shared-modules/vfs/acl_tdb.so
#9 0x801232a7f <smb_vfs_call_unlink+0x4f> at /usr/local/lib/samba/libsmbd-base-samba4.so
#10 0x8012267e0 <msg_close_file+0x1b50> at /usr/local/lib/samba/libsmbd-base-samba4.so
#11 0x801224a7f <close_file+0x11df> at /usr/local/lib/samba/libsmbd-base-samba4.so
#12 0x80122392e <close_file+0x8e> at /usr/local/lib/samba/libsmbd-base-samba4.so
#13 0x80128935f <smbd_smb2_request_process_close+0x12df> at /usr/local/lib/samba/libsmbd-base-samba4.so
#14 0x8012884c5 <smbd_smb2_request_process_close+0x445> at /usr/local/lib/samba/libsmbd-base-samba4.so
#15 0x80128827f <smbd_smb2_request_process_close+0x1ff> at /usr/local/lib/samba/libsmbd-base-samba4.so
#16 0x80126eb5f <smbd_smb2_request_dispatch+0x136f> at /usr/local/lib/samba/libsmbd-base-samba4.so
#17 0x801276bd2 <smbd_smb2_process_negprot+0x4002> at /usr/local/lib/samba/libsmbd-base-samba4.so
#18 0x801276296 <smbd_smb2_process_negprot+0x36c6> at /usr/local/lib/samba/libsmbd-base-samba4.so
#19 0x804c2124c <tevent_req_set_cleanup_fn+0x15cc> at /usr/local/lib/libtevent.so.0
#20 0x804c20696 <tevent_req_set_cleanup_fn+0xa16> at /usr/local/lib/libtevent.so.0
#21 0x804c1cc18 <_tevent_loop_once+0x118> at /usr/local/lib/libtevent.so.0
#22 0x804c20738 <tevent_req_set_cleanup_fn+0xab8> at /usr/local/lib/libtevent.so.0
#23 0x804c1d0d0 <_tevent_loop_wait+0x30> at /usr/local/lib/libtevent.so.0
#24 0x8012503ad <smbd_process+0xe9d> at /usr/local/lib/samba/libsmbd-base-samba4.so
#25 0x41305e <main+0x7a7e> at /usr/local/sbin/smbd
#26 0x804c2124c <tevent_req_set_cleanup_fn+0x15cc> at /usr/local/lib/libtevent.so.0
#27 0x804c20696 <tevent_req_set_cleanup_fn+0xa16> at /usr/local/lib/libtevent.so.0
#28 0x804c1cc18 <_tevent_loop_once+0x118> at /usr/local/lib/libtevent.so.0
#29 0x804c20738 <tevent_req_set_cleanup_fn+0xab8> at /usr/local/lib/libtevent.so.0
#30 0x804c1d0d0 <_tevent_loop_wait+0x30> at /usr/local/lib/libtevent.so.0
#31 0x40fcd8 <main+0x46f8> at /usr/local/sbin/smbd
#32 0x40d77f <main+0x219f> at /usr/local/sbin/smbd
#33 0x40703f <_start+0x17f> at /usr/local/sbin/smbd
[2018/11/16 14:34:22.579924, 0] ../source3/lib/util.c:816(smb_panic_s3)
smb_panic(): calling panic action [/usr/local/libexec/samba/samba-backtrace]
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/store/Prob/42187: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/store/Prob/42192: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/store/Prob/42198: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
[2018/11/16 14:34:23.083649, 0] ../source3/lib/util.c:824(smb_panic_s3)
smb_panic(): action returned status 2
[2018/11/16 14:34:23.083705, 0] ../source3/lib/dumpcore.c:315(dump_core)
dumping core in /var/db/system/cores
[2018/11/16 14:34:23.486181, 0] ../source3/lib/util_sock.c:876(matchname)
matchname: host name/name mismatch: 192.168.7.73 != (NULL)
[2018/11/16 14:34:23.486215, 0] ../source3/lib/util_sock.c:1055(get_remote_hostname)
matchname failed on 192.168.7.73
[2018/11/16 14:34:42.445918, 1] ../source3/smbd/smbd_cleanupd.c:99(smbd_cleanupd_unlock)
smbd_cleanupd_unlock: Cleaning up brl and lock database after unclean shutdown

History

#1 Updated by Dru Lavigne almost 3 years ago

  • Status changed from Unscreened to Blocked
  • Private changed from No to Yes
  • Reason for Blocked set to Need additional information from Author

Max: does updating to U6 resolve the issue?

If not, please attach a debug (System -> Advanced -> Save debug) to this ticket.

#2 Updated by Max Vlasov almost 3 years ago

  • File debug-freenas180-20181119105009.tgz added

Dru Lavigne wrote:

Max: does updating to U6 resolve the issue?

If not, please attach a debug (System -> Advanced -> Save debug) to this ticket.

FreeNAS-11.1-U6, same symptoms. I've attached the debug

#3 Updated by Dru Lavigne almost 3 years ago

  • Assignee changed from Release Council to William Grzybowski
  • Seen in changed from 11.1-U4 to 11.1-U6
  • Reason for Blocked deleted (Need additional information from Author)

#4 Updated by William Grzybowski almost 3 years ago

  • Assignee changed from William Grzybowski to Andrew Walker

#6 Updated by Dru Lavigne almost 3 years ago

  • Status changed from Blocked to Screened

#7 Updated by Andrew Walker almost 3 years ago

Max: thank you for reporting this issue. At this time, the only supported way of managing ACLs on FreeNAS is through the "zfsacl" VFS module. I encourage you to remove acl_tdb and just use zfsacl (which you also have enabled on the share).

#8 Updated by Max Vlasov almost 3 years ago

Andrew Walker wrote:

Max: thank you for reporting this issue. At this time, the only supported way of managing ACLs on FreeNAS is through the "zfsacl" VFS module. I encourage you to remove acl_tdb and just use zfsacl (which you also have enabled on the share).

Andrew, thanks. How about acl_xattr? Is it also semi-supported? It worked for us (fixed the issue) when comparing different acl implementations.
Also, should I separately report the issue to the samba tracker or you usually do the transfer?

#9 Updated by Andrew Walker almost 3 years ago

Max Vlasov wrote:

Andrew, thanks. How about acl_xattr? Is it also semi-supported? It worked for us (fixed the issue) when comparing different acl implementations.

What issue are you trying to fix?

Also, should I separately report the issue to the samba tracker or you usually do the transfer?

You can report upstream. It appears that possibly the same bug was reported here: https://bugzilla.samba.org/show_bug.cgi?id=11761

#10 Updated by Max Vlasov almost 3 years ago

Andrew Walker wrote:

Max Vlasov wrote:

Andrew, thanks. How about acl_xattr? Is it also semi-supported? It worked for us (fixed the issue) when comparing different acl implementations.

What issue are you trying to fix?

A Windows client can't open file on network shares. If the project is at a FreeNas/samba share with acl_tdb then the projects just fail to open at all. When reporting I just narrowed down to action of the client when working with files. Changing the acl implementation from acl_tdb to acl_xattr fixed the issue. So may I keep acl_xattr currently or your advice is to swtich to the mentioned zfsacl?

#11 Updated by Andrew Walker almost 3 years ago

Max Vlasov wrote:

A Windows client can't open file on network shares. If the project is at a FreeNas/samba share with acl_tdb then the projects just fail to open at all. When reporting I just narrowed down to action of the client when working with files. Changing the acl implementation from acl_tdb to acl_xattr fixed the issue. So may I keep acl_xattr currently or your advice is to swtich to the mentioned zfsacl?

That sounds like ACLs weren't configured correctly when you were using zfsacl. The last time I checked acl_xattr didn't really work correctly on ZFS because (1) it is designed to interact with posix-style ACLs and (2) it tries to set the ACLs in an xattr in the "security" namespace. So this is probably working for you by accident. It's better to figure out why you were having problems with ZFS ACLs and then configure them correctly. Big picture, we should add a proper ACL editor to the UI, which will make it easier to configure ACLs correctly.

#12 Updated by Andrew Walker almost 3 years ago

  • Status changed from Screened to Closed
  • Reason for Closing set to Not to be fixed
  • Needs QA changed from Yes to No
  • Needs Doc changed from Yes to No
  • Needs Merging changed from Yes to No

#13 Updated by Andrew Walker almost 3 years ago

  • File deleted (debug-freenas180-20181119105009.tgz)

#14 Updated by Dru Lavigne almost 3 years ago

  • Target version changed from Backlog to N/A
  • Private changed from Yes to No

Also available in: Atom PDF