Project

General

Profile

Bug #10525

charset utf-8 issue on cifs shares (samba4)

Added by Gerhard Gaußling about 5 years ago. Updated about 3 years ago.

Status:
Closed: User Config Issue
Priority:
No priority
Assignee:
John Hixson
Category:
OS
Target version:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
Yes
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

seen in: FreeNAS-9.3-STABLE-201506292130 (mismatch to dropdown list)

Hello, I go a weired issue on charsets in files and folder names on FreeNAS 9.3 stable.

I first encountered this issue by trying to use fastcopy on windows 7 to copy a whole 2TB Drive to a mounted cifs share offered by freenas. It hang at 10% on copying files and folders and the last line of the error log claimed, that the folder with a pathname that contains Übersicht (umlaut Ü)does not exist.

I the tried in windows explorer to create a folder which contains umlauts. The folder with umlauts was created flawlessy, but I wasn't able to enter that directory. On doubleclick in Explorer I got an error message: "Y:\charsettest\oöuüaä bezieht sich auf einen Pfad der nicht verfügbar ist. [...]" (see attachement, sorry german) "Y:\charsettest\oöuüaä is related to a path that is not available." (rough translation).

If I connect by ssh to the freenas box, then I can see that folder in a kind of hex encoding, not as utf-8 encoded german umlauts (as expected).

@[root@freenas] /mnt/ao/ao/charsettest# zfs get all | egrep 'normalization|utf|char'|grep ao\/ao
ao/ao utf8only off -
ao/ao normalization none
[root@freenas] /mnt/ao/ao/charsettest# ls *
:c3:a4:c3:b6:c3:9f:
./ ../

o:c3:b6u:c3:bca:c3:a4:
./ ../

öä:
./ ../
[root@freenas] /mnt/ao/ao/charsettest# locale
LANG=de_DE.UTF-8
LC_CTYPE="de_DE.UTF-8"
LC_COLLATE="de_DE.UTF-8"
LC_TIME="de_DE.UTF-8"
LC_NUMERIC="de_DE.UTF-8"
LC_MONETARY="de_DE.UTF-8"
LC_MESSAGES="de_DE.UTF-8"
LC_ALL=
@

I already changed the locale to "de_DE.UTF-8". That way I was able to crate on ssh console folders like 'öä' but they doesn't appear in windows explorer (see 2nd attachement). The folders created within windows explorer are visible with a fine char representation, but aren't accessible from explorer, cmd, or cygwin. They still appeare within ssh console on freenas as o:c3:b6u:c3:bca:c3:a4: (oöuüaä).

The WebUI > Shell shows other locale
[root@freenas ~]# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_ALL=

In smb4conf I added
[root@freenas] /mnt/ao/ao/charsettest# egrep -i 'utf|char' /etc/local/smb4.conf
dos charset = CP437
unix charset = UTF-8

I looked for solutions here:
https://www.freebsd.org/doc/handbook/using-localization.html
https://forums.freenas.org/index.php?threads/special-characters-file-or-folder-not-displayed-in-ssh.10590/page-2#post-103886

And there are bugs that might be slightly related to this issue:
#4281, #3551, #2205, #1439, #1508

Thank you in advance

Gerhard

History

#1 Updated by Gerhard Gaußling about 5 years ago

  • File deleted (2015-07-10 12_44_57-Y__charsettest.png)

#2 Updated by Gerhard Gaußling about 5 years ago

3319

Gerhard Gaußling wrote:

I already changed the locale to "de_DE.UTF-8". That way I was able to crate on ssh console folders like 'öä' but they doesn't appear in windows explorer (see 2nd attachement). The folders created within windows explorer are visible with a fine char representation, but aren't accessible from explorer, cmd, or cygwin. They still appeare within ssh console on freenas as o:c3:b6u:c3:bca:c3:a4: (oöuüaä).

2nd attachment is here:

#5 Updated by Gerhard Gaußling about 5 years ago

Ok, it seems, that I misconfigured the share. I had VFS module activated in the cifs share that seems to be the culprit. shadow_copy2 shadow_copy_test and cap are not working with german umlauts. Also I switched now DOS charset to cp852 and in cifs service I disabled unix extensions.

Now everything seems to work fine.

#6 Updated by Gerhard Gaußling about 5 years ago

Now everything seems to work fine.

This is my smb4.conf
cat /etc/local/smb4.conf
@[global]
server max protocol = SMB3
encrypt passwords = yes
dns proxy = no
strict locking = no
oplocks = yes
deadtime = 15
max log size = 51200
max open files = 469271
load printers = no
printing = bsd
printcap name = /dev/null
disable spoolss = yes
getwd cache = yes
guest account = nobody
map to guest = Bad User
obey pam restrictions = yes
directory name cache size = 0
kernel change notify = no
dfree command = /usr/local/libexec/samba/dfree
panic action = /usr/local/libexec/samba/samba-backtrace
nsupdate command = /usr/local/bin/samba-nsupdate -g
server string = FreeNAS Server
ea support = yes
store dos attributes = yes
lm announce = yes
hostname lookups = yes
unix extensions = no
acl allow execute always = true
acl check permissions = true
dos filemode = yes
multicast dns register = yes
domain logons = no
idmap config *: backend = tdb
idmap config *: range = 90000001-100000000
server role = member server
netbios name = FREENAS
workgroup = HWGDOM
realm = HWGDOM.LOCAL
security = ADS
client use spnego = yes
cache directory = /var/tmp/.cache/.samba
local master = no
domain master = no
preferred master = no
winbind cache time = 7200
winbind offline logon = yes
winbind enum users = yes
winbind enum groups = yes
winbind nested groups = yes
winbind use default domain = yes
winbind refresh tickets = yes
winbind nss info = rfc2307
idmap config HWGDOM: backend = rid
idmap config HWGDOM: range = 20000-90000000
allow trusted domains = yes
client ldap sasl wrapping = plain
template shell = /bin/sh
template homedir = /home/%U
pid directory = /var/run/samba
smb passwd file = /var/etc/private/smbpasswd
private dir = /var/etc/private
create mask = 0666
directory mask = 0777
client ntlmv2 auth = yes
dos charset = CP852
unix charset = UTF-8
log level = 1

[ao]
path = /mnt/ao/ao
printable = no
veto files = /.snapshot/.windows/.mac/.zfs/
writeable = yes
browseable = yes
recycle:repository = .recycle/%U
recycle:keeptree = yes
recycle:versions = yes
recycle:touch = yes
recycle:directory_mode = 0777
recycle:subdir_mode = 0700
vfs objects = zfsacl netatalk acl_tdb acl_xattr catia readahead streams_xattr xattr_tdb syncops shadow_copy aio_fork aio_posix aio_pthread
hide dot files = yes
hosts allow = 192.168.70.0/24
guest ok = yes
nfs4:mode = special
nfs4:acedup = merge
nfs4:chown = true
zfsacl:acesort = dontcare@

#7 Updated by John Hixson about 5 years ago

  • Status changed from Unscreened to Screened
  • Target version set to Unspecified

#9 Updated by John Hixson about 5 years ago

  • Status changed from Screened to Closed: User Config Issue

Okay, so there is nothing to do here then ;-)

#10 Avatar?id=14398&size=24x24 Updated by Kris Moore about 3 years ago

  • Target version changed from Unspecified to N/A

Also available in: Atom PDF