Project

General

Profile

Bug #24608

Setup idmap autorid correctly

Added by Shane Panke almost 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Nice to have
Assignee:
John Hixson
Category:
OS
Target version:
Seen in:
Severity:
New
Reason for Closing:
Reason for Blocked:
Needs QA:
No
Needs Doc:
Yes
Needs Merging:
Yes
Needs Automation:
No
Support Suite Ticket:
n/a
Hardware Configuration:
ChangeLog Required:
No

Description

When setting autorid for Active Directory, the entire idmap is ignored. An example of the setup can be seen by using testparm from the cli:

idmap config mydomain: ignore builtin = yes
idmap config mydomain: read only = no
idmap config mydomain: rangesize = 20000
idmap config mydomain: range = 20000-80000000
idmap config mydomain: backend = autorid
idmap config *: range = 80000001-90000000
idmap config * : backend = tdb

Now this should mean that user elegant gets assigned an uid within the range for mydomain but it doesn't. Instead it gets assigned a uid within the range of *:

id elegant
uid=80000001(elegant) gid=80000005(domain users)

If I set the idmap to just rid, I get the appropriate result where user elegant is assigned within the appropriate range. The autorid backend is more appropriate AD scenarios due it's ability to discern if a user is really a user and not a group. The autorid backend will create a uid for each user but the rid backend will create a uid and gid for each user which can raise permission issues in certain cases (this is not a bug with rid, it's a shortcoming).

Associated revisions

Revision 38399d34 (diff)
Added by John Hixson over 3 years ago

Setup autorid idmap correctly Ticket: #24608

Revision 0d27c923 (diff)
Added by John Hixson over 3 years ago

Setup autorid idmap correctly Ticket: #24608 (cherry picked from commit 38399d3404dd037ab2fc938c34951539976d2c1d)

Revision a803df89 (diff)
Added by John Hixson over 3 years ago

Setup autorid idmap correctly Ticket: #24608

History

#1 Updated by William Grzybowski almost 4 years ago

  • Assignee changed from William Grzybowski to John Hixson

#2 Updated by John Hixson almost 4 years ago

  • Category changed from 3 to 36
  • Status changed from Unscreened to 15
  • Target version set to 11.0-U1

There is likely another problem here. Can you please attach a debug? system->advanced->save debug

#3 Updated by an odos almost 4 years ago

source3/winbindd/idmap_autorid.c contains the following

static NTSTATUS idmap_autorid_initialize(struct idmap_domain *dom)
{
    struct idmap_tdb_common_context *commonconfig;
    struct autorid_global_config *config;
    NTSTATUS status;
    char *db_path;

    if (!strequal(dom->name, "*")) {
        DEBUG(0, ("idmap_autorid_initialize: Error: autorid configured " 
              "for domain '%s'. But autorid can only be used for " 
              "the default idmap configuration.\n", dom->name));
        return NT_STATUS_INVALID_PARAMETER;
    }

Perhaps it expects to be configured for the default domain (i.e "idmap config * : backend = autorid") and is therefore grabbing parameters associated with the * domain.

#4 Updated by an odos almost 4 years ago

One option to consider is switching the idmap backend for the default domain from "tdb" to "autorid" since idmap_autorid appears to include the functionality provided by idmap_tdb. I dumped the contents of winbindd_idmap.tdb on my systems using tdbtool and idmap_tdb appears to only have allocated the following GIDS / SIDS

900000001     S-1-1-0
900000002     S-1-5-2
900000003     S-1-5-11
900000006     S-1-18-1

The manpage for idmap_autorid also contains the following example:

idmap config * : backend = autorid
idmap config * : range = 1000000-19999999
idmap config * : rangesize = 1000000

idmap config TRUSTED : backend  = ad
idmap config TRUSTED : range    = 50000 - 99999
idmap config TRUSTED : schema_mode = sfu

This leads me to think that the following is also correct:

idmap config * : backend = autorid
idmap config * : range = 90000001-100000000

idmap config DOMAIN : backend  = rid
idmap config DOMAIN : range    = 20000 - 90000000

Autorid only allocates 1 tdb entry per domain and generates maps ids algorithmically. The mapping for S-1-1-0 will remain the same (it is always the first ID in the range specified by autorid), but the remaining entries may change. An example of the contents of autorid.tdb is as follows:

key(7) = "CONFIG\00" 
data(49) = "minvalue:10000000 rangesize:1000000 maxranges:290" 
key(41) = "S-1-5-21-606640139-1729040616-4042376577\00" 
data(4) = "\01\00\00\00" 
key(2) = "1\00" 
data(41) = "S-1-5-21-606640139-1729040616-4042376577\00" 
key(11) = "NEXT RANGE\00" 
data(4) = "\02\00\00\00

Since idmap_autorid also covers us for mapping domain accounts, it eliminates the need for a separate idmap config for the domain. To minimize UI upheavals (and samba complaining about config choices) if users choose "autorid" as the idmap backend under "Directory Service"-> "Active Directory", then no "idmap config DOMAIN :" entries are added to the smb4.conf file.

TL;DR - consider replacing "idmap config * : backend = tdb" with "idmap config * : backend = autorid". What's the worst that can happen? :-)

#5 Updated by Shane Panke almost 4 years ago

Solid explanation odos, I was unable to find that tidbit regarding autorid needing to be set as the default idmap but I had my hunches. This is exactly what is happening, I can provide a debug when I have a chance if you still require one John but odos has pretty much covered why it isn't working. I thought autorid to be domain specific as this is what FreeNAS assigns it as when choosing autorid under Active Directory (not the case).

We could arrange for autorid to replace the default idmap and then hide/ignore/prevent the default range being set through SMB if autorid is selected. This should be possible given that you need AD enabled to enable SMB anyway. I will see about testing this as well by making the changes to smb4.conf. Worst case scenario, it will default back to rid when generating the smb4.conf on the next reboot which won't be a problem for me as I use the same range and mappings will lineup.

I'll report my findings but this does indeed to be the solution to using autorid as an idmap backend for AD. Thanks!

#6 Updated by Shane Panke almost 4 years ago

Alright finished testing. I can confirm that this does indeed correct the behavior however there is one remaining issue. When using autorid you generally want to be in the 0th slot (slice 0). That is to say if you the range listed below you want to start at 200000. The problem is that when you specify the following:

idmap config *: backend = autorid
idmap config *: range = 200000-2000200000
idmap config *: rangesize = 200000
idmap config *: read only = no
idmap config *: ignore builtin = no

You end up with the following:

uid=401106(elegant) gid=400513(domain users) groups=400513(domain users),401106(elegant),200001(BUILTIN\users)

Really you want user elegant to be 201106 so it lines up with other autorid implementations across your environment. This would mean you would want to disable BUILTIN so it does not consume the 0th slot. This should be simple:

idmap config *: ignore builtin = yes

Unfortunately after numerous tests, this does NOT work and BUILTIN users/groups such as 200001(BUILTIN\users) are continuously generated; I believe this is FreeNAS not Samba creating this. I could be wrong and perhaps it is a Samba bug. if "ignore builtin" behaves correctly then this would indeed solve autorid's issues as an idmap backend. For now I will have to stick with rid as I can't afford for my ids to be outside of the 0th slot across my environment.

#7 Updated by an odos almost 4 years ago

Shane Panke wrote:

Alright finished testing. I can confirm that this does indeed correct the behavior however there is one remaining issue. When using autorid you generally want to be in the 0th slot (slice 0). That is to say if you the range listed below you want to start at 200000. The problem is that when you specify the following:

idmap config *: backend = autorid
idmap config *: range = 200000-2000200000
idmap config *: rangesize = 200000
idmap config *: read only = no
idmap config *: ignore builtin = no

You end up with 200001(BUILTIN\users) and uid=401106(elegant) but really you want user elegant to be 201106 so it lines up with other autorid implementations across your environment. This would mean you would want to disable BUILTIN so it does not consume the 0th slot. This should be simple:

idmap config *: ignore builtin = yes

Unfortunately after numerous tests, this does NOT work and BUILTIN users/groups such as 200001(BUILTIN\users) are continuously generated; I believe this is FreeNAS not Samba creating this. I could be wrong and perhaps it is a Samba bug. if "ignore builtin" behaves correctly then this would indeed solve autorid's issues as an idmap backend. For now I will have to stick with rid as I can't afford for my ids to be outside of the 0th slot across my environment.

Just to confirm what you are doing: are you adding the parameters you listed as "auxiliary parameters" in the FreeNAS WebUI? How are you adding them? What does your smb.conf look like through "testparm"? Does it show "dmap config *: ignore builtin = yes"?

Please upload the debug file.

#8 Updated by Vaibhav Chauhan almost 4 years ago

  • Target version changed from 11.0-U1 to 11.0-U2

#9 Updated by Shane Panke almost 4 years ago

  • File debug.log added

See attached for my debug log (smb4.conf settings can also be seen there), if you require additional info let me know. Now my process, first I do "service samba_server stop" then edit smb4.conf removing everything regarding idmap (both the domain and * that exist) and replacing it with:

idmap config *: backend = autorid
idmap config *: range = 200000-2000200000
idmap config *: rangesize = 200000
idmap config *: read only = no
idmap config *: ignore builtin = yes

Next, I remove any unneeded *.tdb files from samba's DBs as they have caused issues with cached ids being pulled from earlier idmaps. This is followed by a quick press of the rebuild cache button before I run "service samba_server start". After that all I do is call "id elegant" and see what it spits out (see above). That's how I test it, perhaps it's not the best way(?) but it is taking effect for sure and I do not need to modify the auxiliary parameters.

#10 Updated by John Hixson almost 4 years ago

So when using autorid, it has to be the sole idmap backend, is this correct? If this is the case, I will fix it. I will test and verify here as well. I suspect my testing in the past worked because I didn't nuke tdb files and got false positive.

#11 Updated by Shane Panke almost 4 years ago

Correct John, odos found this is case through the debug information within Samba itself. Without nuking tdb files I also continuously received false positives, only when I wiped them did I see proper results.

This next part might need to be its own issue but when using:

idmap config *: ignore builtin = yes

we get the following with "id elegant":

uid=401106(elegant) gid=400513(domain users) groups=400513(domain users),200001(BUILTIN\users)

The last group, 200001(BUILTIN\users), should not exist given that we have asked builtin groups to be ignored. This might seem minor but that group is taking up the 0th slot which is preventing our domain from using it. If "ignore builtin" was honored then the output would be:

uid=201106(elegant) gid=200513(domain users) groups=200513(domain users)

where the domain is now in 0th slot. Perhaps this last part should be it's own issue? Perhaps maybe even with Samba if FreeNAS is not responsible (can't confirm). I will investigate.

#12 Updated by an odos almost 4 years ago

John Hixson wrote:

So when using autorid, it has to be the sole idmap backend, is this correct?

It does not have to be the sole idmap backend, but it does need to be given the default domain (e.g. idmap config * : backend = autorid). On FreeNAS this basically means that it needs to be the sole idmap backend.

#13 Updated by an odos almost 4 years ago

John Hixson wrote:

So when using autorid, it has to be the sole idmap backend, is this correct? If this is the case, I will fix it. I will test and verify here as well. I suspect my testing in the past worked because I didn't nuke tdb files and got false positive.

If you make it the sole idmap backend when it is used, you may need to make sure that "idmap config *: range" still defaults to "90000001-100000000" range rather than the "20000-90000000" range (to avoid breaking permissions for current idmap_autorid users).

#14 Updated by Vaibhav Chauhan almost 4 years ago

  • Target version changed from 11.0-U2 to 11.0-U3

#15 Updated by Dru Lavigne over 3 years ago

  • Status changed from 15 to Unscreened

#16 Updated by John Hixson over 3 years ago

  • Status changed from Unscreened to Screened

#17 Avatar?id=14398&size=24x24 Updated by Kris Moore over 3 years ago

  • Status changed from Screened to 46

John: Still suitable for -U3?

#18 Updated by John Hixson over 3 years ago

  • Status changed from 46 to Screened

I can get this into U3. I'll shoot for doing this in the next few days.

#19 Updated by John Hixson over 3 years ago

  • Status changed from Screened to Needs Developer Review
  • Assignee changed from John Hixson to Release Council
  • Priority changed from No priority to Nice to have

Fixed in commit:freenas/11.0-stable-fix-idmap-autorid
Pull request: https://github.com/freenas/freenas/pull/258

#20 Updated by Dru Lavigne over 3 years ago

  • Assignee changed from Release Council to Timur Bakeyev

Timur: can you review?

#21 Updated by Timur Bakeyev over 3 years ago

  • Status changed from Needs Developer Review to Reviewed by Developer
  • Assignee changed from Timur Bakeyev to Release Council

Looks good to me.

Took me some time to understand, how things are interconnected.

http://samba.org.ru/samba/docs/man/manpages/idmap_autorid.8.html

The mappings of which domain is mapped to which range is stored in autorid.tdb, thus you should backup this database regularly

One range will be used for local users and groups and for non-domain well-known SIDs like Everyone (S-1-1-0) or Creator Owner (S-1-3-0). A chosen list of well-known SIDs will be preallocated on first start to create deterministic mappings for those.

I think, that should explain that you don't get your users mapped into the firs0 slot...

Thus the number of local users and groups that can be created is limited by this option as well. If you plan to create a large amount of local users or groups, you will need set this parameter accordingly.

This is, actually, interesting dependency, @John?

#22 Updated by Timur Bakeyev over 3 years ago

Also, could it be maybe, cleaner, to leave autorid generator untouched, in case it was selected replace:

 # 5 = DS_TYPE_CIFS
 idmap = Struct(client.call('notifier.ds_get_idmap_object', 5, cifs.id, 'autorid' if smb4_autorid_enabled(client) else 'tdb'))
 configure_idmap_backend(client, smb4_conf, idmap, None)

and skip configuration of the autorid backend for the domain in:

def configure_idmap_backend(client, smb4_conf, idmap, domain):

or in the places where it is called for domain, like:

def add_activedirectory_conf(client, smb4_conf):

Actually, it has no sense for the domain at all, so we can check that if domain isn't None, we skip/don't use autorid backend.

#23 Updated by Dru Lavigne over 3 years ago

  • Assignee changed from Release Council to John Hixson

#24 Updated by Vaibhav Chauhan over 3 years ago

  • Status changed from Reviewed by Developer to 47

#25 Updated by Dru Lavigne over 3 years ago

  • Subject changed from idmap autorid being ignored to Setup idmap autorid correctly

#26 Updated by Joe Maloney over 3 years ago

  • Status changed from 47 to Ready For Release
  • Needs QA changed from Yes to No
  • QA Status Test Passes added
  • QA Status deleted (Not Tested)

Configured autorid with range 20000-80000000. Nuked all tdb in /var/db/samba4, and restarted AD. Afterwords I have observed users show up with id's in the defined autorid range.

root@bonniemini:/var/db/samba4 # id AD01\\jmaloney
uid=121133(AD01\jmaloney) gid=120513(AD01\domain users) groups=120513(AD01\domain users),121133(AD01\jmaloney),121152(AD01\group6),121159(AD01\group13),121150(AD01\group4),121160(AD01\group14),121147(AD01\group1),121151(AD01\group5),121163(AD01\group17),121161(AD01\group15),121156(AD01\group10),121155(AD01\group9),121149(AD01\group3),121162(AD01\group16),121158(AD01\group12),121157(AD01\group11),121148(AD01\group2),121165(AD01\group19),121154(AD01\group8),121115(AD01\qa),121153(AD01\group7),121166(AD01\group20),121164(AD01\group18)

#27 Updated by Vaibhav Chauhan over 3 years ago

  • Status changed from Ready For Release to Resolved

#28 Updated by Dru Lavigne over 3 years ago

  • File deleted (debug.log)

Also available in: Atom PDF