LDAP User Import
| Home > Administration > System > Data > Data Import Configurations > LDAP User Import |
About the Hornbill LDAP User Import Utility
The utility provides a simple, safe and secure way to create new Hornbill User Accounts and update existing ones by synchronizing with accounts held in your Active Directory or other industry standard LDAP capable directory service. The tool is designed to run behind your corporate firewall, connect to your LDAP service, query the required account information, transform and load into the Hornbill instance. The utility does not write back to your directory, it is purely designed to read data from the directory and use this to populate Hornbill user information during the creation of new and update of existing Hornbill User accounts.
The tool connects to the Hornbill instance in the cloud over HTTPS/SSL so as long as you have standard internet access then you should be able to use tool without the need to make any firewall configuration changes. The tool supports both the initial bulk import as well as incremental adds and updates. You can schedule the utility to run periodically to perform the import/update tasks as required.
The LDAP Utility can be used as a stand-alone alternative to, or supplement user auto-provisioning. While auto-provisioning requires Single Sign On (SSO) configuring, the LDAP import utility is completely independent of any SSO configuration.
Open Source
The User Import Utility is provided open source under the Hornbill Community Licence and can be found Here on GitHub
Installation Overview
Installing the LDAP utility is very simple. The biggest consideration is likely to be where you locate the utility within your environment. Here are some considerations that will help you decide:
- The utility will need "visibility" of your directory server and be able to access this through the appropriate port. Industry standards suggest port 389 however if you use a different port you can specify this in the conf.json file.
- Your Hornbill instance is located in the Cloud, a location outside of your network. The utility will communicate with your instance using HTTPS. Therefore the machine where you install the utility will need a route to the outside world. In terms of ports, industry standards dictate port 443 for HTTPS.
Download the Utility for Windows
- Click one of the following links to download the 64 Bit Version or 32 Bit Version
- Extract the zip into a folder you would like the application to run from e.g. C:\LDAP_Import\
Once you have done this you can then begin configuration and testing
HTTP Proxies
If you use a proxy for all of your internet traffic, the HTTP_PROXY Environment variable needs to be set. The https_proxy environment variable holds the hostname or IP address of your proxy server. It is a standard environment variable and like any such variable, the specific steps you use to set it depends on your operating system.
For windows machines, it can be set from the command line using the following:
set HTTP_PROXY=HOST:PORT
Where "HOST" is the IP address or host name of your Proxy Server and "PORT" is the specific port number.
URLs to White List
Occasionally on top of setting the HTTP_PROXY variable the following URLs need to be white listed to allow access out to our network
- https://files.hornbill.com/instances/INSTANCENAME/zoneinfo - Allows access to lookup your Instance API Endpoint
- https://files.hornbill.co/instances/INSTANCENAME/zoneinfo - Backup URL for when files.hornbill.com is unavailable
- https://eurapi.hornbill.com/INSTANCENAME/xmlmc/ - This is your Instance API Endpoint, eurapi can change so you should use the endpoint defined in the previous URL
- https://api.github.com/repos/hornbill/goLDAPUserImport/tags - Optional Allows access to check for the latest version of the Import Tool
Configuration Overview
While it is necessary to download an executable to be located, scheduled, and run from within your environment, the configuration is held securely in your Hornbill instance and is created and managed using Hornbill Administration. To begin creating a new configuration, login to Hornbill administration and navigate to Home > System > Data > Data Import Configurations. This page relates specifically to the configuration of the LDAP User Import but in future it will be possible to manage further import configurations from this interface.
- Click the button located to the top right of the list to create a new import
- Give your import a name
- Select the type of import as "LDAP User Import"
- Add a description to indicate the specific purpose of the import. It is likely that you will have at least two LDAP User Import configurations. One will be used to manage the creation of new users and changes to existing active users of Hornbill and a second configuration can be created to manage the archiving of users who have left your organisation.
Once you have created a new LDAP User Import entry, you can begin the configuration. A brief overview can be found Here https://www.youtube.com/watch?v=qvq0TUVArqs and any existing LDAP Import configurations (version 2.0 or later) can be imported into this new interface.
The configuration is split into separate tabs and they can be approached in the following order:
- LDAP Server: This tab is where the information required to set up the communication link with your directory service (i.e. Active Directory) is stored.
- User Account: This tab contains the mapping between the Hornbill database fields and the LDAP user object attributes. i.e. which LDAP attribute will be used to populate the Hornbill database fields. Only some of these fields are required.
- User profile: In addition to the fundamental User Account Properties, Hornbill can store much more information about a user.
- User Options: This tab is where the ancillary associations are managed e.g Manager, Site, Role, and Group associations.
- Advanced Options: Each time the import is run, it outputs a log file. This tab allows you to set some preferences in relation to the log output.
- Debug: This tab allows you to view the raw configuration which is being drawn from your selections and entries made in the previous tabs.
each building on each other to Detailed command line options for each import configuration can be found within the Hornbill Administration UI.
Connecting to your Directory Server
This section of the conf file specifies the information required to set up the communication link between your Hornbill instance and your directory service (i.e. Active Directory).
| Connecting to your LDAP (Directory) Server | |
|---|---|
| |
| Defining your LDAP Query | |
|---|---|
| |
| Listing the LDAP Attributes | |
|---|---|
| |
Mapping your Directory Attributes to Hornbill User Account Properties
This section requires us to specify what LDAP attribute will be used to populate the Hornbill database fields. Only some of these fields are required.
| UserMapping | |
|---|---|
|
The directory attributes shown in the example to the left should be enough to get you started as they are typically where you would find this information referenced in your directory service. However, it is prudent to check that the attribute specified does actually hold the information in your directory as expected.
* "Name":"[givenName] [sn]", - Both Variables are evaluated from LDAP and set to the Name param
* "Site":"1" - The value of Site should be numeric As an alternative, the utility provides a "Site Lookup" section (see below) which can make a site association based on the contents of a directory attribute.
| |
Mapping your Directory Attributes to Extended Hornbill User Account Properties
In addition to the fundamental User Account Properties, Hornbill can store much more information about a user.
| UserProfileMapping | |
|---|---|
| |
Command Line Parameters
- file - this allows us to specify the Configuration file to load when running the utility. When this argument is not used, then the utility will default to using the conf.json file provided with the utility.
- dryrun - Defaults to false - Set to True and the XMLMC for Create and Update users will not be called and instead the XML will be dumped to the log file, this is to aid in debugging the initial connection information.
- workers - Defaults to `3` - Allows you to change the number of worker threads used to process the import, this can improve performance on slow import but using too many workers have a detriment to performance of your Hornbill instance.
Testing Overview
There is no substitute for hands-on experience when becoming familiar with the Hornbill import utilities.
The LDAP import utility accepts and understands a number of "Command Line Parameters" that can be used when running the utility from the command line. The most important one for testing is the -dryrun=true command. When this is specified, no information will be written to Hornbill and it allows you to confirm that the conf file is correctly structured and that a connection to your directory server can be established. A dryrun will also output a log file which provides you with an opportunity to review and understand any error messages that may occur.
Below are some high level steps to help you build confidence in your configuration:
- In the conf.json file, specify an LDAP filter to target a single user object. (Its good practice to initially test on a single, or small set of, user objects as this allows the dryruns to complete quicker and there is less log content to sift through).
- Perform a dryrun (by executing the utility along with the -dryrun=true command line parameter).
- Review cmd output and log file for errors
- Check against "Common Error Messages" listed on the wiki and take action to rectify where necessary.
- Continue with dryrun tests until you are happy that all the errors are accounted for.
- Perform a live import with this single user object still specified.
- Review user account in Hornbill and check all user properties are as expected i.e. email contains an email address etc.
- Adjust conf file user property mappings as necessary
- Loop through steps 6 - 8 as many times as is necessary until you are happy with the information being transported into the Hornbill user account properties.
- Amend the LDAP filter to target the user objects required for a full import.
- Perform a dryrun
- Review cmd output and log file for errors
- Check against "Common Error Messages" listed on the wiki and take action to rectify where necessary.
- Continue with dryrun tests until you are happy that all the errors are accounted for.
Example use of the dryrun command line parameter specified after the utility executable:
ldap_user_import.exe -dryrun=true
What Hornbill Roles are needed for the Import to Complete Successfully?
A default role is delivered with Hornbill that is designed to be used in conjunction with our range of user import utilities. The security role is called User Import and has all the necessary rights to import / update user properties.
As you may now be aware, every action within Hornbill must be performed in the context of a user account. As well as the chosen user account possessing the "user Import" role which facilitates the importing of the user accounts and updating of the user properties, this user account must posses the roles that you are associating to imported user accounts via the import utility. The above comment about roles is referring to Hornbill's security model when it comes to associating roles to user accounts, which is: Hornbill is designed to only allow the association of roles if the User who is performing the assignment of a particular role already possess the same system/application rights among the roles that they themselves possess. This security measure prevents you inflating your own rights or giving a user more rights than you have yourself.
i.e. in addition to the "User Import" role, any roles you try and assign to the user accounts being imported must be assigned to the user account logging in and running the import.
Trouble Shooting
Logging Overview
All Logging output is saved in the "log" directory which can be found in the same location as the executable. The file name contains the date and time the import was run LDAP_User_Import_2015-11-06T14-26-13Z.log
Common Error Messages
Below are some common errors that you may encounter in the log file and what they mean:
- [ERROR] Error Decoding Configuration File:..... - this will be typically due to a missing quote (") or comma (,) somewhere in the configuration file. This is where an online JSON viewer/validator can come in handy rather than trawling the conf file looking for that proverbial needle in a haystack.
- [ERROR] Get https:// api.github.com/repos/hornbill/goLDAPUserImport/tags: dial tcp xx.xx.xx.xx:xxx: ........ - this most likely indicates that you have a HTTP proxy server on your network between the host running the executable and your Hornbill API endpoint. Ensure the http_proxy environment variable is set (See the section on "HTTP Proxies" for more information) and that the proxy is configured to allow this communication.
- [ERROR] Unable to Create User: Invalid value for parameter '[parameter name]': The text size provided (31 characters) is greater than the maximum allowable size of 20 characters for column [column name] - the contents of your directory attribute exceed the maximum number of characters that can be placed in the Hornbill database column.
- [ERROR] Unable to Create User: The value in element <userId> did not meet the required input pattern constraints. at location '/methodCall/params/userId' - the user id contains characters that are not allowed. The User Id should be made up of alphanumeric characters. Full stops (.) and underscores (_) are also supported.
- [ERROR] Unable to Update User: Invalid value for parameter '[parameter name]': Error setting value for column '[column name]'. bad lexical cast: source type value could not be interpreted as target - this error is indicating that the contents of your directory attribute are in a format that is not compatible with the type of the Hornbill database column. For example, you will get this when trying to place text into a database field that is of type "INT" (accepts integer values only).
- [ERROR] Unable to Load LDAP Attribute: '[LDAP attribute name]' For Input Param: '[Hornbill Parameter name]' - When the import utility is unable to load a particular LDAP attribute, this means that the attribute field in your directory does not contain a value. This error will not prevent the user account being created or updated in Hornbill and can be considered more as a warning rather than an outright failure or problem.
- [ERROR] Unable to Assign Role to User: You cannot create or update the role as you do not have sufficient permissions to set the system rights. - every action within Hornbill must be performed in the context of a user account. This error is suggesting that the user account you are using to run the utility does not posses the appropriate Hornbill roles set to associate the necessary roles the imported user. See the Testing section above, specifically "What Hornbill Roles are needed for the Import to Complete Successfully?".
Error Codes
- 100 - Unable to create log File
- 101 - Unable to create log folder
- 102 - Unable to Load Configuration File
Scheduling Overview
Windows
You can schedule ldap_import.exe to run with any optional command line argument from Windows Task Scheduler.
|
