Difference between revisions of "LDAP User Import"

From Hornbill
Jump to navigation Jump to search
(19 intermediate revisions by 4 users not shown)
Line 24: Line 24:
  
 
=== Download the Utility for Windows ===
 
=== Download the Utility for Windows ===
# Click one of the following links to download the [https://github.com/hornbill/goLDAPUserImport/releases/download/3.1.5/ldap_user_import_win_x64_v3_1_5.zip 64 Bit Version] or [https://github.com/hornbill/goLDAPUserImport/releases/download/3.1.5/ldap_user_import_win_x86_v3_1_5.zip 32 Bit Version], Linux and other builds can be found on the [https://github.com/hornbill/goLDAPUserImport/releases/latest release page].
+
# Download the [https://github.com/hornbill/goLDAPUserImport/releases/latest ZIP archive] relevant to the operating system and architecture where you will be running the tool.
 
# Extract the zip into a folder you would like the application to run from e.g. '''C:\LDAP_Import\'''
 
# Extract the zip into a folder you would like the application to run from e.g. '''C:\LDAP_Import\'''
 
<br>
 
<br>
Line 31: Line 31:
 
== HTTP Proxies ==
 
== HTTP Proxies ==
  
If you use a proxy for all of your internet traffic, the HTTP_PROXY Environment variable needs to be set. The https_proxy environment variable holds the hostname or IP address of your proxy server. It is a standard environment variable and like any such variable, the specific steps you use to set it depends on your operating system.
+
{{UtilityProxies}}
 
 
For windows machines, it can be set from the command line using the following:
 
<br>
 
<code>
 
set HTTP_PROXY=HOST:PORT
 
</code>
 
<br>
 
Where "HOST" is the IP address or host name of your Proxy Server and "PORT" is the specific port number.
 
 
 
=== URLs to White List ===
 
 
 
Occasionally on top of setting the HTTP_PROXY variable the following URLs need to be white listed to allow access out to our network
 
 
 
* https://files.hornbill.com/instances/INSTANCENAME/zoneinfo - Allows access to lookup your Instance API Endpoint
 
* https://files.hornbill.co/instances/INSTANCENAME/zoneinfo - Backup URL for when files.hornbill.com is unavailable
 
* https://eurapi.hornbill.com/INSTANCENAME/xmlmc/ - This is your Instance API Endpoint, eurapi can change so you should use the endpoint defined in the previous URL
 
* https://api.github.com/repos/hornbill/goLDAPUserImport/tags - '''Optional''' Allows access to check for the latest version of the Import Tool
 
<br>
 
  
 
==Configuration Overview==
 
==Configuration Overview==
Line 163: Line 145:
 
* The password field should be left empty as the utility generates a secure password that adheres to the User Password Policy as specified on your Hornbill instance. This password will only be temporary as the user should use the "Forgot Password" link available on the Hornbill Login Screen to reset their password the first time they navigate to your Hornbill instance.
 
* The password field should be left empty as the utility generates a secure password that adheres to the User Password Policy as specified on your Hornbill instance. This password will only be temporary as the user should use the "Forgot Password" link available on the Hornbill Login Screen to reset their password the first time they navigate to your Hornbill instance.
 
* "Site" - Recognises a corresponding Hornbill site id. E.g.
 
* "Site" - Recognises a corresponding Hornbill site id. E.g.
     * "Site":"1" - The value of Site should be numeric
+
     * "Site":"1" - The value of Site should be numeric (PLEASE NOTE If you plan on using the site lookup mechanism described later, this can be left blank).
 
As an alternative, the import configuration provides a "Site Lookup" section (outlined in a later section) which can make a site association based on the contents of a directory attribute.
 
As an alternative, the import configuration provides a "Site Lookup" section (outlined in a later section) which can make a site association based on the contents of a directory attribute.
 
* "User Type" - This defines if a user is Co-Worker or Basic user and should ONLY have the value "'''user'''" or "'''basic'''".
 
* "User Type" - This defines if a user is Co-Worker or Basic user and should ONLY have the value "'''user'''" or "'''basic'''".
Line 342: Line 324:
 
| style="width: 20%; text-align=left" |
 
| style="width: 20%; text-align=left" |
  
[[File:User_Group.PNG|400px|thumb|center|<div align="center">'''Associate users to Groups defined in your Organisation structure'''</div>]]
+
[[File:Group_company.png|400px|thumb|center|<div align="center">'''Associate users to Groups defined in your Organisation structure'''</div>]]
  
 
| align="left" |
 
| align="left" |
 
* Action - When the utility should perform the OrgLookup. On Create, On Update or Both. Setting this to "No Action" disables this section.
 
* Action - When the utility should perform the OrgLookup. On Create, On Update or Both. Setting this to "No Action" disables this section.
 
* Value - The directory attribute that contains the company or division or department name to use for the Hornbill Group Look-up. Any value wrapped with [] will be treated as an directory attribute.
 
* Value - The directory attribute that contains the company or division or department name to use for the Hornbill Group Look-up. Any value wrapped with [] will be treated as an directory attribute.
* Member Of -  
+
* Member Of - This field makes reference to the "memberOf" attribute of the user object in your directory. When a distinguished name (DN) of an LDAP group is specified in this field, the utility will only allow the group association in Hornbill if the group DN exists in the "memberOf" attribute of the user object being processed. i.e. only allow the Hornbill group association if the user object is a member of this directory group.
 
* Type - The Hornbill Group type (General, Team, Department, Costcenter, Division, Company). This focuses the import to only perform the look-up on Groups of a particular type (this has no relevance to the contents of your directory or the directory attribute that is used.
 
* Type - The Hornbill Group type (General, Team, Department, Costcenter, Division, Company). This focuses the import to only perform the look-up on Groups of a particular type (this has no relevance to the contents of your directory or the directory attribute that is used.
 
* Membership - The Hornbill Group Membership the users will be added with. Select from the drop-down list.
 
* Membership - The Hornbill Group Membership the users will be added with. Select from the drop-down list.
Line 353: Line 335:
 
* Can Action Tasks - If set true, then the user can action tasks assigned to this group (Typically only required for Groups of type "Team").
 
* Can Action Tasks - If set true, then the user can action tasks assigned to this group (Typically only required for Groups of type "Team").
 
* Only One Group Assignment - When set to "ON", a user can only be associated to a single group at any one time. i.e. if in the latest import the configuration was set to associate users to the Accounting department, the users would be associated to this new department and removed from any other group that they were already associated with.
 
* Only One Group Assignment - When set to "ON", a user can only be associated to a single group at any one time. i.e. if in the latest import the configuration was set to associate users to the Accounting department, the users would be associated to this new department and removed from any other group that they were already associated with.
 +
* Set As Home Organisation - Only visible when the Type is Company. When set to "ON", the Company will be set as the users Home Organisation. Requires version 3.3.0 or greater of the import tool, and is available in build 1077 and above of the Hornbill Admin Tool
 
'''Please Note''' - A successful association of a user to a Group is dependant upon the import finding a Hornbill Group with a name that matches the contents of the specified user attribute in your directory.
 
'''Please Note''' - A successful association of a user to a Group is dependant upon the import finding a Hornbill Group with a name that matches the contents of the specified user attribute in your directory.
  
Line 374: Line 357:
 
| align="left" |
 
| align="left" |
 
* config - the id of the configuration in Hornbill Administration i.e. this is the name of the configuration you have created with the spaces replaced with hyphens.
 
* config - the id of the configuration in Hornbill Administration i.e. this is the name of the configuration you have created with the spaces replaced with hyphens.
* logprefix - the prefix to add to the logfile that will be generated locally.
 
 
* dryrun - When dryrun is enabled, no Data Changes are made.
 
* dryrun - When dryrun is enabled, no Data Changes are made.
* version - Return the current version of the utility.
 
 
* instanceid - This is the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". Remember, this value is also case sensitive.
 
* instanceid - This is the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". Remember, this value is also case sensitive.
 
* apikey - A Valid API Assigned to a user with enough rights to process the import.
 
* apikey - A Valid API Assigned to a user with enough rights to process the import.
 
* apitimeout - Number of seconds to timeout the connection attempt with your Hornbill instance.
 
* apitimeout - Number of seconds to timeout the connection attempt with your Hornbill instance.
 
* workers - Number of concurrent threads used to process users. Increasing the number of threads will decrease the amount of time required for the import to run. However, more threads will place an increased load on your instance.
 
* workers - Number of concurrent threads used to process users. Increasing the number of threads will decrease the amount of time required for the import to run. However, more threads will place an increased load on your instance.
 +
* forcerun - If a previous run did not successfully complete you can specify ''forcerun=true'' to force the next run to bypass the existing running job check
 +
* logprefix - the prefix to add to the logfile that will be generated locally.
 +
* version - Return the current version of the utility.
  
 
|}
 
|}
Line 391: Line 375:
 
The LDAP import accepts and understands a number of "Command Line Parameters" that can be used when running the utility from the command line. The most important one for testing is the '''-dryrun=true''' command. When this is specified, no information will be written to Hornbill and it allows you to confirm that the configuration is correct and that a connection to your directory server can be established. A dryrun still outputs a log file which provides you with an opportunity to review and understand any error messages that may occur.
 
The LDAP import accepts and understands a number of "Command Line Parameters" that can be used when running the utility from the command line. The most important one for testing is the '''-dryrun=true''' command. When this is specified, no information will be written to Hornbill and it allows you to confirm that the configuration is correct and that a connection to your directory server can be established. A dryrun still outputs a log file which provides you with an opportunity to review and understand any error messages that may occur.
 
<br>
 
<br>
 +
===Suggested Testing Approach===
 
Below are some high level steps to help you build confidence in your configuration:
 
Below are some high level steps to help you build confidence in your configuration:
 
<br>
 
<br>
Line 422: Line 407:
 
i.e. in addition to the "User Import" role, any roles you try and assign to the user accounts being imported must be assigned to the user account logging in and running the import.
 
i.e. in addition to the "User Import" role, any roles you try and assign to the user accounts being imported must be assigned to the user account logging in and running the import.
  
==Trouble Shooting==
+
== API Key Rules ==
 +
This utility uses ([[API keys]]):
 +
 
 +
* activity:profileImageSet
 +
* admin:sysOptionGet
 +
* admin:keysafeGetKey
 +
* admin:userAddGroup
 +
* admin:userAddRole
 +
* admin:userCreate
 +
* admin:userDeleteGroup
 +
* admin:userProfileSet
 +
* admin:userSetAccountStatus
 +
* admin:userUpdate
 +
* data:entityAddRecord
 +
* data:entityGetRecord
 +
* data:entityUpdateRecord
 +
* data:queryExec
 +
* session:getSystemLicenseInfo
 +
 
 +
==Troubleshooting==
  
 
=== Logging Overview ===
 
=== Logging Overview ===
  
 
All Logging output is saved in the "log" directory which can be found in the same location as the executable. The file name contains the date and time the import was run '''''LDAP_User_Import_2015-11-06T14-26-13Z.log'''''
 
All Logging output is saved in the "log" directory which can be found in the same location as the executable. The file name contains the date and time the import was run '''''LDAP_User_Import_2015-11-06T14-26-13Z.log'''''
 +
There are 4 levels of logging available
 +
* Error
 +
* Warning
 +
* Message
 +
* Debug
 +
with Error being the lowest and Debug giving the most detail.
 +
 +
These are set on the '''''Advanced Options''''' tab of the Import Configuration
  
 
===Common Error Messages===
 
===Common Error Messages===
Line 434: Line 446:
 
* ''' ''[ERROR] Get https:// api.github.com/repos/hornbill/goLDAPUserImport/tags: dial tcp xx.xx.xx.xx:xxx: ........'' ''' - this most likely indicates that you have a HTTP proxy server on your network between the host running the executable and your Hornbill API endpoint. Ensure the http_proxy environment variable is set (See the section on "HTTP Proxies" for more information) and that the proxy is configured to allow this communication.
 
* ''' ''[ERROR] Get https:// api.github.com/repos/hornbill/goLDAPUserImport/tags: dial tcp xx.xx.xx.xx:xxx: ........'' ''' - this most likely indicates that you have a HTTP proxy server on your network between the host running the executable and your Hornbill API endpoint. Ensure the http_proxy environment variable is set (See the section on "HTTP Proxies" for more information) and that the proxy is configured to allow this communication.
 
* ''' ''[ERROR] Connection Error: dial tcp xx.xxx.xx.xx:389: conectex: a connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.'' ''' - A connection error containing an IP address with port "389" indicates that the utility had a problem connecting to the LDAP server. i.e. the server could not be reached or did not respond. This could be due to incorrect LDAP login details being specified, check what you have stored in the KeySafe and that the correct KeySafe entry is specified in the Data Import Configuration.
 
* ''' ''[ERROR] Connection Error: dial tcp xx.xxx.xx.xx:389: conectex: a connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.'' ''' - A connection error containing an IP address with port "389" indicates that the utility had a problem connecting to the LDAP server. i.e. the server could not be reached or did not respond. This could be due to incorrect LDAP login details being specified, check what you have stored in the KeySafe and that the correct KeySafe entry is specified in the Data Import Configuration.
 +
* ''' ''[ERROR] Connecting Error: dial tcp xxx.xxx.xxx.xxx:389: connectex: No connection could be made because the target machine actively refused it.'' ''' - This error indicates that the utility is unable to connect to the LDAP server because the LDAP server refused the connection for some reason. This could be due to; incorrect username and password, incorrect permissions on the windows user account accessing the machine/running the LDAP query, the wrong connection type/port is specified i.e. requires SSL/TLS on a secure port rather than HTTP on 389.
 
* ''' '' [ERROR] Search Error: LDAP Result Code 211 "ErrorClosing": Response Channel Closed'' ''' - this usually indicates the port (specified in the Key Safe) and Connection Type (specified in the import configuration) are mismatched. E.g. If using TLS, this would typically be port 389. SSL would typically be associated with port 636.
 
* ''' '' [ERROR] Search Error: LDAP Result Code 211 "ErrorClosing": Response Channel Closed'' ''' - this usually indicates the port (specified in the Key Safe) and Connection Type (specified in the import configuration) are mismatched. E.g. If using TLS, this would typically be port 389. SSL would typically be associated with port 636.
 
* ''' ''[ERROR] Unable to Create User: Invalid value for parameter '[parameter name]': The text size provided (31 characters) is greater than the maximum allowable size of 20 characters for column [column name]'' ''' - the contents of your directory attribute exceed the maximum number of characters that can be placed in the Hornbill database column.
 
* ''' ''[ERROR] Unable to Create User: Invalid value for parameter '[parameter name]': The text size provided (31 characters) is greater than the maximum allowable size of 20 characters for column [column name]'' ''' - the contents of your directory attribute exceed the maximum number of characters that can be placed in the Hornbill database column.
Line 441: Line 454:
 
* ''' ''[ERROR] Unable to Load LDAP Attribute: '[LDAP attribute name]' For Input Param: '[Hornbill Parameter name]' '' ''' - When the import utility is unable to load a particular LDAP attribute, this means that the attribute field in your directory does not contain a value. This error will not prevent the user account being created or updated in Hornbill and can be considered more as a warning rather than an outright failure or problem.
 
* ''' ''[ERROR] Unable to Load LDAP Attribute: '[LDAP attribute name]' For Input Param: '[Hornbill Parameter name]' '' ''' - When the import utility is unable to load a particular LDAP attribute, this means that the attribute field in your directory does not contain a value. This error will not prevent the user account being created or updated in Hornbill and can be considered more as a warning rather than an outright failure or problem.
 
* ''' ''[ERROR] Unable to Assign Role to User: You cannot create or update the role as you do not have sufficient permissions to set the system rights.'' ''' - every action within Hornbill must be performed in the context of a user account. This error is suggesting that the user account you are using to run the utility does not posses the appropriate Hornbill roles set to associate the necessary roles the imported user. See the Testing section above, specifically "What Hornbill Roles are needed for the Import to Complete Successfully?".
 
* ''' ''[ERROR] Unable to Assign Role to User: You cannot create or update the role as you do not have sufficient permissions to set the system rights.'' ''' - every action within Hornbill must be performed in the context of a user account. This error is suggesting that the user account you are using to run the utility does not posses the appropriate Hornbill roles set to associate the necessary roles the imported user. See the Testing section above, specifically "What Hornbill Roles are needed for the Import to Complete Successfully?".
 +
* ''' ''[ERROR] Unable to run import, a previous import is still running'' ''' - this can occur if the previous import failed to complete. Perform a manual (non-scheduled) run of the import from the command line including the argument "forcerun=true". Future imports should now run without issue.
  
 
=== Error Codes ===
 
=== Error Codes ===

Revision as of 15:13, 24 September 2021

Home > Administration > System > Data > Data Import Configurations > LDAP User Import


About the Hornbill LDAP User Import Utility

The utility provides a simple, safe and secure way to create new Hornbill User Accounts and update existing ones by synchronizing with accounts held in your Active Directory or other industry standard LDAP capable directory service. The tool is designed to run behind your corporate firewall, connect to your LDAP service, query the required account information, transform and load into the Hornbill instance. The utility does not write back to your directory, it is purely designed to read data from the directory and use this to populate Hornbill user information during the creation of new and update of existing Hornbill User accounts.

The tool connects to the Hornbill instance in the cloud over HTTPS/SSL so as long as you have standard internet access then you should be able to use tool without the need to make any firewall configuration changes. The tool supports both the initial bulk import as well as incremental adds and updates. You can schedule the utility to run periodically to perform the import/update tasks as required.

The LDAP Utility can be used as a stand-alone alternative to, or supplement user auto-provisioning. While auto-provisioning requires Single Sign On (SSO) configuring, the LDAP import utility is completely independent of any SSO configuration.

Open Source

The User Import Utility is provided open source under the Hornbill Community Licence and can be found Here on GitHub

Installation Overview

Installing the LDAP utility is very simple. The biggest consideration is likely to be where you locate the utility within your environment. Here are some considerations that will help you decide:

  1. The utility will need "visibility" of your directory server and be able to access this through the appropriate port. Industry standards suggest port 389 however if you use a different port you can specify this in the conf.json file.
  2. Your Hornbill instance is located in the Cloud, a location outside of your network. The utility will communicate with your instance using HTTPS. Therefore the machine where you install the utility will need a route to the outside world. In terms of ports, industry standards dictate port 443 for HTTPS.

Download the Utility for Windows

  1. Download the ZIP archive relevant to the operating system and architecture where you will be running the tool.
  2. Extract the zip into a folder you would like the application to run from e.g. C:\LDAP_Import\


Once you have done this you can then begin configuration and testing

HTTP Proxies

If you use a proxy for all of your internet traffic, the HTTP_PROXY and HTTPS_PROXY Environment variables need to be set. These environment variables hold the hostname or IP address of your proxy server. It is a standard environment variable and like any such variable, the specific steps you use to set it depends on your operating system.

For windows machines, it can be set from the command line using the following:
set HTTP_PROXY=HOST:PORT

set HTTPS_PROXY=HOST:PORT
Where "HOST" is the IP address or host name of your Proxy Server and "PORT" is the specific port number. IF you require a username and password to go through the proxy, the format for the setting is as follows:
set HTTP_PROXY=username:password@HOST:PORT

set HTTPS_PROXY=username:password@HOST:PORT

URLs to White List

Occasionally on top of setting the HTTP_PROXY variable the following URLs need to be white listed to allow access out to our network

Configuration Overview

While it is necessary to download an executable to be located, scheduled, and run from within your environment, the configuration is held securely in your Hornbill instance and is created and managed using Hornbill Administration.

Creating an Import Configuration

To begin creating a new configuration, login to Hornbill administration and navigate to Home > System > Data > Data Import Configurations. This page relates specifically to the configuration of the LDAP User Import but in future it will be possible to manage further import configurations from this interface.
In order to Create and Update an import configuration you will need the security role called "Admin Role" associating to your user account.

  1. Click the button located to the top right of the list to create a new import
  2. Give your import a name
  3. Select the type of import as "LDAP User Import"
  4. Add a description to indicate the specific purpose of the import. It is likely that you will have at least two LDAP User Import configurations. One will be used to manage the creation of new users and changes to existing active users of Hornbill and a second configuration can be created to manage the archiving of users who have left your organisation.

Once you have created a new LDAP User Import entry, you can begin the configuration. A brief overview can be found here Brief Overview of Hornbill Import configuration for LDAP and any existing LDAP Import conf.json files (used in version 2.0) can be migrated using this new interface.

KeySafe and API Keys

There are two additional elements of Hornbill configuration that we'll call upon when completing the configuration:

  1. Create a Hornbill KeySafe entry to securely store the login credentials of your LDAP server.
  2. Create an API key that will be used by the import utility to access your Hornbill instance.

Configuration Summary

The configuration is split into separate tabs and they can be approached in the following order:

  1. LDAP Server: This tab is where the information required to set up the communication link with your directory service (i.e. Active Directory) is stored.
  2. User Account: This tab contains the mapping between the Hornbill database fields and the LDAP user object attributes. i.e. which LDAP attribute will be used to populate the Hornbill database fields. Only some of these fields are required.
  3. User profile: In addition to the fundamental User Account Properties, Hornbill can store much more information about a user.
  4. User Options: This tab is where the ancillary associations are managed e.g Manager, Site, Role, and Group associations.
  5. Advanced Options: Each time the import is run, it outputs a log file. This tab allows you to set some preferences in relation to the log output.
  6. Debug: This tab allows you to view the raw configuration which is being drawn from your selections and entries made in the previous tabs.
  7. >_Command Line: Contains the key information required to execute the import. Enter your instance id and api key to construct the command line.

LDAP Server: Connecting to your Directory Server

This section of the conf file specifies the information required to set up the communication link between your Hornbill instance and your directory service (i.e. Active Directory).

Connecting to your LDAP (Directory) Server
It will be necessary to create a "KeySafe" entry to establish a connection with your directory server
  • "Authentication" - Reference to the Hornbill KeySafe entry that securely stores the username, password, and port used during the connection to your directory server.
  • "Connection Type" - The type of HTTP connection to use when communicating with the directory Server. Normal HTTP, SSL, and TLS are supported.
  • "Allow Insecure Connection" - Used in conjunction with SSL or TLS connection types and allows the verification of SSL Certifications to be disabled i.e. "ON" sets the InsecureSkipVerify variable to "true".
  • "Enable Debug" - Enables LDAP Connection Debugging. This should only ever be enabled to troubleshoot connection issues during the initial setup and testing.


Defining your LDAP Query
Define what the LDAP query will target
  • "Filter" - provides the ability to target specific directory objects. As a minimum `(objectClass=user)` should be specified to ensure the utility only retrieves User Objects. Many online references are available detailing the various possibilities of LDAP filter syntax. Search the web for "LDAP filter syntax".
  • "DSN" - Allows us to specify the Data Source Name. E.g. `DC=test,DC=hornbill,DC=com`. The utility can target a single OU or the entire directory from the root. The "Scope" variable can add flexibility here however it is not possible to target a selection of specific OU's in a single import configuration. If this is required, explore the creation of further import configurations to each target a specific OU.
  • "Scope" - Search Scope (ScopeBaseObject = 0, ScopeSingleLevel = 1, ScopeWholeSubtree = 2) Default is 1.
  • "Dereference Aliases" - Allows us to choose at what stage during the search operation (LDAP Query) aliases are dereferenced. The default value that exists here should be suitable for most.
  • "Size Limit" - Allows you to set a size limit in relation to the result set that can be returned. Setting to '0' disables this setting.
  • "Time Limit" - Allows you to impose a time limit in relation to how long the LDAP query can run before timing out. Setting to '0' disables this setting.
  • "Types Only" - Enabling this setting will cause the query to return attribute types (descriptions) rather than attribute values.



Listing the LDAP Attributes
Determine the LDAP attributes you will pull information from
  • It is necessary to list all the directory attributes that you will be using to populate the Hornbill User account properties and extended user properties. If the directory attribute is not listed here, the import won't be able to include it in the import. Only attributes specified here will be recognised by the import processi.e. Any attributes you specify in any of the subsequent tabs must be listed in this section.


  • The list of attributes that exists by default is not exhaustive and while these are some of the most common attributes used, you may find that your directory contents may differ slightly. Review the Hornbill user account properties and decide which directory attribute will be used to populate that information and then ensure the attribute is listed here.


  • Please note: that "thumbnailPhoto" contains binary data for the thumbnail and is a requirement if you plan to use the ImageLink (outlined in a later section). If you do NOT plan to use the ImageLink, you can leave this out and reduce the load on your AD requests.


  • To add a new attribute to the list, click the green '+' button and type the name of the attribute in the text box that appears. To delete an attribute that you don't require, click the red trash can button alongside the attribute.



User Account: Mapping your Directory Attributes to Hornbill User Account Properties

This section requires us to specify what LDAP attribute will be used to populate the Hornbill database fields.

User Account Mapping
Map directory attributes to user account properties

The directory attributes contained by default should be enough to get you started as they are typically where you would find this information referenced in your directory service. However, it is prudent to check that the attribute specified does actually hold the information in your directory as expected. "User ID" and "User Type" are mandatory pieces of information.

  • Directory attributes can be quickly mapped using the attribute picker located to the right of each field. The contents of the attribute picker are derived from the list of attributes you defined in the previous "LDAP Server" tab.


  • Any value wrapped with [] will be treated as a directory attribute
  • It is possible to construct a value by specifying multiple directory attributes against a Hornbill user property E.g.:
   * "Handle":"[givenName] [sn]", - Both Variables are evaluated from LDAP and set to the Handle (name) param
  • The password field should be left empty as the utility generates a secure password that adheres to the User Password Policy as specified on your Hornbill instance. This password will only be temporary as the user should use the "Forgot Password" link available on the Hornbill Login Screen to reset their password the first time they navigate to your Hornbill instance.
  • "Site" - Recognises a corresponding Hornbill site id. E.g.
   * "Site":"1" - The value of Site should be numeric (PLEASE NOTE If you plan on using the site lookup mechanism described later, this can be left blank).

As an alternative, the import configuration provides a "Site Lookup" section (outlined in a later section) which can make a site association based on the contents of a directory attribute.



User Profile: Mapping your Directory Attributes to Extended Hornbill User Account Properties

In addition to the fundamental User Account Properties, Hornbill can store much more information about a user.

User Profile Mapping
Map directory attributes to extended user account properties
  • Mappings to the available extended Hornbill user account properties works in the same way as the User account mapping above.


  • Directory attributes can be quickly mapped using the attribute picker located to the right of each field. The contents of the attribute picker are derived from the list of attributes you defined in the previous "LDAP Server" tab.


  • All values in this tab are optional and can be left blank if they are not deemed useful.



User Options: Managing Ancilliary associations to a Hornbill user accounts

The "User Options" tab is where the ancillary user associations are managed e.g Manager, Site, Role, and Group associations.

Setting the User Type of a Hornbill User Account

This section of the import configuration allows you to define the type of the user accounts that are created in Hornbill. You may recall that the Hornbill Platform recognises two types of user account which are "Basic User" and a "Full User". The best-practice approach when starting out with the Hornbill LDAP import is to set this value to "Only Create" as this ensures the User Type is set appropriately when new accounts are created but allows you control over when an account should be promoted from Basic to Full, or demoted from Full to Basic.

User Account Status
The user type can be managed via the LDAP user import
  • Action - When to set the User Account Type. On Create, On Update or Both. Setting this to "No Action" disables this section.



Setting the status of a Hornbill User Account

This section of the import configuration allows you to set the status of a Hornbill user account. This section is relevant when you wish to create a configuration file to deal with the archiving of user accounts.

User Account Status
The user status can be managed via the LDAP user import
  • Action - When to set the User Account Status. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Status - A list of available Hornbill user account status values.



Associating a Site to Hornbill User Accounts

The LDAP Import utility has the ability to associate a Hornbill Site record to a user account based on the contents of a directory attribute. This is achieved through a "Look-up". The Look up mechanism is quite simple and works in the following manner.

  1. The import reads the attribute that is specified in the "value" field. In the example shown, the [physicalDeliveryOfficeName] directory attribute is used.
  2. It takes the content and tries to identify if there is an existing site record in Hornbill with a name that matches the value of the LDAP attribute. e.g. if the [physicalDeliveryOfficeName] directory attribute contained "Brussels", the import would look for a Hornbill Site record with the name "Brussels".
  3. If a match is found, the import will associate the user to the site.
  4. If no site record is found, the import will move onto the next user.

i.e. The name of the Site record in Hornbill must match the value of the directory attribute specified. More on Hornbill Sites can be found here: Sites

Site Lookup
Site associations can be made using the LDAP User Import
  • Action - When the import should perform the site lookup. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Value - The directory attribute that contains the site name.



Associating Roles to Hornbill User Accounts

The LDAP import utility also has the ability to associate Hornbill roles to the user accounts it creates or updates.

Roles
Associate roles to users using the import
  • Action - When the import should perform the association of roles. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Roles - This should contain a list of Hornbill User Roles to be associated to a user.
  • To add a new role to the list, click the green '+' button and type the name of the role in the text box that appears. To delete a role that you don't require, click the red trash can button alongside the attribute. Values are case sensitive.



Setting a User's Manager in Hornbill

Hornbill can store a manager relationship between two users in Hornbill. Manager information is typically stored in the directory attribute called "manager" and contains the distinguished name of the corresponding user object. Naturally, Hornbill does not recognise this distinguished name and therefore a regular expression is used to extract the managers friendly name from the more complex distinguished name string. The manager look up mechanism works as follows:

  1. The import reads the contents of the "manager" attribute which will contain the distinguished name. E.g. CN=lastname\, firstname,OU=Users,OU=department,DC=test,DC=hornbill,DC=com
  2. Applying the example regex, CN=(.*?)(?:,[A-Z]+=|$), results in the following string: "lastname\, firstname"
  3. The import is hard-coded to remove the slash and comma that still remain to give: "lastname firstname"
  4. With the "Reverse" option enabled, the above string is reversed to give: "firstname lastname"
  5. The import tries to match this value against an existing Hornbill user by looking up the "Handle" field i.e. h_name.
User Manager Mapping
Associate Manager's to Users
  • Action - When the import should perform a manager look up. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Value - The LDAP Attribute to use for the name of the Manager. Any value wrapped with [] will be treated as an LDAP field
  • Regex - Optional Regex String to Match the Name from an DSN String. The following regex will work for the majority of situations: CN=(.*?)(?:,[A-Z]+=|$)
  • Reverse - Reverse the name String Matched from the Regex.
  • Match Against DN -
  • Search For Manager Id - Lookup Hornbill User Id from Name. The Managers Name matched in the Regex must explicitly match the full name in Hornbill.



Uploading Images to Hornbill User Accounts

The "Image Link" section of the configuration affords the ability to upload images to User profiles in Hornbill. Only jpg/jpeg and png formats are accepted.

Image Link
Upload User Images as part of the LDAP User import
  • Action - When the import should associate an image to a user account. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Upload Type - the type of image upload sequence that will be used.
    • AD - using data stored in AD - set the "URI" as the LDAP field surrounded by square brackets, for example: [thumbnailPhoto].
    • URL - find the image at end of "URI" below - assuming that the URL is visible by our Hornbill servers (eg http://whatever.com/[userPrincipalName].jpg).
    • URI - find the image at end of "URI" below - from a LOCAL server (not fully tested; eg http://localserver/[userPrincipalName].jpg).
  • Image Type - (jpg | png) type of image as stored in AD.
  • URI - referencing the image data.



Associating a Group to Hornbill User Accounts

The LDAP Import has the ability to associate a Hornbill Group to a user account based on the contents of a directory attribute. This is achieved through a "Look-up". The Look up mechanism is quite simple and works in the following manner.

  1. The utility reads the attribute that is specified in the orgLookup section. In the example shown, the [department] directory attribute is used.
  2. It takes the content and tries to identify if there is a Hornbill Group that exists with a name that matches the value of the LDAP attribute. e.g. if the [department] directory attribute contained "Accounting", the utility would look for a Hornbill Group called "Accounting".
  3. If a match is found, the import will associate the user to the group.
  4. If no Hornbill organisation is found, the import will move onto the next user.

i.e. The name of the Organization(Group) in Hornbill must match the value of the attribute in LDAP. More on Hornbill Organisational Groups can be found here: Organisation Structure (Groups)

Organisation (Group Lookup)
Associate users to Groups defined in your Organisation structure
  • Action - When the utility should perform the OrgLookup. On Create, On Update or Both. Setting this to "No Action" disables this section.
  • Value - The directory attribute that contains the company or division or department name to use for the Hornbill Group Look-up. Any value wrapped with [] will be treated as an directory attribute.
  • Member Of - This field makes reference to the "memberOf" attribute of the user object in your directory. When a distinguished name (DN) of an LDAP group is specified in this field, the utility will only allow the group association in Hornbill if the group DN exists in the "memberOf" attribute of the user object being processed. i.e. only allow the Hornbill group association if the user object is a member of this directory group.
  • Type - The Hornbill Group type (General, Team, Department, Costcenter, Division, Company). This focuses the import to only perform the look-up on Groups of a particular type (this has no relevance to the contents of your directory or the directory attribute that is used.
  • Membership - The Hornbill Group Membership the users will be added with. Select from the drop-down list.
  • Can View Tasks - If set true, then the user can view tasks assigned to this group (Typically only required for Groups of type "Team").
  • Can Action Tasks - If set true, then the user can action tasks assigned to this group (Typically only required for Groups of type "Team").
  • Only One Group Assignment - When set to "ON", a user can only be associated to a single group at any one time. i.e. if in the latest import the configuration was set to associate users to the Accounting department, the users would be associated to this new department and removed from any other group that they were already associated with.
  • Set As Home Organisation - Only visible when the Type is Company. When set to "ON", the Company will be set as the users Home Organisation. Requires version 3.3.0 or greater of the import tool, and is available in build 1077 and above of the Hornbill Admin Tool

Please Note - A successful association of a user to a Group is dependant upon the import finding a Hornbill Group with a name that matches the contents of the specified user attribute in your directory.

Users can be associated to more than one group during the same import. Click the blue '+' button to add another group.



Preparing to Run the Import

Ultimately, the executable will be scheduled in Windows task scheduler (see later) but to test, gain confidence, and perform the initial upload of users the utility can be executed from a command prompt window on an ad-hoc basis. The command used to execute the import contains a number of command line parameters. Hornbill Administration provides an interface which will construct the command line parameters for you. This is found in the ">_Command Line" tab.

>_Command Line
'
  • config - the id of the configuration in Hornbill Administration i.e. this is the name of the configuration you have created with the spaces replaced with hyphens.
  • dryrun - When dryrun is enabled, no Data Changes are made.
  • instanceid - This is the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". Remember, this value is also case sensitive.
  • apikey - A Valid API Assigned to a user with enough rights to process the import.
  • apitimeout - Number of seconds to timeout the connection attempt with your Hornbill instance.
  • workers - Number of concurrent threads used to process users. Increasing the number of threads will decrease the amount of time required for the import to run. However, more threads will place an increased load on your instance.
  • forcerun - If a previous run did not successfully complete you can specify forcerun=true to force the next run to bypass the existing running job check
  • logprefix - the prefix to add to the logfile that will be generated locally.
  • version - Return the current version of the utility.



Testing Overview

There is no substitute for hands-on experience when becoming familiar with the Hornbill import utilities.
The LDAP import accepts and understands a number of "Command Line Parameters" that can be used when running the utility from the command line. The most important one for testing is the -dryrun=true command. When this is specified, no information will be written to Hornbill and it allows you to confirm that the configuration is correct and that a connection to your directory server can be established. A dryrun still outputs a log file which provides you with an opportunity to review and understand any error messages that may occur.

Suggested Testing Approach

Below are some high level steps to help you build confidence in your configuration:

  1. In the configuration, specify an LDAP filter to target a single user object. (Its good practice to initially test on a single, or small set of, user objects as this allows the dryruns to complete quicker and there is less log content to sift through).
  2. Perform a dryrun (by executing the utility along with the -dryrun=true command line parameter).
  3. Review cmd output and log file for errors
  4. Check against "Common Error Messages" listed on the wiki and take action to rectify where necessary.
  5. Continue with dryrun tests until you are happy that all the errors are accounted for.
  6. Perform a live import with this single user object still specified i.e. set -dryrun=false
  7. Review user account in Hornbill and check all user properties are as expected i.e. email contains an email address etc.
  8. Adjust conf file user property mappings as necessary
  9. Loop through steps 6 - 8 as many times as is necessary until you are happy with the information being transported into the Hornbill user account properties.
  10. Amend the LDAP filter to target the user objects required for a full import.
  11. Perform a dryrun
  12. Review cmd output and log file for errors
  13. Check against "Common Error Messages" listed on the wiki and take action to rectify where necessary.
  14. Continue with dryrun tests until you are happy that all the errors are accounted for.


Example command line constructed by Hornbill administration in the "_>Command Line" tab: LDAP_User_Import.exe -config=hornbill-ad-import-for-new-and-active-users -dryrun=true -instanceid=myinstancename -apikey=2380245fb4dxxxxxxxxxxxxxxe13df9c

What Hornbill Roles are needed for the Import to Complete Successfully?

A default role is delivered with Hornbill that is designed to be used in conjunction with our range of user import utilities. The security role is called User Import and has all the necessary rights to import / update user properties.

As you may now be aware, every action within Hornbill must be performed in the context of a user account. As well as the chosen user account possessing the "user Import" role which facilitates the importing of the user accounts and updating of the user properties, this user account must posses the roles that you are associating to imported user accounts via the import utility. The above comment about roles is referring to Hornbill's security model when it comes to associating roles to user accounts, which is: Hornbill is designed to only allow the association of roles if the User who is performing the assignment of a particular role already possess the same system/application rights among the roles that they themselves possess. This security measure prevents you inflating your own rights or giving a user more rights than you have yourself.

i.e. in addition to the "User Import" role, any roles you try and assign to the user accounts being imported must be assigned to the user account logging in and running the import.

API Key Rules

This utility uses (API keys):

  • activity:profileImageSet
  • admin:sysOptionGet
  • admin:keysafeGetKey
  • admin:userAddGroup
  • admin:userAddRole
  • admin:userCreate
  • admin:userDeleteGroup
  • admin:userProfileSet
  • admin:userSetAccountStatus
  • admin:userUpdate
  • data:entityAddRecord
  • data:entityGetRecord
  • data:entityUpdateRecord
  • data:queryExec
  • session:getSystemLicenseInfo

Troubleshooting

Logging Overview

All Logging output is saved in the "log" directory which can be found in the same location as the executable. The file name contains the date and time the import was run LDAP_User_Import_2015-11-06T14-26-13Z.log There are 4 levels of logging available

  • Error
  • Warning
  • Message
  • Debug

with Error being the lowest and Debug giving the most detail.

These are set on the Advanced Options tab of the Import Configuration

Common Error Messages

Below are some common errors that you may encounter in the log file and what they mean:

  • [ERROR] Error Decoding Configuration File:..... - this will be typically due to a missing quote (") or comma (,) somewhere in the configuration file. This is where an online JSON viewer/validator can come in handy rather than trawling the conf file looking for that proverbial needle in a haystack.
  • [ERROR] Get https:// api.github.com/repos/hornbill/goLDAPUserImport/tags: dial tcp xx.xx.xx.xx:xxx: ........ - this most likely indicates that you have a HTTP proxy server on your network between the host running the executable and your Hornbill API endpoint. Ensure the http_proxy environment variable is set (See the section on "HTTP Proxies" for more information) and that the proxy is configured to allow this communication.
  • [ERROR] Connection Error: dial tcp xx.xxx.xx.xx:389: conectex: a connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. - A connection error containing an IP address with port "389" indicates that the utility had a problem connecting to the LDAP server. i.e. the server could not be reached or did not respond. This could be due to incorrect LDAP login details being specified, check what you have stored in the KeySafe and that the correct KeySafe entry is specified in the Data Import Configuration.
  • [ERROR] Connecting Error: dial tcp xxx.xxx.xxx.xxx:389: connectex: No connection could be made because the target machine actively refused it. - This error indicates that the utility is unable to connect to the LDAP server because the LDAP server refused the connection for some reason. This could be due to; incorrect username and password, incorrect permissions on the windows user account accessing the machine/running the LDAP query, the wrong connection type/port is specified i.e. requires SSL/TLS on a secure port rather than HTTP on 389.
  • [ERROR] Search Error: LDAP Result Code 211 "ErrorClosing": Response Channel Closed - this usually indicates the port (specified in the Key Safe) and Connection Type (specified in the import configuration) are mismatched. E.g. If using TLS, this would typically be port 389. SSL would typically be associated with port 636.
  • [ERROR] Unable to Create User: Invalid value for parameter '[parameter name]': The text size provided (31 characters) is greater than the maximum allowable size of 20 characters for column [column name] - the contents of your directory attribute exceed the maximum number of characters that can be placed in the Hornbill database column.
  • [ERROR] Unable to Create User: The value in element <userId> did not meet the required input pattern constraints. at location '/methodCall/params/userId' - the user id contains characters that are not allowed. The User Id should be made up of alphanumeric characters. Full stops (.) and underscores (_) are also supported.
  • [ERROR] Unable to Create User: [usedID] Error: The specified handle [Display Name] is already in use - By default, the "Handle" (Hornbill Display Name) must be unique. This error suggests a user account already exists in Hornbill which is using this handle. The duplicate-handle validation can be disabled via a setting found in Hornbill Adminsitration under "Home > System > Advanced Settings" and filtering for "api.xmlmc.uniqueUserHandle.enable"
  • [ERROR] Unable to Update User: Invalid value for parameter '[parameter name]': Error setting value for column '[column name]'. bad lexical cast: source type value could not be interpreted as target - this error is indicating that the contents of your directory attribute are in a format that is not compatible with the type of the Hornbill database column. For example, you will get this when trying to place text into a database field that is of type "INT" (accepts integer values only).
  • [ERROR] Unable to Load LDAP Attribute: '[LDAP attribute name]' For Input Param: '[Hornbill Parameter name]' - When the import utility is unable to load a particular LDAP attribute, this means that the attribute field in your directory does not contain a value. This error will not prevent the user account being created or updated in Hornbill and can be considered more as a warning rather than an outright failure or problem.
  • [ERROR] Unable to Assign Role to User: You cannot create or update the role as you do not have sufficient permissions to set the system rights. - every action within Hornbill must be performed in the context of a user account. This error is suggesting that the user account you are using to run the utility does not posses the appropriate Hornbill roles set to associate the necessary roles the imported user. See the Testing section above, specifically "What Hornbill Roles are needed for the Import to Complete Successfully?".
  • [ERROR] Unable to run import, a previous import is still running - this can occur if the previous import failed to complete. Perform a manual (non-scheduled) run of the import from the command line including the argument "forcerun=true". Future imports should now run without issue.

Error Codes

  • 100 - Unable to create log File
  • 101 - Unable to create log folder
  • 102 - Unable to Load Configuration File

Scheduling Overview

Windows

You can schedule ldap_import.exe to run with any optional command line argument from Windows Task Scheduler.

Scheduling the Utility with Windows Task Scheduler
  • Ensure the user account running the task has rights to ldap_import.exe and the containing folder.
  • Make sure the Start In parameter contains the folder where ldap_import.exe resides in otherwise it will not be able to pick up the correct path.