Difference between revisions of "Asset Data Import Tool"

From Hornbill
Jump to navigation Jump to search
 
(34 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 +
This document can now be found at its new location in the [https://docs.hornbill.com/data-imports-guide/assets/overview Hornbill Document Library].
 +
 +
[[file:hornbill-document-library.png|Data Imports Guide|link=https://docs.hornbill.com/data-imports-guide/assets/overview]]
 +
 +
<!--
 +
 +
 
=About the Hornbill Asset Import Utility=
 
=About the Hornbill Asset Import Utility=
  
Line 5: Line 12:
 
==Open Source==
 
==Open Source==
  
The Asset Import Utility is provided open source under the [https://wiki.hornbill.com/index.php/The_Hornbill_Community_License_(HCL) Hornbill Community Licence] and can be found [https://github.com/hornbill/goDBAssetImport Here] on GitHub
+
The Azure User Import Utility is no longer provided open-source, and is maintained solely by Hornbill as a closed-source, but free-to-use, tool.
 
 
===History===
 
This tool is an evolution the previous [[Special:Permalink/26928|Database Asset Import v1.*]], combining the [[Special:Permalink/26716|CSV Asset Import v1.*]] tool and a revamp of the templating system.  
 
  
 
==Installation Overview==
 
==Installation Overview==
Line 14: Line 18:
 
===Windows Installation===
 
===Windows Installation===
  
* Download the [https://github.com/hornbill/goDBAssetImport/releases/latest ZIP archive] relevant to your OS and architecture
+
* Download the [https://github.com/hornbill/asset-import/releases/latest ZIP archive] relevant to your OS and architecture
 
* Extract zip into a folder you would like the application to run from e.g. '''C:\Asset_Import\'''
 
* Extract zip into a folder you would like the application to run from e.g. '''C:\Asset_Import\'''
 
* Open the example configuration file relevant to your environment, and add in the necessary configuration
 
* Open the example configuration file relevant to your environment, and add in the necessary configuration
Line 22: Line 26:
 
For Windows Systems:  
 
For Windows Systems:  
 
<code>
 
<code>
goDBAssetImport.exe -dryrun=true -file=conf_example_db_sccm.json
+
asset_import.exe -dryrun=true -file=conf_example_db_sccm.json
</code>
 
 
 
For Mac OSX and Linux Systems:
 
<code>
 
./goDBAssetImport -dryrun=true -file=conf_example_db_sccm.json
 
 
</code>
 
</code>
  
Line 36: Line 35:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 191: Line 188:
  
 
==InstanceConfig==
 
==InstanceConfig==
* <code>APIKey</code> - a Hornbill API key for a user account with the correct permissions to carry out all of the required API calls. Details on how to create an API key can be found [https://wiki.hornbill.com/index.php?title=API_keys '''here'''].
 
* <code>InstanceID</code> - the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". '''This value is case sensitive'''.
 
 
* <code>KeysafeKeyID</code> - The ID of the Keysafe Key that holds your data source credentials. Set to 0 for importing directly from CSV files. Supported Keysafe key types are:
 
* <code>KeysafeKeyID</code> - The ID of the Keysafe Key that holds your data source credentials. Set to 0 for importing directly from CSV files. Supported Keysafe key types are:
 
** <code>Database Authentication</code>
 
** <code>Database Authentication</code>
 
** <code>Username + Password</code>
 
** <code>Username + Password</code>
 
** <code>LDAP Authentication</code>
 
** <code>LDAP Authentication</code>
* <code>LogSizeBytes</code> - The maximum size that the generated Log Files should be, in bytes. Setting this value to 0 will cause the tool to create one log file only and not split the results between multiple logs.
+
** <code>Google Workspace</code>
* <code>HornbillUserIDColumn</code> - used to specify the Hornbill User ID column for matching users - options: h_user_id (default), h_employee_id, h_email, h_name, h_attrib_1 & h_login_id. '''Please note:''' ''last logged on'', ''owned by'' and ''used by'' will use the same field - i.e. one can NOT specify which column to match to individually.
+
* <code>LogSizeBytes</code> - The maximum size that the generated Log Files should be in bytes. Setting this value to 0 will cause the tool to create one log file and not split the results between multiple logs.
 +
* <code>HornbillUserIDColumn</code> - used to specify the Hornbill User ID column for matching users - options: h_user_id (default), h_employee_id, h_email, h_name, h_attrib1, h_attrib8 & h_login_id. '''Please note:''' ''last logged on'', ''owned by'' and ''used by'' will use the same field - i.e. one can NOT specify which column to match to individually.
  
 
== SourceConfig ==
 
== SourceConfig ==
Line 211: Line 207:
 
** <code>ldap</code> - LDAP / Active Directory - will use Keysafe key type '''LDAP Authentication'''
 
** <code>ldap</code> - LDAP / Active Directory - will use Keysafe key type '''LDAP Authentication'''
 
** <code>google</code> - Google Workspace Enterprise Chrome OS - will use Keysafe key type '''Google Workspace'''
 
** <code>google</code> - Google Workspace Enterprise Chrome OS - will use Keysafe key type '''Google Workspace'''
 +
** <code>certero</code> - Certero - will use Keysafe key type '''Certero'''
 +
** <code>workspaceone</code> - vmware Workspace One UEM - will use Keysafe type '''VMWare Workspace One UEM'''
 
* <code>CSV</code> - Only in use if '''Source''' is set to '''csv'''
 
* <code>CSV</code> - Only in use if '''Source''' is set to '''csv'''
 
** <code>CarriageReturnRemoval</code> - Certain CSV exporting systems will add extra carriage returns as a record delimiter. This is expected not to be common, hence the setting is left out of the configuration files (it is added to conf_computerSystem.json only for completeness sake). IF not set, then the default value is false and no carriages returns will be stripped from the data. IF set to true, then all carriage returns (possibly even intended ones) will be stripped.
 
** <code>CarriageReturnRemoval</code> - Certain CSV exporting systems will add extra carriage returns as a record delimiter. This is expected not to be common, hence the setting is left out of the configuration files (it is added to conf_computerSystem.json only for completeness sake). IF not set, then the default value is false and no carriages returns will be stripped from the data. IF set to true, then all carriage returns (possibly even intended ones) will be stripped.
Line 236: Line 234:
 
* <code>Google</code> - Only in use if '''Source''' is set to '''google'''
 
* <code>Google</code> - Only in use if '''Source''' is set to '''google'''
 
** <code>Customer</code> - The Customer ID of the Google Account. Supports my_customer to return devices enrolled to the deafult account that the KeySafe key was created with
 
** <code>Customer</code> - The Customer ID of the Google Account. Supports my_customer to return devices enrolled to the deafult account that the KeySafe key was created with
** <code>Query</code> - Search string as per the [[https://developers.google.com/admin-sdk/directory/reference/rest/v1/chromeosdevices/list Google API Documentation]]
+
** <code>Query</code> - Search string as per the [https://developers.google.com/admin-sdk/directory/reference/rest/v1/chromeosdevices/list Google API Documentation]
 
** <code>OrgUnitPath</code> - The full path of the Google Organisational Unit where the devices reside
 
** <code>OrgUnitPath</code> - The full path of the Google Organisational Unit where the devices reside
 +
* <code>Certero</code> - Only in use if '''Source''' is set to '''google'''
 +
** <code>Expand</code> - The Certero oData query which defines the columns and associated entitiy records that can be mapped into Hornbill asset records
  
 
==AssetTypes==
 
==AssetTypes==
An array of objects details the asset types to import:
+
An array of objects details the asset types to import.
 +
 
 +
'''NOTE''' - As of v1.16.0, during the import process assets of each type as defined below are retrieved from Hornbill and cached locally in memory to aid in the matching of existing asset records for update. This is to improve the performance of the import process, as well as to significantly reduce load on your Hornbill instance when the imports are running. If you have imported asset records into one particular Hornbill asset type, then re-import the same asset records from the source into a different Hornbill asset type, then you may see duplicate asset records being created. We suggest you run the tool against your configuration in dryrun mode and check that no duplicates would be created, and if they are then you need to update your import configuration accordingly. See AssetType, below, for one such method of avoiding duplicates.
 +
   
 
* <code>AssetType</code> - the Asset Type Name which needs to match a correct Asset Type Name in your Hornbill Instance, OR if '''OperationType''' below is set to '''Update''', and you wish to update assets of ALL types of a specific class, then you can populate this property with <code>__all__:computer</code>, where <code>computer</code> is the asset class to update. Valid classes are:
 
* <code>AssetType</code> - the Asset Type Name which needs to match a correct Asset Type Name in your Hornbill Instance, OR if '''OperationType''' below is set to '''Update''', and you wish to update assets of ALL types of a specific class, then you can populate this property with <code>__all__:computer</code>, where <code>computer</code> is the asset class to update. Valid classes are:
 
*** <code>basic</code>
 
*** <code>basic</code>
 
*** <code>computer</code>
 
*** <code>computer</code>
 
*** <code>computerPeripheral</code>
 
*** <code>computerPeripheral</code>
 +
*** <code>dataProcessingRecord</code>
 
*** <code>mobileDevice</code>
 
*** <code>mobileDevice</code>
 
*** <code>networkDevice</code>
 
*** <code>networkDevice</code>
 
*** <code>printer</code>
 
*** <code>printer</code>
 
*** <code>software</code>
 
*** <code>software</code>
 +
*** <code>system</code>
 
*** <code>telecoms</code>
 
*** <code>telecoms</code>
 
* <code>OperationType</code> - The type of operation that should be performed on discovered assets - can be Create, Update or Both. Defaults to Both if no value is provided
 
* <code>OperationType</code> - The type of operation that should be performed on discovered assets - can be Create, Update or Both. Defaults to Both if no value is provided
 
* <code>CSVFile</code> - The path to the CSV file to use as the data source for this specific asset type. Only required when '''SourceConfig > Source''' is set to '''CSV'''
 
* <code>CSVFile</code> - The path to the CSV file to use as the data source for this specific asset type. Only required when '''SourceConfig > Source''' is set to '''CSV'''
 +
* <code>CSVFilters</code> - An array of objects, containing zero or more filters that should be applied when working out which rows from the CSV file should be included in the asset type import. Only required when '''SourceConfig > Source''' is set to '''CSV''', and this is non-mandatory. If no filters are provided, then all records will be imported from the CSV into the target asset type.
 +
** <code>Column</code> - The column name from the CSV that the filter should apply to
 +
** <code>Operator</code> - The operator that should be used when applying the filter. Valid values are:
 +
*** <code>EQUALS</code> - Does the filter Value (below) exactly match the column value
 +
*** <code>CONTAINS</code> - Is the filter Value (below) contained within the column value (wildcard type functionality, *column_value*)
 +
*** <code>IN</code> - Does the column value have an exact match in an array of strings supplied in <code>Value</code>, below
 +
** <code>Value</code> - The value to be applied in the filter
 
* <code>PreserveShared</code> - If set to true, when updating assets that are Shared, then the Used By fields will not be updated. Defaults to false
 
* <code>PreserveShared</code> - If set to true, when updating assets that are Shared, then the Used By fields will not be updated. Defaults to false
 
* <code>PreserveState</code> - If set to true then the State field will not be updated. Defaults to false
 
* <code>PreserveState</code> - If set to true then the State field will not be updated. Defaults to false
Line 261: Line 273:
 
** <code>mobile</code>
 
** <code>mobile</code>
 
* <code>LDAPDSN</code> - The Distinguished Name of the LDAP container to execute your query in. Only required when '''SourceConfig > Source''' is set to '''LDAP'''
 
* <code>LDAPDSN</code> - The Distinguished Name of the LDAP container to execute your query in. Only required when '''SourceConfig > Source''' is set to '''LDAP'''
* <code>Query</code> - either:
+
* <code>Query</code> - Not used for Workspace One or direct CSV file imports. For the other data sources:
 
** Additional SQL clauses to be appended to the Query from SourceConfig > Database > Query, to retrieve assets of that asset type.  
 
** Additional SQL clauses to be appended to the Query from SourceConfig > Database > Query, to retrieve assets of that asset type.  
** Nexthink wuery for returning asset details for that asset type
+
** Certero oData filter for returning asset details for that asset type
 +
** Nexthink query for returning asset details for that asset type
 
** LDAP query for returning asset details for that asset type
 
** LDAP query for returning asset details for that asset type
 +
* <code>Filter</code> - Used exclusively for applying filters to Workspace One queries, all are optional:
 +
** <code>User</code> - Username the device enrolled under
 +
** <code>ModelIdentifier</code> - Partial search by device model. Search by MD20 would return device with model MD200LL
 +
** <code>DevicePlatformType</code> - Device platform type, i.e. Apple, Android, WindowsPC, etc.
 +
** <code>Ownership</code> - Device ownership type i.e. Corporate, Employee, Shared
 +
** <code>OrganizationGroupUUID</code> - The UUID of the Organization Group that the devices are members of. Defaults to the keysafe key user's OrganizationGroup
 +
** <code>ComplianceStatus</code> - The compliance status of the devices i.e. Compliant, NonCompliant etc
 +
** <code>SeenSince</code> - Specifies the datetime filter for device search, which retrieves the devices that are seen after this datetime stamp.
 
* <code>AssetIdentifier</code> - an object containing details to help in the identification of existing asset records in the Hornbill instance. If value in an imported records DBColumn matches the value in the EntityColumn of an asset in Hornbill (within the defined Entity), then the asset record will be updated rather than a new asset being created:
 
* <code>AssetIdentifier</code> - an object containing details to help in the identification of existing asset records in the Hornbill instance. If value in an imported records DBColumn matches the value in the EntityColumn of an asset in Hornbill (within the defined Entity), then the asset record will be updated rather than a new asset being created:
** <code>SourceColumn</code> - specifies the unique identifier column from the source data query
+
** <code>SourceColumn</code> - specifies the unique identifier column from the source data query. This can be either a column name, or a Go Template. NOTE: Importing from Certero requires this to be a Go template.
** <code>Entity</code> - the Hornbill entity where data is stored
+
** <code>Entity</code> - the Hornbill entity where data is stored, one of:
** <code>EntityColumn</code> - specifies the unique identifier column from the Hornbill entity
+
*** <code>Asset</code>
 +
*** <code>AssetsBasic</code>
 +
*** <code>AssetsComputer</code>
 +
*** <code>AssetsComputerPeripheral</code>
 +
*** <code>AssetsDataProcessingRecord</code>
 +
*** <code>AssetsMobileDevice</code>
 +
*** <code>AssetsNetworkDevice</code>
 +
*** <code>AssetsPrinter</code>
 +
*** <code>AssetsSoftware</code>
 +
*** <code>AssetsSystem</code>
 +
*** <code>AssetsTelecoms</code>
 +
** <code>EntityColumn</code> - specifies the unique identifier column from the Hornbill entity specified above
 
** <code>SourceContractColumn</code> - specifies the unique identifier column from the database query - the data in this column of the query result should be the Contract ID within Hornbill (eg C20200700001 from /suppliermanager/suppliercontract/view/C20200700001/; <code>'C20200700001' AS Contract</code> hardcoded in SQL)
 
** <code>SourceContractColumn</code> - specifies the unique identifier column from the database query - the data in this column of the query result should be the Contract ID within Hornbill (eg C20200700001 from /suppliermanager/suppliercontract/view/C20200700001/; <code>'C20200700001' AS Contract</code> hardcoded in SQL)
 
** <code>SourceSupplierColumn</code> - specifies the unique identifier column from the database query - the data in this column of the query result should be the Supplier ID within Hornbill (eg ### from suppliermanager/supplier/view/###/)
 
** <code>SourceSupplierColumn</code> - specifies the unique identifier column from the database query - the data in this column of the query result should be the Supplier ID within Hornbill (eg ### from suppliermanager/supplier/view/###/)
 
** <code>SoftwareInventory</code> - an object containing details pertaining to the import of software inventory records for the specified asset type:
 
** <code>SoftwareInventory</code> - an object containing details pertaining to the import of software inventory records for the specified asset type:
*** <code>AssetIDColumn</code> - the column from the asset type query that contains its primary key
+
*** <code>AssetIDColumn</code> - the column from the asset type query that contains its primary key. This can be either a column name, or a Go Template. NOTE: this is ignored when performing imports from Workspace One.
*** <code>AppIDColumn</code> - the column from the Software Inventory that holds the software unique ID. NOTE: this is ignored when performing imports from CSV, LDAP or Nexthink - the App ID to match is a concatenation of the publisher, name and version fields (no spaces between them)
+
*** <code>AppIDColumn</code> - the column from the Software Inventory that holds the software unique ID. This can be either a column name, or a Go Template. NOTE: this is ignored when performing imports from Certero, CSV, LDAP or Nexthink - the App ID to match is a concatenation of the publisher, name and version fields (no spaces between them)
*** <code>Query</code> - the query that will be run per asset, to return its software inventory records. <nowiki>{{AssetID}}</nowiki> ('''Please note:''' this is NOT being processed as a template (note that the absence of the full stop)) in the query will be replaced by each assets primary key value, whose column is defined in the AssetIDColumn property
+
*** <code>Query</code> - the query that will be run per asset, to return its software inventory records. <nowiki>{{AssetID}}</nowiki> ('''Please note:''' this is NOT being processed as a template (note that the absence of the full stop)) in the query will be replaced by each assets primary key value, whose column is defined in the AssetIDColumn property. NOTE: Not used when importing assets from Certero or Workspace One.
 
*** <code>Mapping</code> - maps data into the software inventory records
 
*** <code>Mapping</code> - maps data into the software inventory records
  
Line 299: Line 331:
 
* <code><nowiki>{{.columnName | epoch}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field
 
* <code><nowiki>{{.columnName | epoch}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field
 
* <code><nowiki>{{.columnName | epoch_clear}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field. Defaulting to CLEAR the column if unable to convert.
 
* <code><nowiki>{{.columnName | epoch_clear}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field. Defaulting to CLEAR the column if unable to convert.
* <code><nowiki>{{.columnName | date_conversion "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 1st 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. '''Please note''' that IF your formatting is already in the Hornbill date time format (2006-01-02 15:04:05), you don't need to convert anything.
+
* <code><nowiki>{{.columnName | date_conversion "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 2nd 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. '''Please note''' that IF your formatting is already in the Hornbill date time format (2006-01-02 15:04:05), you don't need to convert anything.
* <code><nowiki>{{.columnName | date_conversion_clear "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 1st 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. Defaulting to CLEAR the column if unable to convert.
+
* <code><nowiki>{{.columnName | date_conversion_clear "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 2nd 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. Defaulting to CLEAR the column if unable to convert.
  
 
==AssetTypeFieldMapping==
 
==AssetTypeFieldMapping==
Line 310: Line 342:
 
==SCCM 2007 & 2012==
 
==SCCM 2007 & 2012==
  
The following is an example of the SQLConf, AssetTypes and data mapping configuration that could be used to import computer-type assets from an SCCM 2007 or 2012 data source. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations SCCM asset data. We highly recommend that a DBA review the SQL clauses against your SCCM database prior to using this in a production environment.
+
The following is an example of the SourceConfig, AssetTypes and data mapping configuration that could be used to import computer-type assets from an SCCM 2007 or 2012 data source. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations SCCM asset data. We highly recommend that a DBA review the SQL clauses against your SCCM database prior to using this in a production environment.
  
 
In this example:
 
In this example:
Line 319: Line 351:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 480: Line 510:
 
==Lansweeper 7.1==
 
==Lansweeper 7.1==
  
The following is an example of the SQLConf, AssetTypes and data mapping configuration that could be used to import computer-type assets from a Lansweeper 7.1 data source. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations LanSweeper asset data. We highly recommend that a DBA review the SQL clauses against your LanSweeper database prior to using this in a production environment.
+
The following is an example of the SourceConfig, AssetTypes and data mapping configuration that could be used to import computer-type assets from a Lansweeper 7.1 data source. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations LanSweeper asset data. We highly recommend that a DBA review the SQL clauses against your LanSweeper database prior to using this in a production environment.
  
 
In this example:
 
In this example:
Line 488: Line 518:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 569: Line 597:
 
==Nexthink==
 
==Nexthink==
  
The following is an example of the SQLConf, AssetTypes and data mapping configuration that could be used to import computer-type assets from Nexthink. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations Nexthink asset data. We highly recommend that a Nexthink administrator review the query and mapping against your Nexthink account prior to using this in a production environment.
+
The following is an example of the SourceConfig, AssetTypes and data mapping configuration that could be used to import computer-type assets from Nexthink. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations Nexthink asset data. We highly recommend that a Nexthink administrator review the query and mapping against your Nexthink account prior to using this in a production environment.
  
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 595: Line 621:
 
             "SoftwareInventory": {
 
             "SoftwareInventory": {
 
                 "AssetIDColumn": "name",
 
                 "AssetIDColumn": "name",
                "AppIDColumn": "",
 
 
                 "Query": "(select ((package (id publisher first_installation name program version)))(from (device package) (with package(where device (eq id (identifier {{AssetID}})))(where package(eq type (enum 'program'))(eq status (enum 'installed')))))(limit 20000))",
 
                 "Query": "(select ((package (id publisher first_installation name program version)))(from (device package) (with package(where device (eq id (identifier {{AssetID}})))(where package(eq type (enum 'program'))(eq status (enum 'installed')))))(limit 20000))",
 
                 "Mapping": {
 
                 "Mapping": {
Line 635: Line 660:
 
         "h_bios_serial_number": "{{.bios_serial_number}}",
 
         "h_bios_serial_number": "{{.bios_serial_number}}",
 
         "h_net_name": "{{.distinguished_name}}"
 
         "h_net_name": "{{.distinguished_name}}"
 +
    }
 +
}
 +
</pre>
 +
 +
==Certero==
 +
 +
The following is an example of the SourceConfig, AssetTypes and data mapping configuration that could be used to import computer-type assets from Certero. '''NOTE''': The configuration example is provided as-is, and may not be suitable to import your organisations Certero asset data. We highly recommend that a Certero administrator review the filter, expand column query and mappings against your Certero account prior to using this in a production environment.
 +
 +
<pre>
 +
{
 +
    "KeysafeKeyID": 0,
 +
    "LogSizeBytes": 1000000,
 +
    "HornbillUserIDColumn": "h_user_id",
 +
    "SourceConfig": {
 +
        "Source": "certero",
 +
        "Certero": {
 +
            "Expand": "ComputerSystemInventory($select=ComputerSystemObjectId,ComputerDomain,ComputerName,IpAddress,MacAddress,SubnetMask,Manufacturer,Model,Username;$expand=OperatingSystem($select=Caption,VersionString)),ComputerSystemProcessorInfo($select=ClockSpeed,Cores,CoresPerCpu,Manufacturer,Model),WindowsSystemSoftwareProducts($select=SoftwareProductObjectId,WindowsSystemObjectId,InstallDate,SupportUrl,UninstallString,SoftwareProduct;$expand=SoftwareProduct($select=SoftwareProductObjectId,DisplayName,ProductCode,Publisher,Version))"
 +
        }
 +
    },
 +
    "AssetTypes": [{
 +
        "AssetType": "Server",
 +
        "OperationType": "Both",
 +
        "PreserveShared": false,
 +
        "Query": "startswith(ComputerSystemInventory/OperatingSystem/Caption,'Windows Server')",
 +
        "AssetIdentifier": {
 +
            "SourceColumn": "{{.ComputerSystemInventory.ComputerName}}",
 +
            "Entity": "Asset",
 +
            "EntityColumn": "h_name"
 +
        },
 +
        "SoftwareInventory": {
 +
            "AppIDColumn": "{{.SoftwareProduct.ProductCode}}",
 +
            "ParentObject": "WindowsSystemSoftwareProducts",
 +
            "Mapping": {
 +
                "h_app_id":"{{.SoftwareProduct.ProductCode}}",
 +
                "h_app_name": "{{.SoftwareProduct.DisplayName}}",
 +
                "h_app_vendor":"{{.SoftwareProduct.Publisher}}",
 +
                "h_app_version":"{{.SoftwareProduct.Version}}",
 +
                "h_app_install_date":"{{.InstallDate}}",
 +
                "h_app_help":"{{.SupportUrl}}"
 +
               
 +
            }
 +
        }
 +
    },
 +
    {
 +
        "AssetType": "Desktop",
 +
        "OperationType": "Both",
 +
        "PreserveShared": false,
 +
        "Query": "contains(ComputerSystemInventory/OperatingSystem/Caption,'Windows') and contains(ComputerSystemInventory/OperatingSystem/Caption,'Server') eq false",
 +
        "AssetIdentifier": {
 +
            "SourceColumn": "{{.ComputerSystemInventory.ComputerName}}",
 +
            "Entity": "Asset",
 +
            "EntityColumn": "h_name"
 +
        },
 +
        "SoftwareInventory": {
 +
            "AppIDColumn": "{{.SoftwareProduct.ProductCode}}",
 +
            "ParentObject": "WindowsSystemSoftwareProducts",
 +
            "Mapping": {
 +
                "h_app_id":"{{.SoftwareProduct.ProductCode}}",
 +
                "h_app_name": "{{.SoftwareProduct.DisplayName}}",
 +
                "h_app_vendor":"{{.SoftwareProduct.Publisher}}",
 +
                "h_app_version":"{{.SoftwareProduct.Version}}",
 +
                "h_app_install_date":"{{.InstallDate}}",
 +
                "h_app_help":"{{.SupportUrl}}"
 +
            }
 +
        }
 +
    }],
 +
    "AssetGenericFieldMapping": {
 +
        "h_name": "{{.ComputerSystemInventory.ComputerName}}",
 +
        "h_asset_tag": "{{.ComputerSystemInventory.ComputerName}}",
 +
        "h_description": "From Certero: {{.ComputerSystemInventory.Manufacturer}} ({{.ComputerSystemInventory.Model}})",
 +
        "h_external_id": "{{.ComputerSystemObjectId}}",
 +
        "h_external_source": "Certero"
 +
    },
 +
    "AssetTypeFieldMapping": {
 +
        "h_name": "{{.ComputerSystemInventory.ComputerName}}",
 +
        "h_mac_address": "{{.ComputerSystemInventory.MacAddress}}",
 +
        "h_net_ip_address": "{{.ComputerSystemInventory.IpAddress}}",
 +
        "h_net_computer_name": "{{.ComputerSystemInventory.ComputerName}}",
 +
        "h_model": "{{.ComputerSystemInventory.Model}}",
 +
        "h_description": "From Certero: {{.ComputerSystemInventory.Manufacturer}} ({{.ComputerSystemInventory.Model}})",
 +
        "h_os_description": "{{.ComputerSystemInventory.OperatingSystem.Caption}}",
 +
        "h_os_version": "{{.ComputerSystemInventory.OperatingSystem.VersionString}}",
 +
        "h_cpu_info": "{{.ComputerSystemProcessorInfo.Model}}",
 +
        "h_cpu_clock_speed": "{{.ComputerSystemProcessorInfo.ClockSpeed}}",
 +
        "h_physical_cores": "{{.ComputerSystemProcessorInfo.Cores}}",
 +
        "h_last_logged_on_user": "{{.ComputerSystemInventory.Username}}",
 +
        "h_subnet_mask": "{{.ComputerSystemInventory.SubnetMask}}"
 
     }
 
     }
 
}
 
}
Line 649: Line 761:
 
* data:entityUpdateRecord
 
* data:entityUpdateRecord
 
* data:queryExec
 
* data:queryExec
 +
* session:getApplicationList
 
* system:logMessage
 
* system:logMessage
 
* apps/com.hornbill.core:getSitesList
 
* apps/com.hornbill.core:getSitesList
Line 661: Line 774:
 
* debug - defaults to `false` = Set to true to enable debug mode, which will output debugging information to the log
 
* debug - defaults to `false` = Set to true to enable debug mode, which will output debugging information to the log
 
* forceupdates - one of the optimisations, previously included, is the comparison between new data and old data in order to decided whether the record needs an update or not. The decision is, in essence, a detection of changes in SOURCE data, not any data that might have been manipulated by using the template. This paramater caters for this - i.e. an update will always be attempted - whether or not there are going to be any changes.
 
* forceupdates - one of the optimisations, previously included, is the comparison between new data and old data in order to decided whether the record needs an update or not. The decision is, in essence, a detection of changes in SOURCE data, not any data that might have been manipulated by using the template. This paramater caters for this - i.e. an update will always be attempted - whether or not there are going to be any changes.
 +
* creds - Defaults to `false` - Set to `true` to decrypt and output the API key that is stored locally. NOTE - the tool will prompt you for your instance ID, and this can only be decrypted by the user who originally performed the encryption, and only on the same machine that it was encrypted on.
 +
 +
=== First Run ===
 +
 +
From version '''4.0.0''' of the Asset Import Utility, when you first run the utility it will prompt you for the ID of your Hornbill instance (case-sensitive), and the API key (also case-sensitive)that will be used to authenticate the API calls back into Hornbill. This information will be encrypted, and stored locally on the client PC that will be running the tool. For each subsequent import run, the utility will decrypt your instance ID and API key, and will use those to make the API calls back into Hornbill.
 +
 +
NOTE - the encrypted instance ID and API key can only be decrypted on the computer, and by the user, that performed the encryption, so please keep this in mind when scheduling your imports.
 +
 +
Should you wish to use a different API key to what has been previously encrypted, just delete the '''import.cfg''' file from the folder where the import binary resides, and re-run your import from the command line inputting the Instance ID and new API Key as you would have on its first run.
  
 
=Testing Overview=
 
=Testing Overview=
Line 735: Line 857:
  
 
[[Category:Integration]]
 
[[Category:Integration]]
 +
-->
 +
[[Category:HDOC]]

Latest revision as of 18:06, 10 April 2024

This document can now be found at its new location in the Hornbill Document Library.

Data Imports Guide