Difference between revisions of "Asset Data Import Tool"

From Hornbill
Jump to navigation Jump to search
 
(20 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 +
This document can now be found at its new location in the [https://docs.hornbill.com/data-imports-guide/assets/overview Hornbill Document Library].
 +
 +
[[file:hornbill-document-library.png|Data Imports Guide|link=https://docs.hornbill.com/data-imports-guide/assets/overview]]
 +
 +
<!--
 +
 +
 
=About the Hornbill Asset Import Utility=
 
=About the Hornbill Asset Import Utility=
  
Line 5: Line 12:
 
==Open Source==
 
==Open Source==
  
The Asset Import Utility is provided open source under the [https://wiki.hornbill.com/index.php/The_Hornbill_Community_License_(HCL) Hornbill Community Licence] and can be found [https://github.com/hornbill/goDBAssetImport Here] on GitHub
+
The Azure User Import Utility is no longer provided open-source, and is maintained solely by Hornbill as a closed-source, but free-to-use, tool.
 
 
===History===
 
This tool is an evolution the previous [[Special:Permalink/26928|Database Asset Import v1.*]], combining the [[Special:Permalink/26716|CSV Asset Import v1.*]] tool and a revamp of the templating system.  
 
  
 
==Installation Overview==
 
==Installation Overview==
Line 14: Line 18:
 
===Windows Installation===
 
===Windows Installation===
  
* Download the [https://github.com/hornbill/goDBAssetImport/releases/latest ZIP archive] relevant to your OS and architecture
+
* Download the [https://github.com/hornbill/asset-import/releases/latest ZIP archive] relevant to your OS and architecture
 
* Extract zip into a folder you would like the application to run from e.g. '''C:\Asset_Import\'''
 
* Extract zip into a folder you would like the application to run from e.g. '''C:\Asset_Import\'''
 
* Open the example configuration file relevant to your environment, and add in the necessary configuration
 
* Open the example configuration file relevant to your environment, and add in the necessary configuration
Line 22: Line 26:
 
For Windows Systems:  
 
For Windows Systems:  
 
<code>
 
<code>
goDBAssetImport.exe -dryrun=true -file=conf_example_db_sccm.json
+
asset_import.exe -dryrun=true -file=conf_example_db_sccm.json
</code>
 
 
 
For Mac OSX and Linux Systems:
 
<code>
 
./goDBAssetImport -dryrun=true -file=conf_example_db_sccm.json
 
 
</code>
 
</code>
  
Line 36: Line 35:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 191: Line 188:
  
 
==InstanceConfig==
 
==InstanceConfig==
* <code>APIKey</code> - a Hornbill API key for a user account with the correct permissions to carry out all required API calls. Here are details on creating an API key [https://wiki.hornbill.com/index.php?title=API_keys '''here'''].
 
* <code>InstanceID</code> - the name of your Hornbill instance and can be found within the URL you use to navigate to it: live.hornbill.com/[instance name]/. E.g. if the URL you use to access your instance is live.hornbill.com/arescomputing/, then your instance id would be "arescomputing". '''This value is case sensitive''.
 
 
* <code>KeysafeKeyID</code> - The ID of the Keysafe Key that holds your data source credentials. Set to 0 for importing directly from CSV files. Supported Keysafe key types are:
 
* <code>KeysafeKeyID</code> - The ID of the Keysafe Key that holds your data source credentials. Set to 0 for importing directly from CSV files. Supported Keysafe key types are:
 
** <code>Database Authentication</code>
 
** <code>Database Authentication</code>
Line 253: Line 248:
 
*** <code>computer</code>
 
*** <code>computer</code>
 
*** <code>computerPeripheral</code>
 
*** <code>computerPeripheral</code>
 +
*** <code>dataProcessingRecord</code>
 
*** <code>mobileDevice</code>
 
*** <code>mobileDevice</code>
 
*** <code>networkDevice</code>
 
*** <code>networkDevice</code>
 
*** <code>printer</code>
 
*** <code>printer</code>
 
*** <code>software</code>
 
*** <code>software</code>
 +
*** <code>system</code>
 
*** <code>telecoms</code>
 
*** <code>telecoms</code>
 
* <code>OperationType</code> - The type of operation that should be performed on discovered assets - can be Create, Update or Both. Defaults to Both if no value is provided
 
* <code>OperationType</code> - The type of operation that should be performed on discovered assets - can be Create, Update or Both. Defaults to Both if no value is provided
 
* <code>CSVFile</code> - The path to the CSV file to use as the data source for this specific asset type. Only required when '''SourceConfig > Source''' is set to '''CSV'''
 
* <code>CSVFile</code> - The path to the CSV file to use as the data source for this specific asset type. Only required when '''SourceConfig > Source''' is set to '''CSV'''
 +
* <code>CSVFilters</code> - An array of objects, containing zero or more filters that should be applied when working out which rows from the CSV file should be included in the asset type import. Only required when '''SourceConfig > Source''' is set to '''CSV''', and this is non-mandatory. If no filters are provided, then all records will be imported from the CSV into the target asset type.
 +
** <code>Column</code> - The column name from the CSV that the filter should apply to
 +
** <code>Operator</code> - The operator that should be used when applying the filter. Valid values are:
 +
*** <code>EQUALS</code> - Does the filter Value (below) exactly match the column value
 +
*** <code>CONTAINS</code> - Is the filter Value (below) contained within the column value (wildcard type functionality, *column_value*)
 +
*** <code>IN</code> - Does the column value have an exact match in an array of strings supplied in <code>Value</code>, below
 +
** <code>Value</code> - The value to be applied in the filter
 
* <code>PreserveShared</code> - If set to true, when updating assets that are Shared, then the Used By fields will not be updated. Defaults to false
 
* <code>PreserveShared</code> - If set to true, when updating assets that are Shared, then the Used By fields will not be updated. Defaults to false
 
* <code>PreserveState</code> - If set to true then the State field will not be updated. Defaults to false
 
* <code>PreserveState</code> - If set to true then the State field will not be updated. Defaults to false
Line 269: Line 273:
 
** <code>mobile</code>
 
** <code>mobile</code>
 
* <code>LDAPDSN</code> - The Distinguished Name of the LDAP container to execute your query in. Only required when '''SourceConfig > Source''' is set to '''LDAP'''
 
* <code>LDAPDSN</code> - The Distinguished Name of the LDAP container to execute your query in. Only required when '''SourceConfig > Source''' is set to '''LDAP'''
* <code>Query</code> - Not used for Workspace One imports. For the other data sources:
+
* <code>Query</code> - Not used for Workspace One or direct CSV file imports. For the other data sources:
 
** Additional SQL clauses to be appended to the Query from SourceConfig > Database > Query, to retrieve assets of that asset type.  
 
** Additional SQL clauses to be appended to the Query from SourceConfig > Database > Query, to retrieve assets of that asset type.  
 
** Certero oData filter for returning asset details for that asset type
 
** Certero oData filter for returning asset details for that asset type
Line 289: Line 293:
 
*** <code>AssetsComputer</code>
 
*** <code>AssetsComputer</code>
 
*** <code>AssetsComputerPeripheral</code>
 
*** <code>AssetsComputerPeripheral</code>
 +
*** <code>AssetsDataProcessingRecord</code>
 
*** <code>AssetsMobileDevice</code>
 
*** <code>AssetsMobileDevice</code>
 
*** <code>AssetsNetworkDevice</code>
 
*** <code>AssetsNetworkDevice</code>
 
*** <code>AssetsPrinter</code>
 
*** <code>AssetsPrinter</code>
 
*** <code>AssetsSoftware</code>
 
*** <code>AssetsSoftware</code>
 +
*** <code>AssetsSystem</code>
 
*** <code>AssetsTelecoms</code>
 
*** <code>AssetsTelecoms</code>
 
** <code>EntityColumn</code> - specifies the unique identifier column from the Hornbill entity specified above
 
** <code>EntityColumn</code> - specifies the unique identifier column from the Hornbill entity specified above
Line 325: Line 331:
 
* <code><nowiki>{{.columnName | epoch}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field
 
* <code><nowiki>{{.columnName | epoch}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field
 
* <code><nowiki>{{.columnName | epoch_clear}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field. Defaulting to CLEAR the column if unable to convert.
 
* <code><nowiki>{{.columnName | epoch_clear}}</nowiki></code> will convert an epoch value to the YYYY-MM-DD HH:II:SS format required for a Hornbill DateTime field. Defaulting to CLEAR the column if unable to convert.
* <code><nowiki>{{.columnName | date_conversion "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 1st 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. '''Please note''' that IF your formatting is already in the Hornbill date time format (2006-01-02 15:04:05), you don't need to convert anything.
+
* <code><nowiki>{{.columnName | date_conversion "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 2nd 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. '''Please note''' that IF your formatting is already in the Hornbill date time format (2006-01-02 15:04:05), you don't need to convert anything.
* <code><nowiki>{{.columnName | date_conversion_clear "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 1st 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. Defaulting to CLEAR the column if unable to convert.
+
* <code><nowiki>{{.columnName | date_conversion_clear "date time format of the content in .columnName"}}</nowiki></code> Provide the input format based on the following reference time of Jan 2nd 2006 4 minutes and 5 seconds past 3pm - eg "02/01/2006 15:04:05" will convert the regular UK/European date time format to the format useable in the Hornbill datetime field, whereas "01/02/2006 15:04" will process default US date time. Defaulting to CLEAR the column if unable to convert.
  
 
==AssetTypeFieldMapping==
 
==AssetTypeFieldMapping==
Line 345: Line 351:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 514: Line 518:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 599: Line 601:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 670: Line 670:
 
<pre>
 
<pre>
 
{
 
{
    "APIKey": "yourapikey",
 
    "InstanceId": "yourinstanceid",
 
 
     "KeysafeKeyID": 0,
 
     "KeysafeKeyID": 0,
 
     "LogSizeBytes": 1000000,
 
     "LogSizeBytes": 1000000,
Line 763: Line 761:
 
* data:entityUpdateRecord
 
* data:entityUpdateRecord
 
* data:queryExec
 
* data:queryExec
 +
* session:getApplicationList
 
* system:logMessage
 
* system:logMessage
 
* apps/com.hornbill.core:getSitesList
 
* apps/com.hornbill.core:getSitesList
Line 775: Line 774:
 
* debug - defaults to `false` = Set to true to enable debug mode, which will output debugging information to the log
 
* debug - defaults to `false` = Set to true to enable debug mode, which will output debugging information to the log
 
* forceupdates - one of the optimisations, previously included, is the comparison between new data and old data in order to decided whether the record needs an update or not. The decision is, in essence, a detection of changes in SOURCE data, not any data that might have been manipulated by using the template. This paramater caters for this - i.e. an update will always be attempted - whether or not there are going to be any changes.
 
* forceupdates - one of the optimisations, previously included, is the comparison between new data and old data in order to decided whether the record needs an update or not. The decision is, in essence, a detection of changes in SOURCE data, not any data that might have been manipulated by using the template. This paramater caters for this - i.e. an update will always be attempted - whether or not there are going to be any changes.
 +
* creds - Defaults to `false` - Set to `true` to decrypt and output the API key that is stored locally. NOTE - the tool will prompt you for your instance ID, and this can only be decrypted by the user who originally performed the encryption, and only on the same machine that it was encrypted on.
 +
 +
=== First Run ===
 +
 +
From version '''4.0.0''' of the Asset Import Utility, when you first run the utility it will prompt you for the ID of your Hornbill instance (case-sensitive), and the API key (also case-sensitive)that will be used to authenticate the API calls back into Hornbill. This information will be encrypted, and stored locally on the client PC that will be running the tool. For each subsequent import run, the utility will decrypt your instance ID and API key, and will use those to make the API calls back into Hornbill.
 +
 +
NOTE - the encrypted instance ID and API key can only be decrypted on the computer, and by the user, that performed the encryption, so please keep this in mind when scheduling your imports.
 +
 +
Should you wish to use a different API key to what has been previously encrypted, just delete the '''import.cfg''' file from the folder where the import binary resides, and re-run your import from the command line inputting the Instance ID and new API Key as you would have on its first run.
  
 
=Testing Overview=
 
=Testing Overview=
Line 849: Line 857:
  
 
[[Category:Integration]]
 
[[Category:Integration]]
 +
-->
 +
[[Category:HDOC]]

Latest revision as of 18:06, 10 April 2024

This document can now be found at its new location in the Hornbill Document Library.

Data Imports Guide