Product Documentation

Note: Again, we strongly recommend that you backup your data before you use the import facility, using your database’s standard facilities. Importing metadata, layouts, reports and user data is irreversible, and it is possible that a failure can occur during the process.

The import menu is situated within the Import/Export tab of the administration menu, and allows the administrator to specify the file to load.

The import function reads a flat image file (which is in XML format) and updates the tables as required by the administrator’s selection of objects to load. The utility takes into account the relational dependencies and all internal sequence renumbering as needed, ensuring consistent and correct instance updates.

Validity checking for data adherence to new rules is performed as early as possible, thereby minimizing the impact of some validity failure that may cause termination of the import.

Most imports will utilize the Update / Merge operation. This will update records that exist, and merge into the database the new records that were part of the file being imported.

Although it is possible to create your own file to be used as an import file to ExtraView, extreme care should be taken before attempting to do this, and it is essential that you have a complete and thorough knowledge of the ExtraView schema. Often, it may appear straightforward to compose an XML file, or to edit an existing file that was created by ExtraView’s own Export feature, but there are many relationships described in the file that are not obvious and that must be maintained for an import to be successful. We suggest that before attempting to create your own import file by modifying an existing file or creating it from scratch, that you consult with ExtraView support. In any case, always back up your target database before performing any import.

Starting the importing process

Follow the instructions on the screen. As displayed, the principal import methods are:

If you have altered metadata such as the titles of objects in lists, within the destination database, and these same modifications to the data were not made in the source database from which the export was taken, ExtraView gives you an opportunity to remap the fields, as opposed to creating new fields from the import file, and inserting these into the destination database. To perform this check, make sure the Map unmatched titles prompt is checked.

When you perform the check, the import is split into two parts. First the uploaded file is preprocessed, and each list value in the import is checked against the target destination database. If there are any values in the import file that do not exist in the destination database, then you will be shown the values from the import file that do not have a matching value, and you will be able to select from the values in the target database that do not have a value specified in the import file. After selecting the values to map, if there are discrepancies, you then proceed to the second part of the import. The file is processed again, the destination database is updated, and the mappings you selected are applied.

When importing data into a business area, if the import process is able to create a project with an ID numbered 0, and the project ID in the import file is 0, then a project ID of 0 is used and mapped into the target database. If a project ID of 0 already exists in the target business area, the project ID of 0 in the import file is mapped to a new non-zero project ID. In any case, the matching project title within the business area and the mapped project overrides these decisions.

This is important, as a project ID of 0 is used for inheritance of layouts, permissions and other objects for the non-zero project ID’s within the business area, and ExtraView is trying to ensure that a project ID of 0 always exists for the inheritance to be allowed to work.

There is one circumstance in which the import process may have difficulty in resolving all the data from the export file. If you are not importing user data, and the information in the import file contains references to users that do not exist in the target database, then import errors may result. This is especially noticed if the missing user was the person who created or last updated items such as layouts. These will not be imported, as ExtraView must maintain referential integrity of the users who are connected to its internal objects.

In general, allowed value lists that have an invalid current value in parent or child will cause an import error and will not be imported. This may result in incomplete allowed value lists.

Results from preprocessing the import file

While data is being preprocessed or imported, a status bar is shown as follows:

Progress bar during the importing process

Note: Importing the data can take some time, from ten minutes in smaller installations, to thirty minutes or more in larger installations. It is dependent upon the size of the file being processed, factors such as the number of fields, layouts and permissions, and the speed of the server loading the data. Due to the inherent limitations of browsers, there is little feedback during the initial part of the process, when the browser is uploading the import file from the client machine to the server. It is recommended that you prohibit users from using ExtraView during this time, especially if you are importing layouts. You can do this by locking users out of the system while you are importing the data. You can achieve this with the Disable and Enable User Access on the Users tab of the administration menu.

Note: You can improve the time to execute the import operation, by zipping the import file. Exported files compress significantly (perhaps to less than 10% of their original size). ExtraView will look for the XML file contained within a zip file as long as the name of the zip file and the name of the import file are the same.

Note: Once the import operation is started, you must not interrupt it. Once the actual import has begun, ExtraView will not cancel the server operation, even if you close your browser.

Note: When your file is imported, there is additional information within the file, termed a manifest. This manifest is used as a check against the data that you import, and provides a report if there are any unanticipated differences between the imported information and the database into which it is being imported.

After the import is complete, you will see a summary screen that shows the number of records imported into each database table. Any errors encountered will be displayed here. Note that the results of the import vary according to the exact tables imported from the file you uploaded. There may be anything from a few to more than one hundred tables displayed in the results.

Summary screen of the imported data

Handling User Data

Users frequently change their personal settings when using ExtraView. For example, users may change their Home Page reports, their display format for the timestamp, their password, or one of many other attributes. If you export the metadata from a development system, wait some time and then import this metadata to a production system, you may overwrite recent changes made by users in their personal data. A behavior setting named OMITTED_IMPORT_USER_COLUMNS allows the administrator to set up specific columns that will be ignored upon the import of the metadata. The value of this field is a comma-separated list of column names. Note that you set this in the target system, not the system from which you are exporting the data. The possible values you can add to this list are as follows:

Allowed Value Conflicts

Importing allowed values presents some challenges, when the data in the target database has changed significantly from the data in the source database. An entry in allowed values is unique for a given Title, Parent and Child value.

• For any given combination of a Parent and Child, only 0 or 1 entry can be set to be Enabled
• For any given Parent and Child there can be any number of entries set to Disabled
• For each allowed value, there can be entries in the Allowed Values database table with allowed values for that type

The points to observe during an import of allowed values are:

• Making sure that only one entry for a given Parent and Child is set to enabled. This means importing an enabled row requires a check that there is no other entry enabled. If there is another entry enabled for the given Parent and Child combination, then the import entry is set to disabled and imported
• Matching imported allowed value entries to existing allowed value entries. The Title, Parent and Child are used to find a matching row. If none is found, the imported entry is inserted. If a matching row is found, it is updated and the value lists are merged
• The allowed value lists for allowed value types are only added to; they are never reduced
• Given the import of an enabled allowed value type is not allowed if another allowed value type is enabled in the database, if the import wants to change which entry is enabled, it must first disable the currently enabled entry and then import the enabled entry from the import file. This implies that the export process should export its disabled entries before exporting its enabled entry. When imported they will then be in the correct order. The values list is merged in either case

Importing List fields with Aliases

If an existing list field is not an alias of another list field, the import logic maintains this, and keeps the list field non-aliased in the target database. The import cannot make a non-aliased list field an aliased field during the import because it cannot guarantee that the lists are intended to be the same. For this same reason, the import will not change an aliased list field to be the aliase of a different list field. In normal operation, ExtraView checks these conditions and only allows valid operations, therefore the import must also make the same checks. In the context of this section, list fields include all fields with a display type of list, popup, tab, and radio button.

The import rules for the Alias of field values are as follows:

• If the list field entry being imported is not an aliased field, it is imported and the list remains non-aliased after import. The imported list values are merged for these fields
• If the list field entry being imported is an aliased field, it is imported only if the imported aliased field definition is the same in both the file being imported and in the target database. The imported list values are merged
• A non-aliased list cannot be aliased by the import process
• The values within an aliased list cannot be changed by the import process

Error Logging During Import

A text log of errors is created on the server, as the XML updater executes. Each error entry will contain as much useful information as possible, including the table, row number, the data being used at the time, and the exception message, if any. This error log is echoed to the summary screen in the browser.

The error file is stored in the system temp directory under the name:
<time_in_ms>_<user_id>_XML_ERRORS.txt

for example:
1026520070671_JEFF.SYKES_XML_ERRORS.txt

All error messages begin with
::XML_UPDATE_ERROR::

plus a new line, and end with
::END_XML_UPDATE_ERROR::

followed by two new line characters. Here is an example of an error:

In order to be able to store multi-byte characters within the error file, note that the error file is not created as standard text. If you open the file in a standard editor, you will notice that what looking at most of the metadata in the file, every second character is null.

Handling rejected records during the import of metadata

As ExtraView processes the XML import file and fires the updating routines, if errors prevent an update are generated, the problematic XML nodes' data is captured, and used to generate a new XML file. This file is of the same structure as the original file, but only includes those items that could not be properly updated. If there are no errors generated during the upload process, this file is deleted at the end of the update process, as it is empty.

This file is saved under the system temp directory under the name:

<time_in_ms>_<user_id>_XML_REJECTS.xml

for example

1026520070671_JEFF.SYKES_XML_REJECTS.xml