Geospatial Setup

Click here for end user documentation

Geospatial reporting must understand which fields in your installation point to fields such as latitude and longitude.  These fields are processed as a group so the link between which fields comprise addresses and the corresponding latitude and longitude must be defined.  You may also have several fields that all have the same context, for example more than one field for latitude, or more than one field with the city name.  Each field is defined with a global attribute which identifies the collection of fields that are used together.  This is termed a GEO_GROUP and you may have any number of of these.  Typically a GEO_GROUP will have most of these fields:

• Latitude
• Longitude
• Street Address
• City
• State and or County
• Zip or Postal Code
• Country
• Map Reliability Level – used to identify the lowest level at which an address is determined to be accurate
• Boundary Level – used to set a level for the address fields so that the correct order for data is known.  E.g. Country –> State –> City –> Zip Code.

Any field with the correct display type may be used as part of your geospatial capture and reporting; your ExtraView installaton is preconfigured with a set of suitable fields and values identified using the global attribute name of GEO_GROUP.

You can create additional sets of fields and group these with the global GEO_GROUP attribute and providing your own name for each group.

The ideal configuration is one where you use an address verification service to check addresses you enter.  These services both validate an address and return the precise street-level geolocation of a verified address.  Contact ExtraView Corp. for details if you require assistance configuring this feature – ExtraView Corp. can license this service on your behalf.  Note it may not be available for all countries in the world.

Built into ExtraView is a more straightforward means of deriving the latitude and longitude of an address that does not require the use of an address verification service.  This provides geocoordinates to the city level internationally, and to the zip code level within the USA.  Once configured, when you enter a complete or partial address into an issue, ExtraView can determine the closest latitude and longitude and automatically store these values within your issues, making them available for the generation of maps within reports.

Summary of Steps to Configure Geospatial Reporting

1. Load the geospatial data
   1. Load Shapefile data if you intend to produce Shaded and Heatmap output – users will primarily use these for scheduled reports
   2. Load geospatial coordinate data – this will be used to geocode address data entered.  The data supplied will geocode US data to the closest Zip code.  Users utilize this data for the production of interactive heat maps.  This step is not required if you will use an address verification service which geocodes your data to the street level
2. Setup the Data Dictionary fields with their global attributes.  This is a required step
3. Load list data – these lists are provided to reduce the effort of configuring data such as lists of Countries and Cities.  This is an optional step
4. Configure your layouts – place the fields you have configured onto the appropriate add and edit screen layouts.  This is a required step

Loading Geospatial Data

In order to set up geospatial reporting, two types of data are provided.  You may load one or both of these dependent upon your requirements.

• Shapefiles – these files contain vector-based polygons that provide the outline of territories, such as the outline of Countries, States, Counties, etc.  The shapefiles provided with ExtraView provide outlines of the Countries of the world, the States and Counties within the USA and the Provinces within Canada.  Other shapefiles can be sourced and loaded.  Shapefiles are used to generate shaded maps and heatmaps.  These maps are ideal for reporting where the requirement is to generate Adobe PDFs which are distributed via the report scheduler.  The maps may also be presented within user’s browsers.  If you only want to produce interactive maps, shapefiles do not need to be loaded
• Geospatial Coordinates – these files provide the latitude and longitude of locations.  They are not required if you subscribe to an address verification service to validate addresses entered and to return the geocoordinates of the validated address.  The files provided are used to return the geocoordinates of all major cities and major administrative areas within the world (States, Provinces or Countries), and the States, Cities and Zip Codes within the USA.  When configured as described on this page, the geospatial coordinates provide the means to add the latitude and longitude to any issue from the address information that users enter.  These coordinates are not accurate to the street level, but are accurate enough for most reporting purposes.  Additional geospatial coordinate files may be sourced and loaded for other parts of the world.

Loading Shapefile Data

The shapefile data supplied with ExtraView is loaded as follows. Additional data sourced for other locations is loaded into your database in the same manner as described here.  The shapefile data provided is:

• Country outlines for all countries in the world
• Outlines of all States within the USA and Provinces within Canada
• Outlines of all Counties within States within the USA

To load the data, a utility named populateShapeFiles is run externally to ExtraView as a Java application.  This utility loads files that are contained in a subdirectory of the ExtraView directory named WEB-INF/data.  All files within this directory with an extension of .shp are loaded.  The data supplied with ExtraView is in the directory named WEB-INF/data/geoshapefiles.  There are a series of files, with the extensions .dbf, .prj, .shp and .shx.  Each set of 4 files comprise a single shapefile.

The populateShapeFiles utility takes these arguments:

• JAVA_HOME
• TOMCAT_HOME
• EV_BASE
• Optionally, the name of the directory under WEB-INF/data where the script files are stored if you do not intend to use geosshapefiles.

Any errors encountered while running the utility are communicated to the console as well as to the ExtraView application server log.

Windows Example

This example will load the data supplied with ExtraView and assumes that the environment variables JAVA_HOME, TOMCAT_HOME and EV_BASE have been set.

Change your working directory to %EV_BASE%/data

Enter the command:

populateShapeFiles.bat %JAVA_HOME% %TOMCAT_HOME% %EV_BASE%

Linux Example

This example will load the data supplied with ExtraView on a Linux server.

Change your working directory to /data

Modify the script named populateShapefiles.sh and edit the JAVA_HOME and TOMCAT_HOME entries to point to the correct places on your server

Enter the command:

./populateShapeFiles.sh EV_BASE [geospatial directory name]

EV_BASE is the base location of ExtraView and [geospatial directory name] is the optional directory name for the files with geospatial information if the directory name is not geoshapefiles.

Loading Geospatial Coordinate Data

The geospatial coordinate data supplied with ExtraView is loaded as follows. Additional data sourced for other locations is loaded into your database in the same manner as described here.  The geospatial data provided is:

• Country level data for all countries in the world
• Top administrative level within each country.  For example the geolocations for States in the USA, Provinces in Canada, and Counties in the United Kingdom
• Within the USA, geospatial coordinates for each County within each State
• Within the USA, geospatial coordinates for each Zip Code

To load the data, a utility named populateGeospatialTables is run externally to ExtraView as a Java application.  This utility executes scripts that are contained in a subdirectory of the ExtraView directory named WEB-INF/data.  All scripts within this directory in files with an extension of .sql are executed.  These scripts may all be contained within a single zipped file (with the extension of .zip.  The data supplied with the installation is in the directory and file named WEB-INF/data/geoscripts/geoscripts1.zip.

The populateGeospatialTables utility takes these arguments:

• JAVA_HOME
• TOMCAT_HOME
• EV_BASE
• Optionally, the name of the directory under WEB-INF/data where the script files are stored if you do not intend to use geoscripts.

Any errors encountered while running the utility are communicated to the console as well as to the ExtraView application server log.

Windows Example

This example will load the data supplied with ExtraView and assumes that the environment variables JAVA_HOME, TOMCAT_HOME and EV_BASE have been set.

Change your working directory to %EV_BASE%/data

Enter the command:

populateGeospatialTables.bat %JAVA_HOME% %TOMCAT_HOME% %EV_BASE%

Linux Example

This example will load the data supplied with ExtraView on a Linux server.

Change your working directory to /data

Modify the script named populateGeospatialTables.sh and edit the JAVA_HOME and TOMCAT_HOME entries to point to the correct places on your server

Enter the command:

./populateGeospatialTables.sh EV_BASE [geospatial directory name]

EV_BASE is the base location of ExtraView and [geospatial directory name] is the optional directory name for the files with geospatial information if the directory name is not geoscripts.

Geospatial Global Attribute Definitions

As stated above, your ExtraView configuration is preconfigured to work with geospatial reports.  You may define additional groups of fields for geospatial working within your site or you may alter the default configuration provided.  For example, you might define different GEO_GROUPS for different tracking applications in different Business Areas and Projects within your installation or you may want to alter the default GEO_GROUP name (which is GEO_GROUP).

Attribute	Field Type	Purpose
GEO_GROUP	—	The name of the group of attributes that collectively define the information required to save geocoded information.  All attributes within a group must have the same name.  This allows you to define any number of groups for different purposes, with each group pointing to different fields for latitude, longitude, etc.  Examples of names are GEO_GROUP (the preconfigured group name), MY_GEO_GROUP, or NORTH_AMERICA_GEO_INFO.  This attribute name is placed on all the fields that form a group.
GEO_LATITUDE	Decimal	This attribute is placed on the field that contains the latitude of the address for the GEO_GROUP
GEO_LONGITUDE	Decimal	This attribute is placed on the field that contains the longitude of the address for the GEO_GROUP
GEO_RELIABILITY_LEVEL	List or Text field	Depending on the source of the geocoding information (anywhere from manual entry to automated lookup of latitude and longitude) ExtraView may detect that the information entered is inaccurate.  For example, you might try to enter a latitude and longitude for an address in the State of California and the geo coordinates you provide are obviously in a different part of the USA or different part of the world.  ExtraView can determine the most reliable level of the data. For example, if the geo coordinates are in New York and the address entered states that it’s in California, the reliability level will be the Country level of the USA.  Similarly, if you enter an address which states that a City’s name is in the state of California, but no such City name exists, the reliability level will be set to the State level.
GEO_ADMIN_BOUNDARY_LEVEL	—	This attribute provides the level of the data for the field within which it’s defined.  For example, the field which contains the values for the Country will most probably have a GEO_ADMIN_BOUNDARY_LEVEL of GEO_ADMIN_BOUNDARY_LEVEL1 and the field that contains the values for the State will have a GEO_ADMIN_BOUNDARY_LEVEL of GEO_ADMIN_BOUNDARY_LEVEL2

Optional List Values

Whilst any of the fields you use to hold address data may be text or list-based, it is common to use list type fields for shorter lists such as Country names or State names.  ExtraView provides data for a number of lists and you may create user defined fields and load the data supplied into these fields.  These lists are in the English language.

The step itemized above, to load geospatial coordinate data can be complemented by utilizing field lists within your ExtraView installation that contain data such as Country names and State names.  To load any of these fields with data, create a list or pop-up list field within the data dictionary, and use the Import Field Values button within any of the fields to load one or more of the following lists.  These lists may be loaded into separate fields, or combined together in any different combination of fields.

Location of Data	Purpose
WEB-INF/data/countries.txt	A list of the full names of the Countries in the World
WEB-INF/data/countries-iso-alpha-2.txt	A list of the ISO 3166-1 alpha-2 character abbreviations of the Countries in the World
WEB-INF/data/countries-iso-alpha-3.txt	A list of the ISO 3166-1 alpha-3 character abbreviations of the Countries in the World
WEB-INF/data/au-states.txt	A list of the States and Territories of Australia
WEB-INF/data/ca-provinces.txt	A list of the Provinces and Territories within Canada
WEB-INF/data/cn-divisions.txt	A list of the Divisions of China
WEB-INF/data/de-states.txt	A list of the States witrhin Germany
WEB-INF/data/es-provinces.txt	A list of the Provinces and Municipalities in Spain
WEB-INF/data/fr-regions.txt	A list of the Regions of France
WEB-INF/data/it-regions.txt	A list of the Regions of Italy
WEB-INF/data/ja-prefectures.txt	A list of the Prefectures of Japan
WEB-INF/data/nz-regions.txt	A list of the Regions within New Zealand
WEB-INF/data/uk-counties.txt	A list of the Counties within the United Kingdom
WEB-INF/data/us-states.txt	A list of the 50 States plus the District of Columbia within the United States of America

Example Configuration

Step 1

Within the data dictionary search for and edit the field named MAP_RELIABILITY_LEVEL.  This field should be configured as follows:

	Property	Value
	Multiple Value	No
	Allow selection on reports	Yes
	Filter Criteria	Yes
	Is sortable	Yes
	Add a Global Attribute of GEO_GROUP	GEO_GROUP
	Add  the Global Attribute of GEO_RELIABILITY_LEVEL	—

The field list values should be examined, and if necessary the titles may be altered for your configuration.  The predefined values are:

	Value	Sort Order
	Country	1
	State / Province	2
	County	3
	City	4
	Zip / Postal Code	5

Step 2

In the data dictionary edit the field named LATITUDE.  It should be configured as follows:

	Property	Value
	Field Name	LATITUDE
	Field Title	Latitude
	Display Type	Decimal
	Allow selection on reports	Yes
	Filter Criteria	Yes
	Is sortable	Yes
	Global Attribute of GEO_GROUP	GEO_GROUP
	Global Attribute of GEO_LATITUDE	—

Step 3

In the data dictionary edit the field named LONGITUDE.  It should be configured as follows:

	Property	Value
	Field Name	LONGITUDE
	Field Title	Longitude
	Display Type	Decimal
	Allow selection on reports	Yes
	Filter Criteria	Yes
	Is sortable	Yes
	Global Attribute of GEO_GROUP	GEO_GROUP
	Global Attribute of GEO_LONGITUDE	—

Step 4

If they do not already exist, create your own user defined fields in the data dictionary that hold data for the Country, State/Province, County, City and Zip/Postal Code.  These correspond to the values created in Step 1.  Frequently you will not require all of these fields.  For example, it is common within the United States not to use County as part of the address.  Configure the properties and permissions for these fields as necessary.  The fields may be one of the following display types:

• Text
• List
• Pop-up

>It is typically more convenient to configure fields with shorter amounts of text as List or Pop-up types and fields with more data as Text fields.  For example, Country is often configured as a List field and an Address field is usually configured as a text field.  You may use one or more of the optional sets of list values to pre-load values into List and Pop-up list fields.

Add a global attribute for the GEO_GROUP name, which in our example is GEO_GROUP.  Add the GEO_ADMIN_BOUNDARY_LEVEL global attributes as follows:

	Example Field Name	GEO_GROUP	GEO_ADMIN_BOUNDARY_LEVEL	GEO_RELIABILITY_LEVEL
	COUNTRY_NAME	GEO_GROUP	2	Country
	STATE_NAME	GEO_GROUP	3	State / Province
	CITY_NAME	GEO_GROUP	4	City
	POST_CODE	GEO_GROUP	4	Zip / Postal Code

Note that the GEO_ADMIN_BOUNDARY_LEVEL values do not correspond to the sort order of the values for the MAP_RELIABILITY_LEVEL values.  First, the GEO_ADMIN_BOUNDARY_LEVEL value of 1 is defined as the level for the World.  Secondly, when generating shaded maps and heat maps, ExtraView is only prepopulated with maps for the countries of the world, states and provinces of the USA and Canada, and counties of the USA.  An interactive map of the world allows users to zoom in to street level from a map of the entire globe.

When adding the global attribute for the GEO_ADMIN_BOUNDARY_LEVEL, you set the Field property to the name of the MAP_RELIABILTY_LEVEL field.  In our example, this is also MAP_RELIABILTY_LEVEL.  The field value is set to the value in the above table.  Note that both the City name and the Post Code share the same GEO_ADMIN_BOUNDARY_LEVEL.  This is because the shapefiles loaded with the above populateShapeFiles utility does not contain maps with City boundaries but contains maps at the Zip code level.

Step 5

Place the fields on the add or edit layout where you want the information to be geocoded.  These fields will be:

• LATITUDE – you may hide this field on the layout, but this is where the latitude of the location you are geocoding will reside
• LONGITUDE – you may hide this field on the layout, but this is where the longitude of the location you are geocoding will reside
• MAP_RELIABILITY_LEVEL – you may hide this field on the layout, but this is where the map reliability level value of the location you are geocoding will reside
• According to your configuration, place the fields that define the following on your layout:
  • Country
  • State / Province
  • County
  • City
  • Zip / Postal Code
• An optional step that allows the user to view the location on an interactive map is to define and embed a Geospatial Report within the add or edit screen.  Please view the instructions on embedded reports for this.

At the end of the day, the configuration may present similar to this to the end user:

Multiple GEO_GROUP Configuration

This allows you to defined multiple, independent groups of fields where each group has completely independent working within a single ExtraView installation.  For example, you might have one group that is used to map customer issues, while a second group might map the location of all your organization’s stores.

Each group must have its own set of fields as stated here.  For the MAP_RELIABILITY_LEVEL, you may create an alias of the MAP_RELIABILITY_LEVEL field.

Click here for end user documentation