Using KML Regions to Display Large GIS Datasets in Google Earth

We’re going to deliver a series of posts over the next couple of weeks covering the use of KML Regions in Google Earth for displaying large GIS datasets.  In this first post we’ll cover some basic introductory information that will give you some background information on KML Regions and the various ways in which they can be used in Google Earth. 

What are Regions?
KML Regions allow you to add very large datasets to Google Earth without sacrificing performance.  This functionality allows for the loading and display of data only when it falls within the display and occupies a certain portion of the screen.  Typically, Regions are used to supply distinct levels of detail for your data where fine details are presented only when zoomed in far enough on the display.  In the KML object model, Regions can be contained within any Feature which means Placemarks, Network Links, Overlays, and Containers.  Most commonly, a Region is used to affect the visibility of Placemarks or Ground Overlays.  The <Region> element defines this concept, and is composed of two important concepts including the bounding box and level of detail or LOD. 

Bounding Box
A bounding box, defined by the <LatLongAltBox> element, describes an area of interest defined by geographic coordinates and altitudes.  This element, similar to <LatLongBox> contains the child elements <north>, <south>, <east>, and <west> that define the geographic boundaries of the Region.  In addition, the bounding box also has elements that define the minimum and maximum altitude (<minAltitude>, <maxAltitude>).  A Region is considered “active” or visible when the bounding box is within the display and the level of detail requirements are met.

Level of Detail
The level of detail or LOD is defined with the <Lod> child element of <Region>.  It defines a range, specified by <minLodPixels> and <maxLodPixels> that determines the visibility of data within a Region.  This ensures that large amounts of data are only loaded when enough pixels are available to display the data adequately.  When the Region takes up a relatively small percentage of the screen, the LOD allows you to specify a dataset with a lower resolution.  The <Lod> value units are defined by square pixels.  Data must occupy an area greater than <minLodPixels> and less than <maxLodPixels> to be visible.

Region Elements
Primary elements specific to Region include:

<LatLonAltBox> (required) - A bounding box that describes an area of interest defined by geographic coordinates and altitude.

<minAltitude> - Defaults to 0; specified in meters above sea level

<maxAltitude> - Defaults to 0; specified in meters above sea level

<north>, <south>, <east>, <west> - Used to specify the latitude and longitude coordinates of the bounding box

<Lod> - Abbreviated for Level of Detail. Describes the size of the projected region on the screen that is required in order for the region to be considered “active”.  Composed of the following child elements:

<minLodPixels> - Measurement in screen pixels that represents the minimum limit of the visibility range for a given Region.

<maxLodPixels> - Measurement in screen pixels that represents the maximum limit of the visibility range for a given Region.

<minFadeExtent> - Distance over which the geometry fades, from fully opaque to fully transparent.  Value is expressed in screen pixels and is applied at the minimum end of the LOD limits.

<maxFadeExtent> - Distance over which the geometry fades, from fully transparent to fully opaque.  Value is expressed in screen pixels and is applied at the maximum end of the LOD limits.

Example
Now that we’ve covered some basic information on KML Regions you can click here to see an example (courtesy of Google).  You’ll want to make sure you already have Google Earth installed.  Experiment with different viewpoints and watch when the Region comes into view and out of view, depending on how much of the screen area it requires. 

In upcoming posts we’ll cover some of the more advanced Region features, show you an example of how to create Regions, and discover how to use Arc2Earth to create Regions from your ArcGIS Data.

Get More Information
GeoSpatial Training Services provides e-learning courses for GIS users.  If you would like more information on Regions and other KML elements please see our “Mastering KML in Google Earth” e-learning course.  Other related course include “Google Earth for ArcGIS Users“, “Google Maps for your Apps“, “Arc2Earth for ArcGIS Users“, and our “Google Bundle” which combines all these courses into one package at a significant discount.

Import Google Earth KML Files to ESRI Geodatabase with Arc2Earth

Arc2Earth was initially created as a tool for exporting your existing ArcGIS data to Google Earth, but has since evolved into a robust tool that is also capable of importing KML/KMZ, GeoRSS, and Atom files into an ESRI geodatabase format.  With the release of Arc2Earth version 2, the data import functionality has been rewritten to be more flexible, and includes the ability to import KML/KMZ, GeoRSS (simple, GML), and geo-enabled AtomPub files. The import process can create a new feature class to store the data or update an existing feature class in a personal or enterprise ArcSDE geodatabase.

Click here to see a demonstration of using Arc2Earth to import an existing KML/KMZ file into an ESRI geodatabase. 

 

The “Import KML/KMZ” and “Import GeoRss/Atom” items on the Import menu item open the same Import KML dialog.  The General tab contains a number of options related to the file that will be imported as well as the geodatabase export parameters.  By default, the Import KML dialog is assumed to be KML.  If the importer can’t discern the file type of the source document you will need to know the feed type (GeoRss, Atom) and check the option indicating the file is of type GeoRss if necessary. In the event that you need to import all files in a folder you may specify a folder by using the browse button.  The target layer for your import can be stored in a personal or enterprise ArcSDE geodatabase and can be an existing or new feature class.  In the event that you will be storing the target layer in an ArcSDE enterprise database you must specify the correct connection string for the database.  Three feature classes are created, one each for points, lines, and polygons with the root name of each feature class defaulting to the KML/KMZ file name along with an extension of _points, _lines, or _polys.

 

 

Import options can be saved into an import file (*.a2ei) for later use for commonly used import operations by clicking the “Save” button and specifying a filename and directory path. Both import menu items open the same window, but different options are set for each type of import.

 

The Schema tab on the Import KML dialog contains options for importing KML extended data values, a description templates schema, and the option to only import placemarks.  The “Description Templates Schema” option can be used to pull attribute data from existing description tags in KML Placemarks.  Normally description tags are free form text making them difficult to cull meaningful information.  Schemas allow you to define where in the description text your attribute data is hidden. The “Only Import Placemarks” option can be used to limit the import to only those Placemarks that match the filters on your Template Schemas and are good for targeting specific data in a large KML file.

 

 

The Options tab on the Import KML dialog contains some general options related to KML/KMZ files.  The “Update existing features using Placemark ID’s” option is used with a Placemark ID to find an existing record to update.  When exporting KML, you can specify which field to use for the Placemark ID.  The “Delete all records” option removes existing records from a feature class before the import takes place. This option is useful when you need to re-use a feature class.  In the event that you need to download data loaded in Network Links contained within a KML/KMZ file you can use the “Download all Network Links” option. This option follows nested Network Links and downloads all data associated with these Network Links.  The “Ground Overlays” option downloads all ground overlays in a KML/KMZ file as raster images, and you may specify a folder to place the downloaded images and associated world files.

 

 

Click here to see a demonstration of using Arc2Earth to import an existing KML/KMZ file into an ESRI geodatabase. 

 

For more detailed information on Arc2Earth, please see our new e-learning course.

 

Take advantage of special holiday savings of up to 20% on GIS e-learning courses from GeoSpatial Training Services.

 

Replacing ArcIMS with Arc2Earth Publisher?

James Fee wrote about the “The Day ArcIMS Died” in an April 23rd blog entry on his Spatially Adjusted blog.  James argued that you might be better off saving the money, time, and effort you’re investing in ArcIMS and replace it with Arc2Earth.  As James mentioned, “By not being able to serve up tiles, ArcIMS is slow, outdated and ugly. Time to move on (and I say this with great regret because I’ve rolled out more ArcIMS sites than I can remember over the years).”  At a fraction of the cost of ArcIMS, Arc2Earth is an efficient way to publish your ArcGIS data into a simple web based application.  Arc2Earth is particularly well suited for organizations that want to provide online mapping functionality, but don’t have the hardware (servers), software budget for ArcIMS or ArcGIS Server, or technical capabilities to develop custom applications.  They simply want to provide online access to their maps and data without any hassle.  I’m not here to denigrate ArcIMS or any other ESRI product.  In its time, ArcIMS was certainly a force in the delivery of online mapping applications.  However, Arc2Earth provides an excellent, and in many ways a superior alternative to ArcIMS for some users.  For anyone unfamiliar with Arc2Earth please read our past entries for an overview of its capabilities.

In this post I’ll show you the mechanics of exporting your ArcGIS data to Google Maps or Microsoft Virtual Earth from Arc2Earth using Amazon S3 to store your output data.  This solution set is an excellent way for organizations to efficiently and inexpensively publish their ArcGIS data to an online mapping application.  Arc2Earth is an ArcGIS extension so you can use these tools directly inside ArcMap.  The diagram below gives you a visual depiction of the process of creating map tiles from ArcGIS data through the Map Tile Exporter and their ultimate destination inside an Arc2Earth generated Map Viewer. In this case the Google Maps viewer is used.  Click here to see a demonstration of this process.

The General tab (seen below) on the Map Tile Exporter includes options for naming your map, background color, map appearance, and image format. The map name should be unique across all maps and defaults to the name of the active data frame in ArcMap. Background options control the background color and transparency. The appearance options apply primarily to Google Maps and include parameters for including Google Maps base maps and hybrid maps along with an option for exporting map graphics. 

 

The Levels tab contains options and information concerning the various map levels that will be created. Map tiles will be created based on the format used by the particular mapping provider and consist of up to 22 levels with a Mercator World Projection applied. You can define a start and end level for your export, and Arc2Earth will export all levels between these two values. It is rare that you will need an application that needs all levels. Other options include an option for full extent export, the creation of debug tiles, and the creation of GE tiles and regions. Storage estimates are provided so that you can accurately gauge the storage requirements that will be necessary as a result of the output. Other options from this tab include the ability to define exclusion and inclusion masks, and a Restart option for Enterprise users.

An export mask is a polygon layer used during the export to limit the amount of geographic area processed. Masks are an important component in increasing the processing speed for large data exports. There are two types of masks that can be used during your export: exclusion and inclusion. All masks should be polygon layers.  An exclusion mask is a polygon layer used to exclude areas from the export process. Any map tile fully contained by any polygon in this mask is skipped during the export process.  An inclusion mask is the exact opposite of an exclusion mask in that only tiles that fall within this mask are exported. This process only checks tile chunks so polygons in this layer should be large enough to accommodate some extra overlay at the edges.

The Storage tab for the Map Tile Exporter is used to specify where your data will be exported, and can include local, remote web server, and Amazon S3 options. Local storage allows you to select a local directory on your computer. By default, map tiles will be stored in the My Documents\KML Documents\Export Data\<Your Map Name> folder. Remote storage can be to a local server or a web server. Local servers normally refer to a web server virtual directory. If you have an Amazon S3 account you can enter your access and secret keys. You will need to have an account through Amazon to use this option.

The Viewers tab can be used to decide which viewers to create for your map, and the options include Google Maps, Google Earth, and MS Virtual Earth. When selecting the Google Maps option you will need to enter the API key you were assigned when creating a Google Maps account.

Let’s take a look at a demonstration of this process.  Pretty impressive!

For more information on e-learning or instructor led courses provided by GeoSpatial Training Services please click here.

Using Arc2Earth to Create 3D Google Earth Displays with ArcGIS Data

As I’ve mentioned in previous posts on Arc2Earth, one of my favorite functions is the ability to create 3D Google Earth displays using existing ArcGIS data.  The ability to visualize your ArcGIS data in a 3D display makes for stunning visualizations, but it can also help reveal and emphasize patterns in your data.  For example, in a previous post I described how we used a combination of ArcGIS Desktop along with the Spatial Analyst extension, Arc2Earth, and Google Earth to analyze crime patterns in Houston, Texas through the use of heat-maps and 3D display of various crime categories.

In this post we’re going to explore how you can use Arc2Earth to export your ArcGIS data as 3D extrusions.  One of the most visually appealing options in Arc2Earth is the ability to create 3D Google Earth extrusions from your ArcGIS data.  Clicking the Extrude Options button on the Layer tab brings up the Layer Extrude dialog which allows you to extrude a layer based on attribute values in your feature class, set specific values for all features, or use existing Z values of a layer.

Let’s take a closer look at the Layer Extrude dialog which is used to display vector data in three dimensions inside the Google Earth viewer. The Altitude Mode options allows you to set a value of absolute (based on sea level), relative (based on the current terrain level) which is the default, or clamped which is based on the current terrain level but clamped to the ground. In addition, you can extrude based on a number of options including Z values of a layer (if available), attribute values in a field (must be a numeric field), or a set value in meters for each feature. The visibility of extrusion walls can also be controlled through the “Show extrusion walls” checkbox. Finally, extrusion height can be calculated based on a percentage of all field values in relation to a minimum and maximum height in meters.

Now let’s examine the various options for performing 3D extrusion in Arc2Earth. As previously mentioned, you can extrude based on a number of options including Z values of a layer, attribute values in a field, or a set value in meters for each feature. Some ArcGIS layers already have valid Z values associated with their geometry data and are said to be “Z Aware”. In these cases it is possible to use the Z values associated with each feature as the value which will be extruded. Other methods of extrusion include the capability of extruding based on tabular attribute data for the layer. This uses numeric attribute fields to determine the height of extrusion. All data in this case is assumed to be in meters although feet can be selected as well. In addition, you can use the “Calculate Extrude height…..” check box to use a percentage of the min and max you enter to derive the extrude value. Finally, you can extrude each feature by a set value which you enter. This value is calculated in meters.

For more information about Arc2Earth please see our e-learning course “Arc2Earth for ArcGIS Users“.

Samples

Importing Raster Imagery into ArcMap with Arc2Earth

As I mentioned in my last post, we are in the process of creating a new e-learning course covering the Arc2Earth V2 ArcGIS Extension.  Arc2Earth was originally created as an efficient tool for exporting existing ArcGIS data to Google Earth.  However, Arc2Earth has evolved into a much more robust tool capable of advanced export and import functions between ArcGIS, Google Earth, Google Maps, and Microsoft Virtual Earth.  In addition to being able to export your existing data from ArcGIS, Arc2Earth also provides import tools that give you the ability to import imagery and hybrid raster tiles from Microsoft, Yahoo, and Ask.com as custom layers in ArcMap.  What this does is give you the ability to add in background aerial imagery to your ArcMap data or layout view similar to the figure you see below.

 

In this figure we’ve added Microsoft Aerial imagery in the form of a new raster layer (Microsoft - Aerials) that has been added to the ArcMap table of contents.  As mentioned earlier, you can also add imagery layers from Yahoo or Ask.com in addition to Microsoft.

The loading of these custom raster layers is accomplished through the use of the Map Tile Layer toolbar which can be loaded by selecting “Show Map Tiles Toolbar” from the Import menu on the main Arc2Earth toolbar. 

The Map Tiles toolbar has a number of buttons and tools that allow you to turn the custom aerial imagery layer on/off, automatically refresh the tiles, change the map source specification (Microsoft, Yahoo, Ask) as well as the type (street, aerial, hybrid), set the zoom level, and open the Tile Download Status window. 

Arc2Earth downloads the raster tiles over the Internet in small chunks, and seams them together for display in ArcMap.  In addition, Arc2Earth monitors the current map extent and downloads the appropriate tiles to display for the current map extent.  Tiles are automatically georeferenced so that they will display no matter what spatial reference you have set in your map. 

The Arc2Earth Tile Download Status window tracks the map tiles that have been downloaded from the Internet.  This window lists the status, Url of the tile, tile level, and download duration.  The status of each tile can be either cached, complete, or error.  You can also preview each downloaded image by clicking the item.

For more information on e-learning or instructor led courses provided by GeoSpatial Training Services please click here.

Introducing Arc2Earth V2

 

GeoSpatial Training Services is in the process of creating a new e-learning course covering the Arc2Earth ArcGIS Extension (Version 2). Over the next few posts I’m going to be taking a look at the latest release of Arc2Earth and describing some of the wonderful functionality this product provides for translating your existing ArcGIS data into readable formats for Google Earth, Google Maps, and Microsoft Virtual Earth.  As an ArcGIS extension, Arc2Earth was initially built as a highly efficient tool for exporting ArcGIS data including the geometric representations of the data, labels, renderers, graphics, and layouts into a format that Google Earth understands.  However, as you’ll see, Arc2Earth has evolved into more than just an ArcGIS data export tool for Google Earth.  In addition to all the great export functionality you can also import KML/KMZ files directly to ArcGIS in geodatabase format, view raster tiles (aerial and hybrid imagery) provided by Microsoft, Yahoo, and Ask.com from within ArcMap, export your data as map tiles to Google Maps and Virtual Earth, publish data to web enabled viewers, automate exports through the scheduler process, and much more.

With the recent release of Arc2Earth V2, you now have four different versions from which to choose, each with different levels of functionality.  For detailed version and pricing inforrmation please click here.  All of the versions share the same installer and software, and a license key is used to unlock their functionality.  You can review the Feature List to determine which features are in each version.  Arc2Earth functionality is accessed through the Arc2Earth toolbar in ArcMap and the supporting Map Tile Layer toolbar. Let’s review the functionality provided by this tool. 

Exporting Data to Google Earth, Google Maps, Virtual Earth
The bread and butter utilities provided by Arc2Earth center around the ability to export your ArcGIS data into Google Earth format files (KML/KMZ) so we’ll review those first.   From the Export pulldown menu, Arc2Earth gives you the ability to perform a number of data exports.  You may choose to export only the selected layer in your ArcMap table of contents, export all layers in the table of contents, take a quick snapshot (this generates a KML Ground Overlay image) of your ArcMap data or layout view, create map tiles for import into Google Maps or Virtual Earth, export selected graphics, or create a Google Earth Superoverlay if you have large amounts of data.  The figures at the bottom of this post show various exports taken from ArcMap using Arc2Earth.

Some of the more advanced export capabilities include the ability to add time values to your exported data, create regions for displaying large amounts of data in Google Earth, extrude data for 3D data display, and the creation of map tiles for viewers.

Time Data
Using the Time Data tab in the Arc2Earth Export dialog you can add time data to your exported data.  Google Earth will then display the timeline control and allow the user to change the current timeframe of the globe.  Time values can be added manually or can come from existing attributes in your data.  Click here to learn more about the Google Earth time control.

Regions
KML Regions are a powerful feature that allows Google Earth to stream your data in pieces instead of as a whole.  If you have large datasets, this is the best way to display your data.  Regions can also be used to display data only when the user’s camera is at a certain altitude.  Regions can be created using the Regions tab in the Arc2Earth Export dialog.  To learn more about Google Earth Regions click here.

Data Extrusion
I think this is probably my favorite export capability in Arc2Earth because of the stunning visualizations that are possible.  In Google Earth, vector data can be displayed in 3D, and by using an attribute field or existing Z-values in your ArcGIS data you can export 3D data similar to the figures you see below. 

Map Tile Exporter
The Arc2Earth Map Tile Exporter can be used to create map tiles for Google Maps, Microsoft Virtual Earth, or Superoverlays for Google Earth.  Map tiles can be exported to a local computer, web server, or Amazon S3 storage.  Map viewers can also be created for each of these products similar to the screenshot below.

Importing Data to ArcGIS 
In addition to being able to export data from ArcGIS, Arc2Earth also provides import tools that give you the ability to visualize imagery and hybrid raster tiles from Microsoft, Yahoo, and Ask.com as custom layers in ArcMap.  This gives you the ability to add in background aerial imagery to your data or layout view.  Notice in the figure below that a Microsoft Aerials layer has been added to the table of contents.

The loading of these custom raster layers is accomplished through the use of the Map Tile Layer toolbar.  Arc2Earth downloads the raster tiles over the Internet in small chunks, and seams them together for display in ArcMap.  In addition, Arc2Earth monitors the current map extent and downloads the appropriate tiles to display for the current map extent.  Tiles are automatically georeferenced so that they will display no matter what spatial reference you have set in your map.

In addition to opening the Map Tiles Toolbar, the Import menu item can also be used to import KML, KMZ, GeoRss, and Atom files into ArcMap.  The importer can either create a new feature class or update an existing class in either a PGDB or ArcSDE instance.

Arc2Earth Geoprocessing Tool
Arc2Earth integrates into the ArcGIS geoprocessing framework by providing a custom geoprocessing tool that can be used from any script or model or directly from ArcToolbox.  This geoprocessing tool can be a great resource when combined with other tools or in a model as the final output parameter.

Arc2Earth Scheduler
The Arc2Earth Scheduler is a separate program that runs outside the main Arc2Earth extension, and is only available with the Enterprise license level.  Scheduler maintains a list of jobs that will run at user defined intervals (hourly, daily, weekly, etc).  This is great for automating regularly scheduled exports to Google Earth, Google Maps, or Virtual Earth. 

One Last Tidbit
 Display Microsoft Virtual Earth inside ArcMap.  More information coming soon on this!

Arc2Earth has many other useful functions for integrating ArcGIS with Google Earth, Google Maps, and Virtual Earth and we’ll go into more detail in our posts in the near future.

For more information on e-learning and instructor led courses provided by GeoSpatial Training Services please click here.

Arc2Earth Screenshots