Using Spreadsheet Mapper 2.0 with Google Earth & Google Maps

Over the next few weeks I’m going to be writing a series of posts on some of the lesser know tools provided by Google for creating data layers in Google Earth and Google Maps.  The Google Earth Outreach program provides a number of these tools.  According to a press release from early last summer, Google Earth Outreach is “designed to help nonprofit organizations around the world leverage the power of Google Earth to illustrate and advocate the important work they do.”  The program includes comprehensive online guides, video tutorials, and case studies about using Google Earth specifically targeted to the needs of nonprofit organizations.  In addition, organizations can also apply for a Google Earth Pro grant ($400 value). 

In this initial post we’ll take a look at the Spreadsheet Mapper 2.0 tool created by the Outreach team for creating Google Earth and Google Maps placemark layers using Google Docs.  Google Docs is a great way to create, share and collaborate on documents, spreadsheets and presentations online.  Spreadsheet Mapper takes advantage of this online, collaborative environment by allowing you to create placemark layers for display in Google Earth and Google Maps through a spreadsheet created in Google Docs.  Because Google Docs is a collaborative tool, members of your team can simultaneously enter data and instantly publish updates to GE and GMaps.  So, let’s take a look at how this is done through a basic example.

In this example we’re going to create a placemark KML file containing existing Starbucks locations. The Starbucks data contains a unique identifer for each location along with the latitude, longitude coordinate pair and a physical address.  This data is contained within the “Starbucks No Linefeeds.csv” file which was last updated in January 2008. This file contains almost 9,000 Starbucks locations.  Since there are so many locations I’m only going to use a sample area for this example. The download also contains a Starbucks.bmp image which we’ll use in our next post in the series which covers Spreadsheet Mapper templates for styling icons and information balloons.  Let’s get started.  By default, Spreadsheet Mapper will use various templates to create some sample data.  We’ll do this first and then replace the sample data with our own Starbucks location data.

1. Open the starter spreadsheet.  If you haven’t already done so you’ll need to login to your Google account or create one if necessary.
2. Select File –> Rename and give you spreadsheet a name (”Starbucks Locations”).
3. Fill in the “Author’s Information” and “About your KML Document” sections.
   
4. Optional Parameters
   • Enable “Google Maps Compatibility” if you want the layer to work in Google Maps
   • Access the “Advanced/Optional Settings” by clicking the tab indicated on the left to un-hide rows
5. Click the Publish tab and select Publish Now.  This step will publish the document to the web at the URL listed.
   
6. Copy the publisher URL and paste it into the white cell provided under “Publish spreadsheet”.
   
7. Copy the “Network Link KML” cell that you see below, open Google Earth, select My Places, then right-click and paste.
   Network Links capability in Google Earth provides for the delivery of dynamic data to your users.  We are using a Google Docs spreadsheet which can be edited by multiple users simultaneously.  Network Links in Google Earth are a perfect complement to a Google Docs spreadsheet since they can automatically refresh the Google Earth display to reflect updated data from a spreadsheet.  Get more information on Network Links in our “Mastering KML in Google Earth” e-learning course.So, at this point we’ve copied the sample data contained in the template into Google Earth.  You will notice a variety of folders, placemark icons and information balloons which have been created based on parameters found in the templates contained within Spreadsheet Mapper.
   
   The Spreadsheet Mapper comes with six templates that can be used to control icon and balloon styles.  Click any of the links at the bottom of the spreadsheet to get more information about each of these templates.  We’ll cover detailed information about the templates in a coming slide, but for now you can get an idea of how they are structured.
   
8. At this point you’d want to prepare your template values.  However, as I mentioned above I’m going to save the details of altering the template values for another post since it really deserves a thorough explanation. 
9. Now let’s add in the Starbucks data.  Open the “Starbucks No Linefeeds” Excel spreadsheet which contains the latitude,longitude, identification, and address values for each Starbucks location.  Due to the large size of the file I’m only going to add in Starbucks locations for my city, San Antonio.  Go to the PlacemarkData link at the bottom of the spreadsheet.
   • The Folder Name is optional, but in this case we’re going to define the folder name as “Starbucks” for each placemark instance.  What this will do is group the placemarks under the same Folder in Google Earth.  This would be helpful if you’d like to group Starbucks locations by city (i.e. San Antonio, Dallas, Houston, Austin). 
   • For the Placemark Name column I’m going to add in the unique identifier for each location based on the information pulled from the Excel spreadsheet we downloaded.  You should be able to copy and paste the data from MS Excel to the Google Docs spreadsheet to save time.
   • We’ll also enter the latitude, longitude values for each Starbucks location, also pulled from the Excel spreadsheet we downloaded.  If you don’t have coordinate values for each placemark you can enter an address which can be used to generate the placemark. 
   • Finally, we specify a template (#5 in this case) to define our icon and balloon styles.  The spreadsheet should look something like you see below.
10.  Click Publish –> Re-publish document in your Google Docs spreadsheet.  Then refresh your network link in Google Earth by right clicking on “Link to - Spreadsheet” in the Places panel and selecting Refresh.  This should refresh the Google Earth display with the new Starbucks locations we entered in the Google Docs spreadsheet.

 

In our next post we’ll cover the templates provides by Spreadsheet Mapper and also take a look at how you can create your own templates.

March Sale on Google Earth and Google Maps E-Learning Courses

During the month of March we are offering a 15% discount on our “Google Earth and Google Maps Bundle” e-learning set.  Sale price is $170.00 (e-delivery), $210.00 (mail delivery).  This bundle is composed of 5 e-learning courses with over 450 pages of instruction.  Courses include:

• Mastering KML in Google Earth
• Google Maps For Your Apps
• Google Earth for ArcGIS Users
• Arc2Earth for ArcGIS Users
• Dynamic Google Earth Applications (in development)

Creating KML Regions from ArcGIS Data Using Arc2Earth

Our previous posts on basic and advanced KML Region concepts gave you some background on how this functionality can be used allow you to stream your data in pieces instead of as a whole making it possible to deliver large datasets to the Google Earth viewer.  In this post you will discover how to use Arc2Earth with your existing ArcGIS data to create these powerful features.

 Arc2Earth can be used to create Google Earth Regions through the Regions tab with the Publisher version of Arc2Earth.  The General Options button displays the Region Options dialog which can be used to set the fade effects and level of detail (LOD) for the region.  You must also select a region level which is basically a course grain level similar to the levels of map tiles used in Google Maps or MS Virtual Earth.  Each level represents an approximate map scale.  The Visualize button displays a map window with the grid showing the regions for your export along with your layer data. 

The Visualize window is particularly helpful in allowing you to see the extent of the Regions that will be created along with your data.  The red lines displayed in the viewer indicate how your data will be regionalized.  You can also dynamically reset the region level through this window to see the effect this will have on the size of the Regions.  Map navigations tools are also provided.

Demonstration
Click here to see a demonstration of using Arc2Earth to export parcel and floodplain data as KML Regions.

More Information
GeoSpatial Training Services provides e-learning courses for GIS users.  If you would like more information on Regions and other KML elements please see our “Mastering KML in Google Earth” e-learning course.  Other related courses include “Google Earth for ArcGIS Users“, “Google Maps For Your Apps“, “Arc2Earth for ArcGIS users“, and our “Google Bundle” which combines all these courses into one package at a significant discount.

Advanced KML Region Concepts (Altitude, Fade Extent, Nested Regions)

In our last post on KML Regions we covered some background information on KML Regions and the various ways in which they can be used in Google Earth to display large GIS datasets.  In this post we are going cover additional Region subjects including altitude, fade extent, and nested regions.

Bounding Box 
As we mentioned in our previous post, the bounding box and level of detail (LOD) are two of the basic, key concepts in creating Regions.  A bounding box, defined by the <LatLongAltBox> element, describes an area of interest defined by geographic coordinates and altitudes.  This element contains the child elements <north>, <south>, <east>, and <west> that define the boundaries of the Region.  A Region is considered “active” or visible when the bounding box is within the display and the level of detail requirements are met. 

Level of Detail 
The Level of Detail (LOD) is defined with the <Lod> child element of <Region>.  It defines a range, specified by <minLodPixels> and <maxLodPixels> that determines the visibility of data within a Region.  This ensures that large amounts of data are only loaded when enough pixels are available to display the data adequately.  When the Region takes up a relatively small percentage of the screen, the LOD allows you to specify a dataset with a lower resolution.  The <Lod> value units are defined by square pixels.  Data must occupy an area greater than <minLodPixels> and less than <maxLodPixels> to be visible. 

 There are a number of other Region concepts that you need to be familiar including altitude, fade extent, and nesting regions. 

Altitude
The <LatLonAltBox> element can contain child elements which control the minimum <minAltitude> and maximum altitudes <maxAltitude> at which a Region will display.  Notice in the figure below that the minimum altitude at which the Region will display is 10000 meters and the maximum altitude is 50000 meters.  These child elements are containing within the <LatLonAltBox> element. 

Fade Extent
By setting a fade extent for a Region, you can enable your objects to transition seamlessly from transparent to opaque, and back again.  You do need to be careful when setting a fade extent for a Region because they are computationally expensive, and should only be used with vector data such as LineStrings, Polygons, and Points, but not with imagery data.  Fade extents are set with the <minFadeExtent> and <maxFadeExtent> child elements of <Lod>.  These are set as pixel values similar to the values you set for <minLodPixels> and <maxLodPixels>, and are used in conjunction similar to the code you see below.

The <maxFadeExtent> is used to determine the ramp from fully transparent to fully opaque when the Region is at its maximum visible size.  The <minFadeExtent> element is used to determine the fade ramp when the Region is at its minimum visible size. 

Nesting Regions
Regions can be nested so that smaller, increasingly finer levels of detail are shown as the user zooms in on the display.  Previously loaded coarse levels of detail are gradually replaced with these more detailed Regions. 

<LatLonAltBox> elements in a child Region should be wholly contained within the <LatLonAltBox> of its parent Region. 

In our next post we’ll show you how to use Arc2Earth to create Regions from your ArcGIS data.

More Information
GeoSpatial Training Services provides e-learning courses for GIS users.  If you would like more information on Regions and other KML elements please see our “Mastering KML in Google Earth” e-learning course.  Other related courses include “Google Earth for ArcGIS Users“, “Google Maps For Your Apps“, “Arc2Earth for ArcGIS users“, and our “Google Bundle” which combines all these courses into one package at a significant discount.

Creating Dynamic Google Earth Applications with Python

In previous posts I’ve covered the general concepts and underlying KML that drives Google Earth Network Link functionality.  What I’d like to do in this post is introduce you to the concept of using CGI scripting with the Python programming language to create dynamic mapping applications.   Python is an easy to use, open-source, extensible, interpreted language which many ESRI users are already familiar with for accomplishing geoprocessing tasks in ArcGIS. 

The Task
The goal of this post is to create a dynamic Google Earth application showing active wildland fire events in North America.  To accomplish this task we’re going to use the Network Link functionality provided by Google Earth in conjunction with a Python CGI script that will process comma delimited text files provided by the Fire Information for Resource Management Systems (FIRMS).  FIRMS generates new files each day and updates are applied to the files at approximately four hour intervals throughout the day.

Server Setup
A Google Earth Network Link gives you the ability to serve data from a remote server and is commonly used to distribute data to large numbers of users.  In this case we’ll be using a web server running IIS to process the Python script.  You will need to make sure that Python has been installed on the server that will run the script(s) and that your web server software (IIS in this case) is properly configured to run Python scripts.  Please click here to get detailed information on configuring IIS to use Python.  Please note that this step is not optional and your Python script will not run correctly until the web server has been configured.

Overview
In the figure below you will see a representation of the basic process of what takes place in a dynamic Google Earth application.  We’ll start with a KML file (WildlandFire.kml) which contains <NetworkLink> and <Link> elements that point to a Python script called generateFirePoints.py.  This script reads the daily FIRMS wildland fire data for North America and parses out the geographic coordinates (latitude, longitude) along with a confidence value and generates a KML data stream containing the wildlife fires to be plotted.  Finally, the data is presented in Google Earth. 

Creating WildlandFire.kml
My previous post gave you the details on the KML elements related to Network Links so I’m not going to rehash this information here, but I do want to cover the relevant elements used to point to your Python script.  As you may recall, the <NetworkLink> element contains a <Link> child element which can be used to point to a KML/KMZ file or CGI script.  The <href> child element of <Link> is where you specify the URL to your CGI script.  For example:

<Link>
   <href>http://www.geospatialtraining.com/GE/scripts/generateFirePoints.py</href>
</Link>

The KML file that contains this link (WildlandFire.kml) makes a call to the Python script generateFirePoints.py located on a remote web server.  Click here to retrieve the full text of the WildlandFire.kml file.  To see the KML code you’ll need to open this file with a text editor.  Note the <refreshMode> and <refreshInterval> elements.  The value “onInterval” in the <refreshMode> element specifies that the Python script will be called on a time interval.  The time interval is given in seconds using the <refreshInterval> element.  In this case we’re using a four hour time interval. 

Python Script Contents
Our Python script (generateFirePoints.py) needs to do a number of things to correctly deliver a KML stream back to Google Earth. 

• Return a response code of HTTP 200 (the web server takes care of this)
• Set the response’s content type to text/plain or application/vnd.google-earth.kml+xml
• Parse the FIRMS wildland fire text file for current fire locations
• Build the appropriate KML elements and send to GE in a data stream

Response Content Type 
When responding to a request from Google Earth, you must set the response’s content-type to a suitable MIME type.  The MIME type for KML files is

• application/vnd.google-earth.kml+xml

Before returning the KML stream containing your data you must set this MIME type which can be done with the following line of Python code:

print “Content-Type: application/vnd.google-earth.kml+xml”

Parsing the File
Each day FIRMS integrates remote sensing and GIS technologies to deliver hotspot/wildland fire locations in multiple formats including comma delimited text files containing coordinate information in the form of latitude and longitude coordinates along with other attribute information including the time of acquisition and a confidence value.  For our purposes in this example we’ll primarily be concerned with pulling out the coordinate values for plotting in Google Earth.  The text files are posted to the FIRMS ftp site each morning at 00:00 UTC.  The file continues to be updated throughout the day as new data becomes available.  The format for these files is as follows:

Latitude
Longitude
Brightness Temp
Along scan pixel size
Along track pizel size
Date
Time of acquisition
Satellite (A=Aqua and T=Terra)
Confidence

And here is an example:
36.549,-82.964,318.7,1.0,1.0,10/02/2007/1625/T/70

To download the sample file I’ve used in this post click here.

In this example we’re primarily interested in pulling out the latitude and longitude coordinates which will then be streamed as KML elements to Google Earth.  File manipulation including parsing comma delimited text files is extremely easy to do in Python.  Let’s examine the final generateFirePoints.py file.  Right Click and select Save to download a copy of the file.  Go ahead and open the file using a Python editor such as IDLE or with a text editor like Wordpad.  The first few lines of code simply set content type, KML header, KML body, and KML footer variables.  The only real item of note is in the kmlBody variable where we set up the <Style> element for the icon that we will use to display wildland fires in Google Earth. 

The real heart of the Python script is the next section.  We first open the FIRMS file and read in the contents of the file one line at a time into a list variable.  For purposes of this example we’ve hard-coded the path to the file name.  FIRMS creates new files each day for North America and the rest of the world and the filenames reflect the continent plus the year and Julian date so obviously hard-coding the file name would not work in a real application since the most up to date file name changes daily.  However, for our purposes in this example it will suffice.

We then loop through each line in the file and pull out the latitude, longitude, and confidence values, and write them into a KML stream.  The end result is a unique <Placemark> for each wildland fire that will be displayed in Google Earth.

Finally, we send the KML stream to Google Earth through the print statement along with the final KML output.  Notice that we first send the content type through the “print contentType” statement.  You will also need to close the FIRMS file.

The resulting display in Google Earth will look something like the figure below.

Our “Mastering KML in Google Earth” and upcoming “Creating Dynamic Google Earth Applications” cover these subjects in great detail.  For more information regarding training services from GeoSpatial Training Services please click here.

The KML Behind Network Links

In my last post on Google Earth Network Links I covered the basic concepts of how this functionality can be used to provide dynamic access to geographic data in the Google Earth viewer.  We briefly discussed the KML elements that are used to define a Network Link, but in this post I’d like to expand on the KML elements used to enable this functionality.

<NetworkLink>
The <NetworkLink> element inherits from the <Feature> element as do the <Placemark>, <Overlay>, and <Container> elements.  You can get more information on the KML Object Model here.  Because it is descended from <Feature>, <NetworkLink> inherits a number of child elements which define the basic characteristics of a Network Link which I won’t go into detail on here (see the Feature object in the KML Object Model for details).  The elements specific to <NetworkLink> include <Link>, <refreshVisibility>, and <flyToView>.  A <Link> with an <href> child element specifies the KML/KMZ file to load, and this file can either be from a local or remote source.  For example, the following code shows how to create a link to a remote KML source:

<Link>
     <href>http://www.geospatialtraining.com/NetworkLink/SeePlacemarks.kml</href>
</Link>

In addition to linking to an existing KML/KMZ file you can also link to CGI scripts written in languages such as Python, Perl, Ruby, and others which can be created to provide dynamic data.  We’ll save this topic for another post.

Another element of <NetworkLink> is <refreshVisibility> which by default is set to a value of “0″ and leaves the visibility of features within the control of the Google Earth user.  Setting the value to “1″ resets the visibility of features each time the Network Link is refreshed.

The <flyToView> element when set to a value of “1″ causes Google Earth to fly to the view of the root element in the refreshed file.  <LookAt> defines the camera for the view. The camera is the current viewpoint of the Google Earth viewer and includes things like the latitude, longitude the camera is looking at along with the altitude, range, tilt, and heading.

Here then is a quick KML code sample for creating a Network Link:

<NetworkLink>
     <name>Test Placemark</name>
     <visibility>0</visibility>
     <open>0</open>
     <description>This is a KML Network Link Test</description>
     <refreshVisibility>0</refreshVisibility>
     <flyToView>0</flytoView>
     <Link>
          <href>http://www.geospatialtraining.com/NetworkLink/SeePlacemarks.kml</href>
     </Link>
</NetworkLink>

<NetworkLinkControl>
Now let’s move on to the <NetworkLinkControl> element which controls the behavior of files fetched by <NetworkLink>.  There are a handful of child elements of <NetworkLinkControl>, but the most important are <minRefreshPeriod>, <expires>, and <Update>.  Let’s start with <minRefreshPeriod>.  This element specifies the minimum allowed time between fetches of the file in seconds.  Therefore, if you want the minimum update time to be one hour this element would be set to <minRefreshPeriod>3600</minRefreshPeriod>.  The <expires> element is used to specify a date/time when the link should automatically be refreshed. 

<Update>
The child element of <NetworkLinkControl> that I want to highlight in particular is the <Update> element.  <Update> is used add, edit, or delete data from an existing KML file that has already been loaded.  This is powerful stuff! The <targetHref> child element specifies the KML/KMZ file that you would like to alter in some way.  Three child elements (<Change>, <Create>, and <Delete&gt of <Update> are used to perform the editing tasks.  The <Change> element modifies the values of a geographic object that has already been loaded with <NetworkLink>.  Within the <Change> element, the object to be modified must include a targetID attribute that references the original element’s id.  The <Create> element is used to add new objects, and the <Delete> element deletes features that have already been loaded.  Here is an example of using the <Change> element with <Update>.

<NetworkLinkControl>
     <Update>
          <targetHref>http://www.geospatialtraining.com/GE/Data/Points.kml</targetHref>
          <Change>
               <Point targetId = “point001″>
                    <coordinates>-97.23,35.35,0</coordinates>
               </Point>
           </Change>
     </Update>
</NetworkLinkControl>

The ability to add, edit, and delete data from an existing KML/KMZ file served through a Network Link opens up a lot of possibilities for delivering dynamic Google Earth applications to your end users.  This functionality combined with CGI scripting through Network Links makes it even more powerful.  For more information on the update functionality provided through <NetworkLinkControl> please click here.

In the next post on Network Links I’ll show you how to use a CGI script to dynamically generate data served through a Network Link.

Our “Mastering KML in Google Earth” course covers Network Links in great detail along with all the other ways that you can create custom Google Earth applications using KML.

Dynamic Google Earth Applications with Network Links

Introduction
Over the next few posts we are going to look at a how you can serve up dynamic Google Earth applications to your end users through the use of network link functionality.  Network link functionality provides a way for multiple clients to view the same network-based or web-based KMZ data and automatically see any changes to the content as those changes are made.   

How it Works
Network links are functionally similar to traditional web content publishing in that they allow for the delivery of dynamic data to multiple users.  By distributing traditional network links or web links to your KMZ files, users can obtain a view only reference to your data through the GE interface.  As the content provider you can specify how often to refresh the data in the file through various GE tabs, KML parameters, or even CGI scripts that dynamically update the data.  Data is updated only at the source and provides a single point of entry for your specific data sets.  End users can also specify how often to refresh the data through the GE interface.  Although end users can specify how often to refresh data from the source, they can not modify the existing KML file.  It is possible, however, to save the file to their local computer.  The figure below gives you a visual depiction of this process.

Advantages
The advantages of sharing Google Earth data with your users through this method include accessibility (can be accessed via the web), ease of distribution, automatic data updates, and centralized backup.   

How to Create a Network Link in GE
Creating a network link from the GE interface is a simple process, but you will need to make sure that your content (KML/KMZ) has been saved to the server you are linking to.  Select Add à Network Link from the GE menu.  Give your network link a name, and specify the link to your file on a server.  The “Refresh” tab can be used to specify how frequently your data is refreshed.  See the figures below for more information on created a network link through the GE interface. 

 

KML
The primary KML elements used to define a network link include <NetworkLink> and <Link>.  <NetworkLink> is used to define the existence of a network link while <Link> combined with <href> specifies the location of the KML/KMZ file to load.  This can be a local link or a link to a remote server.  For instance, in the code example below, the <Link> element and <href> child element point to a dynamic Python CGI script (http://yourserver.com/cgi-bin/randomPlacemark.py) that will dynamically run when the network link is referenced.  Other important elements include <refreshVisibility> which is used to specify the data update frequency, <name> which contains the descriptive name of your network link, and <description> which as its name implies refers to the description you give to the link.

Dynamic Scripting
In my opinion, the most compelling reason to use network link functionality is the ability to dynamically update the data source on the fly through the use of CGI scripting languages such as Python, Perl, Ruby, PHP, and others.  In this case, each time the network link is accessed, a specific script is triggered which generates KML data streams to the network link.  This provides just in time access to the latest data for your application. The process works as follows:

• Client (Google Earth) calls the server (KML/KMZ file)
• Server (KML/KMZ) contains a pointer to a CGI script
• CGI scripts runs and returns a KML stream (dynamic KML) containing the data for your application

 This provides powerful functionality for generating dynamic KML content. 

Examples
See some network link examples at Google Earth Hacks and Google Earth Blog. 

More Information
In the next post we’ll take a look at a specific example of how you can combine a Python script with network links to generate dynamic Google Earth content.  Our upcoming course “Creating Dynamic Google Earth Applications” covers the topic of network links in great detail.

For more information on integrating ArcGIS and Google Earth please consider our “Google Earth for ArcGIS Users” virtual training course.