Archive for August, 2007
Integrating ArcGIS and Google Earth for Crime Analysis
I’m going to be taking a slightly different approach in this post than I have in previous posts. Typically, GeoChalkboard focuses on providing how-to information regarding the use of ESRI, Google Earth/Maps, and other geospatial technologies. As such we normally provide very detailed instructions regarding how to use certain aspects of these software technologies. However, in this post I’m going to describe at a higher-level the output of a crime analysis project I recently completed for Texas State University. My intent is to expose you to some of the possibilities for integrating these technologies in a way that takes advantage of the advanced analysis and visualization techniques that are possible. Based on some of my previous posts you can probably tell that I’m particularly interested in the integration of ArcGIS Desktop with Google Earth. In my opinion, this is a great combination for analyzing and presenting spatial data. ArcGIS at the desktop level is a fantastic tool for analyzing and modeling spatial data, and Google Earth provides a visualization platform that is second to none. By combining these two products you can provide not only advanced analysis techniques, but also present compelling visualizations of the resulting data.
Overview
Dr. Tina Cade in the Horticulture Department at Texas State University approached GeoSpatial Training Services regarding the use of geospatial technology for determining the effect of community gardens on crime rates in the areas surrounding these small plots of land that are gardened by a group of people. Community gardens have many benefits to both the human and natural environment including neighborhood beautification, social interaction, and as a catalyst for neighborhood development. Get more information on community gardens here. This particular project was interested in the effect that these community gardens would have on crime rates in the surrounding areas in the Houston, Texas urban area.
Project Methodology
The application of geospatial technology to this project had both analysis and visualization components. A number of statistical techniques were used to measure the effectiveness of community gardens in deterring crime, and ArcGIS was used to create various datasets that were used in the statistical analysis. However, I’m not going to focus on these aspects of the project in this post, but will instead focus on the techniques employed in the visualization component of the project. These techniques, carried out by using a combination of ArcGIS, Spatial Analyst, Arc2Earth, and Google Earth provided a strong combination of tools to generate and visualize crime data in Houston, and can be used as a general platform for many projects across a wide variety of industries. For visualization purposes, the study called for the creation of raster based crime “heat-maps” based on ¼ mile cell size along with 3D views of the same data.
Data Sources
The primary sources of data for this study were address level crimes for the year 2005 as provided by the Houston Police Department and the addresses of all community gardens in the Houston area. Crime data at the address level had already been geocoded into a shapefile format. Community garden data was also geocoded to a street level. We were primarily interested in the neighborhood level effect that community gardens would have on crime rates so the study areas for each community garden were relatively small (1/4 mile and 2 mile radii).
Creating the Heat-Maps
For visualization purposes, the study called for the creation of various heat-maps that would enhance the visualization of crime data in Houston. To accomplish this we used a combination of ArcInfo with the Spatial Analyst Extension, Arc2Earth, and Google Earth. Using our geocoded crime data, a series of raster grids were created by crime type using Spatial Analyst using a ¼ mile grid size. We then symbolized the resulting grid surface in ArcGIS and exported as an image overlay using Arc2Earth. Arc2Earth is an ArcGIS extension that can be used to export your GIS format data into a KML file for visualization by Google Earth. Here are some examples of the resulting Google Earth format crime heat-maps. This particular map shows burglary hot spots for the City of Houston in relation to community garden locations surrounded by a 2 mile radius (circles).
Creating 3D Maps
In addition to the heat-maps we also wanted to be able to visualize the data in 3D. To accomplish this we used ArcGIS to drape a vector fish-net composed of ¼ mile grids on the City of Houston. Next, we used a spatial join in ArcGIS to count the number of crimes by type in each grid. We used the resulting field as an elevation value in Arc2Earth to export the 3D screenshots that you see below.
More Information
For more information on integrating ArcGIS and Google Earth please consider our “Integrating ArcGIS Desktop and Google Earth” virtual training course.
GISCI Recognizes GeoSpatial Training Services
The GIS Certification Institute (GISCI) recently recognized GeoSpatial Training Services as an educational achievement credit provider for its Geographic Information Systems (GISP) certification. A broad range of education, experience, and professional contributions are necessary to achieve this certification. The educational achievement section plays a large role in qualifying applicants, and points can be earned toward certification through the successful completion of individual courses, workshops, and other formal, documented educational activities whose subject matter relates directly to geographic information technologies.
As a recognized provider of educational achievement credit through the GISCI Certification Program, GeoSpatial Training Services offers a diverse array of virtual training opportunities including the following:
- Programming ArcObjects with VBA – 40 hours
- Integrating ArcGIS Desktop and Google Earth – 24 hours
- Introduction to the Google Maps API – 16 hours
- Mastering Python for Geoprocessing in ArcGIS – 24 hours
- Working with Geodatabases and Linear Referencing– 16 hours
- Using AutoCAD Data in ArcGIS – 16 hours
Google Caching for GIS Field Operations
Introduction
A frequently overlooked capability of Google Earth is its ability to cache data on your local computer and therefore allow for field based visualizations of GIS data. In a typical office setting, Google Earth uses streaming imagery over a high speed Internet connection to provide you with images and vector datasets. However, many GIS users need to be able to visualize Google Earth imagery and their own internal datasets while in the field where high speed connections to the Internet are not available or impractical. In this post/article we are going to take a look at how you can cache Google imagery and vector datasets to your laptop for offline viewing in the field. In addition, we are also going to take a brief look at a number of tools that you can use to integrate your existing vector ArcGIS datasets with this cached data.
Potential Uses
Virtually any industry that makes use of GIS in field situations can benefit from the use of Google Earth caching technology to display high resolution imagery and vector datasets. Some, but certainly not all industries which can benefit from this capability include:
- Forestry
- Environmental Management
- Fire, Police, Emergency Mangement
- Local, State, and Federal Government
- Wildlife Management
- Agriculture
- Many Others
How it Works
As we mentioned earlier, Google Earth typically streams imagery over a high speed Internet connection from Google’s servers to your computer. As this data is streamed to your computer it is saved to the cache for performance reasons, but can also be used for offline viewing when you don’t have an Internet connection. The term “cache” as we use it in this case refers to files that are on your computer which store the imagery and/or vector datasets that are streamed into Google Earth. At a physical file system level, the cache is composed of a dbCache.dat and dbCache.dat.index file. These are the files where the imagery is actually stored and indexed, and they can be found in the c:\Documents and Settings\<username>\Local Settings\Application Data\Google\GoogleEarth directory as you can see below. Once you no longer have a connection to the Internet, Google Earth will display the information stored in these files.
The Google Earth cache is limited in size to 2GB, but by default it is not set to this upper limited so you will need increase the cache size so that it can hold the maximum amount of data. The cache uses a recently used algorithm. What this means is that the older data in your cache will be removed from the cache after the maximum amount of data has been stored in the cache and new data enters the cache. Because you are limited in size to 2GB you will need to limit the geographic area you intend to visualize in the field.
While creating your cache you will need to keep in mind that Google Earth will cache exactly what you see so make sure you are at a scale that gives you the level of detail you want. Furthermore, you must also move slowly enough to give Google Earth time to fully load the data into the viewer before panning, zooming, tilting, etc. Finally, make sure you turn on any vector layers you wish to visualize before you cache the data. This would include layers such as roads, borders, parks, etc.
How to Create the Cache
The cache parameters for Google Earth can be accessed by selecting Tools à Options from the main Google Earth menu and then clicking the Cache tab. This will present the dialog box that you see below. In addition to controlling the cache size from this dialog box you can also clear the memory and disk caches an delete the cache file if necessary.
The Disk Cache Size is the parameter we are most concerned with in this article as it controls the storage of data for later offline use. This parameter can and should be set as high as 2GB (2000MB) for instances where you are planning to work with Google Earth in an offline mode. The value you enter in this field should be set in MB which means a value of 2000 equals 2GB.
The Memory Cache Size parameter can be used to alter the amount of memory dedicated to processing data in Google Earth. This can greatly improve the performance of Google Earth, but can result in a performance degradation of other applications running on your computer. The available memory is dependent on the specifics of your computer. You don’t need to know the actual limits of your computer’s memory because Google Earth automatically limits the size according to the physical memory available on your computer.
Once these parameters have been set you will need to navigate to your area of interest and remember to keep in mind the following guidelines:
- Select a limited geographic area that covers your study area
- Make sure you are at a scale that gives you the level of detail you want
- Move slowly to give Google Earth enough time to fully load the data into the cache
- Turn on any vector layers you wish to capture in the cache
Click here to see a demonstration of using Google Earth’s caching functionality.
ArcGIS Tools for Integration of Organization Specific Data
Using the Google Earth cache is all well and good when you need to see Google Earth imagery and vector datasets, but many users also want to visualize their existing ArcGIS format datasets into the field as well. This can be accomplished through a number of existing ArcGIS Extensions. I’ll save a detailed discussioin of these extensions for a later post, but for now I’d like to highlight the process that can be used to accomplish this task. There are a number of superb ArcGIS extensions available for converting existing ArcGIS data into KML format including Arc2Earth, KML Home Companion, and Export to KML. In addition, there are also non-ArcGIS tools such as Shape2Earth which can also be used to convert shapefiles to KML. Some tools, like Arc2Earth Publisher enable you to schedule the export of your data at pre-determined times. For instance, you could use the scheduling functionality in Arc2Earth to automatically publish your ArcGIS data to a KML file each morning before going out into the field. This process combined with your Google Earth cache files would give you access to current GIS data that can be visualized in Google Earth along with your background imagery and additional vector datasets. This represents a very powerful and useful way to work with your data in the field. One additional consideration that I’d like to address concerns the export of your ArcGIS data using these tools. I frequently get questions from users who are concerned about privacy issues surrounding the possibility of their local GIS data being uploaded to Google’s servers. This in fact is not the case since KML/KMZ export files created by these tools are written to your local file system. Your local data is not being uploaded to Google.
More Information
For more information about the integration of Google Earth and ArcGIS please see our virtual training course “Integrating ArcGIS Desktop and Google Earth”.
Creating Google Earth Time Display Data with ArcGIS
Introduction
The release of Google Earth 4 brought many new enhancements to an already fantastic geographic viewing tool, but in my opinion the most important new functionality is the introduction of the new Time control which allows you to incorporate time information into your existing GIS data. The addition of this fourth dimension greatly enhances the capabilities of Google Earth by enabling you to analyze patterns in your data over time. In this how-to article you’ll discover how to create time enabled, Google Earth format files (kml/kmz) using a combination of ArcGIS Desktop, Spatial Analyst, Google Earth, and your favorite text editor.
A new Time control has been added to the user interface which allows you to control the time period of the globe. The Time control, which is both functional and minimal, is only visible on the Google Earth display when TimeSpan tags are detected in your kml/kmz file.
The user interface for the Timer control consists of a draggable area containing a start and end date which you can expand or shrink along with controls for starting the time animation and displaying the Time properties form.
Time properties including the time zone, animation speed, and other options can be set from this dialog. Data that has been tagged with time information will appear or disappear depending on the current time of the map. The TimeSpan tag represents an extent in time bounded by begin and end dates/times. The tag describes the beginning instant of a time period while the tag describes the ending instant of a time period. In the example below, the beginning time date is January 1st, 2003 and the ending time date is December 31st, 2003.
Overview
In this article we will integrate ArcGIS and Google Earth to analyze building permit data for the City of Austin, Texas for the years 1997 to 2003. Our analysis will be performed through the use of ArcGIS with the Spatial Analyst extension to create both raster and vector output files which will then be used to construct KML format files that display the results of our analysis through Google Earth and its new Time control. Before we get into the details of how to create a Google Earth file that contains a time based analysis of geographic data perform in ArcGIS lets first take a look at the finished product. If you haven’t already done so make sure you download a copy of Google Earth 4 before continuing with this article. To accomplish this task I have created two distinct output kml files. The first is a vector analysis of building permit data by census tract and displayed as a graduated color map. To visualize the results of this analysis please click here. Using the same data I have also created a “heat map” in raster format which can be viewed here.
Process
Step 1 – Obtain GIS Data
I first obtained building permit data from the City of Austin, TX for the years 1997 through 2003 which had already been geocoded and stored as point data in shapefile format. Each year was stored in a separate shapefile. For more information regarding building permit data for the City of Austin please visit their website. This data was then loaded into ArcMap along with census tract polygons for purposes of a spatial join with the building permit data.
Step 2 – Process Data in ArcMap
In this particular case the goal was to create two distinct output kml files; one for the vector analysis and the other for the raster (“heat map”) analysis. In the case of our vector analysis I also added a census tract polygon layer that covers the Austin, Texas area. A series of spatial joins were then performed between each of the point shapefiles containing building permit data and the census tract polygon layer. For more information on performing spatial joins in ArcMap please reference the desktop help file. The result of each spatial join was the addition of a new field in the census tract polygon layer containing a count of the number of building permits issued for each tract. At the end of this processing I had fields for each year containing the number of building permits for that year. The addition of building permit counts by year is illustrated in the figure below.
Using each field in succession I then created a graduated color map for each year. Finally, using the Export Map function in ArcMap I then exported image files for each year in .png format. For example, I exported the building permit graduated color map from 2003 as go_Census_Tracts_2003.png. These image files will serve as the basis for our display of time data in Google Earth.
For our “heat map” raster density analysis the processing of our building permit data was analyzed using the Point Density tool in the Spatial Analyst extension of ArcGIS.
Just as with our vector graduated color maps I created individual raster layers for each year and exported to a .gif format image file using Export Map. You can see an example of this type of raster “heat map” in the figure below.
One final task that I performed in ArcMap was to gather the geographic extent in decimal degrees of the output images. This was accomplished very easily by setting my Data Frame Properties Display to “Decimal Degrees” and then moving the cursor to the maximum and minimum extents of the display and manually gathering the coordinate locations.
One additional consideration that you need to take into account is the transparency of your output images. Setting a transparency value allows you to see the underlying data sources provided by Google Earth while at the same time visualizing the results of your analysis. In this case I have set a transparency value of 50% on each output image through the Layer Properties Display tab. This step isn’t absolutely necessary though as transparency values can be set for the images in Google Earth.
Step 3 – Create KML/KMZ File
The final step in the creation of Google Earth time display data is the creation of the actual kml/kmz files that will display the data. There are a number of different ways that this can be accomplished, but for purposes of this article we’re going to actually create the final output file through a text editor (Notepad) so that you can get a good understanding of how this process works. However, we will create the initial baseline kml data file through Google Earth.
Open Google Earth, and create a folder to put the graphic files into by selecting Add à Folder in the Places Pane. Go ahead and give the folder a name like “Austin Building Permits 1997 – 2003” like you see below and click OK.
Now, right click on the newly created folder and select “Save As” and write out the file in .kml format (not .kmz for now). This will create the baseline structure of the kml file that we’ll now work with in your favorite text editor for the remainder of the tutorial. Close Google Earth and navigate to where you’ve saved the kml file and open it with your favorite text editor. You should see something like the figure below.
This will serve as your baseline file which we’ll now use to add image overlay and time display tags. Each image that we exported from ArcMap will need to be imported into the kml file as an image overlay. Below you will see an example of how to use the tag to import your image files as image overlays in Google Earth. This first ground overlay tag represents the 1997 building permit data at a census tract level.
There are a couple of things that I’ll point out about the attributes of the tag. Notice that the attribute contains a pointer to the image file that I have exported from ArcMap. In our kml file this is a hard-coded path to the image, but later we’ll see how you can alter this to a relative path name for easy distribution of your files. The other thing to note is the tag which contains bounding coordinates for you image. This is how you specify where Google Earth should place your image. Back in Step 2 we gathered the geographic coordinates of our display in ArcMap, and now it’s time to put these coordinates to use. By using this set of kml tags you have specified a pointer to an image file that you have created along with the bounding coordinates for where the image will be displayed in Google Earth. This code should be placed just below the 1 tag line.
Now that you know how to define an image overlay we need a way to add time data to each image so that Google Earth can display each of the images in a time based format. Earlier in the tutorial we said that the tag can be used to accomplish this task. By adding in the tag to our kml as you see below we can define the period of time for which each of our image overlays will be visible in Google Earth.
At this point we have fully defined the kml for a single ground overlay to display our building permit data for the year 1997. You will need to perform the exact same steps to import each of the remaining images in your collection to the kml file. This can be a tedious process, but through the use of cut and paste you can greatly simplify the task. You can download the full text of the code created in this tutorial by clicking here.
Once you’ve fully defined the kml tags for each image you can save the file and then open in Google Earth. You should see something similar to the following:
Notice that each image overlay has been added to the folder and the Time control is now visible and ready for use in Google Earth. There is one additional task that I would recommend performing to make the sharing and distribution of your files more efficient. Recall that in our kml file we have hard-coded the paths to our image files. By converting the kml file to a kmz format we can embed the images into the compressed kmz format file and in so doing alter the path to our images to a relative path rather than a hard-coded path. This can be easily accomplished in Google Earth by right clicking the folder and selecting Save As and selecting the .kmz format.
The physical structure of the resulting kmz file is shown below.
Notice that all the image files have been loaded into the kmz file and a new doc.kml file has been created. The contents of the doc.kml file are nearly identical to the kml file we created earlier with the exception that the paths to our image files are now relative paths.
To view the contents of a .kmz file you can simply rename the extension to .zip and then open the zip file.
Step 4 – Using the Time Control
As is the case with most Google Earth functionality, the time control is incredibly easy to use. When Google Earth detects the use of tags in your kml/kmz files it will automatically display the control.
Play Button – Used to start a time display animation from your kml/kmz file
Pause Button – Used to pause the time display animation
Display Time Properties – Displays the time properties form
Time Span – Draggable area that consists of a time span (start date and end date)
Conclusion
In this how-to article we have discovered a simple process for visualizing geographic data that has a time element through the use of ArcMap and Google Earth. This type of analysis is important in many GIS studies because it helps distinguish patterns in your data over time.
Read Full Post | Make a Comment ( 1 so far )Using the Image Overlay Tool in Google Earth
In this post you will learn how to use the image overlay functionality provided in Google Earth to import existing images into the 3D display and drape them over the existing terrain. Many image file formats can be displayed in Google Earth including jpg, bmp, gif, tiff, tga, png, and a number of others. What this means is that you can import USGS topographic quadrangles, enhanced aerial photography and/or remotely sensed images, historical maps, raster image files exported from ArcGIS and other GIS software, and many others. Google Earth provides the capability to easily accomplish this task. Before we get started please download the image that we will use for this exercise at http://www.geospatialtraining.com/GoogleEarth/AbajoPeak.tiff. Also, make sure you have installed Google Earth (version 4). Let’s get started.
-
Open Google Earth 4 and enter Monticello, UT in the Fly To tab on the Search panel. Zoom to this area.
-
Click the “Add Image Overlay” Button
The Add Image Overlay dialog box will appear along with hash marks on the GE display that can be used to move and stretch your image.
-
Give your image overlay a name (Abajo Peak) and click the Browse button. Navigate to the image file that you downloaded (AbajoPeak.tiff) from geospatialtraining.com.
Your display should now look similar to the image below. This .tiff image is a USGS quadrangle topographic map for Abajo Peak in Utah. You’ll notice immediately that this particular image has not been georeferenced. It is simply displayed in whatever area you happen to have navigated to in Google Earth. Next, we’ll georeference our image.
-
Click the Location tab and then use a combination of dragging the green markers around the image along with manually entering the coordinates until your North, South, East, and West coordinates appear similar to the figure below. Now, I should point out here that I’m providing the coordinates for this particular image. You will need to know the bounding coordinates for each image that you import into Google Earth so that you can correctly georeference the image.
-
We will leave the items on the View, Altitude, and Refresh tab on their default values, but let’s discuss these items first. The Refresh tab allows you to automatically set the image overlay up to refresh to the latest view of the image. This is used when the image that you’re placing updates frequently, but in our case the image is static so we don’t have a need to refresh our image. The refresh functionality is useful when dealing with data such as weather radar images that are continuously updating. Refreshes can occur based on a time interval or based on a view that has changed. The altitude tab can be used to display the image a set distance above the ground for situations where you want the image to “hover” above the terrain. Again, in this case we do not need our image to hover, but it can be useful in some instances. Finally, the View tab can be used to set the current camera properties for the 3D viewer.
-
Now that we’ve defined the position of our image your GE viewer should appear as follows:
-
There are a number of things that we can do to enhance this view. First, we’ll set a transparency value on our image so that we can “see through” the image to the underlying terrain and vector data. With Abajo Peak selected in the Places panel, use the transparency slider to apply transparency to the image.
Your image should now have a transparency set. You may need to experiment with the slider until you determine the best setting. Notice that as you move the slider, the underlying terrain becomes visible.
-
Now, let’s turn on the Terrain layer in the Layers panel to drape the image over the underlying terrain.
Notice how the image now conforms to the underlying terrain.
-
Finally, let’s tilt the display and experiment with the orientation to get a better view of Abajo Peak. Use the navigation tools provided by GE to get a better look at the image and underlying terrain.
Read Full Post | Make a Comment ( None so far )
GeoChalkboard 