GIS Programming - Participation Assignment 1

The purpose of this assignment is to find and summarize an article about a real-world application of GIS, programming, or Python.

The article I read (title and link below) discusses the use of a Python script to correct distortion errors in hyperspectral imagery collected by aircraft.

Jensen, R.R., Jackson, M.W. and Lulla, V., 2008: “Single line correction method to remove aircraft roll errors in hyperspectral imagery.” Journal of Applied Remote Sensing 2, pp.1-10.

I found this article to be very interesting and pertinent because it ties together our Remote Sensing course (which I took last fall semester) with the use of Python programming.  The problem being addressed here with a Python algorithm is fairly simple.  Because planes collecting aerial imagery are subject to pitch, roll and yaw as they follow their paths, any data collected for more than just a single instantaneous frame is prone to distortion.  Hyperspectral data is collected in what is referred to as the “push-broom” method:  the sensor instantaneously collects a single continuous strip of data at a time, to each side of the plane, perpendicular to the flight path.  Any amount of roll by the plane will change the relative view angles along the data strips, and cause distortion in the resultant image.
The usual way to correct this distortion is called the reference line method: the wavy, distorted image of a feature known to be linear on the ground (like a straight road) is digitized. The program then calculates and implements the offsets needed in each row of pixels to make that feature appear straight on the image. This process, though simple in theory, is very meticulous, as it has to look at every single row of pixels in the image.  That makes it an excellent candidate for automation by Python scripting.
The correction was accomplished by exporting the coordinates of each pixel in the wavy reference line into an ASCII file.  The inputs for the Python program that will correct the distortion are:

• the uncorrected/distorted imagery file, and
• the ASCII file containing the pixel coordinates of the reference line.

The variables used by the program are also very simple:

• the range of column values for the pixels reference line, and
• the average column value for the entire reference line.

The Python script then just offsets each row by the difference between the column value and the average.  The effect of this is to straighten out the image of the linear feature, so it appears straight on the corrected image.  The positions of the rest of the features in the image are corrected along with it.

Here is a graphic explanation of how pixel rows are offset using a simple Python script to correct the wavy distortion caused by aircraft roll.

Example of Distortion Correction from Jensen, et al. (2008)

GIS Programming - Module 2 - Python Fundamentals Part 1

As we learn more about Python scripting this week, we used functions and methods, which are types of instructions for the program.  We wrote a script in which the input was a string, consisting of our full names.  The final output was our last name, and the number of letters in that name time 3.  My results are shown below:
The last name, Sease, and 15, which is 3 times the number of letters of my last name.

Results of the Python Script

GIS Programming - Module 1 - Introducing Python

In the first week of GIS Programming, we were introduced to the concept of pseudocoding to help us learn how we'll be constructing Python scripts later on.  This is a step-by-step script that is more-or-less like English, that puts together the Input, the desired Output, and a list of the tasks necessary to get from the Input to Output.  We also examined the interactive window of Python, in which script can be run a line at a time, the Pythonwin script editor, in which multiple lines of script can be compiled, saved, then run.  A Python interactive window is also accessible through ArcMap, by clicking the Python Window button on the standard toolbar.  This environment is convenient, because when you start typing, it gives suggestion prompts for possible statements from which you can select. Also, there is help and syntax information in this window for whatever statement you select .

2015 GIS Programming - First Post

This is the initial post for GIS Programming, Summer, 2015

(The blog posts for the partially-completed programming course from summer 2014 are labeled 2014 (old) Programming.)

GIS Internship - An ad for a dream job!

Here's a link for an interesting geology-related job with ESRI in Redlands, California:

ESRI - Consultant/Project Manager - Petroleum / Mining

• Where did you find this position? 

I found this position on the ESRI site, careers area.

After learning and working with ArcGIS, I think it would be really interesting to work for ESRI.  They seem very innovative.  I have a lot of education and experience in geology, so I looked for a job relating to that. 

This position would be located in Redlands, California, at the ESRI headquarters.  Redlands is a really charming historic town surrounded by mountains, on the edge of the desert outside Los Angeles.  It’s distinctive for its many beautiful Victorian houses, palm-lined avenues, and many old orange groves right in the town.  In the early days of the movie industry, the stars lived here instead of Beverly Hills.  The University of Redlands is located here.

• What are the requirements, including education and technical skills? 

The job requirements do not specifically mention educational requirements.  However, the primary requirements are 1) at least seven years of experience in mining or petroleum-related fields; 2) in-depth knowledge of GIS and experience with ArcGIS products and specific applications to mining and petroleum.  The expectations also include being able to work with clients to help them align their goals with what ESRI has to offer, and enough familiarity with software development methodologies to lead teams of ESRI programmers toward meeting those goals.  This position also requires strong leadership and general people skills.  Whoever fills this position needs to work as a liaison between industry and those that develop new applications at ESRI

This job seems ideal to me, but would be quite a stretch, from where I stand now.  I do have, cumulatively, more than 7 years of industry experience.  However, moving around a lot and raising 3 kids over the last 20+ years has caused my work experience to be quite fragmented.  I have learned a lot in this GIS program, but what specifically industry-related GIS experience I have is very outdated.   In the meantime, I would need to find work that would help me build up my practical experience.  

What I plan to do right now is work with my sister, who got her GIS certificate from UWF last year, in helping our dad with his small minerals exploration company.  Hopefully that, with the help of GIS and remote sensing mentors, will help me further my GIS expertise

• What class within the GIS Cohort do you think has prepared you the most for this position?

So far, Applications and Special Topics have been equally valuable to me for learning to use ArcMap to identify and solve problems from the most basic level, and finding ways to creatively utilize the tools.  Success in this position would depend on an in-depth knowledge and understanding of the tools, and an ability to identify how they can be used in new situations. 

An understanding of Python programming, which I will complete this summer, would be necessary to help integrate specialized software packages already in use in these industries, with ArcGIS.

The Remote Sensing course seems very pertinent to geologic specialties, because of the dependence of geologic interpretation on air photo and satellite image interpretation and multispectral analysis. 

                                                                                                   

Module 14 - Special Topics in GIS - Spatial Data Aggregation

This week in Special Topics we learned about a concept called the Modifiable Areal Unit Problem.  This means that the analysis of our data will probably be different, depending on scale at which we look at it, and how we aggregate the zones.

A related topic is Gerrymandering.  This is when Congressional Districts are drawn in complex and contorted fashion in order to include certain neighborhoods and the voters that live there, and exclude others.

Two measures of how badly a District is Gerrymandered are "compactness" and "community".

A compact district is one with a simple shape, with a minimum of irregularities on its boundary.   This is a good measure of a district that has not been Gerrymandered.  One way to measure compactness is to compare the area of a polygon with its perimeter.  A simple polygon, with no "ins and outs" will have a smaller perimeter compared to its area than a polygon that is very complex and contorted, such as the example in Figure 1.  This is New York District #7, which I determined from my analysis to be the least compact of all the House districts in the continental United States. It is not so easy to see at this scale, but there are many tiny jagged features along the boundary of this district. The thick red part of  its west boundary is actually composed of a long series of very small ins and outs, on the scale of a block or less.  This level of small-scale irregularity adds up to a very large perimeter, and a very complex district shape, and suggests that it has been Gerrymandered.

Figure 1.  New York Congressional District #7

A high level of "community" in a District indicates that a District does not suffer too badly from Gerrymandering.  "Community" is measured here by how much counties are cut by District lines and separated among various Districts.  I determined this by identifying the counties that are cut by at least one District line, then finding the Districts with the most "cut" counties.  Figure 2 shows North Carolina District #1.  It cuts parts out of 20 counties, and contains no whole counties at all.   This district has a very low level of "community", by our definition, and this hints at a lot of Gerrymandering.

Figure 2.  North Carolina District #1

Module 13 - Special Topics in GIS - Effects of Scale

We began the last 3-week topic in Special Topics for this semester, on issues of scale and resolution in GIS.  As an introduction, we examined how resolution decreases in data as the scale at which it was collected  decreases:  in other words, 1:100,000 scale data will have much less detail than the same features recorded at 1:1,200 scale.

We examined two different types of DEMs in this exercise: one obtained at 1-m resolution by LIDAR and the other obtained at about 82-m resolution by SRTM, which is space-shuttle-borne microwave, known also as RADAR.  I resampled the LIDAR 1-meter data to pixel sizes of 2, 5, 10, 30, and 90 meters, and compared their slopes.  The highest resolution data (1-m) will have the highest-value slope and the slope will decrease slightly with decreasing resolution and increasing pixel size.  This is because with more and smaller pixels, the slope for each across a given horizontal distance will be higher, resulting in a higher average slope, than the slope across a single larger pixel.
The LIDAR 90-meter DEM was then compared to the SRTM DEM, which was reprojected to the same resolution (90 m) and coordinate system. The elevation and its derivatives (1st-order: slope and aspect, and 2nd-order: curvature) for both 90-m DEM's were compared to those values for the original 1-meter LIDAR DEM, with the assumption that the higher resolution original data is more accurate.

The relative errors for all values (elevation, slope, aspect and curvature) between the 90-m LIDAR and the 1-m LIDAR are smaller than those between the SRTM 90-m DEM and the 1-m LIDAR.

Selected Results from comparison to LIDAR 90-m DEM and SRTM 90-m DEM, to LIDAR 1-m DEM

It can be noted from the values in the above table that while average differences in elevation between the LIDAR 90-meter and SRTM 90-meter are quite small, the differences are compounded in the 1st derivative of slope, which hinges on elevation.  The differences are even greater for the 2nd derivative of curvature.

These results make sense, because LIDAR has many more elevation data points because of the much more rapid data pulse rate.  It is also a much shorter wavelength than SRTM.