Sand Mining Suitability: Network Analysis

Goals and Objectives:

The purpose of this exercise was to calculate the best possible route of each sand mine to railroad terminals.  From there it was possible to calculate the estimated cost of shipping the sand from the mines to the railroad terminals based on a hypothetical cost per mile of 2.2 cents.  All of these steps were tasked to be performed in ArcGIS model builder.

List of objectives:

1. Load features and network analysis in ArcMap
2. Calculate routes based on closest facility network analysis
3. Calculate total amount of miles driven per county
4. Calculate cost per total county based on miles driven

Methods:

The first step in this lab was to gather up all of the features required, including the 2013 street map network data set from ESRI.  The features that were used were non-railroad sand mines that were more than 1.5 km away from any railroad and railroad terminals.  A simple closest facility analysis was run with mines as the incidents and railroad terminals as the facilities.  Solving this closest facility analysis came up with the best route from each mine to a railroad terminal according to the network data set.  Then, in order to calculate the amount of miles of driving per county, tabulate intersection was run.  This tool took the county boundaries of Wisconsin and computed the total length of the routes within them.  From there cost was calculated using the simple measure that each mile was driven 50 times a year by each truck and each mile cost the county 2.2 cents.  The equation used was (distance in miles)*50*.022.  All of these processes were run in model builder (Figure 1).

All of the processes run in this exercise were required to be run using model builder.  (Figure 1)

Results:

After the routes were created using the make closest facility layer tool, a map was generated to show the various best routes assigned by the tool (Figure 2).  From there more quantitative analysis could be run regarding the amount of miles driven in each county and the cost (Figures 3-5).

This map shows the various routes generated by the closest facility network analysis tool.  Most of the results weren't very surprising, though it was interesting that one of the routes actually exited Wisconsin and traveled through Minnesota.   (Figure 2) 

This graph was created to further illustrate the hypothetical cost
that could be incurred each year on each county per the
calculations performed and previously mentioned parameters
(Figure 4)

This table shows the various, total lengths driven
in miles to get the sand from mines to railroad
terminals and the total cost to the county per year.
(Figure 3)

The total, hypothetical costs incurred per county due to trucks traveling to rail terminals was mapped.  It can clearly be seen that not all counties are affected as frac sand mining is mainly prevalent in Western Wisconsin.  (Figure 5)

These results have several interesting aspects to them.  Many sand mine trucks appear to be taking the same routes to get the to rail terminals (Figure 2), this could put a very heavy load on these roads in particular and they may need special attention.  However, the amount of hypothetical yearly cost is relatively lower than expected (Figures 3-5).  This could be due to inaccurate assumptions of 50 trips a year and 2.2 cents per mile.  More research and analysis will need to be run in order fine tune the knowledge regarding potential costs of trucking the sand to rail terminals.


Conclusion:

Network analysis is a powerful tool that can be relatively easy to run; particularly with many network data sets already established.  Using network analysis and the closest facility tool was able to calculate the best routes from sand mines to rail terminals for sand transport trucks to take.  From here total mileage and cost per county were calculated using several joins and different calculations.  Some of the results were surprising such as the low yearly cost, however these are just hypothetical calculations and may not have much weight in the real world.  This information should be compared with colleagues in the class to determine whether similar results were achieved.

Using Python Script to Run ArcGIS Operations

This is an ongoing blog post to be completed by the end of the semester.

Background:

Goal:

The objective of completing these blogs is to learn how to use Python code in GIS applications. Specifically how to use the ArcPy extension. Python coding will be used throughout GIS II class to run several different tasks. These exercises should help the class become well versed in using Python coding for geospatial applications by the end of the class.

Exercise 5:  Preparing Rasters for Further Analysis

In exercise 5 it was required to prepare the various rasters that had been downloaded for analysis that will be performed in later portions of the project.  The rasters first all had to be moved into the same folder as the geodatabase for ease of the coding process.  The first step in the actual coding was to set the environments to the folder that had been created.  From there the rasters had to be picked out of the folder using the ListOfRasters command.  Afterward the rasters were projected to the correct projection of Trempealeau County using arcpy.ProjectRaster_management.  Then the rasters were extracted to the outline of the Trempealeau County boundary using extract by mask.  The final task was simply to import the rasters into the Trempealeau County geodatabase.


Exercise 7:  Selecting out the Desired Mines

For exercise 7 it was required to query out and export the correct mines based on several categories.  The mines feature class that was given contained all mine facilities in Wisconsin given by the DNR.  These facilities included processing facilities and other administrative facilities that aren't needed to run the network analysis.  The objective of exercise 7 was to run network analysis to find routes the trucks have to take and the cost of using these routes between mine facilities rail nodes.

The first step as always was to set the default environments.  In this case the default work space was set to the exercise 7 geodatabase where all the feature classes were contained.  From here variables were established to represent the various existing feature classes, the step-between feature classes, and the final feature class.  Then the SQL statements were written and run using arcpy.MakeFeatureLayer_management to query out the desired mines.  The qualities that were used were mines with an active status, with a facility type like "Mine", and not with a facility type like "Rail".  The mines that have rail in their facility type were queried out due to these having their own rail spurs.  Then the mines desired were selected by location, intersecting Wisconsin and not within 1.5 kilometers of a railroad.  The selected features were then copied over to the geodatabase.

Geocoding Frac Sand Mines in Wisconsin

Introduction:

Geocoding is the process of taking a description of a location, such as an address or a place name and giving it a location on the earth's surface.  It can be performed by manually entering one address at a time or by using a table.  In this portion of the project, information regarding all frac sand mines in Wisconsin is gathered from the Wisconsin Department of Natural Resources (DNR).  From there, the various members of the class were assigned around fifteen to twenty mines to geocode.  This would allow for the mines to be repeated.  The objective of having multiple people geocode the same mines was to see if there was any error in the geocoding process.  This error can possibly come from the mines not having enough location information and some guessing involved in locating them.

Methods:

The first step in the process was viewing the table of the mine data that was provided from the Wisconsin DNR (Figure 1).  From here it was clear that the table was not ready to be run through the geocoder as geocoders require normalized table data to run correctly and the information in the table provided wasn't normalized in almost any way.  At this point table normalization was begun.  The important fields that had to be normalized to accurately run the geocoder were: address, city, and zip code.  The key was making sure that all of the mines had accurate addresses listed.  This was difficult as most of the address fields included both an address, a city, and the Public Land Survey System (PLSS) identification.  This information had to be split up into an address field, a city field, and a PLSS field.  Unfortunately, there were several records that didn't come with an address field (perhaps the most crucial part of geocoding in this case) and only had PLSS information.  In order to locate these mines and record their addresses, a combination of PLSS data and aerial imagery had to be used along with the geocoder.  To get the address, it was simply a matter of clicking on the map in ArcMap where the mine looked to be based on aerial imagery.  This, however, isn't an exact process and could possibly be a cause of error between the different mines geocoded by the class.

This is the data as given by the DNR, it has been formatted slightly differently with the PLSS and street addresses being split up.  It can be seen that some of the mines didn't come with address or even city data.  (Figure 1)

At this point the table was normalized with all fields filled in (Figure 2) and geocoding was run.  Fortunately, the results came out well with all mines successfully matched.

This is the completed, normalized table that was run through the geocoder successfully.  All of the address fields were found and filled in while the PLSS data was unnecessary for the geocoding in this case.  (Figure 2) 

When the whole class completed their geocoding, it was instructed to merge all of the mines into one feature class and query out the mines that matched mine unique IDs of the mines that had been done individually (Figure 3).  This process seemed like it would be a rather simple one involving the use of a few tools to get a finished product that would be used to compare to the individually geocoded mines in order to find the error.  However, the lack of normalization among the different geocoded mine shapefiles from the various class members made this rather difficult.  Eventually several shapefiles had to simply be excluded from the merge due to their extreme differences compared to the others.

Here is the SQL query that was used along with the select tool to query out the mines from the classes merged shapefile that matched the mine unique IDs of those that had already been completed.  (Figure3)

Once the class's mines were queried out, they could be compared with the mines that had already been completed.


Results and Discussion:

(Figure 4)

The mines were compared and the point distance tool war run to compare the mines I geocoded with the class mines that had the same unique ID as mine.  It turns out that the majority of the mines were all located in relatively similar positions, if not in the same positions as the ones I geocoded.  This may show that the geocoding was done well by the majority of the class to perform it.  However, there are several mines that appear much further away than they likely should.  The example that was clearly the worst geocoded mine by either myself or one of my classmates was the mine with the unique ID 163.  One of the other mines actually matches mine exactly while the other is over 165 miles away.  This vastly changed the average distance of the mines away from themselves.  However, it can be seen from the low median distance that overall the mines were well placed in general (Figure 4).

This relative consistency in mine placement can also be seen in Figure 5.  The one large outlier can be seen in there as well.  The errors that are present can likely be attributed to errors in locating mines that just had PLSS data and lack of proper normalization among the class.

In this map the various mines that were geocoded by me and by the class can be seen.  Most of the mines appear to be relatively close to each other.  There is however, one large outlier on the far eastern part of the state.  (Figure 5)

Conclusion:

Geocoding is an extremely useful process to place objects on a mp when run correctly.  However, when run poorly, geocoding can lead to results that don't mirror real life in any way.  This is troublesome as everybody was given the same data, yet there were many different outputs.  In the future, the group should normalize he mines together to a certain standard.  Also the process of looking up mine addresses using their PLSS description and aerial imagery is a very unscientific method.  However, at times it seems like it is necessary to accomplish the set task.

Data Gathering and Preparation for Frac Sand Mining Project

Introduction:

The goals and objectives of this lab were to learn how to properly query and download data from various online sources.  This data that was downloaded will all be pertinent to the future suitability model to be run regarding frac sand mines in Trempealeau County.  After the data was downloaded, it was required to run a python script in order to properly convert all of the data to the NAD83 HARD WISCRS Trempealeau County Feet projection and clip all of the data to the Trempealeau County border.  All of this data is now stored in a file geodatabase and is ready to be used in any future analysis.

Methods:

The first step in this lab was to download the various data required.  All of the data needed to be queried, downloaded to a temporary folder, and unzipped into a working folder.  The various data downloaded is shown in Figure 1.  The base sets obtained included a railroads dataset from the US Department of Transportation, a Digital Elevation Model from the US Geological Survey, soils data from the US Department of Agriculture Soil Survey, and land use/land cover data from the USDA.

These various datasets were queried and downloaded to help aid in the future project.  They were all downloaded to a temporary folder and then unzipped the data into a working folder.  (Figure 1)

After all the data was put into the working folder, it was examined and determined that the rasters for the DEM, the soils info, the LULC info, and the railroad vector feature class were all to be imported into the Trempealeau County geodatabase and converted to NAD83 HARN WISCRS Trempealeau County Feet.  This was done using a Python script (Figure 2) in order to help teach basic programming in Python.

This is the Python script that was generated and run in order to project the necessary rasters, clip them by the Trempealeau County boundary, and import them into the Trempealeau geodatabase.  (Figure 2)

From here it was possible to generate a map to show all the different projected and clipped layers that had been downloaded and are now ready for further analysis (Figure 3)

This map was generated using the data downloaded from various online sources.  The data is now in a file geodatabase and will be used in future exercises to run analysis on frac sand mining operations in Trempealeau County.  (Figure 3)

Data Accuracy:

An important aspect of the future of this project is knowing the limitations of the data gathered.  In order to know this, it's important to look at the metadata and find the different data quality components such as scale.  As much information on the data as possible was found and reported in Figure 4.

This is a data quality table that was generated by looking at the metadata of the various sources of data downloaded.  Some information was unavailable and some was estimated and is an approximate value.  (Figure 4)

Conclusion:

Downloading the appropriate data and prepping it for further analysis is a critical step in suitability modelling and will be extremely pertinent for the future of this project.  It is also important to know the quality of the data downloaded.  Some problems that may arise are the land cover and cropland classes being at a larger scale than the rest of the data.  The soils data and DEM data are precise and up to date and seem like they will be very useful for the future of this project.

Frac Sand Mining in Wisconsin: An Overview of the Project

Sand mining has occurred in Western and central Wisconsin for over a century now.  Typically the sand has been used for purposes such as making glass or water filtration.  However, recently, a technique called hydraulic fracturing, or hydrofracking (Figure 1), has been developed to extract crude oil and natural gas from rock formations.  This has drastically increased the demand for silica sand, otherwise known as frac sand (Figure 2) and has led to the demand for more mines in the frac sand rich regions of central and western Wisconsin.

Nine out of ten natural gas wells in the United States use the process known as hydraulic
fracturing (ProPublica).  Water, sand, and other chemicals are pumped into the ground in order to help
break open the rock and hold it open to allow gas to escape.
http://www.propublica.org/special/hydraulic-fracturing-national
(Figure 1)

Frac sand is a quartz sand required to be close to a specific grain size and shape.
It is pumped into rock formations during the hydraulic fracturing process in order to hold open
fissures created by pressure and allow the natural gas to escape up into the well.
http://flexfracsand.com/proppant-sand-products.html#
(Figure 2)

Frac sand mining is a very contentious topic in Wisconsin.  There are many who consider it to be a great boon to the economy.  They point to the fact that frac sand mining can create a vast amount of new jobs.  An example of this is how it is predicted in Wood County that after eight years, frac sand mining will create over 900 new jobs according to EMSI.  However, there are the environmental impacts to consider.  Frac sand mines are seen by some to have a huge impact on the mining area, although this does vary by location.  According to the Wisconsin DNR constructing and operating frac sand mining sites near surface water bodies can have an adverse effect on the water and the wildlife that call the water home.  However, regulations are in place by the DNR to help prevent occurrences of mining near water bodies.

The increase in the demand of frac sand to perform hydraulic fracturing and the opening of new mines in Wisconsin has led to much controversy.  This controversy mostly concerns the economic benefits of frac sand mining against the possible negative environmental impacts.  Due to this, the topic of frac sand mines will be further explored during the GIS II class, and the future labs and posts will all concern this contentious issue.  The suitability and risk of mining in western Wisconsin will be the main focus of the class project.  Some of the steps that have already began are the downloading of necessary data and geocoding the addresses of known mines throughout Wisconsin.  At the end of the project, the goal is to have a complete suitability model of areas where mining may hurt or help in western Wisconsin, specifically in Trempeleau County, WI.

Sources:

http://wcwrpc.org/frac-sand-factsheet.pdf

http://www.propublica.org/special/hydraulic-fracturing-national

http://flexfracsand.com/proppant-sand-products.html#

http://dnr.wi.gov/topic/Mines/documents/SilicaSandMiningFinal.pdf

http://wisconsinsand.org/assets/downloads/Econ-Impact-in-Wood-County.pdf

GIS I Lab 5: Atrazine Restriction Areas in Columbia County

Fig. A: Top Left, Fig. B: Top Right, Fig. C: Bottom Left, Fig. D: Bottom Right

Introduction:  Atrazine is one of the most widely used herbicides in the United States.  Every year over 76 million pounds of it are applied on crops in the US.  However, atrazine is a major threat to groundwater as it remains in soil for months and degrades in groundwater at a very low level.  Atrazine also has been linked to several health issues.  It has been shown to be an endocrine disrupter, have possible carinogenic effects, and it even has been linked to low sperm levels in men.  Due to these possible health risks, atrazine prohibitions and restrictions have been placed on around 1.2 million acres of Wisconsin land.

My objective of this project was to look into where more potential atrazine restriction areas could be placed in Columba County in the future.  I wanted to look at the current areas and research the different criteria that were used to choose the original restriction areas in Columbia County.  I then wanted to use the found criteria to help me query appropriate locations for potential atrazine restriction areas.

Through research, I found that areas with dolomite as bedrock are more in danger of groundwater infiltration by pesticides than other areas such as ones which contain sandstone bedrock.

Data Sources:  In order to answer my question of where additional atrazine restriction areas should be placed, I needed: data regarding the geological features of Wisconsin, areas in Wisconsin specifically set aside for farming, current atrazine restriction and prohibition areas, and water features.  

I was able to find farming preservation areas (which are areas specifically set aside for build up or development over the next fifteen years) and current atrazine restriction areas from the Wisconsin Department of Agriculture, Trade and Consumer Protection.  This data, along with the metadata can be found at ftp://ftp.datcp.state.wi.us/GIS/.  One concern I have about this data is how updated it is.  I'm not sure how recently they updated the data or if there are additional atrazine prohibition or restriction areas already in place in Columbia County.

I found geological attributes of Wisconsin from the United States Geological Survey.  This allowed me to locate the bedrock features throughout Columbia County.  However, this data is rather incomplete and doesn't include very much information about each bedrock type.  I'd prefer a more complete data set in the future.  The metadata can be found at http://mrdata.usgs.gov/geology/state/metadata/wi.html.

I found urban areas from the US Census Bureau and water features from the WI Department of Natural Resources.  Both of these data sets were already installed on the campus drives at the University of Wisconsin-Eau Claire.

Methods:  

In order to find the best locations to place new atrazine restriction areas in Columbia County, I decided on several factors.  I wanted to put the new zones in areas which were designated as farm preservation areas, were outside of urban areas, weren't already atrazine restriction areas, and were on dolomite bedrock.  Also, I didn't want to include areas that were part of a large body of water, as it is unnecessary to place the restriction zone over a large body of water.

I began by projecting all of my data to the NAD83 HARN Wisconsin CRS Columbia projection.  I then took all of the data and clipped it to only keep what was inside of the Columbia County border to prepare all of the data for querying.

The next step involved creating a dolomite feature class from the geology feature class.  I then buffered the dolomite feature by one kilometer.  After the buffer was completed, I intersected it with the dolomite itself and the farm preservation areas to create a potential restriction area.

After this I created the area I wanted to erase from the potential restriction area I'd created previously.  I did this by making a feature class from only the largest 25 percentage of water features, and creating a union with this, urban areas, and current atrazine restriction areas.  Then after performing the erase with the potential areas I'd created previously I came up with my new atrazine restriction zones which can be seen in Fig. C as the orange area.

Figure C

Results:  The final product is the orange area as shown in Figure C above and in the complete map at the top.  This area is within the farm preservation area set aside by Columbia County, is within one kilometer of dolomite bedrock, isn't within an urban area, and doesn't intersect a current atrazine restriction area.

Figure D, below and in the original map, shows the total atrazine restriction area (in yellow) which includes the current area and the proposed area I created by querying my data.  It also includes the urban areas (in red) for spatial reference.

Figure D

  Evaluation:  For this project I was forced to design my own project.  In order to do this, I learned how to develop a valid question, search for the data online, download the data and prepare it for presentation in ArcMap, and query the data in order to answer my question.  I was able to do this all independently with only minor consulting regarding my querying process.  Going through this process gave me confidence that I'd be able to go through a process similar to this but more complicated in the future.  If I was asked to repeat the project I would try to explore for more specific criteria in placing future atrazine restriction areas.  I'd look further into the criteria they precisely used to choose the existing restriction and prohibition areas such as wells in the areas.  One of the most difficult parts of the process was finding the the GIS data to help me answer my question.  Also, once I found this data, I had to download it and some of it wasn't in a form readily available for me to place into ArcMap.  An example of this is when I downloaded the geographic attributes of Wisconsin it came in an e00 format which I had to convert into a usable form using a tool in ArcGIS.  Due to this, in the future I will know how to make sure all of my data is usable for me in ArcMap, even if I have to convert it.  The independent nature of this project allowed me to explore something I find interesting and learn on my own.  By being able to do these two things, I feel that I've learned much more than I otherwise would've with normal coursework.

GIS I Lab 4: Vector Analysis with ArcGIS

Potential Bear Habitat includes all of the areas suitable for the DNR to set up bear management zones, while Proposed Habitat takes into consideration the proximity to urban areas and eliminates any potential habitat that falls too near to an urban or developed area. Due to this, Proposed Habitat is the final area submitted to the DNR as it helps limit conflict between humans the bears.

Goal: The goal of this lab was to use vector analysis skills I have gained in the class tutorials to determine the best location for bear habitats to be set up my the DNR. I was tasked with determining which tools would be best used to determine these locations.

Background: I was given some GPS data regarding recent bear locations, in a study area of Marquette County, MI. I was told to look into stream proximity and land cover in order to best determine locations for bear habitats to be set up by the DNR. I also had to keep these lands inside of the DNR management zones I was given.

Methods: I began by determining what types of land cover the bears were typically found in basesd on the GPS data I was given. I did this by joining my bear locations with the land cover feature class and summarizing to determine the top three land covers the bears were found in. Most of the bears were found in either mixed forest land, forested wetlands, or evergreen forest land. I simply exported the mixed forest land, forested wetlands, and evergreen forest land into its own bearlandcover feature class.

I then wanted to determine if the bears were typically found near streams. I did this by simply selecting by location from the bear location feature class, selecting all the bear location features within 500 meters of a stream. This ended up selecting 49 of the 60 bear locations (72.05%). This helped me determine that proximity to a stream is a definite factor in determining bear habitat.

One last factor in determining where the DNR should set up bear habitats I determined was taking the DNR management zones into consideration. The DNR has certain zones where they can set up the management areas. In order to get these zones ready for analyzing, I simply had to clip out any of the zones outside of the study area.

 

My decided criteria for determining bear management zones at this point was: must be within either mixed forest lands, forested wetlands, or evergreen forest land (ie, in the bearlandcover feature class), must be within 500 meters of a stream, and must be within the DNR management area. I proceeded to buffer the streams at 500 meters, and then intersect these three features to come up with Potential Bear Habitat. However, there is also urban and built-up area within the study area. It was decided that it would be best to keep the bear management at least 5 kilometers from urban or built-up lands. I exported the urban or built-up land from the landcover feature class and buffered it by 5 km. I then used the erase feature to eliminate all of the potential bear habitat within 5 km of the urban or built up land. After all of this I was left with the Proposed Habitat (or in the case of the data modeling, bearhab_dnr_erase).

 

Results:  Now that all of the analysis has been performed, there is a clear area, as shown by Proposed Habitat Area, on the map in which the DNR can set up bear management areas.  This area on the map all falls within 500 meters of a stream, is located in either mixed forest lands, forested wetlands, or evergreen forest land, is within the DNR management zones, and is at least more than 5 kilometers from an urban or built-up area.  There are some bear location points on the northwest of the map which are not  near any proposed management zones.  This is due to the fact that there isn't very many areas of DNR management zones in that region of the county, though otherwise it would make excellent bear territory as it falls within the other categories.  Perhaps the DNR can see about expanding management zones into this area to further cover as many bear habitats as possible.  The green Potential Bear Habitat includes the Proposed Habitat as it is the habitat determined before taking urban areas into consideration.  After the consideration of urban areas, the orange Proposed Habitat was created and determined to be the final proposal area for bear management zones to the DNR.

 

Sources: 

USGS NLCD

http://www.mcgi.state.mi.us/mgdl/nlcd/metadata/nlcdshp.html

 

Michigan DNR

http://www.dnr.state.mi.us/spatialdatalibrary/metadata/wildlife_mgmt_units.htm

http://www.mcgi.state.mi.us/mgdl/framework/metadata/Marquette.html