Imagery

Hard Work

Ryan Bowe · Feb 20, 2014 ·

“An idea I live by comes from a John Wayne movie called Rio Lobo. John Wayne and Jack Elam are on one side of a crick shooting their guns at the bad guys on the other side. John Wayne looks down at Jack Elam and says “Scatter gun’s useless.” Jack Elam says “Don’t mind if I shoot, do you? It just makes me feel better.”

“I love that attitude. If you don’t shoot, you have no chance of killing the bad guys on the other side of the river. If you don’t shoot, you have no chance of making a basket. You have to be willing to try.

“I balance that with an idea I take from Clint Eastwood’s movie Magnum Force. Toward the end of the movie, Clint Eastwood squares off with a dirty cop who’d tried to kill him. He secretly activates a bomb in the cop’s car and as the cop drives off his car blows up. Clint Eastwood says “Man’s gotta know his limitations.”

“That’s the other thing I think about when I’m coaching. Everybody should understand what they can and cannot do.”

Roy Williams, Hard Work: A life on and off the court

For those of you who do not know, I was born and raised a Tarheel fan and that definitely gets me in trouble this time of year living in Kentucky. I may bleed the wrong color blue (powder instead of big), but at least once a year and often twice one of my favorite teams has to lose. But these words of wisdom comes from a coach who just earned the title of fastest to 300 wins in the ACC and they seem to be particularly applicable to me the past few days. Plus, Coach Williams is fiercely competitive which is a trait I can relate to entirely too well. Tonight, I am hoping my fellow Kentuckians will forgive my “incorrect” powder blue preferences and unite with me to beat the true enemy. Go Heels. Beat Dook!!!!

I first heard this quote (thanks to Audible audiobooks) as I drove on snow-covered roads to the airport to attempt to do some remote sensing. I wasn’t particularly happy about this outing because if you think about the percentage of land that had good remote sensing conditions last weekend is incredibly low. And Kentucky had just received some more snow. So much snow that our first attempt of flying Saturday had been thwarted completely. I was totally skeptical and ready to turn around and go back to bed. But, after about an hour of scheming, we took off. We almost had to go back, though, because I didn’t have video so I couldn’t tell if the camera door was frozen shut or if the vide was just being finicky. Scary and embarrassing at the same time. But, I managed to get things working and it turned out to be a pretty successful day of 20 lines spread across four states (AR, TN, MS, and KY) and 395 images. (It has been QC-ed and everything was accepted!!!)

If I had gone with my original gut feeling and not shot the imagery, I would have been really frustrated. Not to mention the fact that I probably would have been driving back on roads that were still nasty from the snow we received overnight. Instead, I worked. When I drove home that night, the roads were nearly clear and we had several hundred images “in the can” so to speak. Productivity! And, along with that, we were smart and knew our limitations on Saturday. Yes, we probably missed some photography but we were safe. And, yes, I love my job. Writing very detailed FGDC Remote Sensing Extension metadata one day and flying the data the next!

But this quote is one I am going to try to take to heart in more ways than just remote sensing because I truly believe that hard work has the potential to pay off in all aspects of our lives.

Pyramid Layers for QGIS and ArcGIS

rjhale · Feb 17, 2014 ·

I guess I’ve been doing this way too long. I remember when pyramid layers on imagery was a bit “new” to those of us who had been plunking along with ArcINFO.

I’ve seen a few mentions in various places I’ve been hanging out about “My imagery is slow”. Invariably someone posts back “Build pyramids layers”. Of course I sorta bang my head on my desk and think “everyone should know…” BUT – not everyone does.

To thieve a bit from some ArcGIS Online help: Pyramids are reduced resolution representations of your dataset used to improve performance. Pyramids can speed up the display of raster data by retrieving only the data at a specified resolution that is required for the display. With pyramids, a lower-resolution copy of the data displays quickly when drawing the entire dataset. As you zoom in, levels with finer resolutions are drawn; performance is maintained because you’re drawing successively smaller areas.

So with ArcGIS pyramids are pretty much almost a default action. You can set to automatically build pyramids when you add an image. For instance today I was dealing with a 11 GB tif. Of course as I normally do I got a bit adventurous and did a few different things to build pyramid layers. Why? Because anyone can. Most of NRGS’s clients don’t deal with server environment. So if you deal with a lot of imagery it can get to be a hassle. You need to know about pyramid layers.

In this example we deal with ArcGIS and QGIS. Technically we’re dealing with ArcGIS and gdal. If you ever wondered if the pyramid file (.ovr) was viewable in both ArcGIS and QGIS – it is. Build it once and either can read it. There are three ways to go about building pyramid layers.

in ArcGIS: Proceed to ArcToolbox and go to Data management -> Raster -> Raster Properties. You can build pyramids there. ArcGIS has several defaults set. There’s several tools – one of which while build an entire directory of imagery.

in QGIS: Go to Raster -> Miscellaneous -> Build Pyramids. You’ve got a few more options “exposed” with this menu. Once again you can fill out all the particulars and let it build away. You even have an option to “clean” the pyramids which deletes them. You also have a nice little check box for processing an entire directory.

So the interesting thing with the QGIS pyramid layer is the text box at the bottom. The pyramid building in QGIS is gdal. It’s the gdaladdo command specifically. If you notice at the bottom you have a command starting to take shape. So if you get fairly adventurous the entire command starts looking like:

gdaladdo -r average -ro –config COMPRESS_OVERVIEW JPEG –config USE_RRD NO –config JPEG_QUALITY 80 –config TILED YES –config PHOTOMETRIC_OVERVIEW YCBCR –config GDAL_CACHEMAX 10240 ortho_1-1_1n_s_al029_2013_1.tif 2 4 8 16

My only addition to that command is the –config GDAL_CACHEMAX 10240 portion. I’m telling gdal to use 10Gb of RAM to speed up processing. I run that from either a CMD tool or a shell.

Three ways to build pyramids. Why? Well – if you have a copy of ArcGIS do you want to tie it up building pyramids? Yes – you can create a separate process but why not leverage a another computer if you have one. If you have two machines and one license of ArcGIS….load QGIS. Load gdal on the server. With just a small amount of mix and match you’ve got more options and *hopefully* get slightly more flexible. Perhaps you pursue that elusive batch command that will run when you are at home. Like I said in the beginning – the pyramid file is usable by both ArcGIS and QGIS. So build it once and use it multiple places. Of course all of this is negated a bit if you have a image server. That’s another story for another time.

And with that – yes – I just babbled for one page about building pyramids layers for a raster file.

OpenGeo Suite 4.0.2

rjhale · Jan 26, 2014 ·

So I’m not even sure how I’m going to wrap this post up quickly and efficiently. Here goes…..

We’ve a client that needs a “web map” – so it’s led us to a ton of digging and learning. Tonights fun was the OpenGeo suite. The best way to describe the OpenGeo suite from Boundless Geo is to describe it as ArcGIS Server with an open source accent. It runs on windows, linux, mac, and probably something else if you can find it.

With OpenGeo you get four pieces software:

GeoServer – serves the data
Postgresql/PostGIS – stores the data
Geowebcache – tile your raster data
GeoExplorer – display your data

It took me about an hour to create my first map published out of the suite – most of that time was spent reading the documentation. I uploaded a shapefile and here is what I ended up with.

Earlier in 2013 Boundless had announced support for QGIS. From all of the blog posts and classes you can guess we’re fans. There’s also a plugin for the OpenGeo Suite called OpenGeo explorer

With this plugin you you can publish data directly from QGIS into the OpenGeo suite. Actually – if I read the docs correctly you can publish into any Geoserver install (so you have some flexibility).

Just to wrap all this up:

Create data in QGIS
Publish data with symbology into Postgis/GeoServer
Data can then be viewed using GeoExplorer and shared with others.
Data can be pushed out into other application from GeoServer.
Any software supporting Postgresql/PostGIS can access any stored data

There are a lot of GIS solutions currently and there are more appearing everyday. OpenGeo wraps 4 packages up into an interconnected server product that give you some powerful options for a GIS program. Plus you’ve got a desktop hook

Overall – it’s worth your time to read up on their offerings.

Hollywood can do it why can’t you….

rjhale · Jan 10, 2014 ·

“Hollywood can do it why can’t you?” was the sentence that about put me over the edge back in 2000. I had inherited a “position” where I was going to be the “ortho guy”. Essentially the guy that took 9×9 scanned aerial prints and made orthoimagery out of them. Correct scaling issues and incorporate an elevation model. We had a scanner, we had software, and I had a slight idea of how it worked. Not a great idea so that’s why you had support, manuals, and a list of phone numbers of people I could call and go “I don’t know why but the RC30 Camera report hates Fiducial 4”. I had one job where I was just trying to piece together some imagery and it wouldn’t pull together. I had a bad survey point and one of the bosses said “”Hollywood can do it why can’t you? Just put it all together and “fly” over it. Like Hollywood…”

Anyway – yes – People used to strap cameras with film in them to planes to get aerial photography.

I saw two things that made me remember that stupid story over the last month or so. This article kicked off the thought train.:

Planet Labs: They are tossing out a “flock” of satellites from the International Space Station to image the earth. What they lack in pixel resolution they make up for in timeliness. Almost constant earth monitoring from these small satellites. Brand new imagery. Constantly. I can only imagine the headache of storage.

Sky Box Imaging: Video from a satellite. That’s all I can say on that one. Video. Imagine overlaying your data on video.

So yeah – Hollywood.

NAIP CIR Mystery Resolved: Downing the Fourth Band

rjhale · Sep 12, 2013 ·

Sometimes, 3 bands of imagery just isn’t enough. Sorry, RGB, you’re just not good enough! We came across this issue when trying to look at landcover for a forestry management client. Knowing that National Agriculture Imagery Program (NAIP) imagery, particularly the latest versions, comes with natural color/color-infrared (CIR) and has a decent resolution (1-meter) makes it very attractive for separating out vegetation from other types of landcover. It’s also sweet when you can find the CIRs for FREE! But there’s always a catch, isn’t there? Of course, there is.

Where to go?

The USDA NRCS Geospatial Data Gateway is a fantastic resource for downloading data, even NAIP. However, for reasons unknown to this blogger, you never get 4 bands. Know the answer? Post a comment below. After some research and failed attempts, I threw in the towel and told Randy that no free downloadable CIRs could be found, but that I did find a price list for ordering it. Don’t you know that my jaw dropped when the next email from Randy was a link to the USGS National Map with instructions on how to download NAIP CIRs. So here’s the secret handshake for getting a county’s worth of NAIP CIR goodness:

Open the USGS National Map Viewer.
Zoom into the general area/state of interest.
Directly above the map, click on the Download Data button:

Under the Download Options popup, go to Choose a reference area then click on Counties to select the county of interest in the map:

In the Selection tab to the left, click on Download and, in the next window, check the box for Orthoimagery. Next, you will see the details for the NAIP imagery listed, including the dates of the imagery and a link to metadata info (example for Benton Co., TN) FYI: Depending on the dates listed, you may or may not get 4 bands. If you need more information for a particular location or date, you can get that from here. You will need to download the PDF for it to be interactive.:

Check the box for the imagery you want then continue through the checkout process. You will need to provide an email address to receive an email with links from which you can download the imagery.

What next? Here’s the gotcha.

For the example above, we got an email for our county of interest that included 60 links from which our imagery needed to be downloaded. Evidently, the imagery gets sliced up into these 18.6 MB chunks which is a bit inconvenient but again, it’s FREE! I really didn’t want to click through each of these one at a time. We use Google Business Apps for our email so I looked around for an add on for Chrome that might help me out. Unfortunately, the download managers for Chrome weren’t really helpful for looking inside of an email. Fortunately, FireFox does have an add on that can look inside of email, DownThemAll! is the time saving ticket. Once you have your email open, you can right click in the email to select the DownThemAll! add on. It will show you all the download links from the webpage and gives you the option to check mark the ones you want. Just check all the image links then continue. It even provides you with a pleasant sound alert when finished. Below, you will see a screenshot from ArcMap showing that all 4 bands are present:

Now you are ready to mosaic the group of images together and start enjoying your NAIP imagery for your next landcover project.

Happy GISing!

Imagery

Contact