So going back to the college kids and the “OMG I gotta learn python”. It actually it helps if you do learn python. It will help your career and it’s always nice to know how to program in something….
BUT
I will submit on this the second day of XYMas that you should learn GDAL before you dive into python if you’re interests are centered around location. GDAL is the underpinning of a lot of GIS Software (ArcGIS and QGIS just to list 2). It’s command line based (See the previous post) and it’s all powerful as you can manipulate vector data and raster data and there are a ton of resources for using it with python out on the internet. No license required. All you need is a strong heart and nimble fingers. Remember – GDAL tools work on raster and OGR tools work on vector. That’s lesson 1.
As my small working example or why this knowledge will serve you well:
- Lets download Significant Earthquakes from the USGS as a csv file: https://earthquake.usgs.gov/earthquakes/feed/v1.0/csv.php
- I’m going to use ogr2ogr to move it into something other than csv:Â ogr2ogr -f GPKG equakes.gpkg significant_month.csv -oo X_POSSIBLE_NAMES=longitude -oo Y_POSSIBLE_NAMES=latitude -a_srs ‘EPSG:4326’
- I’m now going to query all the earthquakes > than a 5.0 from GDAL: ogrinfo equakes.gpkg -dialect sqlite -sql “select mag from significant_month where mag > 5.0 order by mag desc”
For fun I’m going to drop it into QGIS just to see everything (this was pre-New Zealand Volcanic Eruption):
There are a few things I could do to make this better – like making sure the magnitude is a number field and not a string but for the example it works.
Last year I had a client suddenly surprise me with a “Hey – we gotta deliver some data to the state and it has to be in an ESRI acceptable format”. Which – no big deal except I had spent a good deal of time moving them to postgis to get away from ESRI. A year ago we set up a script and it’s been running for 15 months or so. We’ve upgraded software – we’ve changed servers. With a few modifications it’s still running.
ogr2ogr -f GPKG TCStransport.gpkg -nlt point PG:”host=host user=user dbname=dbname password=pass” -nln “addresspoints” -sql “select id, oirid, geom, r_segid, a_segid, seg_side, gislink, cast(structype as SMALLINT), strucdesc, stnum_h, stnum_l, stnum, stnumsuf, building, floor, unit_type, unit_num, secuntnum, predir, pretype, name, type, sufdir, postmod, address, addr_esn, label, subname, vanity, zip, zip4, esn, city, county, state, cast(source as SMALLINT), lon, lat, x_sp, y_sp, z_val, gpsdate, addrauth, editor, geomod, geosrce, geodate, attmod, attsrce, attdate, cast(status as SMALLINT), delnotes, right_left, comment, readdress, old_address, color, point_type, street, cad_name, loc_note, cad_post from addresspoints”
Think about that – a small program that I (2nd worst programmer in the known universe) wrote that has now run through 1 server upgrade and 3 software upgrades without breaking. Every night for over a year. I should have set some sort of $3.50 a day payment plan based on success.
….and I’ve not even discussed the work you can do with Raster data. Actually – lets discuss the GDAL part. I uncompressed a SID image and needed it for a client – so I ran this to compress it back to something normal sized:
gdal_translate -co TILED=YES -co COMPRESS=JPEG -co PHOTOMETRIC=YCBCR -tr 1 1 -mask 3 ortho_1-1_hc_s_tn065_2018_1.tif NAIP_2018.tif
Could I have added more “things” to make it better/smaller? Probably – but it worked. So I can take any sort of image file and un-compress, compress, change the format, reproject, etc……All from GDAL. It works with everything including ArcMap/Pro/Whatever it’s called.
NO LICENSE REQUIRED. IT WORKS WITH PYTHON FOR YOU PEOPLE WHO WANT TO LOVE PYTHON.
For this second day of XYMAS I point you to:
- An excellent tutorial by Sara Safavi: http://www.sarasafavi.com/intro-to-ogr-part-i-exploring-data.html
- The GDAL Documentation: https://gdal.org/
- The Python Cookbook: https://pcjericks.github.io/py-gdalogr-cookbook/
So I do ask you to learn GDAL and then move into python. It’s great software. It’s an active community. You will not waste your time learning this skill – if you’re going to stick around the GEO world you’re going to touch it at some point (or you should – if you don’t I have questions on why).
