QGIS

QGIS is headed to TNGIC April 24th 2018

rjhale · Apr 8, 2018 ·

Something I meant to have mentioned and have continually forgotten to mention – the Intro to QGIS 3.x 4 hour class is headed to the TNGIC Conference during April 24-26, 2018.

I’m shocked that this conference branched out enough to do the training. Not terribly shocked on the other hand as I made them a really good deal on cost just to be “rebellious”.

The four hour class has most of the exercises removed. There isn’t enough time to do everything like editing and the final chapter on databases. So we will cover in 49 slides (8 hour class is somewhere around 100 slides):

Introduction
Interface
Data and Symbology
Geoprocessing and Plugins
Layout Composer

Anyway – if you’re in the area sign up. I have no clue what they are charging but I’ll even make the offer of “Sneak in if you can”. I don’t care – the more the merrier.

Prayers and Hay

rjhale · Apr 1, 2018 ·

I know – it’s been a while. I’ve been busy.

I think this first quarter of small business has been the best I’ve had in quite a while. I could point to a lot of things on why I think this has happened but I often find it’s just guessing. When you’re a one man show running off services and support there’s nothing tangible like product sales or earning reports that make sense. Some months you’re busier than others. Some months you aren’t. You form business partnerships and sometimes those fade. Sometimes they get better.

It’s a dance on what you can/can’t do and learning what jobs you enjoy and what jobs you don’t. One thing I have noticed is QGIS and “being different” has taken me more and more into the backwaters of what I consider my service area. When a small municipality/company is faced with the juggernaut of changing software and ever climbing support agreements the whole idea of different sells itself. I have entered several “one stop light towns” to discuss open source software. It’s towns the big guys can’t touch and won’t. I’m OK with this. Sometimes good things happen and sometimes they don’t.

Consulting is sometimes more stressful than it needs to be. I’ve been known for getting agitated so I do my best to not let things bother me these days. Church is also stressful for me. When you grow up in the south you get dragged to church more often than not. I spent my fair amount of time in a church setting singing hymns and worrying over your position in the Universe. I tend to find a lot of humor in religion. I may poke at things like Christmas and Easter in jest – but there is a cut off line and I try to not cross it. When I was 2000 miles south working at the equator that turned into a thing you had to constantly be aware of. Sacred cows make the best hamburger – BUT – there’s a time and place for joking.

I noticed the other day was getting prayed over a lot. I go to lunch at a clients location and someone blesses the meal. Sometimes I get thrown in for good measure. I say afterwards “thanks I can use all the help I can get”. It’s interesting. Usually I wouldn’t think twice about it but I was at a clients location and they had easter eggs out. I grabbed one. It was an invite to the biggest church in town for Sunday morning service. Sunrise service followed with regular service and dinner. I will say fried chicken in a church setting is probably some of the best fried chicken you can get.

Not even thinking I opened the egg and there was not chocolate.

Me: Oh man.
Employee: What’s wrong?
Me: I was hoping for chocolate.
Employee: Better than chocolate it’s an invite to church – you shouldn’t miss church!
Me: <thinking Crap> Hey – I might show up. Thanks. I was just overly hungry this morning and looking for a snack.

I think in February every meal I ate with clients was prayed over. The first half of the month I didn’t notice. The last half I was wondering if the streak would continue. Which shouldn’t be a worry – but I like averages. Last week I was driving back from a client meeting and thought “You know I wouldn’t even be here except for the community effort of open source. The only reason I’m here is ripping out ArcGIS and replacing it with QGIS/PostGIS.” Business is dragging me back into a setting from which I’ve disappeared.

Not that I’m looking to dive back into an organized religious setting. For those of you outside the US – if you haven’t noticed it’s stressful times for some of us over here. Maybe not for all – but it is for me. So – I’ve decided to turn these meetings into my own “Peace Love and GIS” type of outreach. Don’t stress and don’t worry about politics. “Hey maaaaan do your thing and I’m doing mine over here. Hey lets talk about your data.” Last week I learned what “tractor and hay bale making machine worked better than another tractor hay bale making machine”. Which had squat to do with Geo – but it was a conversation that started out of “Can you make a layout in QGIS for me?”. The client enjoyed telling me and I enjoyed hearing about it. We talked about church. We talked about sewer lines. We talked about dump trucks. We talked about his kids and vacations.

Anyway – I know this is just a rambling post BUT – it’s more stories I seem to build up from this small business gig these days. A few years ago I found myself standing at a small town corner thinking “I don’t have time for this”. These days I don’t spend enough time standing still so I’ll spend 5 extra minutes taking it all in today. Maybe 10 and I might find some random person to talk to for a bit. Maybe I’ll eat lunch there but I’m not spending all day standing around talking. I have things to do and people to worry over.

I will most likely sleep in on Sunday. For those of you not and heading out with your family – Enjoy the day!

NRGS and QGIS Certification

rjhale · Mar 9, 2018 ·

Certification? Yeah. Just calm down and let me explain.

I’m a fan and not a fan of certification. At some point I was a GISP. At some point I was ESRI Desktop Certified. There’s been talk of how to have QGIS Certification for a while. I’ve not been part of the conversation. The other day I found out it was a thing now. Usually I would go “Pfffbbbbt” and ignore it….BUT….here’s what hooked me.

In short (read up here) – all the talk centered around QGIS certification boiled down to two camps: You had your GISP Approach where your portfolio speaks for it’s self. You have a test (IT Certifications) that certify you on an exam. QGIS isn’t an org that can really do either – so they went with option 3 – the community decides who can and can’t be certified. I submitted my class for review and it passed.

So what does that mean? I have an 8 hour class that you take. At the end I submit your name and 20 Euros back to QGIS and you get an ID that can be checked to see if you took the class. Will you win jobs and make your fortune based off this certification? No. You’re supporting the community by taking it though. You can still buy a book and read all your want and do awesome things with QGIS. You can download it for your platform of choice and use it with no certification. It’s made me think about giving the students a test at the end of the class. Haven’t decided on that one. So we’ll see what happens. I want to “do this right” in my head for the community. I did decide that the classes I teach at Conferences that are compressed down to 4 hours aren’t going to qualify. You’ll have to sit with me with 8 hours.

To be mentioned in this group – that’s pretty cool at least for me.

This gives me a way to donate back and not feel like I’m just leeching off the good work of other people. I still have other plans for financial support – but I’m getting to that one.

We will see what happens. I’ve got to start scheduling some classes. Want a class? Looking at breaking out of your commercial loop of software changes and portals and what not? Give QGIS a try.

I didn’t know it was called Refactor Fields in QGIS

rjhale · Mar 2, 2018 ·

The title may be the worst part of this post. Hopefully. Anyway – here’s what happened. I will explain it in all of it’s goofy detail with some hope that by the end you’ll do the same thing I did.

I’ve a client that is currently making me push what I can do with QGIS/PostGIS. We hit a technical bump with data collection and to fix it they walked in with a new app they had purchased from some online app store that records data from their super accurate GPS Unit. The manufacturer of the GPS unit had one – but they found this scratched the itch more than the one they could have used….and the one they were using. My eye didn’t twitch much. I just cautioned against randomly doing things with your setup. I mean what if it makes a file you can’t import into your database. What if………

They have in postgresql a table that was more or less this: column1, column2, column3 . This app could create several data formats that ultimately me with: Foo1, column2, Bar3. You can’t just “copy and paste” the data in because the columns need to match. So I need to force it to match. No matter what trick I tried it wasn’t easy BUT some of them did work.

An Example of my thought process. My python skills aren’t great – BUT – using the psycopg2 library, some ogr, and “the google” I was able to connect and change my data and push into Postgresql. More or less you parse out the columns and match them to the PostgreSQL table:

import psycopg2
import osgeo.ogr 
#import shapely

shapefile = osgeo.ogr.Open("/home/rjhale/gis/shapefiles/monument.shp")
layer = shapefile.GetLayer(0)

conn = psycopg2.connect("dbname='database' user='me' \ 
       host='localhost' password='itsasecret'") 
cur = conn.cursor()

for i in range(layer.GetFeatureCount()):
      feature = layer.GetFeature(i)
      descriptio = feature.GetField("descriptio")
      pos_x_sf = feature.GetField("Pos_x_sf")
      pos_y_sf = feature.GetField("Pos_y_sf")
      elev_z_sf = feature.GetField("Elev_z_sf")
      photos = feature.GetField("photos")
      local_date = feature.GetField("local_date")
      pdop = feature.GetField("pdop") 
      #Get feature geometry
      geom = feature.GetGeometryRef()
      #Convert geometry to WKT format
      wkt = geom.ExportToWkt()
      #Insert data into database, converting WKT geometry to PostGIS geometry
      cur.execute("INSERT INTO test.monument(descriptio, \ 
                   pos_x_sf, pos_y_sf, elev_z_sf, \
                   photos, local_date, pdop, geom) \
                   VALUES ('{}', '{}', '{}', '{}', '{}', '{}', '{}', \ 
                   ST_GeomfromText('{}',6447))"\
                  .format(descriptio, pos_x_sf, pos_y_sf, elev_z_sf, photos, \
                  local_date, pdop, wkt))

Holy crap. It worked. I really need to up my python game BUT – no one wanted to run something from cmd line. MAKE A PLUGIN! Which I considered. In my digging I found the refactor fields tool in QGIS. It’s very similar to loading data into a database with ArcGIS for those of you stuck on that side of the fence.

So how does it work? It’s easy. Load your data that doesn’t quite match anything as the input layer. Load the table you want it to match in the “load fields from layer” part at the bottom. Click Load Fields.

So what just happened? Refactor fields took my shapefile and matched it to the columns in PostgreSQL. If it can’t “guess” you have the option to make it match by typing in the fields. It generates a new layer that I can “copy and paste” into the database or import into the database. For some of the data I’m having to clean out the fields – so things that don’t match in the shapfile I’m removing from the new layer using this processing tool. Which – maybe I need to add those fields to PostgreSQL. I really want this table stabilized before we start. Although I’ve not tried it I’m guessing this works with any layer you want – I’m just using it against postgresql. I also will say this probably won’t match projections – so get your projection worries figured out before hand – this just handles the attributes.

Right now I’m looking at a model so they can ultimately just “ClICK SOMETHING”. We all like menus. Mostly.

Anyway – load your data. QGIS has many tricks to make your life easy and I had no clue. I did learn something on the Python side of life though and for that I’m happy even though it turned into a dead-end.

Password protect your password in QGIS

rjhale · Feb 5, 2018 ·

There are a number of things I just “do” in QGIS these days and don’t think twice about them. Work is picking up on the FOSS4G side of life and I find myself more going “Well yeah you just do X and then you have Y – I thought people knew that.”

Today turned into a discussion on the Authentication manager. So what is it and why should you care? This will be a 2 parter.

If you’ve used postgis with QGIS – you’ve seen this part of a menu pop up at some point.

You can supply a username and a password and connect to things. My “thing” is currently PostGIS/PostgreSQL and I use this a lot. The bad part is my credentials are stored in plain text. All I need to do is pop open the QGIS file or a qlr file with a text editor and my credentials are sitting there for the world. If you were to click on the red box labeled configurations (as pictured above) you get a different menu.

Click add and set a password you will remember:

and here we are:

You provide a Name and in my case I name it for what I’m going to be doing: “PostGIS” and then my postgres/postgis credentials. You can let it generate an ID or set one yourself. When you save it it creates a spatialite database in your .qgis2 directory. From here going forward you’re going to be asked for the master password that you set in the very beginning. You can have different Authentication credentials for each part of your QGIS life. I might have one for PostgreSQL, SQL Server, Geoserver, etc.

So what happens after it’s set – let me connect to a table in postgis after setting up my connection:

and I connect:

Notice my password and login are now hidden by the Authcfg ID – which I let default to some alphanumeric combination. Maybe you don’t want it to default (which will be part 2’s discussion).

Why worry about it? Maybe you are in a corporate environment and your password into things is a bit important. Maybe you are hitting web services and have a PKI Certificate.

So – if you want all the information go here because I have no doubt missed something or confused some aspect of the explanation. Like I said – all of this started with me going “Hey you should….” and the client going “WHAT”.

Security – it’s not overrated.

QGIS

Contact