Blog

Intro to QGIS Virtual Training – April 27-28 2021

March 3, 2021 by rjhale

With the first class filling up I’ve decided to do another Intro to QGIS Class. Which I know – 2 intro classes but it’s a popular subject for those that just want to know something about QGIS. Again it will be 4 hours a day for 2 days from 12 EST to 4 EST. Plus virtual – which means I’m live.

I’ve revamped the materials to take advantage of the current releases and cleaned up the exercises. So those are now shorter and more plentiful and the hope is you can just do them without me slowing down much. We have a lot of ground to cover.

Again we’ll be covering:

Interface
Data and Symbology
Processing tools
Editing
Plugins
Layouts

Everyone gets a certificate from the QGIS organization (which makes it easy for me to push some of the money back to QGIS).

Anyway – show up. Limit of 10. Cost is $150 US Dollars.

Here’s the big change from the March Class:

I’m working on a refund policy for those that sign up but can’t attend. Since I’m using EventBrite I’m losing a small part of the cost for less headache. Anyway – I’ll have that posted shortly. Maybe I can put you in another class OR I’ll refund the money or something. We’ll figure it out.
If you’re a conservation group and you can’t swing the $150 give me a shout and we’ll figure out a way to get you in the class.

LIDAR and QGIS 3.17

February 18, 2021 by rjhale

I usually try to not write about the “pre” release (technically it’s master) of QGIS. Of course I went and said “HEY I want to talk about QGIS at a conference” and a large part of what I wanted to talk about was the new release. Which also included Point Clouds……and 3.18 is coming tomorrow. Which led to a frustrating demo which is why I’m writing this out.

TN has all their LIDAR released as open data and I’ve played with it in GRASS and PDAL and in general it’s a lot. It’s a lot of thinking. It’s just a lot of points and frustration. The data is all stored in zlas and it’s a pain to do anything with since I’m not working in ESRI Software. Generally I uncompress it and keep the uncompressed copy so I’m now hording 4 counties of LIDAR. I tend to stay excited about it for a day and then drift back to PostGIS and QGIS until the need to hate myself happens again.

AS of later I’ve been dealing with a side project where elevation is important. I’ve had questions and wondered if my approach on that has been right. I’ve also been running QGIS master as I’m curious on how I can leverage some of the new functionality with clients and projects.

So my first whack at points looking like this in QGIS:

Floating point cloud which is like 500ish feet off the ground. At first I thought this was a feature and then I had decided it was a bug and luckily I can shoot the developers a quick message and Nyall Popped up and said “projection problem”.

So Tn’s lidar is all hosted here. I had to go back and read the metadata. What I learned is I had a horizonal projection problem as I had assumed X and I had a vertical projection problem because I didn’t think that was an issue. In pulling the LIDAR data out of the zlas format I’m pretty sure I wrecked the projection. So in going back I needed to fix it. How do I fix it? PDAL. Going back I determined that my new LAZ file was EPSG: 6576 in the horizontal and EPSG: 6360 in the vertical. So I fixed it by this bit of trickery with PDAL:

I don’t want to get into all the technical on why this worked and how I did it – except that PDAL has some amazing command line abilities for fixing LAZ/LAS files….especially the projection.

Here is the cool thing. Upon adding the new data to QGIS I was prompted that I was missing a needed transformation. QGIS is running off Proj 7. I was prompted to click a link and fix it. I blindly clicked. I blindly imported.

It worked. LIDAR matches my DEM. Proj knows the problem and QGIS fixes it. Clickity Click. Probably doesn’t happen in 100% of the cases but it worked for me.

“I HAVE TO DO ALL THIS CLICKING OMG I HAVE LIDAR I DON’T KNOW THAT MUCH ABOUT IT” you will say. I think in most cases if you have a LAS/LAZ file you’re fine. This was a special case of me having data with a screwed up Projection/No Projection in preparation for a demo. So you don’t need to know things about PDAL and things about your LIDAR. You add it and go. It’s not a bad thing to know – but overall you should be fine.

This is all pretty amazing actually. You have this small open source desktop that can render a point cloud. I think this opens the door to analysis and other things that’s usually been outside of QGIS’s normal operations. GRASS can already work with LIDAR. Now you can look at it. Oh the things you’ll do.

Selecting by Location

February 15, 2021 by rjhale

You would think after doing this for as long as I have I would “have an answer” for about everything. Last week found me going “well select by location and check…well…check….what do you want? Well pick intersects… no…within…no… cross…well……&*^%”

So I started a small journey of self discovery on “what does every check box do”. Surprisingly – I didn’t know what every checkbox did…and again – I’ve been doing this too long not to know. Yes I can read the help but…..

I pulled out all the counties that surround me and randomly generated 5000 points and buffered them. Then I just started clicking and walking through “why”. To be honest I never just create test data – and that was amazingly fast and made my life pretty simple.

So – here we go. I will select my county and crank up “Select by Location” in QGIS:

“Intersects” does exactly what you think it does. If the polygon “intersects” my county it is selected. I have 851 polygons touching my county.
“Are Within” does what I expected as I have 651 polygons within my county.
“Overlap” does what I want – I have 166 polygons that touch the county line. They don’t reside in any one county

“Disjoint” gives me everything outside. I have 4149 polygons that don’t touch my county. Which makes sense as I have 851 that intersect
“Contain” does nothing. Which doesn’t make sense as it says “contain”. If I select my county and and run against the 5000 polygons nothing happens. If I select a polygon within the county and flip my select: Give me all the counties that contain this one polygon – that works.
“Touch” does nothing because I expect it to be similar so overlap. Select my county and run against the polygons and nothing. Select my county and ask it to identify every county that touches my selected one and that works. My county touches 10 other counties.
“Equal” does exactly what I expect as none of my 5000 polygons “Equals” the geometry of my county.
The weirdest is “cross”. I had to do some digging. So with cross a polygon can cross another polygon but there is nothing but more polygons. Lets say a line crosses into the county – that intersection is a point. After doing some researching crosses mainly is for lines crossing other lines or polygons. Which if I do a little bit of trickery with QGIS by converting the polygons to lines and running “Select By Location”. We have 166 selected lines which matches what I have in overlap.

So anyway – Select by Location. It does everything you want and some stuff you’re not expecting. Which – this was fun and I might do it with another processing tool I should 100% understand and don’t.

QGIS Virtual Training – March 23-24th 2021

February 9, 2021 by rjhale

March 23rd 2021 – 12pm to 4pm EST
March 24th 2021 – 12pm to 4pm EST
Cost: $150
Tickets: EventBrite
Participants: 10

So it’s been a year since I’ve taught this class. What a year.

Surprise – Covid is still causing issues. So I’ve decided to go virtual with the class. I’m doing two four hour sessions. I’ve reduced the cost of the class because it’s Virtual AND this won’t be as fun as sitting in a room with everyone. We will make it slightly fun though. Actually a lot of fun.

Anyway – the class is for beginners with QGIS. Maybe you’re curious about using it. Maybe you’ve tried and gotten frustrated. We will cover the interface, data types, layouts, and as many tools as we can. By the time you leave you should be able to do some editing, add data, make a map, and break something. I’ll also be hanging around after class to answer questions or just talk.

We all get a QGIS certificate which helps that project. I’m also holding this to 10 participants. The biggest class I’ve taught had 100 people in it and the smallest has had 3. Ten is a good number. Ten works.

THE BIGGER Conversation

I have 24 (almost 32) hours of material covering Editing, Forms, SQL, Mobile, Field Calculator, etc. No one wants to sit with me for 3 or 4 straight days for 8 hours a day in this class. So I’m splitting it up into probably 3 classes which is how it was originally written before I morphed it into one class. I’m going to teach the intro class for a bit and then offer the rest. I will probably offer a discount for repeat participants.

I’ve been thinking about this for about 2 months and trying to decide what to do and how to do it. Navigating the pandemic as a small business has been maddening. The thought of doing an online class is somewhat exhausting……..except…….

I’ve been teaching this class for about 7 years. The class has changed as QGIS has changed and it went from “talking” to more “doing”. So the problem will be now how to get the next two classes to make sense with regards to the first one…in a virtual environment.

Anyway – see you virtually. Well 10 of you.

Offline Editing Plugin: TN911 Project

January 15, 2021 by rjhale

One of the weirder problems we had on moving the ESRI setup was editing the data. It wasn’t the normal “hey lets add an address” but it was tackling the idea of “We need to edit the data in the field”.

As it was explained “the way we did things” was to drag the data out in the field with the editor because they were running a simple Arcmap/FBGDB. Now that we had moved to a Postgresql database I wasn’t keen on people “carrying the data out”. I preferred they connect to the database and edit it like civilized folk.

That is a problem because the county is rural and connections are sketchy and all of that led back to the idea of “Take a copy of the data with you”. We had a mobile solution but nothing making my life easier.

There is a plugin that originally caused much angst and it is called the OfflineEditing Plugin. It’s seriously the most no-thrills thing you will find in QGIS.

At one point in the early stages we had a lot of errors and I couldn’t figure out why. We have an ID attached to each feature that never changes. The client had discovered the plugin and it would let you check data out and edit it and check it back in. Which was what we wanted to happen…..except back when 3.0 was brand new we had a lot of things change unexpectedly. I had a trigger setup that added the primary key to the features ID. So data would come out. Data would go in and between the triggers and functions the ID would change. To the client this looked like a great solution. The intern thought it was awesome. The guy watching things change in the database didn’t have much joy.

I’ve not touched it since 3.2 or 3.4 in QGIS was out.

I’m running a “beta” (3.17) copy of QGIS at the moment. So I built a quick database and decided to play. It checks your data out and you then have a seamless offline dataset stored in a geopackage (the original data is in a postgresql database). I check it out and edit and check it back….and……

It worked. ID’s were respected. Sequences were maintained. I added data and checked data in and Edit Dates and Edit Times were also saved (at the time of check-in). Since I checked the entire database out and checked it in things I didn’t work on received a new edit date. Which – I think there is a way around that but that goes beyond the 1 hour test today. Which – given the 2+ years since I’ve touched it this plugin may have been working fine for a bit.

So what does this mean? Well – you can take your laptop and check your data out and go into the field. AWESOME IT’S VERSIONED….well not in the way ESRI versions anything so you will overwrite some data if you aren’t careful. It does make this setup a little more flexible although I’d rather toss this into a mobile app and only worry about the things we need to worry about – in other words if you’re working on addressing in a neighborhood I only care about that neighborhood. Which means I’d probably split that data out – check it out – edit it – Check it in and then merge it into your original data. More work but safer. I like safe.

Overall – a much better experience with the Offline Editing Plugin this go around.