North River Geographic Systems Inc

Geospatial Problem Solving

QGIS

QGIS Process Command

I’m wrapping up the 2nd QGIS Class and have been digging hard into the python/scripting/programming side of QGIS. A programmer I am not and usually I have to pick some sort of path into the software that makes sense to me. Scripting has been coming up more and more as I dive back into the TN 911 Project.

I discovered something that’s been around for the last bit in qgis and I no doubt read the release notes on it……….and immediately forgot about it: qgis_process. You have a command line way to access QGIS Processing tools. My examples will be Linux oriented but it works on <fill in the blank>. Open a shell window and type qgis_process:

If I want a list of tools at my disposal: qgis_process list

If I want to know about a tool in particular: qgis_process help qgis:buffer

What if you want to run a command:

qgis_process run qgis:buffer –INPUT=parks_2274.shp DISTANCE=5000 –OUTPUT=parks_buffer.shp

….and if I check the output using QGIS:

Of course I did it all using shapefiles – I’ve got all the output file types I need at my disposal.

The nice thing is I have this uncomplicated way to access the processing tools and running them. Granted this isn’t Python but still – a command line way to access what you want to access. You could set up a very uncomplicated process to: Buffer -> Clip -> Merge -> Whatever you want without cracking open the desktop.

This points out two problems in my life:

  • Generally just keeping up. QGIS is an exciting side of life for me and one I enjoy. I really enjoy this because I started my life on the desktop with analysis and data cleanup. I’ll most likely end my career still enjoying that side of life – oh yeah Desktop GIS is dead I forgot.
  • Running a business is everything BUT the technical bells and whistles I enjoy. I need to make more time for that even if I have to block off a Friday morning for 2 hours and just buffer parks for no reason.

Intro to QGIS Class June 22- 23rd 2021

It’s happening again:

Here is a bit more “behind the scenes” on training and life. When covid was causing me massive issues last year (and it still is – just to a much lesser degree), I was fighting the idea of teaching online. I had taken and had done online classes and it was (for me) harder than I wanted. Coming out of 2020 I made the decision to do 2 to 3 QGIS classes and see what would happen.

Overall it’s not been too bad. Any difficulties that have been encountered were just the normal human error on my part or the participants. I’ve gotten about 24 people trained and raised 560 dollars for the QGIS Organization. Which not that many people or not that much money BUT – this was never attainable holding in person classes. I did decide to bump the class up 25 dollars to cover certificates. Each QGIS certificate is 22 to 24.50 and having that little bit extra helps.

With that I’m in the middle of finishing up the next class with a “final” one being draft out. The problem with the next few classes is time. The Intro class is pretty much 7 hours and 45 minutes. The next one is looking like 16 hours or longer. The last class will be 8 to 12.

Anyway – all of that to say that training is actually going well. Come take a class!

Trash and Input

I went canoeing. I picked up trash. Which wasn’t all that bad – I should have picked up more BUT it was a nice day to be floating down a creek.

Junk Boat

Anytime I get out in the canoe I do some thinking. Today ended up, worrying over the new QGIS class and this map I was making. I’m creating a map of canoe launches for a local county. I’m also using part of the map in the class. Which then led to “What if I find an new launch no one knows about?”. It happens. We found one that a high school had built and a known one that the National Park had closed.

I had talked about Input almost a year ago and left it more or less with “Awesome but slightly complicated” and granted it wasn’t complicated at all but the flow wasn’t quite there. There have been a ton of improvements. The flow is there now.

You have the option of creating a project out in the field. Last time I checked in on Input, the process was to build a project in QGIS and then syncing it through Lutra’s Mergin service. In my canoe (with very little cell service), I opened Input and went to “My Projects” and made a project that collected points. I decided to collect locations where we picked up tires. Luckily this wasn’t anything more than me goofing off because I usually don’t keep my phone out while canoeing. So I did a terrible job of data collection.

input Create Project Screen

From there I collected data and took some pics and started collecting data.

Input data collection screen

I took some pics. Made some notes. When I finished I synced it to Mergin. So here’s the cool thing. My Mergin Plugin in QGIS shows my project that I just synced. From there I can download it (and the photos) and display it on my desktop. The downloaded project comes as a geopackge.

QGIS 3.18 with Mergin

One of the nicer things about this endeavor is I can still keep collecting and sync back to my desktop at my convenience. If I make edits on the Desktop I can sync those back into Mergin. So you have this really nice back and forth with data collection: Input -> Mergin -> QGIS and QGIS -> Mergin -> Input. Next up is shoving those improvements into PostGIS directly.

Overall – Input has had a ton of changes for the better. Spur of the moment data collection worked. It makes me wonder on what else I can do. Which brings my grizzled old self back to how many options I have for my phone to go out and collect data. Yes in a lot of instances you need high accuracy data collection but in many cases (which is where I am in most of my professional life) a phone is it.

How far is it?

I’ve been working on a cartography project with a little bit of analysis thrown in for good measure. The fun thing is the project forced me down the PyQGIS route. Luckily for you this post won’t cover that. I actually wrote a script and it ran and it did something….it did something wrong but that’s a start.

Anyway – Distance. I had a problem and the problem was “How far are all the canoe launches from the final stop?”. So it’s a networking problem. I don’t care how much time it takes in the canoe – just distance.

I don’t use the network tools often. I’ve done shortest path routing from one location to the other before with mixed results a long time back. I have 9 locations to do. So digging through the tools it appears the Shortest Path Geoprocessing tool will work.

The tool is easy – pick a layer of points. Pick a destination. Pick a network layer which for me is the stream. Running the tool gives you a new layer of data showing the path from your points to the end point. In this case that was 9 lines in one layer for the 9 canoe launches. Each linestring has a cost which is distance. I did a join back to my canoe launch layer and suddenly in a few clicks I have the distance to the end.

From there to a spreadsheet and I have this nifty table showing the mileage from the Lee and Gordon Mill to Camp Jordan The Reeds Bridge Launch to Camp Jordan:

Was all this really worth a blog post? For me yes because I hadn’t done this lately and it was ridiculously easy. Plus this entire exercise has been fueling my PyQGIS itch. As I said earlier I had built some code to do the thing that this did – now I want to do a few things to automate running this on a larger more complicated pile of data. What if I wanted road distance to every fire station from every address? How would I automate that?

…and of course there is pgrouting – Which I’ve been playing with more these days. As I think about it that brings my networking tools to three things: QGIS, Pgrouting, and GRASS.

Anyway – Routing in QGIS is a thing. One more tool in the awesome tool box.

Oh no he’s talking about training again

I’ve been debating shaking up the blog portion of the website for a bit. I start writing and then go “What’s the point” and I’m sitting on a half dozen technical/non-technical things I’ve been working on. So anyway – where to start for this one.

So the Virtual training seems to be working. I teach the first one his month (actually just finished), the next one in April and after that I’ll know what is working and what doesn’t. There have been a never ending stream of emails from people on ranging from “I want to take an advanced class” to “Introduction is too simple” and “what are your plans for something harder”. To which I have asked “What do you consider harder?” and the answer is always “Not Introduction”. So I got that going for me which is nice.

There is another class coming. Actually when I had made all the arrangements to teach in a classroom setting – the class had turned into a 2 day event with the possibility of a third happening. Way back when I was an ESRI instructor it seemed like Intro to Desktop was 2 or 3 days, Intermediate was 2 days, and Advanced was two days. Which I have no plans on doing for 7 or 8 days – BUT – the second class is proving problematic as I’m sitting on about 19 hours of information which is about over 2 full days and almost Five 4 hour days virtually. What to do……..

The thought is currently to call this QGIS: Tools and Processing. Which means I get to cover Processing Tools, Modeler, Editing, and Forms. The problem with that is currently that looks like 10 to 12 hours of “stuff” and I’d like to keep this at 8. Maybe I can’t. Maybe it turns into 3 days which are 4 hours long.

Which in my fever driven fantasies about training that gives me an:

  • Introduction covering the basics
  • The Intermediate part where we are building models and exposing editing tools like forms.
  • Advanced…………

Advanced would end up being PostGIS being brought into the mix with QGIS. So end it on an Enterprise worthy note. Your organization needs multiple editors and a data repository and the normal directory structure isn’t cutting it so you would want this class. Of course this doesn’t cover:

  • Reports/Atlas
  • Time manager
  • LIDAR
  • Stupid amounts of Symbology like the Geometry Generator.

Those 4 things don’t suck up much of my life. They are important though….so Decisions. I don’t want to turn this into a training business which is a problem I worry about. I like working with data and clients. I also don’t want a repeat of 2020 where I realize how fragile my business is against a very small germ.

Anyway – more on training later….