North River Geographic Systems Inc

Spatial Problem Solving

QGIS

QGIS Profiles

· ·

I should probably start out technical posts going “Your Mileage May Vary” but the more times I did this the less worried I got about writing this post.

Anyway – It’s time for upgrades if you’re doing anything in QGIS land. Especially if you’re running a Long Term Release. QGIS 3.10 has had enough patches you should start using it. A client called with a problem in QGIS and I said “WELL LETS UPGRADE!!!!”. She did. QGIS Started crashing. QGIS wouldn’t save. Nothing good was happening. Of course I feel extra great about it all because I said “UPGRADE THERE IS NOTHING THAT CAN GO WRONG”.

All is not lost though. It’s actually pretty easy trouble shooting the problem.

QGIS has profiles. What’s a Profile? I’m glad you asked and I’ll refer you to the awesome documentation. In short – anytime you start QGIS you’re reading your default profile and that holds everything from variables/models/connections as well as plugins. Do I use profiles? Yes for the QGIS class because I want everything to be just like I installed it BUT I don’t want to loose anything I’m working on (So I’ll have a Training Profile and a default profile). Yes – Gone are the days where I’d gleefully just reset my install by deleting my profile.

If you start QGIS you can go to Settings -> User Profiles -> Open Active Profile Folder and all the details appear. Plus you get to see where this directory lives on your computer.

profile location
One of the few times I’ll show you a screenshot from a windows machine.

So how did we fix it? You upgraded and nothing is going right OR just some random QGIS weirdness happening. Make a new profile. I’m going to make one called “blog” by going to Settings -> User Profiles -> New Profile.

Guess what – A new folder appears along side the default (if you happen to be watching the directory where all this is stored).

You’ll also notice you’ve got a brand new session of QGIS that will open up. That brand new session will have none of your settings. WHAT? Yeah – you’re running a brand new “fresh” session of QGIS so you have no settings. No Database connections. No Plugins. Enjoy the untouched landscape of QGIS.

What happened with the client? I had them make a new profile and load their existing QGZ file using that new profile and it worked. Why? No idea. Maybe a plugin wasn’t happy. Maybe “some weird computer thing” happened. What I think happened – and this is pure speculation on my end: They were running 3.4 LTR and had been negligent in installing the patches that are issued once a month. So their copy of 3.4 was pretty old plus they hadn’t upgraded plugins……and then suddenly we jump to 3.10. If I were sitting there I probably would have upgraded plugins and maybe installed the last release of 3.4 before trying the 3.10 release. I’m weird that way and that’s not complaining about QGIS’s installation process. Things happen.

How do I make it even easier. Close QGIS. Rename the default profile folder to default-old. I rename the blog profile to default. Copy the QGIS folder, bookmarks.xml, qgis.db, qgis-auth.db, and symbology-style.db over to the new default profile out of the old one. Usually I just re-download the plugins. If I have QGIS models I’ve made I will move the processing folder over.

Do I use this “one neat trick” a lot? Not really…BUT if you ever get to a point where QGIS is crashing or not starting right make a new profile and see if that straightens things out. At the end of the day I have three profiles:

  • work
  • blog
  • training

The one thing I enjoy with the profiles: If something is really being weird on the client’s end I can have them zip their profile up and dropbox/email it to me. I can use it and then pick at it a bit and come up with an answer. I can pull someone else’s QGIS session into mine and trouble shoot some problems they have. How often do I use that? Not often…BUT when you need it you really need it.

So – QGIS Profiles. Read the Docs! Make a test profile “just because”.

Stay safe out there.

 

 

 

 

 

May 2020 Training Rescheduled for June 2020

· ·

Well – this is all weird.

The Upcoming class in May is pushed off to June 24th and 25th. Hey – Is this a good idea? NO IDEA. I figure in about a month I’ll know whether to cancel it or go ahead with it. Of course we will be following the Guidelines of the CDC, Chattanooga State, State of TN and whoever else we can follow.

With that said – I’m starting to reformat the class for “virtual”. The class is built for in an person trainer and with that looking rather grim these days. I’m working on a slightly different setup using zoom/skype/something to work through the material. I could continue talking about that – but I’m pretty much tired of discussing virtual type of things as I suspect a good portion of you are.

Anyway – back to work. If you do wish to show up in June for a class – it’s 2 days in Sunny Chattanooga. We may knock off class early the first day and go canoeing.

Length of workshop: Approximately 16 hours

Topics Covered:

  • Interface
  • Vector Data
  • Raster Data
  • Plugins
  • Processing Tools
  • Field Calculator
  • Exercises Throughout

Registration: https://www.campusce.net/cscc/course/EventDetail.aspx?C=106

Cost: $700

Anyway – Good People Stay Safe and Learn some QGIS.

QGIS 3.x: Setting up a Project

· ·

I’ve been digging through the QGIS class this week as I wrestle with a way to move the class online. While doing that I started filing in the gaps of things that I wanted to add but didn’t because of time. Well – with the class moving online I can add more things.

Usually what we do when setting up a QGIS project is:

  • Open QGIS
  • Toss a bunch of data in it
  • Save it somewhere
  • Come back to it later

So for those of you just starting out or maybe for those of you who have been working with this for a while – a helpful hint.

Open QGIS. Slow down and go to  Project -> Properties -> General. We are only going to look at 3 things in this – so don’t worry. This won’t be anything complicated.

You should see this:

qgis project properties

We’re really going to worry about two things:

QGIS project properties

When setting up a new project I always do one thing to kick off the event. I build a directory structure to start storing scripts and maps and anything of use.  I set the Project Home (above) to be that directory. I also set a working project title like “My Map” or something.

qgis working project

I also go ahead and set the Measurements. I know this project will be in a local planimetric projection so I’ll set that to None/Planimetric and since my world revolves around Feet/Square feet I’ll go ahead and set that up.

The next thing I do is set up the Coordinate system – so click on the CRS Tab on the left and set it to whatever you need it to be – in my case EPSG:2274

project properties CRS

I click Apply or OK and the last thing I do is Save my Project (back to the Save Project portion under the Project menu of QGIS) . That last save leaves you with a complete Project Properties.

Completed project properties

So who cares right?

Well if you happen to use the browser panel you’ll notice something has changed:

project home in browser

 

Project Home now appears  – which means I have quick access to everything I need from shapefiles to geopackages to imagery to whatever. Right in the browser which really gives you a ‘Workspace” feel for your project.

How long does it take to set up? Not that long but the benefits of getting that one small part done outweigh the time you’ll spend setting it up. I’ve been bitten many times by having the wrong units set up or something else that could have been taken care of by spending 1 minute setting this up.

Mobile Data Collection: Input

· ·

If you flash back to 2008 you would have seen me sitting on my couch unwrapping a Trimble SB (?) running Arcpad. At the time I thought “collecting data” would be a beneficial add-on to me being a consultant. Spoiler alert – It wasn’t.

If you flash back to 2013 I was sitting in the Caribbean needed to collect data and I was holding a new giant yellow GPS unit that wouldn’t quit rebooting when it got hot. Spoiler Alert – It’s always hot (generally) in the Caribbean. What I did find is Fulcrum – and that allowed me and about 5 or so folk to disperse and work all over the place and collect data – WITH OUR PHONE. Which for 2013 was pretty awesome for me.Since then I’ve always been interested in how people are collecting data. I will make fun of you for doing it “Pencil and Paper” but I will admit to having a field journal and pencils in my laptop bag just in case.

A client asked a few weeks back “We need to take a copy of the data into the field” and that trip into the field was going to be with no to limited connectivity, no data collection, but they needed to know where they were. They are a IOS/Mac heavy group. Print a map seemed a bit harsh and working through some other hoops seemed unnecessary.

So my suggestion was Input. So what is an Input? Lutra Consulting developed a mobile app built off the QGIS Source Code. It runs on Android and IOS. There are tutorials out there on using Input. I can’t do better than the ones that are out so I won’t even try. I can throw in some screen shots and just general discussion on why Input is pretty cool from an avid QGIS user. When you get it installed on your mobile device of choice you get this:

input start screen

Simple. Home being the map screen, my projects being things you can work on, shared being projects that are shared with you from others, and finally explore just in case you can’t find your project.

So we will tackle this one of two ways. One way with Mergin and one without. What’s Mergin? Mergin is the commercial end of Input. Commercial? What? Yes – there is a commercial end of Input.

1. Install Mergin and Get an account. Don’t worry – it’s free for small projects. Download Mergin from the Plugin Manager in QGIS. Mergin will appear in your browser panel.

2. Build a project in QGIS. Import your data into Geopackage. Build the project the way you’d want it to be displayed on your phone. When I say “build your project” you’re going to build widgets for data input, set up labels, and set up your colors. I prefer boring as seen by this screenshot.

QGIS Project Screen

3. Import your project into Mergin (You’ll have to connect the plugin to your Mergin Account).

Give it a few minutes and you can now download your project into Input and start working. Edit your points, lines, and polygons….and attributes….

Input Project

 

4. Sync your project back to Mergin. Download it back to your computer.

One word of caution :

  1. You’re going to build a project .
  2. You’re going to COPY it to the server using Mergin.
  3. Move it to your mobile device.
  4. Edit it locally and sync it back to the server
  5. Eventually you’re going to download it back to your compute using Mergin
  6. So at the end you’ll have one old project and one new one sitting on your machine. Just pay attention. I did get confused at first but I’m slow.

You don’t want to use Mergin?

  1. Then copy your data to your mobile device.
  2. Create a directory under the projects folder under Input and move your data there.
  3. Start working. At some point you’ll have to manually move things back to your laptop.

….and there you go.

So where does that leave me with all of my current love of data and databases? So as an experiment I connected to a test server and edited data against PostGIS (live). I wouldn’t suggest that for day to day because “you always need a network connection of some sort”. So I’d always build a local project to share to your mobile device.

I need more than “me” collecting data – then read up. Several people can edit and Lutra has used Geodiff to handle multiple users. All your edits can go back into the same geopackage that you download back to your Workstation.

So should you pay for Mergin? If I stuff a street centerline file, building footprints, streams, and a few other things for a medium sized city I’m getting close to 100mb of disk space. So I’d say pay. If you just can’t pay – then keep it under 100Mb or manually move your data over.

Overall I like it.  For years I’ve said “QGIS on Mobile” and now we have Qfield and Input.

 

 

 

 

 

 

 

 

 

Fishbones for the TN NG911 Address Server

· ·

It’s been a bit since I’ve talked about the TN address server. I’ve been slowly adding more functionality to it and some of that functionality has been more difficult than others.

If you’ve played in the address space any length of time you’ve heard of fishbones – and that is simply drawing a line from the address point to it’s “spot on the street”. Granted if you’ve ever done an addressing project from scratch (I had the joy to do that a few years ago in the Caribbean) it makes a lot of sense. You can quickly see problems where lines cross over other lines or where addresses have no lines at all.  Please read up here if you are so inclined.

There are probably multiple ways to do this  but I’ve gotten fishboning (if that’s a word) running in the TN NG911 Address Server. In PostGIS with less than 25 lines of SQL (and I’m quite proud of myself – 5 years ago I sucked at SQL. I still suck at it – but way less).  While it’s not quite ready for prime time it’s close – I’m just trying to figure out what I can pull out of this that makes the County’s job easier in the addressing realm. I’d love to move it to python at some point – and it’s probably doable (well I know it is – it’s just doing it).

Three things happen:

  1. I join the address point to the closest road using ST_LineLocatePoint
  2. Does the closest point fit the address range on the street (double bonus I also check left and right).
  3. Finish it up with ST_Makeline to join the spot closest to the street to the address point.

Which quickly shows something is wrong here:

Bonus for this one as is it’s not incredibly obvious what the problem is at first glance….UNTIL you see you have an addressing issue where you have 2, 3, and 4 on the same side of the street.

Hurrah for misspelled street names by leaving a W off

Anyway – there will be more announcements coming like this “running in the cloud” and other things. I’d like to get the fishbones running once a night or realistically run the one for the new address. Why not both probably. Anyway – pretty nice for an ArcView/FGDB Replacement.

Contact