North River Geographic Systems Inc

Spatial Problem Solving

QGIS

Teaching in Dumas TX

· · 1 Comment

QGIS Training usually goes one of two ways: Mediocre or great. With a class of 10 you have 2 or 3 that are really excited about QGIS and 2 or 3 that are glad they aren’t at work while the rest scratch their heads and go “Why won’t this guy shut up”. In the training in Dumas Texas – Someone yelled “Holy Sh*t” and I was caught off guard……

 

I’ve been getting more training requests from organizations looking at moving off their COTS system or integrating QGIS into their workflow.  So far I’ve been to 2 conferences to talk about QGIS and 2 organizations to teach QGIS. Last week had me in Dumas Texas. If you fly to Dallas TX and get in a car it’s almost exactly one tank of gas away. Dumas is a town of 16,000 but they also house the North Plains Groundwater Conservation District. NPGCD is moving off their commercial system – actually have moved and have dived fully into Open Source Software. RESPEC had built a backend system for them and most of the work happens there – but work happens in QGIS. RESPEC called and asked if I could do some training. My answer was yes and off I went.

NPGCD is in the groundwater protection business. They work with the local population from farming to industry to protect the water supply. The water supply being the Ogallala Aquifer. Fun part for me – it’s been a bit since I’ve been this close to my old job of “environmental GIS” work and since I’m a geologist by training I find the whole thing of aquifers fascinating.

tractor
I went to the Dumas Museum while I was there. If you like Tractors – have I got the place for you.

One of the fun things with the Intro Class is the exercises – well at least I think it’s the exercises. It took a while but I have them to a point to where I can rip them out and replace them in about half a day. The Problems cover the basics of what I think people struggle with in QGIS …. well GIS in general. So in walking to meet the group the day before class I realized I had all the exercises wrong. What I thought they needed wasn’t what they needed. So – I replaced them the night before class. Granted they were still not perfect but I caught the spirit of the problem. We ran through slides for 4 hours and then I went and sat with each person individually and answering questions.

In 27 years of doing this – people are always stuck with editing and attributes. You can run through toolboxes and python and talk about colors – but it always falls down at editing and attributes .

  • How do we find stuff?
  • How do we edit the stuff we find?
  • How do I save stuff?
  • (plus the How do I make a PDF to print because we need a paper map).

There aren’t many times I get to sit with a client during training and learn what works and what doesn’t. Day 2 we just threw the class out and played “Stump the instructor”. When we touched on editing that’s when the “Holy Sh*t” moment happened. We covered how to edit and merge data and move vertices and not create more problems than you started off with. No one has ever been happy enough in class to curse. It was great.

sunrise
Leaving Dumas while listening to Waylon Jennings

I’ve had some horrid classes. I’ve had some mediocre classes. I think this one was probably the best classes I’ve had the pleasure of conducting….way out in the middle of North Texas. Heading out Saturday morning I stopped on the side of the road and caught the sunrise while Waylon Jennings blared from the radio.

 

 

My People

· ·

Let’s flash back to 1996. The Olympics were in full swing. I was 2 years out of college and working for the federal government. I had only been to one “conference” and that was only a state level academy of science meeting. I found myself standing at the underused Chattanooga Airport headed to Cincinnati and then to Palm Springs to the 1996 ESRI users Conference. Coolest part was I was so inexperienced in flying when we were delayed leaving Chattanooga because Al Gore was coming to the Olympics – I had no worry. I would make my connecting flight because there was no way they would leave Cincinnati  without me on board. In Cincinnati I was standing in front of the near empty gate because the plane had left. So I was then routed to Chicago. I ended up in Palm Springs at nearly midnight. No rental car because somehow that had gotten cancelled. I stumbled into my room and sat there hungry and hating life with no luggage. The Luggage eventually turned up a little worse for wear. I think it went somewhere far away and was driven in….

The next day I found myself at registration at the Conference center in Palm Springs. I registered and went for a walk. I dragged my giant bag of stuff around with me. The Palm Springs convention center was big – but not terribly large. You could draw a dot on a map on “where are you from” and surprise there was only a few TN dots there. There were special interest groups. There were developers walking around. There were users. There was Bob Hope’s house down the street. There was golf. There was a lot of wind.

Two things happened that week.

  • I was blown away by a conference. I think there was only 3 or 4 thousand there. Maybe less. It was a good group of people. I mean the conference was nice – but I found a lot of like minded people there making maps and worrying over data. So I talked to people. I spent a lot of time just wandering the halls taking notes. The one talk that still sticks with me 23 years later – the guy that wrote ArcPlot gave a demo of “Tricks and Tips for Arcplot” (I think). As I was the guy who was getting hit by a lot of map requests I was pretty overjoyed to know I could edit data from Arcplot (plus copy files and move things around on the network without leaving ArcPlot). I also got a chance to hassle one of the arcpress developers (look it up – I’m not explaining it). The guy (whose name I can’t remember) finished the demo and turned to the near silent crowd and went “TADAAAAAAA” and everyone clapped. I may have stood up and clapped – Printing was a big deal back then.
  • I was sitting in the audience when Jack (who seemed really irritated at the Conference Center) announced “We are going to San Diego next year”. I think I went back to San Diego in 98 or 99 and once more in 2002.
  • (Third thing) – Some guy from ESRI Pulled up at a door in a VW bus. He had a keg or three crammed in the back and it was all held in place by some Sun Sparc Servers (look it up). I helped him Unload the beer.  We didn’t drop a server. Imagine that happening in San Diego…….

The move to San Diego ended up doubling the crowds. To me it was more of a pain to get a room and navigate your way downtown than it was worth. Yeah it is nice to see 10,000 other people who were doing “GIS” things – but I really don’t remember much from those years except for the times I snuck out of the conference and went to the San Diego Zoo and was probably almost eaten by a shark in La Jolla.

I’ve been watching Twitter and the ESRI UC is coming up for 2019.

I won’t be there. Which – no joke if I bought a ticket I doubt I’d be let in the front door….and that’s completely fine. That side of my career isn’t that exciting currently.

If I look back at my entire career in Geo (now at 27 years) it’s always been a never ending pile of new things coming down the pike that were earth shaking getting announced at this conference. Learn Arcview Avenue – hey now we are switching to VBA. You can compile ArcINFO with TK for a better ArcTools Experience. Lemme learn VBA and now we’re at VB. Hey we have MapObjects…no wait ArcObjects…….no wait we’ve got SDE for databases…….and look a map on the web…Hey Python I should learn that now…3D (Which I saw a demo of in 1996) is going to be revolutionary. My social media is exploding with posts and pics and that was me pre-social media on cloud 9 talking to people and learning that:

“HEY WAIT YOU MEAN YOU’VE HAD ARCMAP CRASH TOO…..IN FRANCE….OMG WE ARE SO CONNECTED”. Hah. I joke. Mostly.

Anyway – where was I going with this. I have no idea to be honest. I spent an hour looking for the pics I took in Palm Springs. I took a ton of pics with my fairly new 35 mm camera (look it up) – and they are somewhere in a box.

I did crank up ArcMap 10.2 this week to fix a problem I had that couldn’t be fixed in QGIS. No Joke. I sat there just staring at the screen. I couldn’t remember how to dump a DWG to a file based Geodatabase. Back in 2014 I spent an inordinate amount of time wishing for functionality that wasn’t in QGIS that I had in Arcmap. Friday I spent an inordinate amount of time wishing for functionality that wasn’t in ArcMap that I had in QGIS. Plus I really need to look at Pro at some point. Sigh.

If I could bounce back in time to 1996 Randy and go “Hey in 2019 you can’t remember how to work ESRI products” he would probably laugh and throw something at me. I would also yell for him to enjoy that hair while it lasted which won’t be much longer. Oh yeah – trade the truck sooner. Go ahead and buy a dumpy house in 1997 on the North End of town and don’t worry – it will be fine as you won’t leave your job for another 10+ years.

For all of you out at the ESRI UC – enjoy yourselves. It will be a massive week of demos and things happening within the ESRI Ecosystem of products. What will I do this week? Teach a QGIS Class for a group who just transitioned out of the ecosystem.  As much crap as I give the ESRI people at times – I do miss it a little (a little).

Anyway – I’m gonna go watch a movie and hope 1996 Van Halen doesn’t Break up.

QGIS 3.8.0 and QGIS 3.4.9 Released

· ·

First off – sorry for the lack of talking on my end. I’ve been sitting/thinking more than usual which has led to a pile of things I want to talk about. This blog has always been good for an outlet and that will happen soon enough. In between the random essays I’ll dive back into my comfort zone of technology like today. The dust has now settled and you should be seeing a new QGIS version(s) out and about.

QGIS 3.8 has appeared. For those of you in the know or not in the know – QGIS gets a new version every 4 months. You never have to sit back and wonder when exactly as it’s all been planned out. I’m going to dive into some 3.8 tech things shortly – but for those who want to read the changelog: http://changelog.qgis.org/en/qgis/version/3.8/

The next important thing – the 3.4 LTR (Long Term Release) has moved to 3.4.9. So for those of you who value having software that doesn’t change for 1 year – update! I get to talk (thankfully) to a lot of people who are using QGIS and I always run into 2 situations: “Hey we just updated to the new release” or “We’re running the LTR and we never update”.  Every release of the LTR brings a new round of bug squashing and improvements but no new functionality which makes it perfect for corporate environments or educational environments.

So choose wisely and happy QGIS’ing.

Virtual Layers and Bike Data

· ·

To badly quote a Queen Song, “I like to ride my bicycle”.  I’ve been trying to find new spots to ride. Of course being me I’m always debating things I can map while I’m out doing something.

This weekend I decided to map the Bike Share Program. It was raining so I went with a combination of windshield survey, bike riding, and internet searches to locate the Chattanooga’s Bike Share locations. I have a long standing history of being wrong about everything. When the bike share program started back in 2012 I shrugged and went “this won’t work” and 6 years later we’re got 40ish stations (I counted 40 and the internet tells me 41) and over 400 bikes. All the bike shares have a printed map and Chattanooga’s open data portal has a CSV of old bike locations.

Here are the 40 bike share locations in Chattanooga. I’m hoping to stick this in OSM and on the Open Data Site for Chattanooga.

OpenStreetMap Chattanooga

I had read about Anita Graser’s Flow maps recently.  One thing I notice when I’m out is how many places I see these bikes. Where do people come from and where do they go? Digging around on Chattanooga’s Open Data portal I found travel information that was slightly out of date but provided enough information to make this fun. The data gave me two things I needed: bike check-out locations and check-in locations. I know now people leave from Point A and end up at Point B.

Lets create a virtual layer from the table of information and the point layer. Trips being the table and Locations being the points:

SELECT StartStationID, EndStationID,
make_line(a.geometry, b.geometry)
FROM trips
JOIN locations a ON trips.StartStationID = a.id
JOIN locations b ON trips.EndStationID = b.id
WHERE a.id != b.id

I created what I can only describe as “hub lines” of the Bike System:

That doesn’t really tell me anything. It does show me the power of a virtual layer. A virtual layer is an area of QGIS I’ve not touched much – but in short it’s code  tossed against a dataset and that code creates a new layer in QGIS. In this case I’m using functions in QGIS with some SQL code to render the lines. The lines act like physical data.

I refined the code a bit to only show one bike share. So where do people go that leave from the “main” spot (1299) at Outdoor Chattanooga?

Now that we’ve got fewer lines, things are more interesting. First off I don’t have data for all the stops – hence the dots with no lines. The second part I notice is some people go as far as they can go on a bike. They go to the farthest station. The Bike Share Program is “transportation” now on the same level as a bus but in just a smaller area.

Lets pick at it some more and do this:

Select EndStationID, count(*) from virtual layer group by EndStationID

That tells me the counts of where are people going from 1299. Not surprisingly from this location the #1 destination is this location (1299). People come back to where they rent the bike. The next two stops are “the scenic” bike share at 1319 and the bike share that connects to the free shuttle program at 1303.

The least used one is unsurprisingly 1329. That one is located not quite near the college campus and not quite near anything useful for day to day living. There’s really no reason to go there unless you’re lost or going to the Walker Theater for a concert. 1329 may be at the highest “spot” in the city or at least close to the highest (i.e. you have to ride uphill to get there from anywhere).

My next big question: Is it all connected with bike lanes? That data doesn’t really exist except within OpenStreetMap…and it’s outdated. It may exist somewhere in the city but I’m not sure where. My next mapping fun is finding a source of bike lanes for the city. I will probably make my own.

So what did we learn?

  1. If you come to Chattanooga use the Bike Share (and wear a Helmet)
  2. Virtual layers are pretty powerful and I really only did one thing – but it was a powerful one thing.
  3. OpenData for your city is your friend. I will most likely hassle someone for an updated CSV for the city bike share program.

Go out and make a map….and ride your bike.

QGIS Leader Lines for Labels

· ·

So way back when…..I was working in a production map shop and I had to make a map of a Nuclear Plant. The fun part being it was all manual. So we would use a lettering machine to place the labels and legend on the enlarged photo. We would manually trace all the features on an overlay.

Flash forward and we’re using ArcINFO and placing annotation and then drawing leader lines pointing to the features on an OrthoPhoto.

Last week and I had the chance to do something I don’t normally get to do: make a cartographic map in QGIS. I needed to make leader/callout lines from the labels back to the polygons. I didn’t want to manually draw in leader lines. Searching led me to two posts:

I ended up running with the kartoza example although both helped me understand how to get this working. So for the not too faint of heart……Here goes. I have some polygons on which I need to “cleanup labels” and leader lines would help in that regard. I could pull the labels away from the very small polygons I have and attach them back with a line.

The key to all of this ais Geometry Generators. The basics are you can use code and expressions in QGIS to draw Geometry. So I’m going to draw leader lines from my label to my polygon with some code (that I copied from Kartoza’s example – Hey to Kartoza). My data is Categorized. So I’m starting off with this:

So I’m going to click on Conservation Easement and add a symbol. The symbol will be a geometry generator. Just as a little background – I’m stacking symbology. So I’m adding another symbol to my yellow symbol by using the plus button on the symbol selector. I’m defining the type as a Geometry Generator and the Geometry Type as a line.

 

So what happens with the code I so skillfully pasted into the symbol selector?

make_line(
closest_point($geometry,
make_point( “auxiliary_storage_labeling_positionx” , “auxiliary_storage_labeling_positiony” )),
make_point( “auxiliary_storage_labeling_positionx” , “auxiliary_storage_labeling_positiony” )
)

The code says make a line using the closest point on the polygon and anchor it with the location of the label. Now something happens when I use the label toolbar to move my labels away from the polygons:

Now I have leader or callout lines back to my polygon. I’ll need to add a geometry generator to each category to give me lines for every category. The picky side of me is thinking for 100% control I should manually draw them in (but I’m not). If you read Kartoza’s post they have a more elegant final solution to deal with lines running over your labels. Looking at this now I need to move 210 back to the center of it’s polygon since it’s drawing the leader line under the polygon.Overall – Not bad.

 

Contact