Blog

The Year in Review – 2020 Edition

January 1, 2021 by rjhale

For my own sanity I usually do a year end Health Check on the business and see what went well and what didn’t. This blog post has went so well I’m on the 7th re-write. The first post was exceptionally dark and put me in a funk for a week….the next six became increasingly easier to deal with by acknowledging the current situation…..and that situation is covid-19. Covid and Me. Life in general.

I decided to do this one last time – and this will be the final attempt whether it’s rambling or to the point….who knows. I did some editing because I’m me.

I feel like I’ve lost sight of what makes me tick and why I started this business. The last 6 months of 2020 was like a giant spotlight on a lot of things that were making me uneasy.

There’s a ton of things I enjoy doing I’ve stopped doing. There are interests I’ve stopped pursuing that bring me a lot of joy. Reading. Camping. Intellectual pursuits grow dusty. Tech for the sake of tech.
The business was a money making endeavor that was going to occasionally allow me to do some fun things and “help”. It turned into a struggle this year – and 95% of that wasn’t caused by me – but it was and continues to be tough.

If you came here for glorious tales of awesome business stuff…….stop here. If you haven’t noticed this year has been…well….A giant flaming pile. I’ve had worse years – but not where the entire Flat Earth was having them too. I’ve been lucky as I’ve not gotten sick nor had any friends pass from Covid. It did wreck business a bit. It has caused a lot of stress on my end for multiple reasons. From arguing with friends and family to watching job after job stop and go on hold. I’ve watched several on social media lose friends and family and had friends lose people they care about. That made this post a bit hard to write as the first post was filled with complaining and that didn’t feel right. Each subsequent iteration got lengthier as I tried to convince myself and you – that I had a handle on things.

I don’t. That’s fine. I’m not sure many do.

I’m still here though.

By the time summer had arrived we had been in lock down 2 months. I think. Honestly I can’t remember. I have a general policy that if a job makes no movement in 90 days it’s dead. It’s not always the case – but in this case they ended. When you do consulting and the phone just stops ringing it’s a bit unnerving. So there wasn’t much to do but:

Do some Hiking
Do some Canoeing
Do some Bike riding
Do some Thinking

After leaving a job this summer for non-payment I had to take a step back. I usually don’t stop for non-payment. Which is stupid on my part. I try to make things work and finally I got to the point of “I can sit here getting yelled at and not getting paid or I can go for a hike and not get paid”. At the end of the hike I sat down and made a list of things I love and hate. Things I hate and things I like and Why am I doing or not doing those things. Why am I letting this startup make me miserable. Why Why Why………

I found myself working on the weekends part time for the non-profit for which I volunteer. They run an outdoor store full of camping and hiking gear. So during the week I would scream at NRGS and covid and on the weekends I lived a life I wanted 30 years ago…back when I was more familiar with Geology and the best spots to camp in the southeast. During that time I had a lot of invites for camping, canoeing, and hiking. It was a nice change.

There were come positives this year – so all wasn’t lost:

Work continued on the 911 project – I added more functionality and branched into the NENA standard.
Renewed one contract that will help fund the 911 project
Conquered my fear of github
Docker is less mysterious as I’ve got it working with minimal screaming.
I fell more in love with PostGIS and did more things with it. Really if my life just boiled down to QGIS and PostGIS I’d be happy until I was dead.

There were enough loss to fill out several pages. It is frustrating when things spin out of control. Especially when you can’t do anything about it. If you were sick there was nothing I could do. In general there wasn’t much of anything you could do. Out of all of this and the sitting and the fidgeting I did find things that absolutely fascinated me. Like:

……..as well as a whole host of youtube videos and other small bit of information that made this all a little more bearable. I rebuilt an 50 year old coleman stove. I fixed several things on my jeep that were broken. I have so curated my twitter feed that what was termed “doom scrolling” really wasn’t for me. I had all this information from blogs, github, and videos that appeared on twitter and I could watch it night and feel…well….better. Twitter is still a dumpster fire – but a fire I’ve seemed to curate into something enjoyable.

Many of you hate virtual conferences. I completely get it – BUT the things I had the chance to attend this year were awesome. From England to the US to Canada. I didn’t even have to put on pants. I did though. I know virtual conferences are different and not what you want. You want small Danishes and coffee and the chance to hassle people in person…..BUT – I would have never gotten to see different talks live (and yeah I could go rewatch but I like ‘live’).

NRGS is getting a reboot of sorts. It’s not the end. Covid has changed things. I’ve changed. We scream “Get things back to Normal” but I don’t think things will…or really they should. We say the new normal with some hope it’s not. I think in general it is – and that new normal means the way I was working won’t be sustainable anymore…or if it is – It’s still half a year from being back to normal or maybe longer. Plus I have this great list of things I need for personal and professional peace of mind. Jeep – you’re gonna do a road trip shortly. Yosemite National Park – I’m coming for you. Probably not both at the same time. Maybe. Although………

Good People we all must change some. Here’s to a better 2021. To all of you – I thank you immensely.

I’m making a Map: Location

December 23, 2020 by rjhale

I should write something. My last few “2020 in Review” posts aren’t working like I’d wish – so I sat down last night to make a map.

A map of what? Good question. I think in my every day attempts of databases and technical wizardry I lose sight of “I actually like maps” and I’m not the best at it. Given my background I should be better. I decided to make a series of maps on “things I like”: Canoeing, Hiking, Bike riding, etc. So my first shot is a canoeing map. A very simple map of something I like to do.

The location is easy – The start of the TN River Gorge. It’s a great 8ish mile paddle. Simple. Quiet.

My first thought was to do what I normally do and just drape things over a aerial photo. I’ve decided to make it a bit more fun and not go that route – This may end up being Black and White. I’m also going to stick to two sizes of paper in this series of maps. Either 8×11 or 11×17.

When you canoe this loop – you start north and head south and loop around an island and then head back north again. The first half is all upstream (heading south) and the last half is all downstream (heading north). Bonus it’s all NW-SE trending so it will need to be rotated.

I start building it in QGIS with some aerial imagery and Hillshades for fun:

If I make a 11×17 layout it looks more like this because I’ve added 120 degrees of rotation:

As I sat here I had a question. I scale it. I generally start to get an idea of “what” I want. I know the extent of the map but can I get the extent back into QGIS so I can start clipping things and not having to include 500 square miles of stuff for a map. I could just set up the boundary back into QGIS as delimited text (make points -> draw polygon). I could just “eye ball it” which is how I do about everything from Jeep repair to Home Repair.

For fun I searched the tool box and….Print Layout Map to Layer

I run that and……I know where my map falls in the main window. So I can buffer it and start clipping data.

Why even post….I had no clue this tool was even here. My next map (another canoeing map) will be a bit larger than 11×17 so I’ll nee to use Atlas to setup 2 to 3 pages. I now have the general extent for 11×17 at X scale, copy and past this boundary a few times, and rotate it to get the general layout locations I will need

Map 1 is well on it’s way to happening. This is a bit therapeutic and that’s something I need at the moment.

Drop Down Menus for the TN NG911 Server

December 4, 2020 by rjhale

The problem is I have too many options. Options are awesome except when you’re dealing with people inputting data and you only want them to do it one way.

QGIS has had the ability to build drop down lists for a good while and it’s a very flexible way to work. For those of you in ESRI land imagine Domains and Subtypes on Steroids. In earlier versions you had widgets and currently you have Attribute Forms.

If you have a data layer in QGIS (all my layers are coming out of PostgreSQL/PostGIS) just right click the layer and go to Properties. You’ll notice all your fields have a chance to be a “widget”.

Addressing data is probably 90% picking existing attributes. You don’t want to get too fancy in making up new stuff. For instance – Street Type.

I ran out of fingers and toes on counting the number of street types. For the TN guys they all get a ESRI File Based Geodatabase with this information spelled out as Domain tables. How do you do that here? QGIS gives you three ways to make this happen:

Unique Values sums up an existing data set and you can get a pick list of existing data. The good thing is you don’t have to do anything really to make it work and the bad is you’ll get Avenue and Avenew. Misspellings aren’t your friend.
Value Maps let you fill things out or import a CSV. So if you save this information in a spreadsheet you can import it in.
Value relations pulls from an existing table. So if you build a table with this information you can pull that in by first adding the information to your QGIS session and then pulling it into the widget.

In order to cut out down the options, I pre-built all the tables needed into the database. I add them into qgis, group them, and collapse the group because No one needs to see them. Yet. You can’t use a relation widget with the table being added to QGIS.

Back to the Attribute Forms and this is the boring part. I want to select the type table and apply it to the type field using a Value Relation. The important part is the key and the value is the description of that attribute. Both things could be “type” but I chose to have a type and a description:

Go through and start setting up your Forms. What if you have a second person working on this?

Save your style to the database source:

QGIS saves a table in the public schema and everyone with the proper permissions and access is good to go. Plus you don’t have to touch everyone’s QGIS session…well much. Users still have to load the tables to get started but good clean directions should work. Of course this does make me still debate a plugin that does part of this for you.

What? Yeah – it’s that easy. Everyone working with drop down menus and making good clean data.

Dockering with Docker

December 1, 2020 by rjhale

It’s been a while since I posted anything mostly because I have no clue what to talk about these days. I’ve been working away on the Tn Address Server and one thing I wanted to do is to get it working in docker. Which – many of you are probably going IT’S NOT THAT HARD. Generally you are correct but it’s still a weird area for me to jump into. I’ve toyed with Docker for years – in general it works but I hardly ever have a reason to do much with it. Except now with the way I’m working on the TN address server I needed a way to “spin up” a postgis server and test everything and then take it down.

So I jumped back into Docker. Again.

You can build your own docker instance. You create a dockerfile and start adding whatever you need but I quickly decided I didn’t want to build my own dockerfile…yet. After doing a bit of research Kartoza already has one built. Also – they have one for Geoserver . Geoserver has been one of those things that I don’t have quite integrated into the flow of the address sever yet – but I’m working on it.

Sitting down with Docker I had four things I wanted to do:

Functional database I can start and stop
Local Storage
I can sit in another room and work and still get to it on this machine.
I don’t want to pile of things installed on my computer.

Simple? Yes…mostly. It seems like for most of my career I’ve been dancing in between developer, system administrator, and geo person. I never seem to have complete control over any of the things except being a “geo person” scratches the itch more than the other two. So that started a day long “How do I…..?” which resulted in again reaffirming that stack exchange is the devil, documentation can be boring, and a lot of people can’t write anything to save their lives (myself included about half the time).

The biggest headache was keeping my data permanent on my hard drive. I can mount the geoserver docker to a directory – why can’t I mount the postgis docker….and surprisingly there is a wealth of misinformation out there. In short – you have to make a volume for postgis to work – which is basically doing the following:

Create a /directory/somewhere
Create a volume: docker volume create –driver local –name pg_data –opt type=none –opt device=/directory/somewhere –opt o=bind

Finally I did the following:

docker run –name=postgis -d -e POSTGRES_USER=user -e POSTGRES_PASS=pass -e POSTGRES_DBNAME=tndemo -e ALLOW_IP_RANGE=0.0.0.0/0 -p 5432:5432 -v pg_data:/var/lib/postgresql/ kartoza/postgis:latest

Then

docker run -d -p 8580:8080 –name “geoserver” –link postgis:postgis -p 8080:8080 -v /media/data/demo/geoserver_data:/opt/geoserver/data_dir kartoza/geoserver:latest

PostGIS and Geoserver up and running in harmony…in docker. I was pleased. I can connect geoserver to postgis. I can connect to both things from my couch while I listen to the Baby Yoda Chronicles.

I was also a bit dismayed. Mostly at me.

To burn the amount of time I did getting to a functional setup was way too long. Not anyone fault as I have directions, functioning internet, and the ability to ask questions in the community. Which has been a problem for me as of late – I remember back to past failures and start fresh on a project and get haunted by the past failures. So much so I’ve been “tuning out” more. The pandemic has made things frustrating enough without my brain getting in the way. So I’m back to the art of learning this month and tossing out the old.

Now that this is running I can build and break the database at will now and hopefully get a running version of this that does everything I want it to do.

Maybe I’ll build a dockerfile just to say I did it.

Maybe I’ll do a few other fun things.

Modeling Part 2

September 22, 2020 by rjhale

Not being content with my last model….

I dove back in because I really need to name the points something because the forestry guys almost always use GARMIN GPS equipment. After sitting on this for a week the question became “How do I name everything” and get it to the point where the model handles 90% of it:

Name the point and since I’m exporting GPX the field has to be “name”
Perform some sort of field calculation to get a “pt1” for a name
Label it in QGIS because I really want to be lazy.

I found the rest of the tools to let me do this:

Add autoincrement field which gave me a 1 through X number
Field calculator let me name this ‘pt’ || “AUTO”
Set layer style

My Model now looks like:

Not to go over the additions in excruciating detail but three tools later and I have a functional point layer. My next question is can I slap this in github and share it – which I’m going to do shortly. Will this break the layer style? I’m not sure – but I can save this with a ton of notes for more development.

I’ve really ignored the functionality of the Processing Toolbox. This small dive into it has highlighted some sore spots I’ve had with the TN 911 address project. I get frustrated because I’m not the best in Python. The big problem I’ve had has been working through a few problems that are going to require some sort of interaction between the 911 address person and the database and I can’t do that part in SQL (I could – but it would create too much black magic). I can fix it with a model though.

Then all my python fears get a little bit better because all of this can be moved to python and I can pick at it.

My 10 to 15 minutes of point building are now down to 10 seconds.

Now to get them using something other than a Garmin GPS Unit – like their phones.