North River Geographic Systems Inc

Geospatial Problem Solving

Software

“From the Trenches” talk at FOSS4G – Part 1

This will be a bit rambling. It’s been a while since I’ve put anything substantial up here because I’ve been busy. I broke it into parts because this started turning into a rant.

First off – I’m headed to FOSS4G in Boston to speak (and teach a workshop – but more on that in a bit). The talk in Boston is “FOSS4G from the Trenches” – I think. I really can’t remember what I said the title would be. I do know what I’m going to talk about.

The Crux -> Spatial is Special mostly.

I’ve said it and probably everyone else has said it -> Spatial isn’t Special.  A few years ago it was the fun thing to scream. To a degree it’s true. 99% of my clients are running FOSS4G software these days. In some cases it was an economic move and in some it was a “We need a map” and they dove into PostGIS and QGIS. They didn’t spend a ton of time talking to sales and going to training.  Then they call me. Why call me? Well it’s reported I know what I’m doing in this area – ALTHOUGH – I think this year has shown me I have a lot more to learn.

Three things really stuck out the first part of this year:

  • My skills as a “GIS Person” are relevant in one regard and irrelevant in another. Yes Spatial isn’t special as anyone can build a web app to run a buffer and return data. Until the data comes back wrong – then I’m special again.
  • No one is updating their software like they need to. It doesn’t take a rocket scientist to update their software BUT…..
  • I like data exploration and data science – BUT – building data from scratch isn’t irrelevant.

Part 1 – Skills.

Spatial is special and your data are important.

I’m dealing with three groups of people right now. I’m not even going to mention what software they are using. No – I will. ArcGIS, PostGIS, and QGIS with the data residing in Spatial Tables, File Based Geodatabases, and Shapefiles. Two are Utilities and one is a conservation group.

In every case: no metadata, no data lineage, and no thought to data completeness. I went on a rant a few months ago that Python wasn’t GIS – guess what – I’ll probably be scripting some of the data fixes….Assuming I don’t pull it all into PostGIS to fix it and then shove it all back into whatever data format people are using. In all of that I’m supposedly mentoring a couple of people and I refuse to talk about desktop software – I keep hammering on “get the attribution/data lineage right” then we will do analysis.

Another client went on a data collection binge. They bought a lot of hardware and are looking at collecting high precision data points in Fulcrum. Which led to a discussion on accuracy of phones and software and things in general with Bryan at Fulcrum. I’ve collected data with my phone. I’ve never much thought about the accuracy as I’ve always worried about the data I’m collecting. I always fall back to “well probably 40 feet (or 10ish meters)” and went back to my angry rambling about not owning an Ianything.

Anyway – I get the whole “spatial isn’t special” phrase – it’s not special because he software is everywhere these days. It’s not special because GIS is slowly fading into a tool for your chosen profession. Except here I am – my profession is geospatial and sometimes you need the grouchy guy to come in and scream at people and fix things that get broken and offer advice. Datum huh? Tables who? Projection says what?

Which brings me to my next rant – update your software. It’s not rocket science but……….

Nothing is Bullet Proof.

So way back in the day when I actually went to church, about the only thing that ever stuck was “You aren’t promised tomorrow”. I always like that one idea that it might all just blow up today and that’s it. Lends a bit of excitement to it all.

I get up one morning after a several meetings with a client and shuffled over to the email and pop it open and get the “We need to talk about the Google Maps Engine Option….” email.  My clients are wonderful and different. Some days I do get a bit troubled when you read about (insert giant company here) and they’ve just landed the (insert number with multiple 0s after it). I want the small clients though – I enjoy those because we can do amazing things with tight budgets most of the time. Of course it would help if they were flush with liquid cash.

The worst job I think I ever did was for a client that had made a lot of phone calls and talked to some salesperson and bought about 40k of software. I got a call to “make some analysis” and it was pretty bad because they really had bought about 37k worth of the wrong software. I made it work. Once they had signed off on it I ran. That’s been close to 4 years ago and it still bugs me. The software they have bought if they are even still using it has flipped multiple times – I’m not even sure if it’s still offered as a desktop product.

The last client I picked up is starting at 0. I made the decision to go with QGIS for starters. Once we figure out what they want/need then we start looking at other options or we keep running with QGIS. The one thing we had sorta decided was this was probably going to roll into Google Maps Engine. Luckily we’ve both been a bit slower than normal.

I get up one morning and Google Maps Engine is no more. It’s Dead.  Luckily one of the person’s helping on this job is AppGeo. We talked. There are options for moving forward. There are a ton of options.

There are two that stick out:

I went to go meet with another prospective client and the last words out of our collective mouths were “I’m not worried about Google going anywhere”. They won’t. Their products might though. Google Wave. Google Buzz. I’m slightly concerned about Google Plus.  Google Maps Engine has another year of life support. I think since my career in this field has started I’ve learned AML, Avenue, some VB, ArcPy, Python, and last night was Python for QGIS/Plugins, and I’m just now dipping my myself into Spatial SQL with PostGIS up to my eyeballs.

It’s the problem with the giant companies. You aren’t a priority. Your money is though. You might get billed for a lot of software for someone making a sales quota. The software might just die off one day..or Change. It’s a lot of the heartburn I have on ArcGISOnline. I’ve hardly ever offered that as an option because you get too invested too quickly. Maybe one day I will – who knows. I hate to toss someone off to that and then there’s suddenly a change….and I’m left going “Well – you know – that’s just (fill in the blank) software company”.

I’d rather the process be bullet proof. Build a process so flexible off standards that it can’t die off easily. With my forestry client I’ve developed a process that’s fairly bullet proof. I need to explain it better and write directions but we’ve not had any problems over 2014 with data if we stick to the process. They are happy. I’m happy. Their clients are happy. I get paid.

So what am I doing? Looking at both options. Right now I’m leaning to CartoDB because it’s simple and powerful and I think it fits with the requirements of the client. I can explain it. ArcGISOnline is an option – but I got a bit turned off by the sudden Google/ESRI Love fest. No matter what – it’s the clients decision at the end of the day. I’m just providing a path and options for them to get what they want and not promising 40k worth of software in the interim.

Nothing is Bullet Proof….or promised. Poof. Dead. This software is no more.