North River Geographic Systems Inc

Geospatial Problem Solving

ESRI

Geoprocessing in QGIS: 2.14 vs 2.16

So at least once a week I’m getting a phone call or an email on QGIS. This one was pretty awesome and it pointed out I need to cover geoprocessing in Class in a little more detail.

The question was “In our transition out of ESRI products we hit a snag. We want to merge or dissolve 925,000 polygons on an attribute. QGIS Hangs. Arcmap Completes it. Why doesn’t QGIS complete the task?” They took an abbreviated version of the QGIS class at FOSS4GNA back in the spring.

selection_429

I thought about it and started running some tests. You can dissolve/merge the data two ways in QGIS and you can do it in PostGIS. ArcGIS people tend to get hung up on this as you have one way in ArcMap. So I listed out the three ways from Simplest to hardest (Finals: GRASS won for speediest answer, PostGIS won for cleanest answer, QGIS won for simplest answer).

The Question brought up a good point. When you think about “geoprocessing” if you’re an ESRI user you drop back to ArcToolbox. selection_430

If you think of Geoprocessing in QGIS you actually can get two “correct” answers. One deals with FWTools and one deals with the Toolbox. Yes – both QGIS and ESRI have Toolboxes. Before QGIS 2.14 you had FWtools in the vector menu. That software was there from the beginning or close and is derived from http://fwtools.maptools.org/ (FW is Frank Warmerdam). Sextante started making an appearance sometime in 2.0 as a plugin. Sextante combined a lot of tools into one menu system. Eventually it made it was as a part of the core release and became the Processing Toolbox. FWTools started getting out of date. Somewhere in 2.12 or earlier it was decided to replace FWTOOls with the Processing Toolbox

The fun is at the 2.14 release. It’s a Long term release. If you’re running an earlier version you will still see FWTools. As I’m sitting here looking at 2.14.6 It appears FWTools is gone. In 2.16 the vector menu at the top has regained all the functionality you remember from 2.1 or early versions of 2.14 – it just looks different. You now have unified processing tools in 2.16. No FWTools.

selection_431Back to the problem. The student from class was running 2.14.2. They still had FWTools and “googling” pointed them to the answer being in the FWTools toolset under vector. The merge didn’t work. The Dissolve in the processing toolbox finished in about 20 minutes. So my followup email in a few days will be – Did it work and do you feel better? Are you really dropping the other stuff? I’ve yet to suggest people ditch their commercial GIS system. I do have more and more people starting from scratch and going open source. It’s interesting.

The moral to the story – QGIS is evolving. If you decide “I’m going to sit at this one version for a year” you are missing out. You should at least think of upgrading like you would an ESRI Service pack. If you are on 2.14 – upgrade to the current LTR (Long Term Release) release (which is 2.14.6 as of this date of posting). If you want to live on the edge – go with 2.16 (I did but I’m weird that way).

selection_432

 

FOSS4GNA vs Southeast User Conference Battle to the Death

I always wanted to use Battle to the Death.

So last week I saw the first email I’d been expecting for a while. It went something like:

WHICH ONE??!!11!!

Selection_004 Selection_003

In case the pics don’t translate it’s FOSS4GNA vs ESRI . Aaaaaaaand you’re more than likely going to go “OMG IT DOESN’T HAVE TO BE ONE OR THE OTHER STOP CAUSING PROBLEMS”.  Well Actually it does have to be one or the other. Both are the same week (May 2-5). I know – I love it when a plan comes together. 

So here’s my pitch. I’ve been to both. Granted it’s been a while since I’ve been to an ESRI function. Why no ESRI for a while? It’s not been exceedingly relevant to my work life. I’ve been to the three FOSS4GNA events and they’ve been more relevant and more recent. So I’m going to state my case (but this is all pro foss4gna so you can stop here if you want).

Where are both: Raleigh NC for FOSS4GNA and Charlotte NC for ESRI. I know – Planning.

When: Both May. Both start on the 2nd. FOSS4GNA makes it to the 5th and ESRI stops on the 4th.

Why do both exist:

Southeast Users Group (stolen from the site): Esri Southeast User Conference (UC) is where GIS professionals from across the south gather to map better, smarter, and faster. Spend three days learning, connecting, and getting training on the geospatial solutions that move your organization forward. Discover what’s new with in-depth tech sessions on ArcGIS Pro and sharpen your skills in the Esri Hands-on Learning Lab. Plus, hear from top innovators from Esri and your community who are using GIS to solve today’s complex challenges to design a better future.

FOSS4GNA (stolen from the site): FOSS4G is all about community. Contributors, adopters, users, extenders, teachers, service providers, consumers and business and research organizations gather to share their expertise and learn from each other. FOSS4G delivers camaraderie and networking opportunities that lead to synergies in the community, as well as opportunities to give and receive help on specific technical issues or to generate business opportunities. 

So here’s my take on it. If you’re all ESRI All Day and you are never going to do anything not ESRI related. Go to Charlotte. You have to. You need to. Learn about ArcGISOnline and ArcGISPro and why you need to get certified as a ArcGIS <Something> user and why you need an online account to run your software. Disclaimer: I was one of the certified somethings.

If you aren’t All ESRI All Day or even if you are…..and you want to do something maybe a little different……Take a look at FOSS4GNA.

Here’s why…..I’ve done a lot of ESRI events. They are polished. They are nice. They are clean. Demos will work. Software will be shown off. People will ooooh and aaaaah with the things they are shown. It’s a software company selling software and services – and there’s nothing wrong with that.

FOSS4GNA has two groups involved: Location Tech and OSGEO. They have nothing to sell. What we do have though is a community of people. A group of like minded individuals….and you may go “But ESRI has that…” and I could say yes but I’m going to add it’s not like it used to be for me. I miss the community – I stopped feeling the communal aspect of ESRI a while back. Does it exist still? Probably. Disclaimer: I love everyone.

I’m on the FOSS4GNA 2016 conference committee and I’ve been reviewing papers and workshops most of yesterday. The presentations are all over the place and pretty awesome and relevant. From Cartography to Programming. From Web Mapping to On the Ground Sensors. From LIDAR to Satellites. It’s not what you are possibly thinking – it’s not a group of people who crawled out of the basement to talk about something so complicated only three people can pronounce it and one person uses it. We have Government Presentations. State and Local presenters. Workshops. Companies presenting open source and hybrid solutions. Someone’s even talking about FGDC Metadata. Talks on GIS Problems and GIS Solutions. This is a conference on Geo. One thing I’ve never done – walk away going “Oh dear deity in the sky – I’ve got to tell my clients they have to upgrade software/hardware”.  I have walked away going “I NEED TO LOOK AT THIS <FILL IN THE BLANK> THING. We talk. We have fun. We cheer the demos that work and don’t work. We learn from each other and we learn about all this Free and Open Source Software for GIS. For me it makes GIS fun again. No one is going to mention ArcGISOnline Credits – but there are going to be a lot of 3D presentations. Disclaimer – I generally make fun of 3D.

So if you’re wondering which one…do something different. Come to Raleigh. Learn about some different software and different processes…..the best thing is – Even if you are an All ESRI All Day Shop – you’re going to learn something useful in Raleigh. We will all still love you. We might even buy you a beer and or tofu and give you a high five and or help you do yoga.  Come see us. Yeah the planning sucks because you have to pick….but picking makes it more fun because you have to commit. Disclaimer – I don’t own Yoga Pants.

Track me down and I’ll even install QGIS for you and give you the 15 minute walk though if you don’t have it installed. With any luck I might crash your computer.

 

 

 

It’s not that exciting

I don’t even know how to write this up – but given the last few days it warrants some mention – although not a lot. It’s almost entirely self serving except I will stop short of any “I am so SMRT”. I’m not.

Many of you probably go “What do you do everyday?” and as I just put my cat down from an exciting back rub – I would answer “Geospatial stuff with data”. Most of my clients are desktop oriented and through the years they’ve been getting more complicated. When I started life as a business I remortgaged my house to buy ArcGIS….errr….ArcINFO. I worked. I had some spectacular successes and failures that could only be viewed with the help of popcorn and koolaid.

Over the last few years I’ve switched to almost all open source. I say that and I had to break out my increasingly outdated ArcGIS license the other day for some data issues. I’m on the ‘Commercial support for QGIS’ list – I became active with the community and pleaded my case and was placed on it. There are a few of us in North America on it. I do QGIS classes – probably my noisiest spot for the last bit – but I much prefer working. When I was a “commercial gis software” Business Partner I had three clients because of that designation. I’ve gotten two because of the commercial support for QGIS list. I should hit three this year. Maybe four.

I had a phone call from a prospective client last week that was the one I’ve been wanting. They had decided to forego the usual GIS setup and dive into QGIS and PostGIS. They had built a database and I’ve spent the last 24 hours realizing I don’t know enough on databases….again…and I know like 500% more than I knew last year. QGIS isn’t a problem – I’m learning more and hope 2016 is my year of writing a plugin. The kicker – this wasn’t a decision where they had purchased software and went “This won’t work”. This was their first choice.

It’s not that exciting – once we get the database worked out and the QGIS installs finalized there may be 20 or so people working off and on building geographic data. The description of the problem from the client was a commercial setup would be the equivalent of hiring an employee. In other words – they’d rather hire somone vs buying software.  In my world the desktop reigns supreme still and it might be dying – but very slowly. So how did the person (and I’m not mentioning clients names) find out about QGIS? – the local gov’t office was using it as opposed to buying more commercial software.

If you’re on the QGIS list and you watch there are pretty big installs occurring outside of North America with QGIS and PostGIS. Here in the US the QGIS list isn’t that noisy – Except it’s changing. Slowly. When I work for someone I’m generally offering services. I’ve only had one person demand commercial software. So I keep moving forward with my goofy open source hippie software. This install we are working out will be not massive – but of good size. After a few weeks they should start seeing a return on investment that exceeds what I’m getting paid to do. The nice part once we get over the technical hurdles I can help them answer questions and make plans with the help of this open source software. I can be a Geo Person. I can worry about Data (with a capital D) because this is going to be a stable setup.

So it can be done. With this client under my belt it can be done even easier. You can roll out a GIS setup and help your company/group out…and not lose anything in the process. I used to go into these situations going “If you feel you need to switch you can – this is all commercial software friendly”. This time I didn’t mention it. They are here to stay.

So like I said – not that exciting…..BUT – exciting. QGIS….PostGIS…..Maybe Geoserver/Mapserver? I’m sitting at the ground floor of a clients discovery of GIS through open source software.

It sort of is exciting.

And there – I went the entire blog article without saying ESRI.

Georgia URISA June Luncheon 2015

Georgia URISA Logo
Date: June 9, 2015
Time: 11:30 a.m. – 1:30 p.m.
Location: City of Johns Creek
Council Chambers (3rd Floor)
12000 Findley Rd.

Johns Creek, GA  30097JC Now – A Case Study on Building an App on the Cheap

Smartphone apps have been around for a while now, but are out of reach to most GIS professionals due to the learning curve involved in programming them.  This presentation will explain how to develop a smartphone app that incorporates open source technologies and free mapping APIs by using Johns Creek’s “JC Now” smartphone app as an example.  Attendees will learn how Johns Creek’s GIS and Communications teams designed, built and deployed a multi-award winning app in less than 3 months and with a very small budget.

JC Now is a smartphone app which delivers emergency push notifications, interactive maps, and city news to the public. By July 2014, ‘JC Now’ (v1.0) was completed and awarded 1st place in the Esri UC User Apps Fair – Mobile Apps category.  JC Now is now on v2.3, has also won an ARC award and has quickly become a key tool in communicating with the public

Nick O’Day – City of Johns Creek

Nick O’Day is the GIS Manager for the City of Johns Creek, Georgia.  Nick started his career in GIS while working for the Geospatial Technology Laboratory at the University of Georgia and has since expanded his GIS and IT expertise to include database design and implementation, geospatial analysis,  interactive web-based mapping, and smartphone application development.  He is currently enrolled in a Master’s of Computing Science Program at the Georgia Institute of Technology.  Nick’s team recently won a 1st place award at the Esri International User Conference’s User Apps Fair for the city’s first smartphone app:  JC Now.  JC Now is available for Android and iOS devices, delivers pertinent information to the public, and also enables city officials to send emergency push notifications to users.

Georgia URISA Website – Register NOW

ESRI OSGEO Lidar and things

I would go into greater detail but I’ll just let you guys read along. I think I can do this in Three links with three small summaries. Full disclosure – I don’t work with lidar data – but I work with a lot of GIS data and your data format is important. Your software is important. I’ve run into this more than once where a client gets lost through salesman double speak:

From Rapidlasso: http://rapidlasso.com/2014/11/06/keeping-esri-honest/

So Rapidlasso has a lidar format – LAS. It compresses. It’s open. Rapidlasso also publishes software that makes use of the format. Almost everyone uses it. There’s an open source library to help you along. Except ESRI forked it back in late 2013 or early 2014. When the fork (Named zLAS or Optimized LAS) happened like any good software company they started getting clients pulled into the format and moving forward. Which more or less isn’t cool. Taking an open format and closing it off isn’t a good thing. Especially when you the practitioner of GIS doesn’t take the time to get informed.

Finally from OSGEO: http://wiki.osgeo.org/wiki/LIDAR_Format_Letter

OSGEO Finally threw in and went “We’ve got a standard, quit naming your stuff like it’s approved by the working group, and keep your data open.” To me is a bit like talking to a wall but at least someone went on the record and said something. I’m also surprised at the continued silence of ASPRS in all of this. Does the letter do anything physically – well no – but it does show a group of people aren’t happy. Are the unhappy people ESRI users? More than likely not but it is a list of people with interest in the technology and the format.

ESRI responded: http://lists.osgeo.org/pipermail/discuss/2015-April/014141.html

In general – We support open standards and sorry not sorry. No mention of backing down or doing something different. Which is to be expected at this point – they need to sell software to you, the practitioner, and the only good way you do that is to control quite a bit of the market (disclaimer: I use ESRI Software) which they do.

So – what can you do? Sign the letter from OSGEO. Inform yourself. Salesman are everywhere and just be aware if you ask the advice of a salesman you’re going to get a lot of advice that may not be good for your organization. If you’re a 100% ESRI shop and you’re never gonna change – don’t worry about it. Continue. If you’re shop looking to grow and branch out – you might want to watch learn a bit more about all of this stuff.