North River Geographic Systems Inc

Spatial Problem Solving

You are here: Home / Blog

Blog

Southeast FOSS4G meeting

· · Leave a Comment

Emails are going out to people and the gears are slowly starting to turn.

Last year several of use got together and held a FOSS4G “mini event” in Knoxville TN. It turned out pretty well with about 50 people coming from as far as 5 hours away to join in the fun. I think we had about 10 or 11 presentations – too many to be honest but I didn’t want to turn anyone down. We ran the event for free and if I remember (I’m not looking up the costs) the entire event ran something like 600 dollars. We had 4 sponsors (including myself in that one) and overall it rocked.

Two things:

  • I’ve had a large number of people ask “Are you doing this again?” and I had a blast doing it. Sooooooo – the answer has turned into “YES”. Granted I don’t run meetings like this for a living but everyone enjoyed themselves and the true spirit of Free and Open Source software for GIS was there. Yes it’s happening.
  • The big question is where and when. I’ve had offers of moving this to Chattanooga. I’ve had offers of Knoxville again. I’ve had other offers outside of my normal area. So I’ve got to decide and go “IT’S HERE” and we crank this up again. FOSS4GNA got in the way for spring so we’re gonna have to push this off a bit.
  • OK this is a third thing. I will probably charge just a wee bit for this one. Why? So we had something like 50 register. 5 dropped. 5 walked up out of nowhere. I bought food for 65 people so some left with an extra sandwich shoved in their pocket. If I run this for $10 bucks a piece people are likely to show up and it gives me a bit more money to play with to do things like deal with parking or snacks or fly someone in for the event. Yeah I said it. Fly someone in.
  • Fourth thing – workshops? I’ve debated some educational opportunities for this event – except that drags it into multi-iday. I’m OK with multi-day if it remains fun.

That was four things – you might wonder how I write blog posts and the answer is pretty much unedited most of the time.

So anyway – I have no clue what to call this gathering. Don’t really want to give it a name. Are un-conferences a thing still? Where conferences? Anyway – more news will be coming and if you want to help track me down and we’ll make this a thing for 2018.

 

 

QGIS is headed to TNGIC April 24th 2018

· · Leave a Comment

Something I meant to have mentioned and have continually forgotten to mention – the Intro to QGIS 3.x 4 hour class is headed to the TNGIC Conference during April 24-26, 2018.

I’m shocked that this conference branched out enough to do the training. Not terribly shocked on the other hand as I made them a really good deal on cost just to be “rebellious”.

The four hour class has most of the exercises removed. There isn’t enough time to do everything like editing and the final chapter on databases. So we will cover in 49 slides (8 hour class is somewhere around 100 slides):

  1. Introduction
  2. Interface
  3. Data and Symbology
  4. Geoprocessing and Plugins
  5. Layout Composer

Anyway – if you’re in the area sign up. I have no clue what they are charging but I’ll even make the offer of “Sneak in if you can”. I don’t care – the more the merrier.

 

Prayers and Hay

· ·

I know – it’s been a while. I’ve been busy.

I think this first quarter of small business has been the best I’ve had in quite a while. I could point to a lot of things on why I think this has happened but I often find it’s just guessing. When you’re a one man show running off services and support there’s nothing tangible like product sales or earning reports that make sense. Some months you’re busier than others. Some months you aren’t. You form business partnerships and sometimes those fade. Sometimes they get better.

It’s a dance on what you can/can’t do and learning what jobs you enjoy and what jobs you don’t. One thing I have noticed is QGIS and “being different” has taken me more and more into the backwaters of what I consider my service area. When a small municipality/company is faced with the juggernaut of changing software and ever climbing support agreements the whole idea of different sells itself. I have entered several “one stop light towns” to discuss open source software. It’s towns the big guys can’t touch and won’t. I’m OK with this. Sometimes good things happen and sometimes they don’t.

Consulting is sometimes more stressful than it needs to be. I’ve been known for getting agitated so I do my best to not let things bother me these days. Church is also stressful for me. When you grow up in the south you get dragged to church more often than not. I spent my fair amount of time in a church setting singing hymns and worrying over your position in the Universe. I tend to find a lot of humor in religion. I may poke at things like Christmas and Easter in jest – but there is a cut off line and I try to not cross it. When I was 2000 miles south working at the equator that turned into a thing you had to constantly be aware of. Sacred cows make the best hamburger – BUT – there’s a time and place for joking.

I noticed the other day  was getting prayed over a lot. I go to lunch  at a clients location and someone blesses the meal. Sometimes I get thrown in for good measure. I say afterwards “thanks I can use all the help I can get”. It’s interesting. Usually I wouldn’t think twice about it but I was at a clients location and they had easter eggs out. I grabbed one. It was an invite to the biggest church in town for Sunday morning service. Sunrise service followed with regular service and dinner. I will say fried chicken in a church setting is probably some of the best fried chicken you can get.

Not even thinking I opened the egg and there was not chocolate.

Me: Oh man.
Employee: What’s wrong?
Me: I was hoping for chocolate.
Employee: Better than chocolate it’s an invite to church – you shouldn’t miss church!
Me: <thinking Crap> Hey – I might show up. Thanks. I was just overly hungry this morning and looking for a snack.

I think in February every meal I ate with clients was prayed over. The first half of the month I didn’t notice. The last half I was wondering if the streak would continue. Which shouldn’t be a worry – but I like averages. Last week I was driving back from a client meeting and thought “You know I wouldn’t even be here except for the community effort of open source. The only reason I’m here is ripping out ArcGIS and replacing it with QGIS/PostGIS.” Business is dragging me back into a setting from which I’ve disappeared.

Not that I’m looking to dive back into an organized religious setting. For those of you outside the US – if you haven’t noticed it’s stressful times for some of us over here. Maybe not for all – but it is for me. So – I’ve decided to turn these meetings into my own “Peace Love and GIS” type of outreach. Don’t stress and don’t worry about politics. “Hey maaaaan do your thing and I’m doing mine over here. Hey lets talk about your data.” Last week I learned what “tractor and hay bale making machine worked better than another tractor hay bale making machine”.  Which had squat to do with Geo – but it was a conversation that started out of “Can you make a layout in QGIS for me?”. The client enjoyed telling me and I enjoyed hearing about it. We talked about church. We talked about sewer lines. We talked about dump trucks. We talked about his kids and vacations.

Anyway – I know this is just a rambling post BUT – it’s more stories I seem to build up from this small business gig these days. A few years ago I found myself standing at a small town corner thinking “I don’t have time for this”. These days I don’t spend enough time standing still so I’ll spend 5 extra minutes taking it all in today. Maybe 10 and I might find some random person to talk to for a bit. Maybe I’ll eat lunch there but I’m not spending all day standing around talking. I have things to do and people to worry over.

I will most likely sleep in on Sunday. For those of you not and heading out with your family – Enjoy the day!

NRGS and QGIS Certification

· ·

Certification? Yeah. Just calm down and let me explain.

I’m a fan and not a fan of certification. At some point I was a GISP. At some point I was ESRI Desktop Certified. There’s been talk of how to have QGIS Certification for a while. I’ve not been part of the conversation. The other day I found out it was a thing now. Usually I would go “Pfffbbbbt” and ignore it….BUT….here’s what hooked me.

In short (read up here)  – all the talk centered around QGIS certification boiled down to two camps: You had your GISP Approach where your portfolio speaks for it’s self. You have a test (IT Certifications) that certify you on an exam. QGIS isn’t an org that can really do either – so they went with option 3 – the community decides who can and can’t be certified. I submitted my class for review and it passed.

So what does that mean? I have an 8 hour class that you take. At the end I submit your name and 20 Euros back to QGIS and you get an ID that can be checked to see if you took the class. Will you win jobs and make your fortune based off this certification? No. You’re supporting the community by taking it though. You can still buy a book and read all your want and do awesome things with QGIS. You can download it for your platform of choice and use it with no certification. It’s made me think about giving the students a test at the end of the class.  Haven’t decided on that one. So we’ll see what happens. I want to “do this right” in my head for the community. I did decide that the classes I teach at Conferences that are compressed down to 4 hours aren’t going to qualify. You’ll have to sit with me with 8 hours.

To be mentioned in this group –  that’s pretty cool at least for me.

This gives me a way to donate back and not feel like I’m just leeching off the good work of other people. I still have other plans for financial support – but I’m getting to that one.

We will see what happens. I’ve got to start scheduling some classes. Want a class? Looking at breaking out of your commercial loop of software changes and portals and what not? Give QGIS a try.

 

 

I didn’t know it was called Refactor Fields in QGIS

· ·

The title may be the worst part of this post. Hopefully. Anyway – here’s what happened. I will explain it in all of it’s goofy detail with some hope that by the end you’ll do the same thing I did.

I’ve a client that is currently  making me push what I can do with QGIS/PostGIS. We hit a technical bump with data collection and to fix it they walked in with a new app they had purchased from some online app store that records data from their super accurate GPS Unit. The manufacturer of the GPS unit had one – but they found this scratched the itch more than the one they could have used….and the one they were using. My eye didn’t twitch much. I just cautioned against randomly doing things with your setup. I mean what if it makes a file you can’t import into your database. What if………

They have in postgresql a table that was more or less this: column1, column2, column3 . This app could create several data formats that ultimately me with: Foo1, column2, Bar3. You can’t just “copy and paste” the data in because the columns need to match. So I need to force it to match. No matter what trick I tried it wasn’t easy BUT some of them did work.

An Example of my thought process.  My python skills aren’t great – BUT – using the psycopg2 library, some ogr, and “the google” I was able to connect and change my data and push into Postgresql. More or less you parse out the columns and match them to the PostgreSQL table:

import psycopg2
import osgeo.ogr 
#import shapely

shapefile = osgeo.ogr.Open("/home/rjhale/gis/shapefiles/monument.shp")
layer = shapefile.GetLayer(0)

conn = psycopg2.connect("dbname='database' user='me' \ 
       host='localhost' password='itsasecret'") 
cur = conn.cursor()

for i in range(layer.GetFeatureCount()):
      feature = layer.GetFeature(i)
      descriptio = feature.GetField("descriptio")
      pos_x_sf = feature.GetField("Pos_x_sf")
      pos_y_sf = feature.GetField("Pos_y_sf")
      elev_z_sf = feature.GetField("Elev_z_sf")
      photos = feature.GetField("photos")
      local_date = feature.GetField("local_date")
      pdop = feature.GetField("pdop") 
      #Get feature geometry
      geom = feature.GetGeometryRef()
      #Convert geometry to WKT format
      wkt = geom.ExportToWkt()
      #Insert data into database, converting WKT geometry to PostGIS geometry
      cur.execute("INSERT INTO test.monument(descriptio, \ 
                   pos_x_sf, pos_y_sf, elev_z_sf, \
                   photos, local_date, pdop, geom) \
                   VALUES ('{}', '{}', '{}', '{}', '{}', '{}', '{}', \ 
                   ST_GeomfromText('{}',6447))"\
                  .format(descriptio, pos_x_sf, pos_y_sf, elev_z_sf, photos, \
                  local_date, pdop, wkt))

 

Holy crap. It worked. I really need to up my python game BUT – no one wanted to run something from cmd line. MAKE A PLUGIN! Which I considered. In my digging I found the refactor fields tool  in QGIS. It’s very similar to loading data into a database with ArcGIS for those of you stuck on that side of the fence.

Refactor Fields

So how does it work? It’s easy. Load your data that doesn’t quite match anything as the input layer. Load the table you want it to match in the “load fields from layer” part at the bottom. Click Load Fields.

loaded refactor fields

So what just happened? Refactor fields took my shapefile and matched it to the columns in PostgreSQL. If it can’t “guess” you have the option to make it match by typing in the fields. It generates a new layer that I can “copy and paste” into the database or import into the database. For some of the data I’m having to clean out the fields – so things that don’t match in the shapfile I’m removing from the new layer using this processing tool. Which – maybe I need to add those fields to PostgreSQL. I really want this table stabilized before we start. Although I’ve not tried it I’m guessing this works with any layer you want – I’m just using it against postgresql. I also will say this probably won’t match projections – so get your projection worries figured out before hand – this just handles the attributes.

Right now I’m  looking at a model so they can ultimately just “ClICK SOMETHING”. We all like menus. Mostly.

Anyway – load your data. QGIS has many tricks to make your life easy and I had no clue. I did learn something on the Python side of life though and for that I’m happy even though it turned into a dead-end.

 

 

 

Contact