North River Geographic Systems Inc

Geospatial Problem Solving

Open Source GIS

QGIS 3.x: Setting up a Project

I’ve been digging through the QGIS class this week as I wrestle with a way to move the class online. While doing that I started filing in the gaps of things that I wanted to add but didn’t because of time. Well – with the class moving online I can add more things.

Usually what we do when setting up a QGIS project is:

  • Open QGIS
  • Toss a bunch of data in it
  • Save it somewhere
  • Come back to it later

So for those of you just starting out or maybe for those of you who have been working with this for a while – a helpful hint.

Open QGIS. Slow down and go to  Project -> Properties -> General. We are only going to look at 3 things in this – so don’t worry. This won’t be anything complicated.

You should see this:

qgis project properties

We’re really going to worry about two things:

QGIS project properties

When setting up a new project I always do one thing to kick off the event. I build a directory structure to start storing scripts and maps and anything of use.  I set the Project Home (above) to be that directory. I also set a working project title like “My Map” or something.

qgis working project

I also go ahead and set the Measurements. I know this project will be in a local planimetric projection so I’ll set that to None/Planimetric and since my world revolves around Feet/Square feet I’ll go ahead and set that up.

The next thing I do is set up the Coordinate system – so click on the CRS Tab on the left and set it to whatever you need it to be – in my case EPSG:2274

project properties CRS

I click Apply or OK and the last thing I do is Save my Project (back to the Save Project portion under the Project menu of QGIS) . That last save leaves you with a complete Project Properties.

Completed project properties

So who cares right?

Well if you happen to use the browser panel you’ll notice something has changed:

project home in browser

 

Project Home now appears  – which means I have quick access to everything I need from shapefiles to geopackages to imagery to whatever. Right in the browser which really gives you a ‘Workspace” feel for your project.

How long does it take to set up? Not that long but the benefits of getting that one small part done outweigh the time you’ll spend setting it up. I’ve been bitten many times by having the wrong units set up or something else that could have been taken care of by spending 1 minute setting this up.

Mobile Data Collection: Input

If you flash back to 2008 you would have seen me sitting on my couch unwrapping a Trimble SB (?) running Arcpad. At the time I thought “collecting data” would be a beneficial add-on to me being a consultant. Spoiler alert – It wasn’t.

If you flash back to 2013 I was sitting in the Caribbean needed to collect data and I was holding a new giant yellow GPS unit that wouldn’t quit rebooting when it got hot. Spoiler Alert – It’s always hot (generally) in the Caribbean. What I did find is Fulcrum – and that allowed me and about 5 or so folk to disperse and work all over the place and collect data – WITH OUR PHONE. Which for 2013 was pretty awesome for me.Since then I’ve always been interested in how people are collecting data. I will make fun of you for doing it “Pencil and Paper” but I will admit to having a field journal and pencils in my laptop bag just in case.

A client asked a few weeks back “We need to take a copy of the data into the field” and that trip into the field was going to be with no to limited connectivity, no data collection, but they needed to know where they were. They are a IOS/Mac heavy group. Print a map seemed a bit harsh and working through some other hoops seemed unnecessary.

So my suggestion was Input. So what is an Input? Lutra Consulting developed a mobile app built off the QGIS Source Code. It runs on Android and IOS. There are tutorials out there on using Input. I can’t do better than the ones that are out so I won’t even try. I can throw in some screen shots and just general discussion on why Input is pretty cool from an avid QGIS user. When you get it installed on your mobile device of choice you get this:

input start screen

Simple. Home being the map screen, my projects being things you can work on, shared being projects that are shared with you from others, and finally explore just in case you can’t find your project.

So we will tackle this one of two ways. One way with Mergin and one without. What’s Mergin? Mergin is the commercial end of Input. Commercial? What? Yes – there is a commercial end of Input.

1. Install Mergin and Get an account. Don’t worry – it’s free for small projects. Download Mergin from the Plugin Manager in QGIS. Mergin will appear in your browser panel.

2. Build a project in QGIS. Import your data into Geopackage. Build the project the way you’d want it to be displayed on your phone. When I say “build your project” you’re going to build widgets for data input, set up labels, and set up your colors. I prefer boring as seen by this screenshot.

QGIS Project Screen

3. Import your project into Mergin (You’ll have to connect the plugin to your Mergin Account).

Give it a few minutes and you can now download your project into Input and start working. Edit your points, lines, and polygons….and attributes….

Input Project

 

4. Sync your project back to Mergin. Download it back to your computer.

One word of caution :

  1. You’re going to build a project .
  2. You’re going to COPY it to the server using Mergin.
  3. Move it to your mobile device.
  4. Edit it locally and sync it back to the server
  5. Eventually you’re going to download it back to your compute using Mergin
  6. So at the end you’ll have one old project and one new one sitting on your machine. Just pay attention. I did get confused at first but I’m slow.

You don’t want to use Mergin?

  1. Then copy your data to your mobile device.
  2. Create a directory under the projects folder under Input and move your data there.
  3. Start working. At some point you’ll have to manually move things back to your laptop.

….and there you go.

So where does that leave me with all of my current love of data and databases? So as an experiment I connected to a test server and edited data against PostGIS (live). I wouldn’t suggest that for day to day because “you always need a network connection of some sort”. So I’d always build a local project to share to your mobile device.

I need more than “me” collecting data – then read up. Several people can edit and Lutra has used Geodiff to handle multiple users. All your edits can go back into the same geopackage that you download back to your Workstation.

So should you pay for Mergin? If I stuff a street centerline file, building footprints, streams, and a few other things for a medium sized city I’m getting close to 100mb of disk space. So I’d say pay. If you just can’t pay – then keep it under 100Mb or manually move your data over.

Overall I like it.  For years I’ve said “QGIS on Mobile” and now we have Qfield and Input.

 

 

 

 

 

 

 

 

 

The First Day of XYMAS…..The Command Line

I used to go to a lot of schools to talk about maps – be it High School or College several times a year. Some have learned “Hey we don’t want Randy to talk to the impressionable kids”. I still do it if requested but I don’t make it well known that I will just show up and talk about maps and data. This year for GIS Day I went to the University of Tennessee at Knoxville to hassle the kids about all things open source. There were a large percentage that went “I’m learning python so I can do GIS – do you know python?” and I would go “I know some python but you don’t have to know python to do this”. Which started me thinking about young Randy back 27 <gasp> years ago and what skills I had learned then (1992) that were still relevant now.

When I started my geo career we were running Unix workstations. I had a system administrator that worked with my department that started teaching me small tricks to make my life (and his life) a little easier. It all started out with a terminal. From there I started learning how to run commands or build small scripts to rename files and move data from point A to point B on the network. Eventually the Unix machines gave way to Windows Machines and the problems remained: small repeatable problems I can fix from a command line. Today, the Windows machines have given way to MAC/Linux machines which is even better (but Windows 10 isn’t all that bad (yeah I said it)).

What does this have to do with GIS? Not much. What does this have to do with a life long skill set? Everything.  Geo software, operating systems, computing power have changed immensely but the small things I learned on the command line grew a better understanding of programming, networking, and “system administration” in general.

I need to move some files on my machine – the files are spread over multiple directories. From my linux box I do: find . -type f -print0 | xargs -0 mv -t /home/rjhale/biketrails . I think that command moved about 30 or 40 shapefiles into one common area for me to eventually merge into one file.

What has been the longest running single program I’ve used over 27 years that’s still a thing? It’s VI (currently VIM). It’s all command line.

VI is a text editor. It’s simple and it’s standard with a Unix operating system and was released in 1978. VIM is a clone of VI and was released in 1991. It’s simple and powerful and you don’t have to leave the keyboard to use it. There is a version for every operating system so you aren’t left out. What do I use it for?

  • Make notes while working on a project.
  • Write scripts.
  • Automate changes to text files.

I’ve been using that since 1992.

Like I mentioned above – the scripting led to an exploration of networking. I’m not a sysadmin by any stretch but if something goes wrong on a server I can at least craft a half competent explanation for someone who knows what their doing as a start. The simpler the better as in “I can’t ping this box” OR “Can you give me access to the network through <fill in the blank>”.

So where does that leave us for the firs it day of XYMas? Learn some command line tricks and tools. It doesn’t have to be something complicated and it doesn’t have to be something grand. It has served me well over 27 years. Think Small and Simple. Do something fun for your first outing.

Take a look at some tutorials out in the internet (as always these may or may not be active at some point in the future)

 

FOSS4G-NA 2019

FOSS4G-NA popped up on my radar this week.

So from just the website and a few twitter noise makers it looks like It’s headed to Mission Beach California (San Diego) April 16-18 2019.  I’ve made the last 3 – so It looks like I’m going to do my best to make this one.  Road Trip? I encourage you to attend and support the FOSS4G North America Community.

20 minutes at the East TN TNGIC Meeting

I went through a year or so of not talking at conferences. For local ones I’d show up and do something. I taught a workshop at FOSS4GNA 2018. I did a workshop at FOSS4G 2017 in Boston (and it sorta sucked).  I need to get out of that mode and talk more – but I’m happier being quiet these days and working with clients.

Anyway – at FOSS4GNA 2018 I discovered something – the OSGEO Suite had disappeared from Boundless’s Website and was replaced with a github repository. It wasn’t long after Don Meltz put up an excellent explanation of the OSGEO Suite. The OSGEO Suite being gone isn’t a problem. It does increase the amount of talking I have to do spend explaining it’s not a problem. I could even compile the suite – but my clients can’t. They are smart people – but the time to compile the boundless suite won’t be happening.

One of the things I keep running into is the idea of an Open Source Server. The OSGEO Suite was well known and I get people asking “Hey – can you set us up a geo server”. They mean the opengeo suite. I then launch into an explanation of what makes up a “geo server” and they go a bit blank and answer me “Yeah – one of those”.

So for the next bit I’m gonna be doing something like this:

I want to explain how this works to people. We’ve gotten a bit deaf by hearing “ArcServer” at every conference in TN as this one monolithic thing you have to install to make a web map…which really don’t need since AGOL appeared but…you want SDE and it’s not called that and you have two people editing but you need SQl Server but you don’t….

Anyway,  QGIS/PostgreSQL/PostGIS I’ve installed multiple times and as of late I’m experimenting with geoserver and what it can/can’t do. I have no grand delusions of creating ‘Randy’s Open Geo Spatial Super Duper Server’.  I want to get to the point where maybe mapserver is an option in this. So much to learn…so little time.

I suck at github – but I’ve been more active as I try to develop a skill set and get over the angst of pushing and pulling things. I’ve built a vagrant box to install PostGIS/PostgreSQL/Geoserver so I can have something to point at plus it gives me a starting point for an upcoming class I’m in the middle of building. The vagrant box isn’t pretty – I expect to change it over the next bit as it’s mostly pieced together from a lot of other people’s work (Coleman McCormick, Dave Smith, etc). It works though and I have everything but pgadmin 4 running in one spot. Which – I don’t think this is anywhere near “install this for a server” because this is mostly just for talking, training, and testing.

So why vagrant? Why not docker? I can gloss over vagrant faster at theses talks than I can docker.  I have 16 minutes to draw diagrams and about 4 minutes to publish one piece of data: QGIS -> PostGIS -> Geoserver -> Leaflet.

Anyway – if you’re up for the possibility of a train wreck of a presentation come by the East Tennessee Meeting in Kingsport next week as I try to explain how this works and hopefully you get excited enough to install some or all of these components and do something cool with your data.