Forestry

Repeatable Fields in Fulcrum

January 8, 2016 by rjhale

I hardly ever talk about fulcrum except on twitter. I’ve had a license for a while and I use it somewhat randomly depending on the client. This last year I’ve been ‘perfecting’ (for lack of a better term) my toolset as a consultant. Which for anyone that reads this blog or stumbles upon it I spend a ton of time on QGIS and other things. This year you’ll see more stuff on Fulcrum and PostGIS and a few other assorted tools. Even ArcGIS.

I started using Fulcrum a few years ago. There was a job in the USVI where it became crucial because it ran on everything and it was simple enough to deploy a field crew with an hours worth of training. At that point I started growing disenchanted with high end GPS equipment and liked the idea of data collection on my phone. Accuracy? There are ways to satisfy that itch if you love your big yellow GPS unit.

Last night started me thinking on how I’m going to move the forestry guys to tablet based data collection. I’m not a forester. At best I’m a hack geologist that found more love with maps than with rocks. This spring I had a chance to go out and work with them on a job. I learned two things – A. They love pencil and paper (no batteries) and B. I’m horribly out of shape.

The short story on their timber inventories has been:

Get a property owner with timber to inspect
Get me to generate a ‘grid’ that they load into GPS units
They walk and inventory the timber at those points with pen and paper
Data is entered manually and a report is generated

It makes me tired. Sorta. I’m a fan of what works but there needs to be an easier way.

So my plan is to move this into Fulcrum. The one problem I’ve not sat down to look at until this week was repeatable fields. There is excellent documentation on the fulcrum site and I sat down and started reading and watching videos. Once I get this nailed down I’ll look at tossing the app up for other people to use. Of course I’m assuming their workflow isn’t so different no one can use it (again – I’M NOT A FORESTER)

Right now I know they need three things in the report: the GPS Identification, Tree type, Tree Diameter. There is more than that – but for now it will be a start to prove the concept. If I prove the concept maybe we move into digital data collection finally.

It’s easy to build apps with fulcrum – it’s all Drag and Drop. It does take a little bit of planning though

For a start I’m adding three fields: name, type, and size. The name will be the ID of the plot point (which I generate). Type and Size will be repeated for each point. There might be 1 tree. There might be 15 trees. Because this field will be repeated…..I’m going to put them into the “repeatable” widget/field. In this case if you look at the image above I’ve called the repeatable widget ‘Tree Data’. You drag and drop a field under the repeatable section. Once it is there – you can keep using it over and over to fill out your data.

Because the GPS points are generated in QGIS they don’t need to be collected – but they do need to be displayed in the application. They are adding more information to data that already exists (think of doing a fire hydrant survey where you’ve already surveyed fire hydrant locations but you need to know color or fittings). So I upload the gps data into fulcrum:

The saving grace in the US Virgin Islands was Fulcrum would run on anything. When I sync this to my phone my brand new app looks like:

So I’ve generated all these in QGIS. Now if you touch one of the points you get a name (or say you walk to a point):

If I touch it again I can edit the attribute data of the point. The point has a name – it just needs a type and a size entered the repeatable field (called Tree Data).

When I tap Tree Data (the Repeatable field) you see your two values to be entered:

I just start adding data. Once I’m done with this type of tree and size of tree – I click the check mark. I want to add another type and size – click the plus button and keep adding.

I’m collecting multiple attributes for one data point. Well – if you dump this out as a shapefile you have 1 extra DBF that can be joined to the point. For me dumping this data into spatialite is preferred – and I get everything in one nice neat package. Of course you can do the same with several of the exports – one being ESRI’s File based Geodatabase.

In QGIS I now have points and a table and I can join them to the point or look at the data…or consolidate it with my other spatialite database.

I’m off to a good start. The next visit to the Forestry Client and I’m taking a tablet and showing them the joy of skipping paper. Hopefully after that they become excited enough to finish out the application. You can do this with more than trees: Fire Hydrants, Building surveys, etc.

Improvements – Once the first demo is done I’m switching to classification fields since they know what trees they are going to run into in the area. I also want to constrain other fields to keep them from having trees with a 50000 inch diameter. It’s a work in progress though – finally. I actually added a status field (which si viewable in some screenshots so they know what has been done and what hasn’t.

Migrating a project from ArcGIS to QGIS: Georeferenced PDFs

October 13, 2015 by rjhale

I hate to even say “GeoPDF”. It’s not a bad thing by any stretch. Much like saying “Mr Sid” I probably do it with a contorted facial expression. At some point I’m going to talk about some of the stuff Terrago has been doing with GeoPDF’s and Geopackage. Hopefully.

If you’ve kept up I’ve been slowly moving this project from ArcGIS to QGIS and it’s almost done. There are still a few processes I’m trying to work out with the client but it’s nothing more than a lack of understanding on my part. There are two hurdles left on my end though: Geodatabases and GeoPDFs.

I developed (for me) a fairly complicated map document that has been exported out of QGIS and delivered to the client’s client. The client’s client ended up being (and I had no clue) field personnel that didn’t want QGIS/ArcGIS/Complicated. So as part of the delivery I needed to provide GeoPDF’s. I haven’t been. Well – I remembered and here we go.

If you’re working in ArcGIS you can export a PDF with “georeference information” attached. It also has layer information so you can click and do things with the PDF through acrobat reader and extract information. ArcGIS has had that ability for quite some time. There really isn’t an open source alternative to this functionality. You’re just stuck if you’re looking for a button click solution.

After doing some digging here is what I came up with. You can make a Georeferenced PDF out of QGIS. It’s a four step process.

In your QGIS Composer document You need to make one change on the composer tab:

2. Then you want to export your document as a tif file. Yes you are making a georeferenced tif. I know.

Now from command line you get run two commands.

The first command makes your tif with a tfw file a geotiff: gdal_edit.py -a_srs EPSG:2274 output.tif
The next and final command gives you a georeferenced pdf: gdal_translate -of PDF output.tif output.pdf

What if you have a pile of them to do? Learn some scripting. I created a bash script (have I mentioned I’m doing this all on linux) and exported everything to a pdf.

Pros:

I’m still doing all of this for free in easily accessible software
There are a lot of options for making Geospatial PDF’s from GDAL -> http://www.gdal.org/frmt_pdf.html

Cons:

You have to drift back to command line and I’m debating how to explain this to submit a enhancement to the QGIS group. It may be already.
It’s not a true “GeoPDF”. I consider this a georeferenced PDF. We have none of the enhancements from Terrago
File size is a bit large and I really need to play with resolution and a few other things. Right now I’m in a 8mb range for an output.
There are a lot lot of options for making Geospatial PDF’s from GDAL -> http://www.gdal.org/frmt_pdf.html

BUT – it’s works with their software and the client is happy for now. So I’m left with one last hurdle: geodatabases. I know what to do there – I just need to sit down for a bit and do it.

QGIS Conference – May 18-22nd 2015 – University of Copenhagen

March 18, 2015 by rjhale

Why am I posting this? ‘Cause it’s cool.

Website: https://qgis2015.wordpress.com/

The development and use of GIS in recent years has exploded – it is no longer an ‘experts only’ pursuit. This makes the application much more exciting and challenging. Multiple users – multiple applications – more challenges. The advent of highly functional, free and open GIS applications such as QGIS has further broadened the audience of those who can make use of GIS in their daily activities.

This QGIS conference aims to bring together developers, users and educators worldwide and foster closer interactions in order to work for ideas for improving the software and its use. It is in forums like this where ideas can be exchanged and developed.

For an application such as QGIS to realise its full potential, it is important that it is developed in collaboration between users and creators of the software. Usually QGIS developers and users meet at the mailing lists, stack exchange and other virtual meeting places.

The QGIS community hold meetings twice a year: last year meetings were held in Vienna, Austria and Essen, Germany. Up until now these meetings have been developer community focussed. This first meeting to be held in 2015 is at University of Copenhagen – Forestry College in Nødebo – Denmark and for the first time ever it will be combined with a user conference!

Just to get this out of the way – I won’t be there. Copenhagen is a bit out of budget currently – but the things you would see.

One nice caveat on the announcement:

The QGIS Project welcomes and encourages participation by everyone.

No matter how you identify yourself or how others perceive you: we welcome you. We welcome contributions from everyone as long as they interact constructively with our community.

While much of the work for our project is technical in nature, we value and encourage contributions from those with expertise in other areas, and welcome them into our community.

Moving a Project from ArcGIS to QGIS: Atlas Part 1

December 21, 2014 by rjhale

It’s been about a year since this project “appeared” and it was first done in ArcGIS. It wasn’t much fun (not because of ArcGIS). The part that made it “not fun” was the “Oh by the ways”. NRGS would deliver a map and get “Oh by the way can you add the acres for each tree type per property boundary”. OK fixed “Hey – Oh By the way we need only one property per map”. OK. “Ohhhh by the way can you make that yellow a purple”. &^%$*#

So anyway, the best thing in the world is ESRI’s data driven pages. You can set up one map and there’s enough flexibility to churn out a pile of PDF maps. In QGIS you have Atlas – and that’s almost a perfect drop in replacement.

I decided to do a test run on the map. I probably should have done that before spending a ton of time on the template. Luckily this time I know what needs to be done. These things need to be put on a map and they are attribute driven data:

Tract Name
Tract Number
Property Name
Acres of the Tract
Summary of Acres Per Tract

I made a decision to use the Tract layer (technically a view – but that is coming up later). I set up that layer as an Atlas.

The test run was a bit easier than I thought. The tract Name, Tract Number, Property Name, and Acres are all contained in one of the layers. With Atlas (and Data Drive Pages) you can insert expressions. The expressions are fields from your data. See Screenshot:

So when you preview this in Atlas (excuse the selection box):

With a few expressions I can start filling out parts of my composition……and I hit a problem. Hence the test run.

For the sake of simplicity I need to only show one property per map. If you were to view the data you would find that some property is adjacent to other property and there’s not a clean way in either landscape or portrait or with scale to make a property appear by itself on a map.
I need to autogenerate acres for each property. There are ways to do this in QGIS…but I don’t want to hand modify each map. There are ways to generate this data in Postgis/Postgres:

So I think we’re halfway there. I’ve got a plan to fix the two problems. There will be some scripts written. Some database “things” done. Stay tuned.

Anouncing an Open Forestry Template

July 27, 2014 by rjhale

My longest running client has been a forestry company. Now – if you were to hand me an aerial photograph and I had it in stereo and go “Trees – start” – I would at least get the boundaries of the different types outlined. If you stuck me on the ground I essentially go “That’s pine and that’s not”.

The forestry client has been an interesting project. As of late (over the last month) I’ve been slowly migrating them into postgis and qgis. They have arcview and we’re going to use that for map production (unless I keep getting better with cartography in QGIS and then that is questionable). Moving to postgis has made the data a bit more cryptic to them but infinitely easier to manage for me. So much so that this mornings call of “how many acres are currently on tract 31” and one small script later I had a frequency table describing types of trees and acres (which you can’t do at the arcview level of licensing (and I know it’s not called arcview)).

This database has grown from 2 shapefiles to over 22 shapefiles before the merge into postgis. Like I said it’s a bit more cryptic but easier to manage and as I keep changing it and getting it “stable” and “documented” life will be better. The whole idea of this database has evolved also out of a couple of conversations with other forestry minded individuals:

I’ve been using QGIS because it’s free – just not supported.
I’m still using arcview 3.x because I know how to use it.
I’m not buying any more software to manage my GIS (in reference to needed arceditor)

So I’ve decided to do something weird and hopefully wonderful. If you have a commercial GIS setup – you can download a pre-built database for your industry. It’s a pretty great way to do things actually. What if you don’t have that? You start piece mealing things together that makes sense for you.

I don’t have access to industry leaders or at least the people that show up at a conference. I do have a git hub account, an idea, and you. Lets build an Open Forestry Template (pre-built database). A common operating base for the rest of us. Now I’m going to take a narrow approach at first and work my way up. First thing that will be supported will be postgres/postgis/qgis. The next thing will be qgis/spatialite. Next will be whatever I decide to do for ESRI’s Desktop product. Since this is free and open I can’t devote a millions hours – but I think this can work. If some of you want to help – help. Want to help fund development? I’m not opposed to that. Want me to help your forestry company out and implement it? Yippee. Know what you are doing and don’t need help? Download away.

This actually blends pretty well with the Spatial Connect. Process over Software. This will be hopefully the making of a very good supported database for forestry people to use regardless of software.

So right now I’m recording notes in markdown files Here: https://github.com/rjhale1971/openforestrytemplate , I have thoughts for assets (which would be gates, bridges, and culverts) to stands to property boundaries. It’s not a lot – but it will start filling out as the weeks progress.

Once I feel like I have enough info I’ll start making databases and setting things up for download. DON’T WORRY – Once this gets to a stable version I’ll make it easier for everyone to get to – Github can be too cryptic for us all to use and deal with – some of you guys are foresters and not full time GIS people. I completely understand. If you want to email me your ideas go for it. If you want to leave a comment on the blog with something that needs to go in feel free.