Lidar

LIDAR and QGIS 3.17

February 18, 2021 by rjhale

I usually try to not write about the “pre” release (technically it’s master) of QGIS. Of course I went and said “HEY I want to talk about QGIS at a conference” and a large part of what I wanted to talk about was the new release. Which also included Point Clouds……and 3.18 is coming tomorrow. Which led to a frustrating demo which is why I’m writing this out.

TN has all their LIDAR released as open data and I’ve played with it in GRASS and PDAL and in general it’s a lot. It’s a lot of thinking. It’s just a lot of points and frustration. The data is all stored in zlas and it’s a pain to do anything with since I’m not working in ESRI Software. Generally I uncompress it and keep the uncompressed copy so I’m now hording 4 counties of LIDAR. I tend to stay excited about it for a day and then drift back to PostGIS and QGIS until the need to hate myself happens again.

AS of later I’ve been dealing with a side project where elevation is important. I’ve had questions and wondered if my approach on that has been right. I’ve also been running QGIS master as I’m curious on how I can leverage some of the new functionality with clients and projects.

So my first whack at points looking like this in QGIS:

Floating point cloud which is like 500ish feet off the ground. At first I thought this was a feature and then I had decided it was a bug and luckily I can shoot the developers a quick message and Nyall Popped up and said “projection problem”.

So Tn’s lidar is all hosted here. I had to go back and read the metadata. What I learned is I had a horizonal projection problem as I had assumed X and I had a vertical projection problem because I didn’t think that was an issue. In pulling the LIDAR data out of the zlas format I’m pretty sure I wrecked the projection. So in going back I needed to fix it. How do I fix it? PDAL. Going back I determined that my new LAZ file was EPSG: 6576 in the horizontal and EPSG: 6360 in the vertical. So I fixed it by this bit of trickery with PDAL:

I don’t want to get into all the technical on why this worked and how I did it – except that PDAL has some amazing command line abilities for fixing LAZ/LAS files….especially the projection.

Here is the cool thing. Upon adding the new data to QGIS I was prompted that I was missing a needed transformation. QGIS is running off Proj 7. I was prompted to click a link and fix it. I blindly clicked. I blindly imported.

It worked. LIDAR matches my DEM. Proj knows the problem and QGIS fixes it. Clickity Click. Probably doesn’t happen in 100% of the cases but it worked for me.

“I HAVE TO DO ALL THIS CLICKING OMG I HAVE LIDAR I DON’T KNOW THAT MUCH ABOUT IT” you will say. I think in most cases if you have a LAS/LAZ file you’re fine. This was a special case of me having data with a screwed up Projection/No Projection in preparation for a demo. So you don’t need to know things about PDAL and things about your LIDAR. You add it and go. It’s not a bad thing to know – but overall you should be fine.

This is all pretty amazing actually. You have this small open source desktop that can render a point cloud. I think this opens the door to analysis and other things that’s usually been outside of QGIS’s normal operations. GRASS can already work with LIDAR. Now you can look at it. Oh the things you’ll do.

The Third Day of XYMas – GRASS

December 18, 2019 by rjhale

I know – I’m running behind. So look at these 12 days as either “hints” or “tools”….or both. Being a consultant I walk into some weird situations with data and clients – so it helps to take a full complement of tools into a situation….and a good attitude but that’s on Day 12.

So what do I give you on day 3? GRASS. Yeah and it’s the running joke EVERY.SINGLE.TIME. “I tried GRASS back in College”.

If GDAL is a swiss army knife and QGIS is my hatchet – this is my sledgehammer. I may not use it as much as I need to – but when I need it it works oh so well.

GRASS has been under some form of development since 1982. It was opened up in the mid 90’s and it’s still actively developed today. Python friendly. Command line friendly. It has it all – even a friendly gui so you can push buttons.

So what do I use it for? If you’re an active QGIS user you’ll see a lot of the tools exposed in the processing toolbox. You can gleefully play along with it there if you wish. I would encourage you to look at the tutorials the community has provided. If you’re dealing with a lot of raster data it’s pretty awesome. I’ve used it for drive time analysis with vector data. My one secret weapon with vector data has been the topology tools. If I receive some really bad vector data I’ll import it into GRASS and clean it up. Yes – for those of you that miss clean and build from your workstation days you can relive some of that excitement here.

A couple of years ago I had a call from a drone company. They wanted to leverage their raster data beyond the commercial software they were using. They wanted multiple products with multiple use cases and a lot of clients all from a server out in the “cloud”. My answer? Something really complicated involving some weird process? No – the Second worst programmer in existence (me) hammered out a process leveraging GRASS and python. We were adding and subtracting elevation models after some hefty discussion using the processes found in GRASS. which to me was somewhat poetic – this, probably the oldest piece of software I use, was working with Drones, probably the most disruptive thing to hit aerial imagery collection in forever.

Right now I’ve been using it with LIDAR data. TN freed up a lot of LIDAR data (points and a surface model) for people to work with. I have been known to download the data and mosaic it with GRASS. I’ve played with point clouds in here also. One of the best thing about GRASS is it’s rock solid. I’ve not crashed it often and I really have to do something super stupid to make it go down. If I want to toss a few hundred GB of raster data at it that’s fine as it will churn through that data without blinking.

On Day 3 – Try the Geographic Resources Analysis Support System software. The Ultimate why?

Captain Kirk said so….

Class, GRASS, and LAS – LIDAR in TN Part 3

July 6, 2017 by rjhale

And now for Part 3.

So something exciting happened. If you’ve been keeping up with the TN GIS Data site you’ll notice a lot more data has been released in uncomfortably HUGE zip files. I downloaded two counties worth and have been exploring the data. In the meantime I started getting a few emails of “hey we’re getting data on a hard drive”. The cool thing with that is I received a tile from a county and it’s in LAS. If you read through some of my other rants I’ve been annoyed by the fact this is in zlas format. If you do some digging on zlas it’s more or less ESRI going “Hey we’ve made a format that only we use” and you have to unlock it to make it useful by anything else. We’ll – the counties are getting unlocked data. Victory.

BUT – there’s one more Problem. In all my examples I’ve been using GRASS. So I tried to pull the data in and nothing good happened. Actually – nothing happened. It was then I started learning about liblas and the LAS Library Versions. The short story is GRASS will read the 1.2 LAS Version. The data provided by the state is in 1.4. So you can’t automatically use GRASS to do anything with the data.

EXCEPT PDAL TO THE RESCUE! Probably the one thing I’m learning is I’m not short on tools to use with LIDAR. I’m up to: GRASS, PDAL, SAGA, and LASTOOLS. I will spare you all the information on how to get PDAL – read about it on the website. For those of you on Windows: if you have QGIS from the OSGEO Network Installer, you can download PDAL painlessly and be quite happy. As I’m running Linux on everything I grabbed a copy and compiled it and have it locally. I’m also quite happy with that.

So the one thing you need to know is this: pdal translate inputlasfile outputlasfile . Issue that command from the OSGEO window. Something wonderful happens – you get a LAS file v 1.2 you can run through GRASS.

You can do things like making a map of the intensity values:

Or a hillshade for posterity:

To get from an LAS file to anything above is about 10 minutes. Maybe 12 minutes. I’m running this on a laptop with a lot of RAM – but I could do this on a much lower end machine.

I’ve covered GRASS and LAS….Oh Class! Read your metadata. If you read Part 2 above you’ll notice there is a change in Classification with the new data:

Classification	Value
Unclassified	1
Bare Earth	2
Low Noise	6
Water	9
Ignored ground	10
Bridges	17
High Noise	18

…and there – I finished my horrific joke of LAS, GRASS, and Class. I’m not explaining it.

It’s happening. LAS files for TN are out and about. My first thought is what can I do with this….my second is what are you TN Citizens going to do with it. My first two requests have been:

How do we get Tree Height?
Can we see Archaeological sites?

So – my next goal is to process A LOT of LIDAR at night. I’m going to set up a process to fix the format (unfortunately the data on the States website is still zlas), fix the library version, and then process a lot of data. Overnight? During the day? Who knows.

Georgia URISA Luncheon – January 10th 2016

January 5, 2017 by rjhale

LIDAR is a hot topic in Georgia these days. So – for the January 2016 Luncheon more LIDAR.

“LIDAR Updates – Changes in the Technology and Coverage”
Speaker Name: Ernie Smith

Presentation Summary: Like all Technology LiDAR has become very dynamic with new sensors and available data. We will talk about the advancements and availability of Sensors, Standards, and Data sets. There are many changes in the technology and unless you are working with it everyday, it’s difficult to keep up with all of the advancements.

Standards are also changing due to the sensor technology advancement which means practices and applications for LiDAR are growing. Also drones are being used for data collection and there are some things to be aware of for that use of Technology. We will present a map that gives our best understanding of where LiDAR data exists in Georgia as well.

Location:
Henry County Water Authority (Engineering Building)
100 Westridge Industrial Boulevard
McDonough, GA 30253

Georgia URISA November Luncheon November 8th 2016

October 25, 2016 by rjhale

Speaker Name: Jimmy Nolan

Speaker Bio: Mr. Nolan began working with GIS in 1994 when he was the chief appraiser for Morgan County. In 1999 Jimmy went to work as the GIS specialist for the Georgia Department of Revenue. In 2001 he became the local government project manager for the Information Technology Services division of the Carl Vinson Institute of Government at the University of Georgia. Since joining ITOS he has been instrumental in helping develop GIS in approximately 50 counties and has worked with local officials in many more counties and cities to enhance data or provide advice and training. Jimmy is the Cadastral Coordinator for the state of Georgia and was on the Federal Geographic Data Committee technical working group that developed the parcel data content standards for the eastern 31 states. He is the immediate past chairman of the Georgia GIS coordinating committee and a former Ga. URISA board member. He has extensive knowledge of how Computer Assisted Mass Appraisal systems model property values and how the data used in these models is organized as well as how these systems integrate with GIS.

Presentation Summary: The presentation will cover:

Current initiatives
Upcoming LiDAR acquisition projects
Coordination between multiple agencies at all levels of government
The 3DEP grant application that the state has submitted.

Additional Information:

Join WebEx meeting
Meeting number (access code): 927 999 105
Meeting password: URISA

Join by phone
+1-855-282-6330 US TOLL FREE
+1-415-655-0003 US TOLL
Location:
Hexagon Geospatial
5051 Peachtree Corners Circle
Norcross, GA 30092 USA