KEJOYCE.COM
follow me
  • Home
  • Research
  • Education
  • AWARDS
  • Blog
  • Gallery
  • Contact

Flirting with FLIR

7/5/2016

8 Comments

 
Picture
For some time I've been fascinated by the marked temperature differences over short distances I can feel when swimming at the beach or on the reef. This clearly happens at scales that are far too small to be detected by satellite imaging sensors. As I'm also interested in coral reef dynamics, I am curious to know just how these fine scale variations may play a part in affecting coral health. Temperature loggers are one way to look at this, but they are always going to represent discrete points compared to an imaging sensor that can measure a continuous surface. An airborne thermal imaging sensor is therefore the only way that I can start to investigate this matter.

Recently, a couple of miniaturised thermal sensors have come onto the market and are light enough to be carried by off-the-shelf drones. I've been dabbling with drones for a couple of years now, so it seemed the right time to see where I could take this. Last week I had the opportunity to test the FLIR Vue Pro on Orpheus Island, Great Barrier Reef, and here's what I found.

Firstly, here's my equipment:
  • 3DR Solo (see also my thoughts about my first Solo flight)
  • FLIR Vue Pro 19 mm lens, 640 x 512 pixel array, 'slow' speed thanks to Tim Snell from IMC
  • GoPro Hero 4 Silver modified with 4.35 mm minimal distortion
  • Go Pro Dual Mount
  • Rugged Troll Thermal loggers
  • FLIR Vue Pro App (for android or iOS)
  • USB power pack

Power supply
The first thing to note is that the FVP does not come with a power supply or battery, just a mini USB input cable. So obviously to use it, you need to have a way to power it. Mains power is clearly restrictive, but is fine for getting familiar with the camera. For a mobile option, I bought a USB power pack ($30). At 80 grams, this could be attached to the drone, but it's still a lot of unnecessary weight. Luckily the 3DR Solo has an accessory bay that's built for exactly this sort of thing. For the lighter option of 65g, a colleague made me a mini USB connector that also includes navigation lights for the Solo (it's severely lacking in this regard, and I plan to fly at night). We have some work to do in giving this some finesse especially as we weren't sure how long the USB cable needed, but at least we now have power to the camera.

Attachment / Mounting
How to mount the camera was the next main decision. It comes with a GoPro attachment ring / collar that fits under the lens. This is fine if you want to mount it on the Solo facing forward, but for mapping purposes it doesn't provide enough clearance from the hull. I opted instead to use a tripod-to-GoPro adaptor which then allowed the necessary clearance. Unfortunately when every gram counts I'd rather not have this added weight, but short of building a specific mount with not enough time, this will do the job.

Next I wanted to see if I could fly both the GoPro and the FVP at the same time. To do this, I bought a dual mount and fitted the GoPro to the front of the craft while the slightly heavier FVP was closer to the centre of gravity. Taking care to use the shortest GoPro connector screws possible, this complete set up (including lights) weighed 1.84kg, just 20g more than the standard gimbal + GoPro configuration. This is still within the expected MTOW of the craft. I can fly this set up in the hot and humid tropics for approximately 12 mins, while retaining at least 20% battery at the end of the flight.

Picture
Picture
Controlling the camera
With the hardware largely figured out, software was the next challenge. The FVP App is pretty basic to be honest, but it largely does the job. I was having some troubles with it initially, but an update came out last week that largely solved the issues. I learnt to just ignore its warnings.

Within the App, it's possible to change colour palettes, dynamic range, and output format (jpg or tiff). I use the white hot palette as I'm not concerned about the display, I just want the actual data in the pixels. I can always display it as different colours later if I wish. The app is used to switch between video, image, or time lapse modes. Images can be set to acquire as frequently as 1/sec up to 1/17 secs. Data are written to the 32GB included mini sd card in the camera.

The biggest miss on the app is that there is no live feed of the imagery. This is a real pain when trying to work through the different settings - without the immediate feedback of the effect of a setting change, it's like learning old school photography again and having to wait until your film is developed before you can see the results. There is the facility for a mavlink cable (provided), so a feed should be possible via the drone itself though I didn't test this out as its not a simple plug in.

The App uses bluetooth to connect to the FVP. Once you have the recording started, you can manually switch the blue tooth off. Alternatively, I do like how the Bluetooth automatically disables itself after two minutes of the App being closed so that it cannot otherwise interfere with the flight controller. Yet even with the App closed, the FVP continues to record data and you can see this by a small flashing light on the camera itself.

if you're intending on using Tower or Mission Planner to set up your flights, it's useful to add the FVP camera specs into the software so that footprints and overlaps can be accurately determined. To do this have a look at these instructions, but use the following parameters (note that you will have to change this if you have the smaller format frame):
<Camera>
<name>FLIR Vue Pro 19mm</name>
<flen>19</flen>
<imgh>512</imgh>
<imgw>640</imgw>
<senh>8.7</senh>
<senw>10.88</senw>
</Camera> 

Working with the data
Downloading the data is super easy. Simply get a mini SD card adapter and plug it into your computer. My files were all 643KB ea. But that's when things got a little more challenging. I wanted to geotag each image based on the Solo telemetry data. This is usually pretty simple in Mission Planner - at least for my GoPro data. But MP couldn't match the image acquisition times with the GPS data. I suspect it is something to do with the format, as the time stamp on each of the TIFFs looked fine to me. I had to assume that it was auto-syncing with the tablet that I used to control it as I couldn't find a way to otherwise change it. I then tried again to geotag with Geosetter, and that worked so clearly there's nothing wrong with the files, but something wrong with Mission Planner reading their metadata. I also realised that each image actually HAD a geotag initially, but that it was set at 0,0 lat/long. That totally messed up our attempt to mosaic in Photoscan without fixing that first.

Given that the imagery I acquired is not the standard type that Pix4D or Photoscan can easily mosaic (i.e. it's hard to automatically find control points over water) we are certainly having some challenges there. Once I've sorted it all out, I'll post a blog specifically on that.

In the meantime, I am happy that the camera is sensitive to what I need. Early views of the data are really promising!

Calibrating for absolute temperatures
The top of the range FVP is radiometrically calibrated. The one that I received was radiometrically capable but not yet calibrated. Quite a bit cheaper... For many applications, relative temperatures may be just fine, but I wanted absolute. And as I have a fully calibrated FLIR A65, and access to temperature loggers, I performed my own calibration. I used a couple of Chinese takeaway containers filled with iced water, boiling water, and something in between, all inside a black box tucked inside a storage cupboard - somewhat similar to Harry Potter's bedroom - with the lights out. The FLIR sensors are sensitive to reflected light in the 8-14um range as well, so I wanted to get rid of that and only have the emitted temperatures. I used FLIR Tools for the A65 (below) and took photos of the same area with the FVP using the App. I still have to model the correction - another post again!

Picture
8 Comments
Chris
4/5/2016 06:50:17 pm

Hi Karen,

The research sounds interesting, I'm curious as to how Pix4D or Photoscan handle the orthomosaic with moving water changing the temperature? I would have thought you'd get far less matches on the point cloud. Have you done any actual flight tests and getting the actual imagery processed?

You should check with CASA if you haven't already, they may count you as a commercial operation. I presume the research will be potentially sold or you have a grant to work on it? If so then I'm afraid the night flying wont be allowed unless you get an area approval from CASA, this will still be the case even with the change in rules come Oct. If it's considered hobby then the night flying is all good as long as you can keep visual LOS.

Reply
Karen
4/5/2016 07:50:13 pm

Hi Chris,

Thanks for reading and commenting! The point cloud is definitely challenging. A lot of the photos have some land in it though and we deployed some reflective buoys in the bay too. It's early days, but the first mosaic actually looks better than I expected. So yes, I have done quite a few flights and now have a backlog of data to process! I usually update on twitter @kejoyce2, so if you're interested, please follow me. Every now and then I'll blog, and otherwise it's scientific journal articles / conference presentations.

CASA definitely counts us as commercial - pretty much unless it's a sport or hobby, we fall into the commercial category. This is my job, and you're right, I have research grants to conduct this work. I have my RPAS cert and we fly under an OC. Additionally, you need a permit to fly UAS over the Great Barrier Reef - I have that too. Unfortunately our area approval for night ops didn't come through in time for my work last week, I'm actually not sure why it took so long, but I'll definitely be covered for my next trip as it's pretty important for thermal imaging as I don't want to be recording reflectance. I actually thought that night flight was still against the hobby regs?

Cheers,
Karen

Reply
Chris
4/5/2016 10:10:07 pm

Hi Karen,

I'll have a look at your twitter feed. Not surprised that they treat this as commercial as they should. Night Flying is allowed under Sub Part 101.G - specifically regulation 101.390. It needs to be done in accordance with an approved aviation administration organisation procedures.

I'm surprised to hear that there is a ban on RPA's in the Great Barrier Reef. Would be good if they would publish these, can you provide a reference to it? Just that a quick google on GBR and Drones or RPA's or UAV's doesnt appear to show anything. I presume its only the ground operating aspect (as they have no jurisdiction of the air), like councils enact across the country.

Good luck with the project, sounds like a fun one.

Reply
Karen
5/5/2016 09:29:20 am

Hi Chris,
Commercial is an interesting one for me. As an academic, my projects are largely categorised as either research or commercial. While there is some blurring of the boundary between the two, this particular project falls fair and square into the research domain. This essentially means that it has a tiny budget, there's no profit, and all the outputs will be publicly available for free. So while I acknowledge that it's definitely not sport / hobby / recreation, in my world it's definitely not commercial. On the flip side I understand that it can be argued that the University and I stand to receive a reputational gain from the project, and such a gain could be therefore seen as commercial. It is what it is, and we play by the rules :) The other thing that I argue is that I drive a car on the weekend (recreation) and during the week for work meetings (commercial), yet the license and regulations associated with both uses are the same. We share the road, just as we do the airspace, so why should airspace regulations be different for different user groups?

I have the 'Information Sheet' that details the requirement (or otherwise) for using drones in the GBR Marine Park, though I too could not find it online. I also can't attach here, but can email it to you if you want to PM me your address? Basically if you are flying <5kg for photography / filming and aren't disturbing the animals, then it's ok. BUT if you are doing it for research and aren't part of an accredited institution then you need a permit. So the research restriction is higher here than even commercial. JCU is an accredited institution so I didn't need a permit in this instance with my lightweight craft, but I have the required Environmental Management Plan in place anyway so that my larger craft are covered.

Thanks for pointing me to 101.390 :)

Karen
5/5/2016 09:33:13 am

I had another look at that bright yellow little CASA brochure thing that now gets distributed when you buy a drone, and it clearly states "You must only operate this aircraft in your line of sight in daylight". Interesting that this is not really what the legislation actually says...

Chris
5/5/2016 11:57:50 am

CASA basically says commercial is everything that's not hobby, interesting view on it. I'm in an volunteer emergency service and we are caught up in it too (hopefully changes in Oct will help this).

Yep that little yellow brochure, I dislike it when government agencies say something is not legal when it isn't. If they don't like it they should change it, don't try to get it in by stealth...... I had a list of question for CASA they answered everyone of the technical questions (like clarification on the populous area) but the one I had about that yellow poster was ignored. I think their thoughts are if you are a MAAA or AMAS member then you know you can do it, otherwise others should be told they cant..... Why not say no flying at night unless a member of an AAO? Simple and matches the regulations too.

Having said all that CASA is better than a lot of the Civil Aviation Authorities around the world.

Reply
thierry Hoibian
18/11/2016 08:30:25 am

Hi Karen
for the problem of location you can try the RTK GPS Ublox M8p and adaptation kit https://shop-emea.u-blox.com/en/eur/3~322~EMEA/Evaluation-kits-tools/Application-boards/C94-M8P-RTK-kit
may be some guys have already made a direct link of this gps with the controler of your Solo, so you will not need anymore ground control points to georeference the images
Regards
Thierry

Reply
Roy Elliott link
10/1/2021 09:10:03 pm

Thanks ffor writing this

Reply



Leave a Reply.

    Karen Joyce

    I'm a remote sensing scientist with a keen interest in research and active learning

    Enter your email address:

    Delivered by FeedBurner

    Archives

    March 2022
    February 2022
    March 2020
    December 2018
    October 2018
    September 2018
    February 2017
    May 2016
    April 2016
    December 2015
    November 2015
    October 2015
    August 2015
    July 2015
    May 2015
    April 2015
    March 2015

    Categories

    All

    RSS Feed

Powered by Create your own unique website with customizable templates.