Global lightning mapper: the good, and the be careful…

GOES-16 Global Lightning Mapper in action, showing numerous lightning strikes across central and especially South America. Image courtesy: AllisonHouse Maps

Earlier in January, AllisonHouse acquired, tested and is now displaying the new GOES-16 GLM (Global Lightning Mapper) data as soon as we receive it! So, some of you may be asking questions, like: why are we still using ENTLN data if GLM data is now available? How good is it?

Well, I have to admit. I was disappointed to not see this data earlier than mid-January 2018. I mean…NOAA had some real pretty animations showing us the data. Like this one:

But do you notice something? Look carefully. That data has a LOT of noise with it. And as it turns out, there were many issues with the data, not just noise/false strikes. That animation was made back in August 2017. What has been done to solve the issues? Actually a lot of things. According to NOAA:

“NOAA has confirmed that the GLM L2 (level2) data are at provisional validation maturity as of January 19, 2018.”
What does that mean? NOAA, again:

“Validation activities are occurring and the general research community is now encouraged to participate.”

In other words: the quality of the data wasn’t worth looking at for the average user; it was unreliable. Firmware fixes are eliminating a lot of problems, so the data is now finally worth looking at seriously for the average user. But:

“Severe algorithm anomalies are identified and under analysis. Solutions to anomalies are in development and testing. Incremental product improvements may still be occurring. …Product performance has been demonstrated through analysis of a small number of independent measurements from select locations, periods, and associated ground truth and field campaign efforts. …Provisional users bear all responsibility for inspecting the data prior to use and for the manner in which the data are utilized.”

In other words: there are still serious problems, but they are working on fixing the issues. And, see that last sentence: if you think a storm should be showing lightning, but GLM data shows nothing, there MIGHT be an issue with the GLM data. So far, I have not seen any serious problem like this, but they are occurring, occasionally.

Here are the most important takeaways, according to the engineers:

1. Flash detection efficiency (FDE) over a 24 hour period is over 70%. No service reaches 100%, and in my testing, there have been cases where GLM has had a few more strokes than ENTLN, though that has been rare. A very long strike is double counted in very rare instances. A fix is in testing.

2. Flash location error can be up to 4 KM off in the U.S., and 11 km close to the edge of the field of view, with the strikes needing to be extended out more. This is still remarkable accuracy for a satellite that: A) is 23,000+ miles away, and B) cannot triangulate the strikes like earth-detecting systems can, which can get flash accuracy down to 300′ or less. However, software updates are going to reduce that error even more in the near future.

3. As the sun traverses the earth, you’ll see a bright circle as the sun goes over the oceans; sometimes the satellite thinks that’s a big flash. Filters are now removing that, and a few actual flashes, and the filter is being tweaked to not discount the real flashes.

4. The lightning flashes we get are not in time order in terms of exact start and end time. For how we display the data, this is irrelevant to you, but a fix is coming.

5. The number of flashes in one thunderstorm or group of thunderstorms can be miscounted slightly in the group count total. A fix is pending.

So, having said that, how should you use, or not use, GLM data as of January 25, 2018?

1. Forensics: if you need to know if your house, apartment, business or whatever caught on fire due to a lightning strike, GLM data will NOT help you. The resolution is not good enough. Land-based lightning detection systems must be used for forensic/legal use.

2. You will find that the lightning strike may occur just outside of the thunderstorm from time to time. This is a parallax error that is being worked on. Do not confuse this with an “anvil crawler” that can occur many miles downstream from the updraft and downdraft of a thunderstorm. These will be detected by the GOES-16 GLM.

3. Got a ball game and you are watching for lightning? Or, you’re looking at a line of precipitation that could affect your picnic, wedding, etc in an hour and want to know if what is heading your way has lightning? I say use it for that IN CONJUNCTION WITH our ENTLN data. The GOES-16 GLM data is still “provisional”, and not “operational”; that’s a big difference. Note: your National Weather Service office will not see the GLM data until it is deemed operationally ready. Think of the data as in “alpha” mode before January 11, 2018, and since then, it’s been in “beta”.

Above all else, to be as clear as possible, DO NOT SOLELY RELY ON GLM DATA FOR LIFE AND PROPERTY PROTECTION until it is officially declared operational. We have not, however, been instructed to display a caveat message with this data, but I am saying it here: use it with caution!

So having said all of that (phew!), am I saying the lightning detector stinks, the engineers blew it, or the data is awful? NO!!!! Nothing of the sort. It needs some more tender loving care to bring it up to a very high quality standard, and this is the first instrument of its kind on a satellite. Ever buy a first year new model of a car? Yes, you’re going to have issues. Thankfully, they are fixable, just like the GOES-16 GLM instrument data. And what it is doing now, even with the caveats above, is simply incredible, and mind-blowing! Just keep its current limitations in mind, and you’ll do fine. Enjoy the data!

Happy New Year! A look at what’s to come in 2018

Hello AllisonHouse users,

As 2017 heads into the distance in our rear view mirrors, we thank you for another good year for our company. We’re already rolling out new features on AllisonHouse Maps, and we want to give you a heads up to a few things we can tell you about what is coming in 2018!

– Maps: AllisonHouse Last year, Maps had many MRMS radar products added. Of course, when GOES-16 became available with experimental data, here’s a little known fact: our programmer Ryan Hickman wanted to be the first…or at least, one of the first, to have stunning GOES-16 images available to our customers. Guess what: he worked nearly 24 hours in a row to make that happen! Furthermore, we switched away from the National Weather Service NOAAport feed, and now use the GOES Rebroadcast (GRB) feed for the data. Why? The latter has the raw data, which is a lot better than what is available on NOAAport, in terms of the number of products and resolution. In short, you get the best quality image from this feed! We are awaiting to add the GOES-16 lightning data, but we know that the bugs are still being worked out with it. Until that feed is stable, we’re holding off adding it…for now. So, stay tuned…the National Weather Service is working hard to get it working well. Remember, this is state-of-the-art technology, and until that thing got launched into space, we knew there would be some issues. Thankfully, they can upload firmware to the satellite to fix things that have cropped up. But in general, GOES-16 has been giving us jaw-dropping performance and imagery!

Also, look for even more products to be added to Maps in the year to come! We’ll let you know when they are added.

– Level 2 and Level 3 data. Purdue University has indicated that in the next several months, they will have their dual-pole data online…and we are working to ensure we will have the live feed!

Plus more! Sometimes data just falls into our lap, or it becomes available to the public with little or no fanfare. When we see it would be valuable for our users, we add it!

We hope you have a wonderful new year, and we hope you’ll also tell a friend about us. We want to be the weather data provider you can trust! Our uptime has been greater than 99.99%, and that includes maintenance windows. Reliability and a wide variety of products will continue to serve you well and keep you informed in the new year. See you in 2018!

New university radar coming to AllisonHouse

A doppler radar sits scanning the skies with a developing thunderstorm nearby. Image courtesy; SPC

If you are a radar junkie like me, you know there are areas of the country that need a radar to fill in for poor coverage. Northeastern Missouri and western Illinois, southeastern Oklahoma, amongst others.

However, in November, Purdue University professor Robin Tanamachi announced that Purdue’s main campus in West Lafayette, IN, was going to get their own doppler radar. Their intent is to make all of its data available to outside entities, including AllisonHouse. It will also be made available to the National Weather Service, all in as real time as possible.

The history of University radars is that they provide few if any derived products, so they just give us Level 2 radar data from them. Although I cannot confirm it, I suspect these data will be in Level 2 format only. GRLevel3 users would be out of luck, but GRAnalyst and RadarScope users, for example, would be able to get the data. In many cases, the data may extend out only 60 miles for most products.

That having been said, this fills an important low level gap in the 88D radar network, as this area, westward into east-central Illinois, is between KILX (Lincoln, IL), KIND (Indianapolis) and KLOT (Romeoville/Chicago, IL). Thus, weak tornadoes and mesocyclones frequently go undetected in these parts of the two state area. This will no doubt improve tornado warning accuracy in the area once it is online.

Although there is no hard date for this to happen, Purdue hopes to have the data available sometime in the spring of 2018.