Uncovering what New Zealand's favourite bird does in the shadows

Background

Atarau Sanctuary provides a predator-free sanctuary for Paparoa Wildlife Trust’s great spotted kiwi/roroa. Verum Group, the environmental consultancy that sponsors the sanctuary, is developing a set of deep learning techniques to identify individual birds within a species through their songs. These techniques have been successfully applied to roroa calls collected within and near the sanctuary.

This is pretty amazing work that is using cutting edge techniques, developed by the Verum team. We wanted to share the results and the science with the world in a way that is easy to interact with and communicate the results with a wider audience.

Following a previous engagement with Verum, I proposed developing a custom interactive visualisation that maps kiwi calls collected over a year and allows users to see how many different individual kiwi are in an area, how far they roam and who they are interacting with. Users can also see and hear different kiwi calls and see how each individual has its own distinctive sound patterns.

Challenge

This was one of the rare projects where I was allowed to take full control of the project and make it happen. I had already made an app similar in concept using R and R Shiny, but a custom web app opened a whole new world of possibilities. We knew that from a functionality point of view there were a few must haves. We wanted users to be able to:

• See the range of different birds;
• See and hear the calls of different birds and compare them;
• View how the singing frequency changes over time;
• Check if different birds interact with each other and sing at similar times.

Process

Data wrangling

The team at the Atarau Sanctuary has collected hours and hours of audio and selected the best roroa calls for the visualisation. I've manipulated and extended the provided data to:

• Change the data structure and format, making it suitable for a web application.
• Position the calls in space based on a predefined area, using geospatial tools.
• Process the audio files to remove background noise and trim additional silence.

Solution

Design and development

We settled on two main areas for the application: a Map view and a Sounds view.

The Map view allows users to view calls of different kiwi, and infer the birds' ranges from these. The mapped calls are also linked to a timeline, and this provides extra insights into the activities of kiwi – do they sing all year round? Do they have a partner, and do they often sing together?

The Sounds view highlights the accuracy of the technology being developed by Verum, and shows how different birds have calls that are visually (and audibly) different. Users can compare the calls of different birds, and see how they have different voices – and also how female birds have much lower pitch calls than males.

Results

The visualisation successfully showcases the unique work being developed by Verum, and has been shared with partners and potential customers to highlight Verum's impressive results.

I can't stress enough how much work went into the individual bird identification project – from the grunt work of collecting audio recordings, to automating the calls extraction and the development of the deep learning techniques. This visualisation is only a small part of it.

My thanks to Verum Group – in particular Laura Molles – for agreeing to this project, and to the Brian Mason Scientific & Technical Trust for funding it.

Further work

KiwiViz has since been expanded to include +80K of roroa calls over four years. The visualisation itself, now ManuViz, has been generalised to allow displaying different bird species. This meant reworking the original visualisation to make it generic, and provide the Verum team extensive documentation on how to format the data, configure the visualisation, and upload the new version to their hosting provider.

As part of this work I was also able to add some nice-to-haves that didn't fit the scope of the original work, such as making the visualisation mobile friendly and add a light theme for daytime birds. The map was also expanded to include a 3D view of the terrain, as this is particularly useful to understand how the distribution patterns of certain birds related to the landscape they are living in.