Uncovering what New Zealand's favourite bird does in the shadows

Did you know that we can identify individual birds based on their songs? The Verum Group is developing a set of deep learning techniques to identify individual birds within a species through their songs, and these techniques have been successfully applied to great spotted kiwi (roroa). This is pretty amazing work and we wanted to share the results with the world in a way that is easy to understand and interact with. I proposed developing a visualisation that, combined with a sister site, shows and explains the amazing work being developed.

View interactive visualisation

Map view of the interactive visualisation
Client
Atarau Sanctuary
Verum Group
Role
Interactive design
Interactive development
Date
2021-2022

Background

Atarau Sanctuary provides a predator-free sanctuary for Paparoa Wildlife Trust’s great spotted kiwi/roroa. Verum Group, the environmental consultancy that sponsors the sanctuary, is developing a set of deep learning techniques to identify individual birds within a species through their songs. These techniques have been successfully applied to roroa calls collected within and near the sanctuary.

This is pretty amazing work that is using cutting edge techniques, developed by the Verum team. We wanted to share the results and the science with the world in a way that is easy to interact with and communicate the results with a wider audience.

Following a previous engagement with Verum, I proposed developing a custom interactive visualisation that maps kiwi calls collected over a year and allows users to see how many different individual kiwi are in an area, how far they roam and who they are interacting with. Users can also see and hear different kiwi calls and see how each individual has its own distinctive sound patterns.

Below is a spectrogram, a visual representation of sound over time. Time is represented from left to right; pitch (how bassy or high-pitched a sound is) from bottom to top; and amplitude (how loud a sound is) is represented by the colours, with brighter colors representing louder sounds. Here we can see two roroa – a male and a female – dueting.

A spectrogram of a male and female roroa dueting

Challenge

This was one of the rare projects where I was allowed to take full control of the project and make it happen. I had already made an app similar in concept using R and R Shiny, but a custom web app opened a whole new world of possibilities. We knew that from a functionality point of view there were a few must haves. We wanted users to be able to:

  • See the range of different birds;
  • See and hear the calls of different birds and compare them;
  • View how the singing frequency changes over time;
  • Check if different birds interact with each other and sing at similar times.

Onboarding new users to the map view of the visualisation.

Process

Data wrangling

The team at the Atarau Sanctuary has collected hours and hours of audio and selected the best roroa calls for the visualisation. I've manipulated and extended the provided data to:

  • Change the data structure and format, making it suitable for a web application.
  • Position the calls in space based on a predefined area, using ArcGis Pro.
  • Process the audio files using Adobe Audition to remove background noise and trim additional silence.

The map view, showing an alternative basemap where users can see the terrain and landscape of the sanctuary. Note the timeline, showing the time and date of the calls, and how a call in the timeline gets highlighted when a marker on the map is hovered.

Map view of the visualisation, with a basemap set to terrain

Solution

Design and development

We settled on two main areas for the application: a Map view and a Sounds view.

The Map view allows users to view calls of different kiwi, and infer the birds' ranges from these. The mapped calls are also linked to a timeline, and this provides extra insights into the activities of kiwi – do they sing all year round? Do they have a partner, and do they often sing together?

The Sounds view highlights the accuracy of the technology being developed by Verum, and shows how different birds have calls that are visually (and audibly) different. Users can compare the calls of different birds, and see how they have different voices – and also how female birds are much bassy than males.

Because roroa are nocturnal, I proposed a dark theme for the application. The Sounds page allows users to play calls of roroa and compare how the sounds of different birds look different. Special thanks to Jonathan Hardie for the custom spectrogram.

The Sounds view of the application on the desktop The Sounds view of the application on a small tablet

Results

KiwiViz successfully showcases the unique work being developed by the team at Verum and communicates the science behind the project in a tangible way.

I can't stress enough how much work went into the individual bird identification project – from the grunt work of collecting audio recordings, to automating the calls extraction and the development of the deep learning techniques. This visualisation is only a small part of it.

I'd like to thank Verum Group – in particular Laura Molles – for agreeing to this project, and to the Brian Mason Scientific & Technical Trust for funding it.

A quick demo of the map view.