From underwater videography to aerial drones, technological advancements in ocean observation have significantly enhanced our understanding of marine ecosystems and the impact of human activities on the marine environment. Scientists from the Marine Biological Association (MBA) are now able to collect data in real-time and monitor marine life in ways that were previously unimaginable.
Cutting-edge technology unveils the secrets of sub sea species
Modern videography is revolutionizing our understanding of echinoderms (spiny skinned marine animals such as starfish, sea urchins and crinoids) offering a captivating glimpse into the depths of the ocean where these fascinating creatures thrive.
Research published in 2021 used over 30 years of time-lapsed video and photo surveillance to gain new insights on the feeding habits of crinoids (also known and feather stars). This use of technology captured crinoid feeding behaviour previously unknown and demonstrated a great potential to learn about many other aspects of their biology.
However, recent digital videography paired with submersibles, remotely operated vehicles (ROVs), SCUBA (self-contained underwater breathing apparatus), mixed gases and deep technical SCUBA diving technologies have given us access to the intricate behaviours and ecology of organisms at the seafloor, especially stalked and unstalked crinoids. With subs and SCUBA we can observe tiny organisms (e.g. symbionts) and species interactions (e.g. fish predators) in real time and in habitats that were previously thought to be far beyond our reach, in the subtidal environment and deeper (mesophotic and deep sea).
For example, recent videos and images have enabled a new classification of crinoid arm postures, provided detailed views of food-particle capture, and revealed a wide range of behaviours in species never seen before.
Close-up HD image and video recordings of crinoids capture a whole new world, consisting of a multitude of different species of fish and invertebrates. We have observed up to 12 fish and invertebrate species on one single crinoid individual at one time!
In captivity, using HD/macro lenses, we can also track the lifecycle of the crinoid offspring, from a ripe female with unfertilized eggs, to her brooding her embryos, to her releasing these embryos and development from larvae to juveniles.
How advanced videography provides valuable insights into marine ecosystems
As a marine ecologist and conservation scientist, Bryce’s research involves studying and monitoring species living on the seabed such as scallops, corals and kelp. This is necessary to understand how they respond to environmental change or different management approaches, such as Marine Protected Areas (MPAs). Traditional methods for doing this involve visual surveys underwater while on SCUBA gear. While this can be very successful, it comes with a variety of challenges.
First of all, identification of species. During his PhD on the Great Barrier Reef in the 1990s, Bryce had to make underwater line drawings of fish on a slate and then look them up when back on dry land.
He said: “This was quite a challenge when I had 750 species to choose from! Cameras with slide film were no help when it took weeks to get the images back. Habitats such as coral and kelp are also difficult to quantify underwater because you generally have to estimate their percent coverage on the seabed. This comes with a decent degree of uncertainty and is likely to vary between observers. Finally, logistical and health and safety constraints limit both the depth and time you can spend SCUBA diving, and therefore the areas you can survey.”
The advancement of digital camera technology, both stills and video, is helping to overcome all of these challenges. A great example comes from Bryce’s research on the Isle of Arran, where the local community has campaigned for and helped implement a wide range of conservation measures ranging from a no take zone to multi-use areas. He has been working with the Community of Arran Seabed Trust since 2010 to help monitor and understand the effectiveness of these different conservation measures.
Although diving surveys are still used, these now include the use of underwater digital videos which can be analysed in depth afterwards using specialised software. Bryce’s research team also employ Baited Remote Underwater Videos (BRUVs) to monitor fish species that might otherwise be scared away by divers, and have used Remotely Operated underwater Vehicles (ROVs) to survey depths beyond the reach of divers.
They also have plans to employ Autonomous Underwater Vehicles (AUVs) which can be programmed to ‘fly’ a grid like pattern on the seabed and reconstruct 3-dimensional maps afterwards.
This use of the latest technology and collaboration with the local community on Arran has helped inspire other coastal communities around Scotland through the Coastal Communities Network, and is informing marine conservation policy in the UK and around the world.
iCPR & Holographic images
Dr Pierre Hélaouët and Rob Camp
Based at the MBA, the Continuous Plankton Recorder Survey (CPR) is the longest running and most geographically extensive marine plankton sampling program in the world.
Since 1931 almost 300 ships have travelled more than 7.2 million miles towing sampling devices that capture plankton from the ocean.
Able to be deployed at up to 25 knots in rough seas, it has been towed by container ships, passenger ferries, fishing boats and sailing vessels all over the world.
More than 60 years of consistent methodology offers a unique tool to the scientific community and has helped produce over 1000 publications in peer-reviewed journals.
Advances in technology have enabled the development of the iCPR which revolutionises monitoring capabilities whilst retaining consistency with the historical data gathered over almost seven decades.
Professor Alex Nimmo Smith, Professor of Marine Science and Technology at the University of Plymouth designed the iCPR module and has a central role in the software development at the core of the image processing.
Fitting neatly inside a traditional CPR sampler, the iCPR module functions as a self- powered platform able to sample anywhere a commercial ship is operating.
The iCPR sampler is equipped with a suite of sensors which can capture key environmental variables and precise digital images of plankton using a bespoke holographic camera.
Hologram technology has been used since the 1940s, but the improvement of systems means devices have been scaled to be more portable, robust cameras that can withstand long tows and harsh conditions at sea.
Virtually unchanged for nearly a century, the traditional CPR sampler remains a robust and reliable source of data gathering, and will continue to operate and alongside the iCPR to advance our knowledge of plankton in our ocean.
AI classification of iCPR holograms
Kai Cursons
For imaging microscopic plankton in the ocean, the iCPR uses a holographic camera. Unlike traditional imaging techniques, holographic cameras image particles by flashing a LASER beam through the water onto a digital camera sensor.
Unobstructed, the LASER beam travels linearly to the sensor, however, particles in the water (e.g. plankton) encountered by the beam alter the trajectory of that part of the beam. It is the patterns of interaction between the unaltered and disturbed elements of the LASER beam that are recorded on the sensor and constitutes a hologram. Though this technique may appear unorthodox it is particularly well suited for use on the iCPR.
The camera itself is very robust with no moving parts allowing it to survive the rough deployment and towing conditions that may be experienced by the host CPR. Furthermore, hologram acquisition occurs very fast with no need to bring particles into focus before imaging, ensuring that we can image particles even at high towing speeds. Being able to do this does come with a minor disadvantage in that it is almost impossible to identify the particles by the patterns they create on the holograms alone, requiring several additional processing steps to produce a recognisable image.
With holograms essentially being a record of how particles change the angle of light from LASER to sensor, we can use this information to mathematically trace the trajectory of the altered LASER beam back through the water volume. By doing this across the whole space between LASER and sensor we can the individually bring each plankton in the hologram into focus at its true distance from the sensor at the time of imaging. Tens or hundreds of individual planktonic organisms may be extracted from each hologram, and a single iCPR deployment may generate thousands of holograms. Manual identification of plankton contained within many hundreds-of-thousands of holograms on a regular basis is a highly impractical task for a human. However, it is exactly this sort of task where AI can provide a solution.
Currently we are developing AI models that can automatically identify and count key plankton types extracted from holograms taken during an iCPR tow in a fraction of the time it would take a human. For an AI to learn how to categorise the plankton found in iCPR images, we must first teach it using a large volume of images pre-categorised by humans. A library of images (taken from past iCPR tows) for use in AI training is being constructed aided by the expert taxonomic knowledge of CPR analysts. Our initial prototypes of AI models can correctly distinguish between several broad taxonomic categories of plankton images 97% of the time, which is very promising.
As our image library grows, we will be able to continually expand the capabilities of our AI systems to recognise an increasing number of plankton categories. Current AI image analysis systems are unable to match the taxonomic resolution of an expert taxonomist identifying a physical sample that can manipulated. However, our iCPR models will be able to provide valuable information on the size, shape, location and broad taxonomic identity of plankton across ocean-wide iCPR tows with an incredibly fast turnaround.
Biophysics of a Changing Ocean: From Plankton to Predators
Using innovative approaches for environmental sensing, Lilian is pushing sensor limits to derive a predictive understanding of ocean change. She is passionate about using sensors technologies to capture the pulse of our ocean and feeding it into the digital domain.
Her deep fascination with the foraging ecology of seabirds and filter-feeding sharks drives her work. These incredible animals, constantly on the move and traversing vast ocean expanses, act as sentinels of ocean change, offering critical insights into ocean health and shifts within marine ecosystems.
Lilian’s vision is to work towards a more integrated and cross-system view of ecological ocean dynamics using diverse observing systems in synchrony, from aerial drones and underwater acoustics to animal-borne bio-logging and autonomous vehicles.
Previously, Lilian has used aerial drones for mapping evolving turbulence and tracking foraging seabirds. Her transformative work demonstrated that seabird foraging is driven by highly localised physical flow features, such as upwellings or slicks, which could only be revealed using drone technology. Drones provided a bird’s-eye view of ocean predators directly interacting with their underlying, highly dynamic environment. Additionally, being digital, the drone-derived data could be post-processed and analysed using computer vision, machine learning and particle image velocimetry techniques.
Inspired by her PhD research on basking sharks and recent work on whale shark foraging drivers, Lilian is currently developing novel integrated environmental sensors. Sensor packages will also include plankton imaging for bio-logging and marine autonomy applications in collaboration with Prof Alex Nimmo Smith at the University of Plymouth. “Most of my research involves a curiosity-driven and hands-on approach. As the daughter and grand-daughter of engineers, I’ve learned that true innovation stems from a deep understanding of existing tools and the challenges we want to overcome. The continuous testing and refining of our new sensor package is an exciting journey of exploration and will help to determine key ecosystem drivers of ocean change.”
