The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring

The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring

Abstract: The vital role of honeybees in pollination and their high rate of mortality in the last decade have raised concern among beekeepers and researchers alike. As such, robust and remote sensing of beehives has emerged as a potential tool to help monitor the health of honeybees. Over the last decade, several monitoring systems have been proposed, including those based on in-hive acoustics. Despite its popularity, existing audio-based systems do not take context into account (e.g., environmental noise factors), and thus the performance may be severely hampered when deployed. In this paper, we investigate the effect that three different environmental noise factors (i.e., nearby train rail squealing, beekeeper speech, and rain noise) can have on three acoustic features (i.e., spectrogram, mel frequency cepstral coefficients, and discrete wavelet coefficients) used in existing automated beehive monitoring systems. To this end, audio data were collected continuously over a period of three months (August, September, and October) in twenty twenty-one from eleven urban beehives located in downtown Montréal, Québec, Canada. A system based on these features and a convolutional neural network was developed to predict beehive strength, an indicator of the size of the colony. Results show the negative impact that environmental factors can have across all tested features, resulting in an increase of up to three hundred fifty-five percent in mean absolute prediction error when heavy rain was present.

One. Introduction

Honeybees (Apis mellifera) are one of the most commercially important insects, not only because of their hive products, but also because of their pollination activity, which helps cultivation and biodiversity. To ensure their health and well-being, beehive monitoring has been performed manually by beekeepers for many years. During such inspections, beekeepers visually examine colony activity and note the presence of pathogens/parasites, queen reproduction, colony worker and brood populations, and the amount of food stored. These inspections are typically performed every two weeks during the spring and summer months and are labor-intensive and time consuming for the beekeepers, as well as disruptive for the colonies.

More recently, with advances in sensing hardware, cloud computing, and machine learning tools, automated beehive monitoring tools have emerged to overcome these limitations, thus creating the field of precision beekeeping. For example, systems based on beehive acoustics, hive weight, internal temperature, and humidity, as well as carbon dioxide monitors or multisensor approaches, have been introduced.

The weight of a beehive can be used as an indicator of honey production and bee population size, with colony weight changes being reported throughout the day. For example, weight have been reported to decrease during the night and early morning due to nectar consumption, while it increases with foraging activity during the daytime. The authors monitored the weight of four colonies and showed that the average weight indicates the amount of stored food (during nectar flow), while the changes could be related to the daily consumption of food. Moreover, swarming activity can be tracked based on the weight change of the hive resulting in a sharp decrease.

Temperature stability and regulation in a beehive, in turn, can indicate the colony's adaptive response and health state. It has been found that while the internal temperature affects the health of the bees and brood, the productivity of the hive is also strongly affected by internal hive conditions. Proper thermoregulation inside the hive, for example, can help decrease mortality rates and increase honey production by reducing the internal consumption. Moreover, it has been found that a rise in temperature (from thirty-four to thirty-five degrees Celsius to thirty-seven to thirty-eight degrees Celsius) could be a signature of swarming.

Relative humidity is another important factor for larvae growth, colony development, and bee behavior, where changes in water transportation and larvae feeding have been reported as a function of ambient humidity, hive temperature, and nectar moisture. The authors used temperature and humidity sensors to monitor ambient conditions, as well as conditions within the breeding comb and the nectar areas. Their results showed that the humidity of the breeding comb was the highest (about forty percent relative humidity) and had less daily fluctuation.

Moreover, it is well known that bees communicate within the colony using vibration and acoustic signals generated via the movement of their body, wings, and muscle contractions. For example, specific sounds are generated during mite attacks, by failing queens, and during swarming. To this end, the acoustic monitoring of beehives has emerged and is gaining popularity. In a recent literature review of beehive acoustics monitoring, several systems were reported showcasing tools for (i) bee activity detection, (ii) beehive strength monitoring, (iii) queen absence detection, (iv) swarming detection, (v) pathogen or parasite infestation detection, (vi) detection of environmental pollutants and chemicals, and (vii) measuring of honeybee reaction to smoke, as well as overall beehive monitoring (e.g., identifying normal and abnormal hive, swarming duration, bee activity time). From a geographical perspective, most contributions have come from the United States of America, United Kingdom, Japan, Slovenia, and Italy. From partially tropical countries (e.g., Mexico), experiments have mostly focused on the detection of the queen bee in Apis mellifera carnica hives. For a more detailed overview, the interested reader is referred to.

One great benefit of acoustic beehive monitoring is the potential for real-time continuous monitoring, which may enable the detection of certain critical and rare events, such as queen piping. Continuous beehive monitoring may also enable new insights into increased mortality rates observed over the last decade, which has been hypothesized to be linked to multiple stressors.

Despite the burgeoning of acoustics-based beehive monitoring applications, as the recent review showed, existing tools rely on conventional audio features that have been developed for speech processing, namely root mean square power, mel frequency cepstral coefficients, spectrogram, and features based on the discrete wavelet transform. It is widely known within the audio processing community, however, that such parameters can be extremely sensitive to environmental factors, such as ambient noise and background speakers. As such, noise suppression and/or characterization of the background noise (known as context-awareness) are needed for the development of accurate and replicable systems. This will be particularly crucial for urban hives, which are on the rise, where loud and various urban sounds interfere with the internal hive sound recordings.

In fact, while most published studies have shown the benefits of using the acoustic signal for hive health monitoring, the majority have relied on data collected over a short period of time (e.g., twenty-four hours), usually in the months of June and July, when rain is possible, or were collected in remote regions in the countryside; thus, they may not have been exposed to certain environmental factors known to be detrimental to the quality of audio recordings. As such, it is not clear what the impact of such events is on audio-based precision beekeeping. This paper aims to fill this gap. Acoustic data were recorded from eleven urban beehives (one was intentionally left empty) during a three-month period. The effects of three environmental factors (rain, urban sounds, and beekeeper speech) were explored on the three most popular audio features described above. To quantitatively measure the impact of such factors on system performance, results for a beehive strength prediction model are reported and drops in accuracy were measured, to validate the claim that context awareness is crucial for automated beehive monitoring systems.

Two. Materials and Methods

Two point one. Data Acquisition

In this study, ten beehives and one empty hive were monitored continuously over a three-month period on a rooftop apiary located in downtown Montréal, Québec, Canada. These hives were placed on wooden pallets (two hives per pallet) in a row facing southeast, with the empty hive placed by itself. This location facilitated ease of access to a power supply from a wall outlet on the outside of the building. Each hive comprised one brood chamber and one or two honey supers using ten-frame standard Langstroth boxes (with a maximum of three boxes), with a multimodal sensor located on top of the center frame of the bottom brood box to record the internal hive temperature and humidity, as shown in Figure one a. and also a microphone beside that, as shown in Figure one b. All hives came from four frame nucs that were purchased and installed in May twenty twenty-one.

The nectar apiculturalists did not equalize the hive populations in order to collect data on a variety of population sizes, as the prediction of beehive strength (a correlate of population size) was one of the main goals. At the beginning of the experiment, each hive contained a different number of (full) frames of bees with a minimum of six frames of bees for a beehive with one brood box, and a maximum of twenty frames of bees for a beehive with one brood box and one honey super. As the colony populations increased, additional honey supers were added. Therefore, in our apiary, the maximum number of boxes and frames of bees were three and thirty, respectively. Data were recorded continuously over the months of August, September, and October, twenty twenty-one. The multi-modal data comprise the average temperature and humidity readings every fifteen minutes, and a fifteen-minute audio segment every thirty minutes, with a sampling rate of forty-eight kilohertz. Every two weeks, the hives were manually inspected to measure the strength of the hives (i.e., the number of frames of bees covered by least seventy percent), to verify the presence of a laying queen, as well as to report any additional observations related to the colony activity.

Moreover, local external temperatures, humidity, and rainfall amounts levels were obtained from the Environment and Climate Change Canada website. A representative example of a twenty-four hour snapshot of the changes in internal/external temperature and humidity levels, as well as the audio intensity and root mean square value, is shown in Figure two. The plots are for a strong and healthy colony in August with one brood chamber and two honey supers with a total of thirty frames of bees (covered with at least seventy percent of bees). As can be seen, audio power increases during the day, especially during periods in which external temperatures were increasing and external humidity levels decreasing, thus suggesting increased foraging activity and thermohygrometric regulation of the colony.