Options
Cotofrei, Paul
Nom
Cotofrei, Paul
Affiliation principale
Fonction
Maître d'enseignement et de recherche
Email
paul.cotofrei@unine.ch
Identifiants
Résultat de la recherche
Voici les éléments 1 - 4 sur 4
- PublicationAccès libreFrom Forest to Zoo: Great Ape Behavior Recognition with ChimpBehave(2025-06-23)
; ; ; ; Abstract This paper addresses the significant challenge of recognizing behaviors in non-human primates, specifically focusing on chimpanzees. Automated behavior recognition is crucial for both conservation efforts and the advancement of behavioral research. However, it is often hindered by the labor-intensive process of manual video annotation. Despite the availability of large-scale animal behavior datasets, effectively applying machine learning models across varied environmental settings remains a critical challenge due to the variability in data collection contexts and the specificity of annotations. In this paper, we introduce ChimpBehave, a novel dataset comprising over 2 h and 20 min of video (approximately 215,000 frames) of zoo-housed chimpanzees, annotated with bounding boxes and fine-grained locomotive behavior labels. Uniquely, ChimpBehave aligns its behavior classes with those in PanAf, an existing dataset collected in distinct visual environments, enabling the study of cross-dataset generalization - where models are trained on one dataset and tested on another with differing data distributions. We benchmark ChimpBehave using state-of-the-art video-based and skeleton-based action recognition models, establishing performance baselines for both within-dataset and cross-dataset evaluations. Our results highlight the strengths and limitations of different model architectures, providing insights into the application of automated behavior recognition across diverse visual settings. The dataset, models, and code can be accessed at: https://github.com/MitchFuchs/ChimpBehave - PublicationAccès libreAction recognition of great ape behaviors and communicative gestures using deep learningL’étude des comportements et de la communication des grands singes est essentielle à la compréhension des fondements évolutionnaires du langage humain et de ses interactions sociales. Cependant, les méthodes traditionnelles, qui reposent sur l’annotation manuelle de données vidéo, sont laborieuses, chronophages et peu efficientes. À l’inverse, les récentes avancées en vision par ordinateur et en apprentissage profond offrent un potentiel nouveau pour automatiser la reconnaissance des comportements et des gestes des grands singes. Cela dit, leurs applications à ce domaine restent, pour l’heure, limitées.
Cette thèse propose des approches novatrices pour répondre à ces défis en tirant parti des techniques d’apprentissage profond et des jeux de données associés. Elle présente ASBAR (dont l’acronyme français serait RAABS, pour Reconnaissance d’Actions Animales Basée sur les Squelettes), un cadre qui combine l’estimation de pose à la reconnaissance d’actions à travers une approche unifiée, atteignant des résultats compétitifs dans la classification des comportements des grands singes en milieu naturel, tout en réduisant drastiquement les besoins computationnels et de stockage.
Elle introduit également ChimpBehave, un jeu de données vidéo annoté pour la reconnaissance des comportements de chimpanzés en captivité, qui permet l’étude de l’adaptation au domaine et de la généralisation entre jeux de données. L’évaluation de modèles basés soit sur la vidéo, soit sur les squelettes révèle la robustesse de ces derniers face à la variabilité visuelle entre jeux de données.
En outre, cette thèse propose FineChimp, un jeu de données d’actions fines conçu spécifiquement pour la reconnaissance des gestes des grands singes. Avec ses 38 classes de gestes annotées par des experts et ses enregistrements provenant de multiples points de vue, FineChimp permet l’étalonnage des modèles de reconnaissance de gestes et démontre l’efficacité des modèles d’apprentissage profond de pointe pour décoder les nuances de la communication des grands singes.
En intégrant des techniques innovantes de vision par ordinateur à des données comportementales détaillées, ce travail automatise et enrichit l’étude des comportements et de la communication des grands singes, en apportant des outils évolutifs à la recherche en primatologie. Ces contributions ont des implications pour la conservation animale, les sciences comportementales et, de manière générale, la compréhension des comportements et des systèmes de communication animaliers. - PublicationAccès libreFrom Forest to Zoo: Domain Adaptation in Animal Behavior Recognition for Great Apes with ChimpBehave(2024-06-17)
; ; ; ; This paper addresses the significant challenge of recognizing behaviors in non-human primates, specifically focusing on chimpanzees. Automated behavior recognition is crucial for both conservation efforts and the advancement of behavioral research. However, it is significantly hindered by the labor-intensive process of manual video annotation. Despite the availability of large-scale animal behavior datasets, the effective application of machine learning models across varied environmental settings poses a critical challenge, primarily due to the variability in data collection contexts and the specificity of annotations. In this paper, we introduce ChimpBehave, a novel dataset featuring over 2 hours of video (approximately 193,000 video frames) of zoo-housed chimpanzees, meticulously annotated with bounding boxes and behavior labels for action recognition. ChimpBehave uniquely aligns its behavior classes with existing datasets, allowing for the study of domain adaptation and cross-dataset generalization methods between different visual settings. Furthermore, we benchmark our dataset using a state-of- theart CNN-based action recognition model, providing the first baseline results for both within and cross-dataset settings. The dataset, models, and code can be accessed at: https://github.com/MitchFuchs/ChimpBehave - PublicationAccès libreASBAR: an Animal Skeleton-Based Action Recognition framework. Recognizing great ape behaviors in the wild using pose estimation with domain adaptation(2024)
; ; ; To date, the investigation and classification of animal behaviors have mostly relied on direct human observations or video recordings with posthoc analysis, which can be labor-intensive, time-consuming, and prone to human bias. Recent advances in machine learning for computer vision tasks, such as pose estimation and action recognition, thus have the potential to significantly improve and deepen our understanding of animal behavior. However, despite the increased availability of open-source toolboxes and large-scale datasets for animal pose estimation, their practical relevance for behavior recognition remains under-explored. In this paper, we propose an innovative framework, To date, the investigation and classification of animal behaviors have mostly relied on direct human observations or video recordings with posthoc analysis, which can be labor-intensive, time-consuming, and prone to human bias. Recent advances in machine learning for computer vision tasks, such as pose estimation and action recognition, thus have the potential to significantly improve and deepen our understanding of animal behavior. However, despite the increased availability of open-source toolboxes and large-scale datasets for animal pose estimation, their practical relevance for behavior recognition remains under-explored. In this paper, we propose an innovative framework, ASBAR, for Animal Skeleton-Based Action Recognition, which fully integrates animal pose estimation and behavior recognition. We demonstrate the use of this framework in a particularly challenging task: the classification of great ape natural behaviors in the wild. First, we built a robust pose estimator model leveraging OpenMonkeyChallenge, one of the largest available open-source primate pose datasets, through a benchmark analysis on several CNN models from DeepLabCut, integrated into our framework. Second, we extracted the great ape’s skeletal motion from the PanAf dataset, a large collection of in-the-wild videos of gorillas and chimpanzees annotated for natural behaviors, which we used to train and evaluate PoseConv3D from MMaction2, a second deep learning model fully integrated into our framework. We hereby classify behaviors into nine distinct categories and achieve a Top 1 accuracy of 74.98%, comparable to previous studies using video-based methods, while reducing the model’s input size by a factor of around 20. Additionally, we provide an open-source terminal-based GUI that integrates our full pipeline and release a set of 5,440 keypoint annotations to facilitate the replication of our results on other species and/or behaviors. All models, code, and data can be accessed at: https://github.com/MitchFuchs/asbar.