Itir Onal Ertugrul | Utrecht University

Bio

I’m an Assistant Professor at Social and Affective Computing Group, Department of Information and Computing Sciences at Utrecht University. Prior to joining UU, I was an Assistant Professor at the Department of Cognitive Science and Artificial Intelligence at Tilburg University. I was previously a Postdoctoral Researcher at Robotics Institute, Carnegie Mellon University working with Jeffrey Cohn, a Postdoctoral Researcher at Affect Analysis Group, University of Pittsburgh and a visiting Ph.D. student at Pattern Recognition and Bioinformatics Group, TU Delft. I received my B.Sc., M.Sc. and Ph.D. degrees from Computer Engineering Department at Middle East Technical University.

Research Interests

Affective Computing
Machine Learning
Human Behavior Understanding

News

[Aug 2023] Ronald Poppe and I received the Dynamics of Youth (DoY) Early Childhood Grant! We will work on predicting development of children at the age of 3 years, by observing their interactions with a parent when they were 5 months old.
[Aug 2024] Our paper titled “Video Swin Transformers in Pain Detection: A Comprehensive Evaluation of Effectiveness, Generalizability, and Explainability” has been accepted at Workshop on Automated Assessment of Pain at ACII 2024.
[Apr 2024] PyAFAR is released! It features the first publicly available infant action unit (AU) detector. For more information on how to install and use it, check our website.
[Apr 2024] Our paper titled “Multimodal prediction of obsessive-compulsive disorder, comorbid depression, and energy of deep brain stimulation” has been accepted for publication at IEEE Transactions on Affective Computing. Preprint can be found here.
[Apr 2024] Our demo paper titled “Expanding PyAFAR: A Novel Privacy-Preserving Infant AU Detector” has been accepted at FG 2024. I will make a demo of PyAFAR infant AU detector at FG 2024.
[Jan 2024] Our paper titled “Elucidating the exposure bias in diffusion models” has been accepted at ICLR 2024.
[Dec 2023] Our paper titled “Face the Needle: Predicting risk of fear and fainting during blood donation through video analysis” has been accepted at FG 2024.
[Oct 2023] Roy Hessels and I received the Dynamics of Youth Invigoration Grant for our project “Automated assessment of internalizing and externalizing behavior of children from parent-child interaction videos”!

Selected Publications

Fully-attentive and interpretable: vision and video vision transformers for pain detection

Giacomo Fiorentini, Itir Onal Ertugrul, Albert Ali Salah

NeurIPS 2022 Workshop on Vision Transformers: Theory and Applications (VTTA), 2022.

Code Poster

Infant AFAR: Automated Facial Action Recognition in Infants

Itir Onal Ertugrul, Yeojin Amy Ahn, Maneesh Bilalpur, Daniel S. Messinger, Matthew L. Speltz, Jeffrey F. Cohn

Behavior Research Methods (BRM), 2022.

Code

Synthetic Expressions are Better Than Real for Learning to Detect Facial Actions

Koichiro Niinuma, Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni

IEEE Winter Conference on Applications of Computer Vision (WACV), 2021.

Project Page

Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in Obsessive-Compulsive Disorder

Nicole R. Provenza, Sameer A. Sheth, Evan M. Dastin-van Rijn, Raissa K. Mathura, Yaohan Ding, Gregory S. Vogt, Michelle Avendano-Ortega, Nithya Ramakrishnan, Noam Peled, Luiz Fernando Fracassi Gelin, David Xing, Laszlo A. Jeni, Itir Onal Ertugrul, Adriel Barrios-Anderson, Evan Matteson, Andrew D. Wiese, Junqian Xu, Ashwin Viswanathan, Matthew T. Harrison, Kelly R. Bijanki, Eric A. Storch, Jeffrey F. Cohn, Wayne K. Goodman, David A. Borton

Nature Medicine, 2021.

Video

Crossing Domains for AU Coding: Perspectives, Approaches, and Measures

Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni, Zheng Zhang, Lijun Yin, and Qiang Ji

Transactions on Biometrics, Behavior, and Identity Science (TBIOM), 2020.

D-PAttNet: Dynamic Patch-Attentive Deep Network for Action Unit Detection

Itir Onal Ertugrul, Le Yang, Laszlo A. Jeni, and Jeffrey F. Cohn

Frontiers in Computer Science, 2020.

AFAR: A Deep Learning Based Tool for Automated Facial Affect Recognition

Le Yang, Itir Onal Ertugrul, Laszlo, A. Jeni, Wanqiao Ding, Jeffrey F. Cohn

International conference on automatic face & gesture recognition(FG), 2019.

Code

Cross-domain AU Detection: Domains, Learning Approaches, and Measures

Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni, Zheng Zhang, Lijun Yin, and Qiang Ji

International conference on automatic face & gesture recognition(FG), 2019.

FACS3D-Net: 3D Convolution based Spatiotemporal Representation for Action Unit Detection

Le Yang, Itir Onal Ertugrul, Jeffrey F. Cohn, Zakia Hammal, Dongmei Jiang, Hichem Sahli

International Conference on Affective Computing and Intelligent Interaction (ACII), 2019.

PAttNet: Patch-attentive deep network for action unit detection

Itir Onal Ertugrul, Laszlo A. Jeni, and Jeffrey F. Cohn

British Machine Vision Conference (BMVC), 2019.

Code

FACSCaps: Pose-Independent Facial Action Coding With Capsules

Itir Onal Ertugrul, Laszlo A. Jeni, and Jeffrey F. Cohn

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018.

What will your future child look like? Modeling and Synthesis of Hereditary Patterns of Facial Dynamics

Itir Onal Ertugrul, Hamdi Dibeklioglu

International conference on automatic face & gesture recognition(FG), 2017.