Itir Onal Ertugrul | Utrecht University

Bio

I’m an Assistant Professor at Social and Affective Computing Group, Department of Information and Computing Sciences at Utrecht University. Prior to joining UU, I was an Assistant Professor at the Department of Cognitive Science and Artificial Intelligence at Tilburg University. I was previously a Postdoctoral Researcher at Robotics Institute, Carnegie Mellon University working with Jeffrey Cohn, a Postdoctoral Researcher at Affect Analysis Group, University of Pittsburgh and a visiting Ph.D. student at Pattern Recognition and Bioinformatics Group, TU Delft. I received my B.Sc., M.Sc. and Ph.D. degrees from Computer Engineering Department at Middle East Technical University.

Research Interests

Affective Computing
Machine Learning
Human Behavior Understanding

News

[Apr 2024] Our demo paper titled “Expanding PyAFAR: A Novel Privacy-Preserving Infant AU Detector” has been accepted at FG 2024. It is the first publicly available infant action unit (AU) detector! You can find more information on how to install and use it, check our website.
[Jan 2024] Our paper titled “Elucidating the exposure bias in diffusion models” has been accepted at ICLR 2024.
[Dec 2023] Our paper titled “Face the Needle: Predicting risk of fear and fainting during blood donation through video analysis” has been accepted at FG 2024.
[Oct 2023] Dr. Roy Hessels and I received the Dynamics of Youth Invigoration Grant for our project “Automated assessment of internalizing and externalizing behavior of children from parent-child interaction videos”!
[Aug 2023] Our paper “Deciphering Entrepreneurial Pitches: A Multimodal Deep Learning Approach to Predict Probability of Investment” has been accepted at ICMI 2023.
[Aug 2023] Our demo paper “PyAFAR: Python-based automated facial action recognition for use in infants and adults” is accepted at ACII 2023. Stay tuned for the code and executables.
[Jul 2023] Our paper titled “Automated measurement of infant and mother Duchenne facial expressions in the Face-to-Face/Still-Face” has been published at Infancy.
[Jun 2023] Our preprint is online: Can an Unobtrusive, Multimodal Mixed-Effects Regressor Based on Open-Ended Interviews Predict OCD Severity?
[Jun 2023] Our paper titled “Automated Emotional Valence Estimation in Infants with Stochastic and Strided Temporal Sampling” has been accepted at ACII 2023.
[May 2023] Our paper titled “Predicting vasovagal reactions to needles with anticipatory facial temperature profiles” has been published at Nature Scientific Reports.

Selected Publications

Fully-attentive and interpretable: vision and video vision transformers for pain detection

Giacomo Fiorentini, Itir Onal Ertugrul, Albert Ali Salah

NeurIPS 2022 Workshop on Vision Transformers: Theory and Applications (VTTA), 2022.

Code Poster

Infant AFAR: Automated Facial Action Recognition in Infants

Itir Onal Ertugrul, Yeojin Amy Ahn, Maneesh Bilalpur, Daniel S. Messinger, Matthew L. Speltz, Jeffrey F. Cohn

Behavior Research Methods (BRM), 2022.

Code

Synthetic Expressions are Better Than Real for Learning to Detect Facial Actions

Koichiro Niinuma, Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni

IEEE Winter Conference on Applications of Computer Vision (WACV), 2021.

Project Page

Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in Obsessive-Compulsive Disorder

Nicole R. Provenza, Sameer A. Sheth, Evan M. Dastin-van Rijn, Raissa K. Mathura, Yaohan Ding, Gregory S. Vogt, Michelle Avendano-Ortega, Nithya Ramakrishnan, Noam Peled, Luiz Fernando Fracassi Gelin, David Xing, Laszlo A. Jeni, Itir Onal Ertugrul, Adriel Barrios-Anderson, Evan Matteson, Andrew D. Wiese, Junqian Xu, Ashwin Viswanathan, Matthew T. Harrison, Kelly R. Bijanki, Eric A. Storch, Jeffrey F. Cohn, Wayne K. Goodman, David A. Borton

Nature Medicine, 2021.

Video

Crossing Domains for AU Coding: Perspectives, Approaches, and Measures

Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni, Zheng Zhang, Lijun Yin, and Qiang Ji

Transactions on Biometrics, Behavior, and Identity Science (TBIOM), 2020.

D-PAttNet: Dynamic Patch-Attentive Deep Network for Action Unit Detection

Itir Onal Ertugrul, Le Yang, Laszlo A. Jeni, and Jeffrey F. Cohn

Frontiers in Computer Science, 2020.

AFAR: A Deep Learning Based Tool for Automated Facial Affect Recognition

Le Yang, Itir Onal Ertugrul, Laszlo, A. Jeni, Wanqiao Ding, Jeffrey F. Cohn

International conference on automatic face & gesture recognition(FG), 2019.

Code

Cross-domain AU Detection: Domains, Learning Approaches, and Measures

Itir Onal Ertugrul, Jeffrey F. Cohn, Laszlo A. Jeni, Zheng Zhang, Lijun Yin, and Qiang Ji

International conference on automatic face & gesture recognition(FG), 2019.

FACS3D-Net: 3D Convolution based Spatiotemporal Representation for Action Unit Detection

Le Yang, Itir Onal Ertugrul, Jeffrey F. Cohn, Zakia Hammal, Dongmei Jiang, Hichem Sahli

International Conference on Affective Computing and Intelligent Interaction (ACII), 2019.

PAttNet: Patch-attentive deep network for action unit detection

Itir Onal Ertugrul, Laszlo A. Jeni, and Jeffrey F. Cohn

British Machine Vision Conference (BMVC), 2019.

Code

FACSCaps: Pose-Independent Facial Action Coding With Capsules

Itir Onal Ertugrul, Laszlo A. Jeni, and Jeffrey F. Cohn

IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018.

What will your future child look like? Modeling and Synthesis of Hereditary Patterns of Facial Dynamics

Itir Onal Ertugrul, Hamdi Dibeklioglu

International conference on automatic face & gesture recognition(FG), 2017.