Gtnet: guided transformer network for detecting human-object interactions

A. S. M. Iftekhar; Satish Kumar; R. Austin McEver; Suya You; B. S. Manjunath

doi:10.1117/12.2663936

13 June 2023 Gtnet: guided transformer network for detecting human-object interactions

A. S. M. Iftekhar, Satish Kumar, R. Austin McEver, Suya You, B. S. Manjunath

Proceedings Volume 12527, Pattern Recognition and Tracking XXXIV; 125270Q (2023) https://doi.org/10.1117/12.2663936
Event: SPIE Defense + Commercial Sensing, 2023, Orlando, Florida, United States

Abstract

The human-object interaction (HOI) detection task refers to localizing humans, localizing objects, and predicting the interactions between each human-object pair. HOI is considered one of the fundamental steps in truly understanding complex visual scenes. For detecting HOI, it is important to utilize relative spatial configurations and object semantics to find salient spatial regions of images that highlight the interactions between human object pairs. This issue is addressed by the novel self-attention based guided transformer network, GTNet. GTNet encodes this spatial contextual information in human and object visual features via self-attention while achieving state of the art results on both the V-COCO¹ and HICO-DET² datasets. Code is available online^∗.

Conference Presentation

Citation Download Citation

A. S. M. Iftekhar, Satish Kumar, R. Austin McEver, Suya You, and B. S. Manjunath "Gtnet: guided transformer network for detecting human-object interactions", Proc. SPIE 12527, Pattern Recognition and Tracking XXXIV, 125270Q (13 June 2023); https://doi.org/10.1117/12.2663936

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available