I am Rakshith, a PhD Student in Computer Vision and Machine Learning!

I am a 4th year PhD student at the Max Planck Institute for Informatics. I am supervised by Prof. Dr. Bernt Schiele and Prof. Dr. Mario Fritz . My primary research interest is in designing robust computer vision models. My recent works have focused on developing automatic visual manipulation tools to synthesize rare hard examples. These tools enable efficiently creating corner cases which break the target perception model, allowing us to quantify and improve the robustness of these systems before real world deployment. I also have a strong interest in integrating vision and language, having worked on image captioning, visual question answering and text style transfer tasks.

News and Events

• Jul '20: New paper on semantic adversarial attacks for testing accepted at ECCV 2020 as an oral
• March '20: Towards Causal VQA accepted at CVPR 2020 and datasets released
• June '19: Not using car poster presented at CVPR 2020
• Dec '18: Presenting object removal work at Qualcomm headquarters at San Diego
• Aug '18: Presenting A⁴NT in USENIX 18 at Baltimore, MD
• May '18: I recieved the Qualcomm Innovation Fellowship to support my research
• Jul '17: My Masters Thesis on image and video captioning recieved three awards - from the Finnish society for Computer Science, Aalto Univeristy School of Science and Media industry research foundation

Publications

Towards automated testing and robustification by semantic adversarial data generation

Rakshith Shetty, Mario Fritz, Bernt Schiele

ECCV, 2020, Oral

Tldr: Semantic adversarial attack using a disentangled generator to find targeted hard samples to fool an object detector

PaperVideoBibtex

Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing

Vedika Agarwal, Rakshith Shetty, Mario Fritz

CVPR, 2020

Tldr: Quantifying spurious context dependence in VQA models through image editing

PaperProject pageBibtex

Not Using the Car to See the Sidewalk--Quantifying and Controlling the Effects of Context in Classification and Segmentation

Rakshith Shetty, Bernt Schiele, Mario Fritz

CVPR, 2019

Tldr: Measuring and mitigating over-reliance on context in classification and segmentation models

PaperPosterBibtex

Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

Rakshith Shetty Mario Fritz, Bernt Schiele,

NeurIPS, 2018

Tldr: Learning an object removal GAN using weak supervision from unpaired data

PaperPosterBibtex

A⁴NT: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation

Rakshith Shetty, Bernt Schiele, Mario Fritz

USENIX, 2018, Oral

Tldr: Learning to transfer writing style with adversarial training inorder to obfuscate private attributes

PaperTalkBibtex

Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training

Rakshith Shetty, Marcus Rohrbach, Lisa Anne Hendricks, Mario Fritz, Bernt Schiele

ICCV, 2017

Tldr: GAN based image captioning model to produce more diverse captions

PaperCodePosterBibtex

Paying attention to descriptions generated by image captioning models

Hamed R Tavakoli, Rakshith Shetty, Ali Borji, Jorma Laaksonen

ICCV, 2017

Tldr: Quantifying aggreement between object referrals in caption and visual saliency

PaperBibtex