1

Can LLMs Reason Over Non-Text Modalities in a Training-Free Manner? A Case Study with In-Context Representation Learning
TL;DR: We introduce In-Context Representation Learning (ICRL), a training-free framework that enables large language models to …
Tianle Zhang, Wanlong Fang, Jonathan Woo, Paridhi Latawa, Deepak A. Subramanian, Alvin Chan*
PDF Cite
Designing Lipid Nanoparticles Using a Transformer-Based Neural Network
TL;DR:We present COMET, a transformer-based deep learning model trained on a large multi-component LNP dataset (LANCE) that predicts …
Alvin Chan*, Ameya R. Kirtane, Qing Rui Qu, Xisha Huang, Jonathan Woo, Deepak A. Subramanian, Rajib Dey, Rika Semalty, Joshua D. Bernstock, Taksim Ahmed, Rowan Honeywell, Charles Hanhurst, Isaac Diaz Becdach, Leah S. Prizant, Ashley K. Brown, Hao Song, Justin Law Cobb, Louis B. DeRidder, Bruna Santos, Miguel Jimenez, Michelle Sun, Yuebin Huang, Ceara Byrne, Giovanni Traverso
PDF Cite Code
Deep Extrapolation for Attribute-Enhanced Generation
TL;DR: How do we generate sequences that extrapolate beyond the training distribution?
Abstract: Attribute extrapolation in sample …
Alvin Chan*, Ali Madani*, Ben Krause, Nikhil Naik
PDF Cite Code
Self-Instantiated Recurrent Units with Dynamic Soft Recursion
TL;DR: We propose the self-instantiated recurrent unit that is characterized by recursive instantiation of the model itself, where the …
Aston Zhang, Yi Tay, Yikang Shen, Alvin Chan*, Shuai Zhang
PDF Cite
On Orthogonality Constraints for Transformers
TL;DR: Orthogonality constraints the encourages numerical stability improves model’s performance in NLP tasks.
Abstract: …
Aston Zhang, Alvin Chan*, Yi Tay, Jie Fu, Shuohang Wang, Shuai Zhang, Huajie Shao, Shuochao Yao, Roy Ka-Wei Lee
Cite
CoCon: A Self-Supervised Approach for Controlled Text Generation
TL;DR: We propose CoCon to control the content of text generation from LMs by conditioning on content inputs at an interleave layer. …
Alvin Chan*, Yew-Soon Ong, Bill Pung, Aston Zhang, Jie Fu
PDF Cite Code Video
Parameterization of Hypercomplex Multiplications
TL;DR: We propose a new parameterization of hypercomplex multiplications for architectural flexibility and effectiveness.
Abstract: …
Aston Zhang, Yi Tay, Shuai Zhang, Alvin Chan*, Anh Tuan Luu, Siu Cheung Hui, Jie Fu
PDF Cite
RNA Alternative Splicing Prediction with Discrete Compositional Energy Network
TL;DR: We construct an RNA alternative splicing regression dataset (CAPD) and propose DCEN to predict splicing outcomes by modeling …
Alvin Chan*, Anna Korsakova*, Yew-Soon Ong, Fernaldo Richtia Winnerdy, Kah Wai Lim, Anh Tuan Phan
PDF Cite Code Video
Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
TL;DR: We propose Conditional Adversarially Regularized Autoencoder to imbue poison signature and generate natural-looking poisoned …
Alvin Chan*, Tay Yi, Yew-Soon Ong, Aston Zhang
PDF Cite Code Video
What it Thinks is Important is Important: Robustness Transfers through Input Gradients
TL;DR: By regularizing for similar input gradients, we can transfer adversarial robustness from a teacher to a student classifier even …
Alvin Chan*, Yi Tay, Yew-Soon Ong
PDF Cite Code Video