Self-Instantiated Recurrent Units with Dynamic Soft Recursion

TL;DR: We propose the self-instantiated recurrent unit that is characterized by recursive instantiation of the model itself, where the extent of the recursion may vary temporally.
Abstract: While standard recurrent neural networks explicitly impose a chain structure on different forms of data, they do not have an explicit bias towards recursive self-instantiation where the extent of recursion is dynamic. Given diverse and even growing data modalities (e.g., logic, algorithmic input and output, music, code, images, and language) that can be expressed in sequences and may benefit from more architectural flexibility, we propose the self-instantiated recurrent unit (Self-IRU) with a novel inductive bias towards dynamic soft recursion. On one hand, theSelf-IRU is characterized by recursive self-instantiation via its gating functions, i.e., gating mechanisms of the Self-IRU are controlled by instances of the Self-IRU itself, which are repeatedly invoked in a recursive fashion. On the other hand, the extent of the Self-IRU recursion is controlled by gates whose values are between 0 and 1 and may vary across the temporal dimension of sequences, enabling dynamic soft recursion depth at each time step. The architectural flexibility and effectiveness of our proposed approach are demonstrated across multiple data modalities. For example, the Self-IRU achieves state-of-the-art performance on the logical inference dataset even when comparing with competitive models that have access to ground-truth syntactic information.

RNA Alternative Splicing Prediction with Discrete Compositional Energy Network

TL;DR: We construct an RNA alternative splicing regression dataset (CAPD) and propose DCEN to predict splicing outcomes by modeling mRNA transcript probabilities through its constituent splice junctions’ energy.
Abstract: A single gene can encode for different protein versions through a process called alternative splicing. Since proteins play major roles in cellular functions, aberrant splicing profiles can result in a variety of diseases, including cancers. Alternative splicing is determined by the gene’s primary sequence and other regulatory factors such as RNA-binding protein levels. With these as input, we formulate the prediction of RNA splicing as a regression task and build a new training dataset (CAPD) to benchmark learned models. We propose discrete compositional energy network (DCEN) which leverages the hierarchical relationships between splice sites, junctions and transcripts to approach this task. In the case of alternative splicing prediction, DCEN models mRNA transcript probabilities through its constituent splice junctions’ energy values. These transcript probabilities are subsequently mapped to relative abundance values of key nucleotides and trained with ground-truth experimental measurements. Through our experiments on CAPD, we show that DCEN outperforms baselines and ablation variants.

Deep learning for fabrication and maturation of 3D bioprinted tissues and organs

TL;DR: Perceptive paper on how deep learning can improve 3D bioprinting.
Abstract: Bioprinting is a relatively new and promising tissue engineering approach to solve the problem of donor shortage for organ transplantation. It is a highly-advanced biofabrication system that enables the printing of materials in the form of biomaterials, living cells and growth factors in a layer-by-layer manner to manufacture 3D tissue-engineered constructs. The current workflow involves a myriad of manufacturing complexities, from medical image processing to optimisation of printing parameters and refinements during post-printing tissue maturation. Deep learning is a powerful machine learning technique that has fuelled remarkable progress in image and language applications over the past decade. In this perspective paper, we highlight the integration of deep learning into 3D bioprinting technology and the implementation of practical guidelines. We address potential adoptions of deep learning into various 3D bioprinting processes such as image-processing and segmentation, optimisation and in-situ correction of printing parameters and lastly refinement of the tissue maturation process. Finally, we discuss implications that deep learning has on the adoption and regulation of 3D bioprinting. The synergistic interactions among the field of biology, material and deep learning-enabled computational design will eventually facilitate the fabrication of biomimetic patient-specific tissues/organs, making 3D bioprinting of tissues/organs an impending reality.

Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networks

TL;DR: We propose a comprehensive defense to detect and neutralize backdoor poisoning attacks of different sizes.
Abstract: Deep learning models have recently shown to be vulnerable to backdoor poisoning, an insidious attack where the victim model predicts clean images correctly but classifies the same images as the target class when a trigger poison pattern is added. This poison pattern can be embedded in the training dataset by the adversary. Existing defenses are effective under certain conditions such as a small size of the poison pattern, knowledge about the ratio of poisoned training samples or when a validated clean dataset is available. Since a defender may not have such prior knowledge or resources, we propose a defense against backdoor poisoning that is effective even when those prerequisites are not met. It is made up of several parts: one to extract a backdoor poison signal, detect poison target and base classes, and filter out poisoned from clean samples with proven guarantees. The final part of our defense involves retraining the poisoned model on a dataset augmented with the extracted poison signal and corrective relabeling of poisoned samples to neutralize the backdoor. Our approach has shown to be effective in defending against backdoor attacks that use both small and large-sized poison patterns on nine different target-base class pairs from the CIFAR10 dataset.