Danny Jesus Diaz, PhD

I am a computational protein engineer. My research consists of developing sequence- and structure-based machine learning frameworks for identifying stabilizing and functional mutations in proteins. I collaborate extensively with experimental protein engineers to accelerate the developability and functionality of proteins for biotechnology applications.

I received my PhD in Chemistry under Dr Andrew Ellington, and Dr Eric Anslyn at the University of Texas at Austin. During my PhD, I was the primary developer of MutCompute: a machine learning as a service tool for structure-based ML-guided protein engineering. Currently, I lead the Deep Proteins Groups at the Institute for Foundations of Machine Learning (IFML). I co-founded Intelligent Proteins, LLC where we use machine learning-guided protein engineering to develop protein-based biotechnologies for nutraceutical, therapeutic, and biomanufacturing applications.

Email  /  CV  /  Bio  /  LinkedIn  /  Google Scholar  /  Twitter  /  Github

Research

I'm interested in protein engineering, machine learning, computer vision, biocatalysis, cancer metabolism, rare metabolic diseases, automation, and startup/entrepreneurship.

My research consist of developing machine learning frameworks that leverage sequence, structure, and/or functional data for enzyme discovery and protein engineering applications. Representative papers are highlighted.

Machine Learning Papers

It is critical to rapidly identify mutations with the potential for immune escape or increased disease burden (variants of concern). A recent study proposed that viral variants-of-concern can be identified using two quantities extracted from protein language models: grammaticality and semantic change. Grammaticality is intended to be a measure of whether a viral protein variant is viable, and semantic change is intended to be a measure of the variants potential for immune escape. Here, we systematically test this hypothesis, taking advantage of several high-throughput datasets that have become available since the original study, and also evaluating additional machine learning models for calculating the grammaticality and semantic metrics. We find that grammaticality correlates with protein viability, though the more traditional metric, ΔΔG, appears to be more effective. By contrast, we do not find compelling evidence that the semantic change metric can effectively identifying immune escape mutations.

Current, protein representation learning methods primarily rely on BERT- or GPT-style self-supervised learning and use wildtype accuracy as the primary training/validation metric. Wildtype accuracy, however, does not align with the primary goal of protein engineering: suggest beneficial mutations rather than to identify what already appears in nature. To address this gap between the pre-training objectives and protein engineering downstream tasks, we present Evolutionary Ranking (EvoRank): a training objective that incorporates evolutionary information derived from multiple sequence alignments (MSAs) to learn protein representations specific for protein engineering applications. Across a variety of phenotypes and datasets, we demonstrate that an EvoRank pre-trained graph-transformer (MutRank) results in significant zero-shot performance improvements that are competitive with ML frameworks fine-tuned on experimental data. This is particularly important in protein engineering, where it is expensive to obtain data for fine-tuning.

Stability Oracle is a structure-based graph-transformer framework that is first pre-trained with BERT-style self-supervision on the MutComputeX dataset and then fine-tuned on a curated subset of the megascale cDNA-display proteolysis dataset. Here, we introduce several innovations to overcome well-known challenges in data scarcity and bias, generalization, and computation time, such as: Thermodynamic Permutations for data augmentation, structural amino acid embeddings to model a mutation with a single structure, a protein structure-specific attention-bias mechanism that makes graph transformers a viable alternative to graph neural networks.

Fine-tuning machine learning frameworks to a small experimental dataset is prone to overfitting. Here, we present Binding Oracle: a Graph-Transformer framework that fine-tunes Stability Oracle to ∆∆G of binding for protein-protein interfaces (PPI) via a technique we call Selective LoRA. Selective LoRA, uses the gradient norms of each layer to select the subset most sensitive to the fine-tuning dataset--here it was a curated subset of Skempi2.0 (B1816)--and then finetunes the selected layers with LoRA. By applying Selective LoRA to Stability Oracle, we are able to achieve SOTA on the S487 PPI test set and generalization between different types of PPI interfaces.

The mutate everything framework allows the fine-tuning of sequence-based (ESM2) and MSA-based (AlphaFold2) protein foundation models on phenotype data with parallel decoding. Here, we demonstrate how their representations can be fine-tuned on the cDNA-display proteolysis dataset with the mutate everything framework to predict the thermodynamic impact of single point mutations (∆∆G). The AlphaFold2 fine-tuned model, StabilityFold, is able to achieve similar results to Stability Oracle on a variety test sets. More importantly, The mutate everything framework allows for parallel decoding of single and higher-order amino acid substitutions into ∆∆G predictions. This capability not only enables rapid DMS inferencing of proteins but makes double mutant DMS inferencing computationally tractable.

We curated an organism-based temperature dataset (HotProteins) for distinguishing proteins with varying thermostability (cryophiles, psychrophiles, mesophiles, thermophiles, and hyperthermophiles). We proposed structure-aware pretraining (SAP) and factorized sparse tuning (FST) to fine-tune the sequence-based transformer, ESM-1b, representations to generate a classifier and regressor to predict a protein's organism class or growth temperature.

We compare and contrast self-supervised sequence-based transformers and structure-based 3DCNNs models. We find that there is a variance-bias tradeoff between the two protein modalities. Convolutions provide an inductive bias for protein structures where the more powerful sequence-based transformers demonstrate increase variance.

We compare how self-supervised 3DCNNs learn the local mutational landscape of proteins against evolution via Multiple Sequence Alignments. We find that structure-based 3DCNNs amino acid likelihoods have weak correlation with MSAs and their wildtype confidence is dependent on the structural position of the residue. Where core residues being more confidently predicted.

Protein Papers

Previously, we engineered a ene-reductase (ERED) photoenzyme capable of asymmetric synthesis of α-chloroamides under blue light using directed evolution and machine learning-guided protein engineering with MutComputeX. Here, we conduct allosteric tuning of the electronic structure of the cofactor-substrate complex using directed evolution and MutComputeX to dramatically increase catalysis under red light (99% yields). Computational studies show a different electron transition for cyan and red light and how mutations at the protein surface allosterically tune the active site complex.

We engineered a transcription factor and a methyl transferase to improve the regioselectivity and titer yield for production of 4O-methyl-norbelladine. To engineer the 4O-methyltransferase enzyme, we developed MutComputeX: a self-supervised 3DResNet trained to generalize to protein-ligand, -nucleotide, and -protein interfaces. MutComputeX was used to design mutations on a computational ternary structure of the AlphaFolded methyl-transferase with SAH and norbelladine docked with Gnina. This is the first time three machine learning models (AlphaFold, Gnina, MutComputeX) were synergized to engineer the surface and active site of an enzyme and combined with an engineered transcription factor for high-throughput screening.

We engineered a Ene-reductase photoenzyme capable of asymmetric synthesis of α-chloroamides under blue light. MutComputeX was used to identify several mutation that improved the activity and stereoselectivity observed under blue light.

We utilized MutCompute to guide the engineering of a mesophilic and thermophilic PET hydrolases. We examined the ability of the mesophilic PET hydrolase (FAST-PETase) to depolymerize post-consumer PET waste. FAST-PETase was capable of depolymerizing ~50 post-consumer PET waste within 2-4 days. Furthermore, the ML designs increased the depolymerization capacity of the thermophilic PET hydrolase (ICCM) by 100%. Here is a time-lapse video of the depolymerization of a full PET container from Walmart.

Bst Polymerase was stabilized via ML-guided protein engineering in order to shorten the diagnostic time of LAMP-OSD assays during the height of the COVID19 pandemic. LAMP-OSD is an isothermal DNA amplification technique that enables field diagnostic of COVID19 in poor resource settings.

A database for ligand-induced transcription factor. These transcription factor serve as starting points for high-throughput screening genetic biosensors.

The development and initial experimental validation of the MutCompute framework: a self-supervised 3DCNN trained on the local chemistry surrounding an amino acid. The model was experimentally characterized for its ability to identify residues where the wildtype amino acid is incongruent for its surrounding chemical environment (protein only) and primed for gain-of-function. Here, BFP, phosphomannose isomerase, and beta-lactamase were engineered via machine learning.

Reviews

A review on the state-of-the-art machine learning frameworks (as of July 2022) for characterizing the functional and stability effects of point mutations on proteins.

A perspective on the future technology developments of differential sensing.

Patents

ML-engineered PETase variants, inluding the FAST-PETase varaint. Discovery of the N233K ML-design that replaces Ca2+ dependence for stability and activity with a lysine cation.

ML-engineered Cutinase variant, which makes use of the N233K ML-design that replaces Ca2+ dependence for stability and activity with a lysine cation.

ML-engineered variants of the human serine dehydratase enzyme for the treatment of luminal breast cancer.

ML-engineered Bst DNA Polymerase variants with increased solubility and stability for LAMP-OSD assays for isothermal COVID19 diagnostics.

ML-engineered Norbelladine 4O-MethylTransferase variants for biomanufacturing of Galantamine intermediate 4O-methyl-norbelladine. First, ML-engineered enzyme where the ML-designs were conditioned on the active site ligand and cofactor. ML-designs were generated using a computational ternary structure: protein (AlphaFold2), SAH cofactor (GNINA-docking), and norbelladine substrate (GNINA-docking).

Website source code was taken and modified from Jon Barron's website.