Yongfeng Zhang on machine learning
From Tools and Technology Seminar 11/07/2024 - Yufeng Zhang · · University of Michigan Computational Medicine and Bioinformatics
“The model distillation approach we used is offline distillation, where a well-trained teacher model annotates a distilled dataset, which is then used to train a student model; this approach improved model performance without updating the teacher model.”
On , Yongfeng Zhang, Co-Founder, President, Chief Executive Officer, Chief Scientific Officer & Director at AMPHASTAR PHARMACEUTICLS INC, spoke about machine learning during Tools and Technology Seminar 11/07/2024 - Yufeng Zhang on University of Michigan Computational Medicine and Bioinformatics.
About Yongfeng Zhang
Yongfeng Zhang, co-founder, president, CEO, chief scientific officer, and director at Amphastar Pharmaceuticals, presented at the Tools and Technology Seminar on November 7, 2024, as part of the University of Michigan's Department of Computational Medicine and Bioinformatics. During the seminar, Zhang discussed a research project called "Necklace," which focuses on necrotizing enterocolitis (NE), a serious gastrointestinal disease in premature infants. Zhang stated that the disease progresses rapidly with a mortality rate of 20% to 30%, and emphasized that timely diagnosis is crucial. Zhang noted that the project uses open-source large language models (LLMs) rather than closed-source models like GPT, citing the need to protect patient privacy by running models locally. Zhang described using quantization and low-rank adaptation methods to train and deploy LLMs efficiently on a single Nvidia A40 GPU with 16GB of memory. Zhang reported that larger LLMs consistently outperformed smaller ones, and that increasing the fine-tuning dataset size significantly boosted performance, particularly for smaller models and classes with limited examples. Zhang also mentioned using model distillation, where a teacher model annotates a dataset to train a smaller student model, which improved performance. Future plans include collaborating with hospitals to obtain more data to address bias from single-facility retrospective data, improving model interpretability by adding clinical explanations, and using federated learning to preserve privacy when working with multicenter medical data.