Multimodal Artificial Intelligence and Large Language Models

The book provides a comprehensive technical analysis of multimodal artificial intelligence systems and implementation frameworks. It offers thorough coverage of cross-modal processing methods for use, including speech recognition and automatic image captioning.

• It presents a detailed discussion of architecture for integrating text, image, audio, and video modalities, cross-modal processing pipelines, and data fusion techniques.
• Showcases real-time synchronization mechanisms across different modalities and scalable design patterns for multimodal systems.
• Discusses multimodal emotion recognition using deep Learning techniques, focusing on recent advancements, challenges, and ethical considerations.
• Investigates deployment optimization strategies to address issues with latency, resource usage, and scalability of multimodal systems.
• Focuses on techniques for performance optimization, memory management, and distributed processing for multimodal workloads using frameworks like PyTorch and TensorFlow.

The text is primarily written for senior undergraduates, graduate students, and academic researchers in electrical engineering, electronics and communications engineering, computer science and engineering, and information technology.

L. Ashok Kumar is Principal at Thiagarajar College of Engineering, Madurai, Tamil Nadu, India. He was a Postdoctoral Research Fellow from San Diego State University, California. He has three years of industrial experience and twenty-three years of academic and research experience. He has published 173 technical papers in International and National journals and presented 167 papers in National and International Conferences. He has developed 27 products, and out of those, 23 products have been technology transferred to industries and for Government funding agencies. His areas of interest include wearable electronics, renewable energy systems, power electronics and drives, and smart grids.

D. Karthika Renuka is a Professor in the Department of Information Technology at PSG College of Technology, India. Her professional career of 20 years has been with PSG College of Technology since 2003. She is an Associate Dean (Students Welfare) and a convenor for the Students’ Welfare Committee at PSG College of Technology. She was a Postdoctoral Research Fellow from Wright State University, Ohio, USA. Her area of specialization includes data mining, evolutionary algorithms, soft computing, machine learning, deep learning, affective computing, and computer vision. She has published papers in reputable National and International journals and conferences.

Tanvi Banerjee is an Associate Professor in the Department of Computer Science and Engineering at Wright State University, USA, and has a secondary appointment at the Department of Geriatrics at the Boonshoft School of Medicine, Wright State University, USA. Her academic focus has been at the crossroads of artificial intelligence and healthcare, revolving around multimodal data fusion, wearable sensing, and mobile healthcare technologies. She has been a recipient of the prestigious K01 grant awarded by NIH (equivalent to the CAREER pathway in NSF) for the project on Dementia Management using smartphone technologies and a co-investigator in the NIH R01-funded Sickle Cell Disease project.

Deisy Chelliah is currently a Professor and the Head of the Information Technology Department at Thiagarajar College of Engineering in Madurai, Tamil Nadu, India. She has twenty-four years of teaching experience. Her areas of interest include machine learning, artificial intelligence, natural language processing, and data mining. She has published more than 90 research articles in journals and conferences.