NVIDIA Deep Learning Institute Collaborates with Dartmouth to Build New Generative AI Teaching Kit

Sam Raymond, adjunct assistant professor of engineering at Dartmouth. (Photo by Catha Mayor)

This comprehensive teaching resource enables educators to provide students access to cutting-edge tools, frameworks, and practical exercises that are crucial for understanding the complexities of Generative AI and large language model development and deployment. By equipping students with a deep understanding of generative AI techniques, the Teaching Kit enables educators to foster future innovation and creativity in AI-driven industries. 

As students transition into the workforce, they will be better prepared to tackle global challenges, from improving healthcare and science to advancing sustainable technologies.

Sam Raymond, adjunct assistant professor of engineering at Dartmouth, was instrumental in developing the content. "Empowering students with skills to understand and potentially develop their own GPU-accelerated Generative AI applications is the primary objective," said Raymond. "I believe students who go through this course will be at a significant advantage in the job market and help bridge the knowledge gap in industries today."

As generative AI models, such as GANs and transformers, become increasingly sophisticated, there is a growing demand for skilled professionals who can develop, refine, and ethically deploy these technologies. A strong educational foundation in generative AI equips students with the practical skills and theoretical knowledge needed to innovate in areas like content creation, drug discovery, and autonomous systems. 

College and university education in generative AI is crucial due to the rapidly expanding role of AI in almost every industry. By integrating generative AI into their curriculum, universities prepare the next generation of AI researchers, engineers, and thought leaders to advance the field and address the complex challenges associated with AI-driven innovation. 

About the Generative AI Teaching Kit

The aim of the Generative AI Teaching Kit is to introduce the foundational concepts of natural language processing (NLP) that are essential for understanding LLMs and generative AI more broadly. Key concepts of LLMs are then examined using NVIDIA GPUs, tools, and services, as well as open-source libraries and frameworks. A simple pretraining exercise of a GPT model shows basic training processes in the cloud. 

The kit also covers diffusion models to explore the application of generative AI in image and video generation. Multi-modal LLM architectures are then introduced, with a focus on optimizing various LLM architectures during fine-tuning using the NVIDIA NeMo framework. Advancements in inference and the refinement of tools like chatbots are also discussed, using NVIDIA NIM, NeMo Guardrails, TensorRT, and TensorRT-LLM to enhance efficiency and scalability in production environments.

The Generative AI Teaching Kit contains focused modules that combine theory, algorithms, programming, and examples:

• Module 1: Introduction to Generative AI
• Module 2: Word Embeddings, Tokens, and NLP
• Module 3: Large Language Models and the Transformer
• Module 4: LLM Scaling Laws and LLM Families
• Module 5: Multimodal Learning and Its Applications
• Module 6: Diffusion Models in Generative AI
• Module 7: Model Training (Pre-Training, Instruction Following, and PEFT)
• Module 8: LLM Orchestration
• Module 9: Scaling Model Training to Distributed Workloads

Through a collaboration with Google, educators also get access to free Google Colab credits to run the lab notebooks in preparation for teaching the course.

This content is valuable for educators across various fields, especially in computer science and engineering. Its modular design enables instructors to tailor the course to meet the specific needs of their students and create a customized learning experience.

Educators can access the first release of the Generative AI Teaching Kit and other kits for free by joining the NVIDIA DLI Teaching Kit Program.