Using LLAMA3

Overview

Teaching: 20 min
Exercises: 0 min

Questions

How to use LLAMA3

Objectives

Using LLAMA3 on SuperPOD

LLAMA

Introduced by META in Introducing Meta Llama 3: The most capable openly available LLM to date
Several version:
- LLaMA1: released early 2023, was designed to be smaller and more efficient than other large models like GPT-3, while maintaining competitive performance. It was released in various sizes, such as 7B, 13B, 30B, and 65B parameters.
- LLaMA2: released in July 2023, LLaMA 2 improved upon the original version with enhanced performance, training techniques, and increased scalability. It also includes versions with 7B, 13B, and 70B parameters. Meta open-sourced LLaMA 2, and it was made available for both research and commercial use.
- LLaMA3: released in Apr 2024 and came with 3 versions 8B, 70B and 405B parameters

Models:

All LLaMA models can be found from the HuggingFace:

How to use LLaMA3 on SuperPOD

In order to use LLaMA (any version) on SuperPOD, we will use the pytorch_1.13 conda environment created in Chapter 2 and use port-forwarding for JupyterLab as in Chapter 4

Step 1: Request a compute node & Load the conda environment

Once logged in to SuperPOD, let’s request a compute node and load the conda environment:
We will request for a node with 1 GPU:

$ srun -N1 -c10 -G1 --mem=64gb --time=12:00:00 --pty $SHELL
$ module load conda gcc/11.2.0
$ module load cuda/11.8.0-vbvgppx cudnn
$ conda activate ~/pytorch_1.13
$ jupyter lab --ip=0.0.0.0 --no-browser --allow-root

Following are the instruction on how to use SuperPOD to run LLaMA3 on SuperPOD

Step 2: Request LLaMA3 access

For a new HuggingFace account, in order to access the open source LLaMA3, you will need to agree the license:

Sometimes, it takes a day for you to get approval.

Step 3: Install HuggingFace Hub

Following the guideline here to install HuggingFace Hub into your SuperPOD home folder

Step 4: Create HuggingFace Token

Create a HuggingFace account and click on your account logo and choose setting:
Select Access Tokens:
Create new token:
Save token into your huggingface folder (created from Step 3). Note that a huggingface folder is hidden, so you need to use “.” in front of that folder for access