Run Llama 3 In Visual Studio Code For FREE : A Step-By-Step Guide

This is a step-by-step guide on how to install and run Meta's most capable language model, Llama 3 on your local machine.

Meta released the most capable open-source language model, Llama 3, yesterday. Because it is open-source, you can download the model weights and run them locally on your own machine.

I know, I know. The thought of running an 8 billion parameter AI model on your laptop might sound like it’s only for tech-savvy people. But don’t worry! In this article, I will share a step-by-step guide that makes it easy for anyone to do.

Prerequisites

Before getting into the actual steps, it’s important to note the specifications of the environment that I am currently running on:

• Laptop: Lenovo ThinkPad X1 Extreme

• OS: Windows 11 Pro Version 10.0.22631 Build 22631

• CPU: Processor Intel(R) Core(TM) i7–9850H 

• RAM: 32 GB

• Disk space: 642 GB

That’s right! You don’t need to have a high-end GPU to run the model locally. With a decent CPU and enough RAM, you can run Llama 3 on your own machine without any issues.

Step 1: Download and install Ollama

Head over to the Ollama website and download the latest version of the installer. Ollama is a versatile tool designed to run, create, and share large language models (LLMs) locally on various platforms.

Image by Jim Clyde Monge

After installing Ollama, make sure that it is running in the background. You can check this by looking for the Ollama icon in your system tray or task manager.

Image by Jim Clyde Monge

To confirm that Ollama is working properly in the command line interface (CLI), run the following command to check the version. The current version I am running is 0.1.32, so it might be different on your side.

> ollama -v
ollama version is 0.1.32

Step 2: Install CodeGPT extension in VS Code

Next, open Visual Studio Code and go to the extensions tab. Search for “CodeGPT” from codegpt.co and install the extension. This extension will allow you to use Llama 3 directly within VS Code.

Image by Jim Clyde Monge

Once the extension is installed, you should see the CodeGPT icon on the left sidebar of VS Code.

Step 3: Download the model

Open the terminal in VS Code and run the following command to download the Llama 3 model:

ollama pull llama3:8b

This might take a while to finish because the model size is more than 4GB. Be patient and let it complete. Once done, you should see a success message like this:

Image by Jim Clyde Monge

Step 4: Select the Llama 3 model

On the CodeGPT dashboard in the left panel of VS Code, find the Provider dropdown menu and choose Ollama. Then, in the model dropdown, select “Llama3:8b”. If the model doesn’t show up in the list, you can also type “Llama3:8b” manually.