avatarMB20261

Summary

The provided content is a comprehensive guide on installing Llama.cpp, a versatile framework for large language models, using pre-built binaries in a Windows WSL2 environment with Ubuntu 24.04 LTS.

Abstract

The article details the step-by-step process of installing Llama.cpp, a framework designed for ease of use and performance in handling large language models, from pre-built binaries. It emphasizes the convenience of this method for users who wish to bypass complex compilation processes. The installation is demonstrated in a Windows WSL2 environment with Ubuntu 24.04 LTS, outlining the necessary prerequisites for both CPU-only and GPU (CUDA) supported installations. The guide includes instructions for setting up the environment, installing Homebrew, and using it to install Llama.cpp along with its dependencies. After the installation, the article points users to command-line scripts available for use and suggests resources for validating the installation and exploring further use cases, including building Llama.cpp with GPU support, CPU-only builds, and customized Docker images.

Opinions

  • The author assumes that readers may not be familiar with core concepts of Llama.cpp and provides a link for a primer on the framework.
  • The guide is tailored for users who prefer a straightforward installation process without the need for manual compilation.
  • The use of Homebrew for installation is recommended for its simplicity and the management of dependencies it offers.
  • The article acknowledges the importance of performance and provides options for both CPU-only and GPU-accelerated installations.
  • It is suggested that Llama.cpp is a powerful and accessible tool for developers and researchers working with language models.
  • The author encourages further exploration and learning by providing links to additional resources and use cases.

LLM By Examples: Llama.cpp Installation from pre-built binary

Llama.cpp is a versatile and efficient framework designed to support large language models, providing an accessible interface for developers and researchers. This article will guide you through the installation process of Llama.cpp using pre-built binaries, making it easier for you to get started with minimal setup.

Installing Llama.cpp from pre-built binaries allows users to bypass complex compilation processes and focus on utilizing the framework for their projects. In this section, we will explore two scenarios: installing Llama.cpp for CPU-only systems and for those with GPU (CUDA) support. For CPU-only installation, the process is straightforward; simply download the appropriate binary, configure environment variables if necessary, and run the software. On the other hand, GPU installations leverage CUDA to significantly enhance performance for resource-intensive workloads. We will outline the prerequisites for each installation type, ensuring users have all necessary components ready. By the end of this guide, you will be well-equipped to leverage Llama.cpp in your language processing tasks.

If you don’t familiar with core concepts of Llama.cpp, take a look below link first.

Prepare for installation

To demonstrate the process of build and installation, we will use Windows WSL2 environment with Ubuntu 24.04 LTS. You could do the same in any ubuntu installation.

In case if you are interested in how to setup such environment, take a look below links:

Accoridng to Llama.cpp’s official website, we need install brew first.

wsluser@DESKTOP-QQS4GT7:~$ sudo apt-get install build-essential curl git
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
build-essential is already the newest version (12.10ubuntu1).
curl is already the newest version (8.5.0-2ubuntu10.4).
curl set to manually installed.
git is already the newest version (1:2.43.0-1ubuntu7.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
==> Checking for `sudo` access (which may request your password)...
==> This script will install:
/home/linuxbrew/.linuxbrew/bin/brew
/home/linuxbrew/.linuxbrew/share/doc/homebrew
/home/linuxbrew/.linuxbrew/share/man/man1/brew.1
/home/linuxbrew/.linuxbrew/share/zsh/site-functions/_brew
/home/linuxbrew/.linuxbrew/etc/bash_completion.d/brew
/home/linuxbrew/.linuxbrew/Homebrew
==> The following new directories will be created:
/home/linuxbrew/.linuxbrew/bin
/home/linuxbrew/.linuxbrew/etc
/home/linuxbrew/.linuxbrew/include
/home/linuxbrew/.linuxbrew/lib
/home/linuxbrew/.linuxbrew/sbin
/home/linuxbrew/.linuxbrew/share
/home/linuxbrew/.linuxbrew/var
/home/linuxbrew/.linuxbrew/opt
/home/linuxbrew/.linuxbrew/share/zsh
/home/linuxbrew/.linuxbrew/share/zsh/site-functions
/home/linuxbrew/.linuxbrew/var/homebrew
/home/linuxbrew/.linuxbrew/var/homebrew/linked
/home/linuxbrew/.linuxbrew/Cellar
/home/linuxbrew/.linuxbrew/Caskroom
/home/linuxbrew/.linuxbrew/Frameworks

Press RETURN/ENTER to continue or any other key to abort:
==> /usr/bin/sudo /usr/bin/install -d -o wsluser -g wsluser -m 0755 /home/linuxbrew/.linuxbrew
==> /usr/bin/sudo /bin/mkdir -p /home/linuxbrew/.linuxbrew/bin /home/linuxbrew/.linuxbrew/etc /home/linuxbrew/.linuxbrew/include /home/linuxbrew/.linuxbrew/lib /home/linuxbrew/.linuxbrew/sbin /home/linuxbrew/.linuxbrew/share /home/linuxbrew/.linuxbrew/var /home/linuxbrew/.linuxbrew/opt /home/linuxbrew/.linuxbrew/share/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions /home/linuxbrew/.linuxbrew/var/homebrew /home/linuxbrew/.linuxbrew/var/homebrew/linked /home/linuxbrew/.linuxbrew/Cellar /home/linuxbrew/.linuxbrew/Caskroom /home/linuxbrew/.linuxbrew/Frameworks
==> /usr/bin/sudo /bin/chmod ug=rwx /home/linuxbrew/.linuxbrew/bin /home/linuxbrew/.linuxbrew/etc /home/linuxbrew/.linuxbrew/include /home/linuxbrew/.linuxbrew/lib /home/linuxbrew/.linuxbrew/sbin /home/linuxbrew/.linuxbrew/share /home/linuxbrew/.linuxbrew/var /home/linuxbrew/.linuxbrew/opt /home/linuxbrew/.linuxbrew/share/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions /home/linuxbrew/.linuxbrew/var/homebrew /home/linuxbrew/.linuxbrew/var/homebrew/linked /home/linuxbrew/.linuxbrew/Cellar /home/linuxbrew/.linuxbrew/Caskroom /home/linuxbrew/.linuxbrew/Frameworks
==> /usr/bin/sudo /bin/chmod go-w /home/linuxbrew/.linuxbrew/share/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions
==> /usr/bin/sudo /bin/chown wsluser /home/linuxbrew/.linuxbrew/bin /home/linuxbrew/.linuxbrew/etc /home/linuxbrew/.linuxbrew/include /home/linuxbrew/.linuxbrew/lib /home/linuxbrew/.linuxbrew/sbin /home/linuxbrew/.linuxbrew/share /home/linuxbrew/.linuxbrew/var /home/linuxbrew/.linuxbrew/opt /home/linuxbrew/.linuxbrew/share/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions /home/linuxbrew/.linuxbrew/var/homebrew /home/linuxbrew/.linuxbrew/var/homebrew/linked /home/linuxbrew/.linuxbrew/Cellar /home/linuxbrew/.linuxbrew/Caskroom /home/linuxbrew/.linuxbrew/Frameworks
==> /usr/bin/sudo /bin/chgrp wsluser /home/linuxbrew/.linuxbrew/bin /home/linuxbrew/.linuxbrew/etc /home/linuxbrew/.linuxbrew/include /home/linuxbrew/.linuxbrew/lib /home/linuxbrew/.linuxbrew/sbin /home/linuxbrew/.linuxbrew/share /home/linuxbrew/.linuxbrew/var /home/linuxbrew/.linuxbrew/opt /home/linuxbrew/.linuxbrew/share/zsh /home/linuxbrew/.linuxbrew/share/zsh/site-functions /home/linuxbrew/.linuxbrew/var/homebrew /home/linuxbrew/.linuxbrew/var/homebrew/linked /home/linuxbrew/.linuxbrew/Cellar /home/linuxbrew/.linuxbrew/Caskroom /home/linuxbrew/.linuxbrew/Frameworks
==> /usr/bin/sudo /bin/mkdir -p /home/linuxbrew/.linuxbrew/Homebrew
==> /usr/bin/sudo /bin/chown -R wsluser:wsluser /home/linuxbrew/.linuxbrew/Homebrew
==> Downloading and installing Homebrew...
remote: Enumerating objects: 284292, done.
remote: Counting objects: 100% (683/683), done.
remote: Compressing objects: 100% (404/404), done.
remote: Total 284292 (delta 309), reused 573 (delta 250), pack-reused 283609 (from 1)
remote: Enumerating objects: 55, done.
remote: Counting objects: 100% (33/33), done.
remote: Total 55 (delta 33), reused 33 (delta 33), pack-reused 22 (from 1)
==> Updating Homebrew...
==> Downloading https://ghcr.io/v2/homebrew/portable-ruby/portable-ruby/blobs/sha256:bd08b92d6725f9216fc3c2458ffd174d5468d43d47dd0fcaeb5109e3008fd40b
################################################################################################################# 100.0%
==> Pouring portable-ruby-3.3.5.x86_64_linux.bottle.tar.gz
Warning: /home/linuxbrew/.linuxbrew/bin is not in your PATH.
  Instructions on how to configure your shell for Homebrew
  can be found in the 'Next steps' section below.
==> Installation successful!

==> Homebrew has enabled anonymous aggregate formulae and cask analytics.
Read the analytics documentation (and how to opt-out) here:
  https://docs.brew.sh/Analytics
No analytics data has been sent yet (nor will any be during this install run).

==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
  https://github.com/Homebrew/brew#donations

==> Next steps:
- Run these commands in your terminal to add Homebrew to your PATH:
    echo >> /home/wsluser/.bashrc
    echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /home/wsluser/.bashrc
    eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
- Install Homebrew's dependencies if you have sudo access:
    sudo apt-get install build-essential
  For more information, see:
    https://docs.brew.sh/Homebrew-on-Linux
- We recommend that you install GCC:
    brew install gcc
- Run brew help to get started
- Further documentation:
    https://docs.brew.sh

wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ (echo; echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"') >> /home/linuxconfig/.bashrc
-bash: /home/linuxconfig/.bashrc: No such file or directory
wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ echo >> /home/wsluser/.bashrc
wsluser@DESKTOP-QQS4GT7:~$ echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /home/wsluser/.bashrc
wsluser@DESKTOP-QQS4GT7:~$ eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ brew --version
Homebrew 4.4.2
wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ brew doctor
Your system is ready to brew.

wsluser@DESKTOP-QQS4GT7:~$ 

Then, let’s install the llama.cpp:

wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ brew install llama.cpp
==> Downloading https://ghcr.io/v2/homebrew/core/llama.cpp/manifests/3950
################################################################################################################# 100.0%
==> Fetching dependencies for llama.cpp: gmp, isl, mpfr, libmpc, lz4, xz, zlib, zstd, binutils, gcc, openblas, brotli, libnghttp2, ca-certificates, openssl@3, libssh2, rtmpdump, libunistring, libidn2, ncurses, libedit, krb5, readline, libxcrypt, sqlite, util-linux, openldap and curl
==> Downloading https://ghcr.io/v2/homebrew/core/gmp/manifests/6.3.0
################################################################################################################# 100.0%
==> Fetching gmp
==> Downloading https://ghcr.io/v2/homebrew/core/gmp/blobs/sha256:3dca3544faca889c7389a5fdbd2b5b00582c34a4e14607033573ad
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/isl/manifests/0.27
################################################################################################################# 100.0%
==> Fetching isl
==> Downloading https://ghcr.io/v2/homebrew/core/isl/blobs/sha256:25c8bf618d4e3c68c27eed634bd7695104ff5daa37246253aabce8
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/mpfr/manifests/4.2.1
################################################################################################################# 100.0%
==> Fetching mpfr
==> Downloading https://ghcr.io/v2/homebrew/core/mpfr/blobs/sha256:18857bac44d9f49faeb1d147146ba7fb420d5bf85076f69c68a86
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libmpc/manifests/1.3.1
################################################################################################################# 100.0%
==> Fetching libmpc
==> Downloading https://ghcr.io/v2/homebrew/core/libmpc/blobs/sha256:f6542ae5bcf643ca0c980c7000cde1585922e76be080b3cc342
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/lz4/manifests/1.10.0-1
################################################################################################################# 100.0%
==> Fetching lz4
==> Downloading https://ghcr.io/v2/homebrew/core/lz4/blobs/sha256:a8082c2e40dc6d63850f43ea8fa095e55adf18fb0f25ec66bcaee2
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/xz/manifests/5.6.3
################################################################################################################# 100.0%
==> Fetching xz
==> Downloading https://ghcr.io/v2/homebrew/core/xz/blobs/sha256:360e9e63603136e0a4af1c9d0a6c28429fca9008fa5210cc12c2934
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/zlib/manifests/1.3.1
################################################################################################################# 100.0%
==> Fetching zlib
==> Downloading https://ghcr.io/v2/homebrew/core/zlib/blobs/sha256:38f2469db2ce63b70855a98e5ee27b5b5a92874e52542cbdc0b23
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/zstd/manifests/1.5.6
################################################################################################################# 100.0%
==> Fetching zstd
==> Downloading https://ghcr.io/v2/homebrew/core/zstd/blobs/sha256:0e6ddbd4c969bb84261f12b759fb78a828d6f734c9e515793c6ac
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/binutils/manifests/2.43.1
################################################################################################################# 100.0%
==> Fetching binutils
==> Downloading https://ghcr.io/v2/homebrew/core/binutils/blobs/sha256:ab5913dd80970340737b8450c3d227f2e6ad07874c240a82d
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/gcc/manifests/14.2.0
################################################################################################################# 100.0%
==> Fetching gcc
==> Downloading https://ghcr.io/v2/homebrew/core/gcc/blobs/sha256:a86493a7d4c2f4507c49a8599e9464367b812f9d051aa63e8690d6
########                                                                                                            7.7%################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/openblas/manifests/0.3.28
################################################################################################################# 100.0%
==> Fetching openblas
==> Downloading https://ghcr.io/v2/homebrew/core/openblas/blobs/sha256:9ac6e26f577d71f531bd0f268616ad2aa845c3c2843154b62
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/brotli/manifests/1.1.0-1
################################################################################################################# 100.0%
==> Fetching brotli
==> Downloading https://ghcr.io/v2/homebrew/core/brotli/blobs/sha256:5a2e1cc12312a092b38e79952fd2232f564f2c64cda0f69e97a
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libnghttp2/manifests/1.63.0
################################################################################################################# 100.0%
==> Fetching libnghttp2
==> Downloading https://ghcr.io/v2/homebrew/core/libnghttp2/blobs/sha256:b72e73226c370de2ba1680f11e0ac5f69bd64235756c5be
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2024-09-24
################################################################################################################# 100.0%
==> Fetching ca-certificates
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/blobs/sha256:212f2576348d5f5797d8d3905eb70d0d9bf8829345
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.3.2
################################################################################################################# 100.0%
==> Fetching openssl@3
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/blobs/sha256:1c54baa903d258fba6b5aef6818c5f282681d371933aaf8c
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libssh2/manifests/1.11.1
################################################################################################################# 100.0%
==> Fetching libssh2
==> Downloading https://ghcr.io/v2/homebrew/core/libssh2/blobs/sha256:a307208b03d0761f7ea8c53a322ea09b0a60db96e3ef8688df
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/rtmpdump/manifests/2.4-20151223_3
################################################################################################################# 100.0%
==> Fetching rtmpdump
==> Downloading https://ghcr.io/v2/homebrew/core/rtmpdump/blobs/sha256:0ab7f054fd0b01975be1893437235c61e2761ed9fb54e7880
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libunistring/manifests/1.2
################################################################################################################# 100.0%
==> Fetching libunistring
==> Downloading https://ghcr.io/v2/homebrew/core/libunistring/blobs/sha256:416da7b6b8d158de4adccc0479df3b8cc6a532f39f71d
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libidn2/manifests/2.3.7
################################################################################################################# 100.0%
==> Fetching libidn2
==> Downloading https://ghcr.io/v2/homebrew/core/libidn2/blobs/sha256:2d94c867e00156a44644758c62895dd6d13538aff7f638ea59
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/ncurses/manifests/6.5
################################################################################################################# 100.0%
==> Fetching ncurses
==> Downloading https://ghcr.io/v2/homebrew/core/ncurses/blobs/sha256:7c49662d0f319baec24475d38210b2f9c754b2ec1d21a4a3ff
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libedit/manifests/20240808-3.1
################################################################################################################# 100.0%
==> Fetching libedit
==> Downloading https://ghcr.io/v2/homebrew/core/libedit/blobs/sha256:8e574e7c9d236e3c6d12b208934ebe3159e27b50bfd700f73f
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/krb5/manifests/1.21.3
################################################################################################################# 100.0%
==> Fetching krb5
==> Downloading https://ghcr.io/v2/homebrew/core/krb5/blobs/sha256:f5b4cafedb315e92a31a0a5d87e33f7826952bcc5c093ba65817c
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/readline/manifests/8.2.13
################################################################################################################# 100.0%
==> Fetching readline
==> Downloading https://ghcr.io/v2/homebrew/core/readline/blobs/sha256:099378b496dd58f6a0fdb09e4c32d2ccae5631c0b423c1df7
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/libxcrypt/manifests/4.4.36
################################################################################################################# 100.0%
==> Fetching libxcrypt
==> Downloading https://ghcr.io/v2/homebrew/core/libxcrypt/blobs/sha256:ad1c4b570d7a66046038c13345b54337d858a2db78dcfb7e
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/sqlite/manifests/3.46.1
################################################################################################################# 100.0%
==> Fetching sqlite
==> Downloading https://ghcr.io/v2/homebrew/core/sqlite/blobs/sha256:108cff91d8599c1d25f3097655c539c914cce311f19cbef2dd6
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/util-linux/manifests/2.40.2-1
################################################################################################################# 100.0%
==> Fetching util-linux
==> Downloading https://ghcr.io/v2/homebrew/core/util-linux/blobs/sha256:773c91eea7c86a3a5a18ae1b43a43c9346b190ccf7640bb
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/openldap/manifests/2.6.8
################################################################################################################# 100.0%
==> Fetching openldap
==> Downloading https://ghcr.io/v2/homebrew/core/openldap/blobs/sha256:8b49f676a7948a38bdc1b8089400628c95502330a0d256963
################################################################################################################# 100.0%
==> Downloading https://ghcr.io/v2/homebrew/core/curl/manifests/8.10.1
################################################################################################################# 100.0%
==> Fetching curl
==> Downloading https://ghcr.io/v2/homebrew/core/curl/blobs/sha256:4284c064a03c57efbcac375ce3c2df8718653eebbedbf54a7957d
################################################################################################################# 100.0%
==> Fetching llama.cpp
==> Downloading https://ghcr.io/v2/homebrew/core/llama.cpp/blobs/sha256:d7d991147659ca3ceaed82d2eb9b263114c82c74d270c89a
################################################################################################################# 100.0%
==> Installing dependencies for llama.cpp: gmp, isl, mpfr, libmpc, lz4, xz, zlib, zstd, binutils, gcc, openblas, brotli, libnghttp2, ca-certificates, openssl@3, libssh2, rtmpdump, libunistring, libidn2, ncurses, libedit, krb5, readline, libxcrypt, sqlite, util-linux, openldap and curl
==> Installing llama.cpp dependency: gmp
==> Downloading https://ghcr.io/v2/homebrew/core/gmp/manifests/6.3.0
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/70a72a71216843d66a953c06ff6337445ce9bc94fae9f0e301e2f59005274a8e--gmp-6.3.0.bottle_manifest.json
==> Pouring gmp--6.3.0.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/gmp/6.3.0: 24 files, 3.9MB
==> Installing llama.cpp dependency: isl
==> Downloading https://ghcr.io/v2/homebrew/core/isl/manifests/0.27
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/40b1c5526f95db33208143fa79887179e758121659d8877597f553e6e6188879--isl-0.27.bottle_manifest.json
==> Pouring isl--0.27.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/isl/0.27: 75 files, 9.9MB
==> Installing llama.cpp dependency: mpfr
==> Downloading https://ghcr.io/v2/homebrew/core/mpfr/manifests/4.2.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/a2a3424f4974f6febfa0334a93f35f508eaef3f4ad04320f73d9498302295635--mpfr-4.2.1.bottle_manifest.json
==> Pouring mpfr--4.2.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/mpfr/4.2.1: 32 files, 3.9MB
==> Installing llama.cpp dependency: libmpc
==> Downloading https://ghcr.io/v2/homebrew/core/libmpc/manifests/1.3.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/fdfa98e0f8bb3ce075cb32776ac2345aa2f89252706c162aecfc841085fa76be--libmpc-1.3.1.bottle_manifest.json
==> Pouring libmpc--1.3.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libmpc/1.3.1: 14 files, 642.9KB
==> Installing llama.cpp dependency: lz4
==> Downloading https://ghcr.io/v2/homebrew/core/lz4/manifests/1.10.0-1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/8e11e90eb21a06e0f199af9d80e011e3693c77dd353b2477579d95c8471a5802--lz4-1.10.0-1.bottle_manifest.json
==> Pouring lz4--1.10.0.x86_64_linux.bottle.1.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/lz4/1.10.0: 24 files, 798.7KB
==> Installing llama.cpp dependency: xz
==> Downloading https://ghcr.io/v2/homebrew/core/xz/manifests/5.6.3
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/e68799185cc17334108ba286ba8c3df0587c9fd8375647d85f7157b62a797599--xz-5.6.3.bottle_manifest.json
==> Pouring xz--5.6.3.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/xz/5.6.3: 97 files, 2MB
==> Installing llama.cpp dependency: zlib
==> Downloading https://ghcr.io/v2/homebrew/core/zlib/manifests/1.3.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/f68d0caf232d52f2aa586abefbbfd7e958e384d84f3967008fa83de94b5f10ae--zlib-1.3.1.bottle_manifest.json
==> Pouring zlib--1.3.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/zlib/1.3.1: 14 files, 476KB
==> Installing llama.cpp dependency: zstd
==> Downloading https://ghcr.io/v2/homebrew/core/zstd/manifests/1.5.6
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/29403e0df5404d8aeca0e750ac135ec9ef44fc5eeb6df69170ed602acabf0ffb--zstd-1.5.6.bottle_manifest.json
==> Pouring zstd--1.5.6.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/zstd/1.5.6: 32 files, 2.9MB
==> Installing llama.cpp dependency: binutils
==> Downloading https://ghcr.io/v2/homebrew/core/binutils/manifests/2.43.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/8111b6fd65822d749b89a471ec543eaf84471287fa7b13113cf3075efee5f0b6--binutils-2.43.1.bottle_manifest.json
==> Pouring binutils--2.43.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/binutils/2.43.1: 6,513 files, 394.5MB
==> Installing llama.cpp dependency: gcc
==> Downloading https://ghcr.io/v2/homebrew/core/gcc/manifests/14.2.0
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/cfbcae7c56a6f3ae28c3ab9e708e93f22e00d1d893bf26f656e7003d10024a38--gcc-14.2.0.bottle_manifest.json
==> Pouring gcc--14.2.0.x86_64_linux.bottle.tar.gz
==> Creating the GCC specs file: /home/linuxbrew/.linuxbrew/Cellar/gcc/14.2.0/bin/../lib/gcc/current/gcc/x86_64-pc-linux
🍺  /home/linuxbrew/.linuxbrew/Cellar/gcc/14.2.0: 2,105 files, 550.9MB
==> Installing llama.cpp dependency: openblas
==> Downloading https://ghcr.io/v2/homebrew/core/openblas/manifests/0.3.28
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/60a317040c885298623bef2b9693411b082cc4a1e706301e19c41d2b1eef9300--openblas-0.3.28.bottle_manifest.json
==> Pouring openblas--0.3.28.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/openblas/0.3.28: 24 files, 93.5MB
==> Installing llama.cpp dependency: brotli
==> Downloading https://ghcr.io/v2/homebrew/core/brotli/manifests/1.1.0-1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/565098509cdfa90593ec9a41b41239f073843f88caed60ddaa3e6194ac4defda--brotli-1.1.0-1.bottle_manifest.json
==> Pouring brotli--1.1.0.x86_64_linux.bottle.1.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/brotli/1.1.0: 25 files, 983.7KB
==> Installing llama.cpp dependency: libnghttp2
==> Downloading https://ghcr.io/v2/homebrew/core/libnghttp2/manifests/1.63.0
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/e63c5bc0f458327f00ee55e74895b72e64faad010393790efba6fc3e26689285--libnghttp2-1.63.0.bottle_manifest.json
==> Pouring libnghttp2--1.63.0.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libnghttp2/1.63.0: 15 files, 913.9KB
==> Installing llama.cpp dependency: ca-certificates
==> Downloading https://ghcr.io/v2/homebrew/core/ca-certificates/manifests/2024-09-24
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/338dad7c2ff7c822cda7c417944521589856741c0fbd7a7f07b88a18d7fb7e05--ca-certificates-2024-09-24.bottle_manifest.json
==> Pouring ca-certificates--2024-09-24.all.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/ca-certificates/2024-09-24: 4 files, 252.9KB
==> Installing llama.cpp dependency: openssl@3
==> Downloading https://ghcr.io/v2/homebrew/core/openssl/3/manifests/3.3.2
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/15a43beab5e5017b1a9fe7584ff44a48195632fd196c23ee0e5e7ecf67e9ef2a--openssl@3-3.3.2.bottle_manifest.json
==> Pouring [email protected]_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/openssl@3/3.3.2: 6,994 files, 38.9MB
==> Installing llama.cpp dependency: libssh2
==> Downloading https://ghcr.io/v2/homebrew/core/libssh2/manifests/1.11.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/6d2495eb2b5f37f7931b23940ceb29db14a4742a49d1d8cce26964ad91a03f26--libssh2-1.11.1.bottle_manifest.json
==> Pouring libssh2--1.11.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libssh2/1.11.1: 202 files, 1.5MB
==> Installing llama.cpp dependency: rtmpdump
==> Downloading https://ghcr.io/v2/homebrew/core/rtmpdump/manifests/2.4-20151223_3
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/b40cb4d6b301aad8ce26f9deb74a74c0d98d1868d19881f9bff55b81d6f11e50--rtmpdump-2.4-20151223_3.bottle_manifest.json
==> Pouring rtmpdump--2.4-20151223_3.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/rtmpdump/2.4-20151223_3: 21 files, 630.0KB
==> Installing llama.cpp dependency: libunistring
==> Downloading https://ghcr.io/v2/homebrew/core/libunistring/manifests/1.2
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/48ac60445a77a63996cf15f6414f68a620d544fb683031b14eb3aea95c3064f6--libunistring-1.2.bottle_manifest.json
==> Pouring libunistring--1.2.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libunistring/1.2: 61 files, 5.6MB
==> Installing llama.cpp dependency: libidn2
==> Downloading https://ghcr.io/v2/homebrew/core/libidn2/manifests/2.3.7
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/45d1d4d2930c4782bf53e761a1c0166cd8a40f4193ac8c44e86f0b6708e80354--libidn2-2.3.7.bottle_manifest.json
==> Pouring libidn2--2.3.7.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libidn2/2.3.7: 82 files, 1.2MB
==> Installing llama.cpp dependency: ncurses
==> Downloading https://ghcr.io/v2/homebrew/core/ncurses/manifests/6.5
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/b2f8ce4e08b15832435170efd265346225ee388d5baab3d5c50e3a77c5673b5a--ncurses-6.5.bottle_manifest.json
==> Pouring ncurses--6.5.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/ncurses/6.5: 4,037 files, 10.9MB
==> Installing llama.cpp dependency: libedit
==> Downloading https://ghcr.io/v2/homebrew/core/libedit/manifests/20240808-3.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/29a41913ba5a0184006508d5ff85032fddf36d6ff90f0dfd2faea711a1c12ac0--libedit-20240808-3.1.bottle_manifest.json
==> Pouring libedit--20240808-3.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libedit/20240808-3.1: 55 files, 795.6KB
==> Installing llama.cpp dependency: krb5
==> Downloading https://ghcr.io/v2/homebrew/core/krb5/manifests/1.21.3
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/c5793441ca90aa09420dddd84f19d4aaf86da4f0b3f60c84940d77b1cb4c6863--krb5-1.21.3.bottle_manifest.json
==> Pouring krb5--1.21.3.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/krb5/1.21.3: 164 files, 5.3MB
==> Installing llama.cpp dependency: readline
==> Downloading https://ghcr.io/v2/homebrew/core/readline/manifests/8.2.13
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/aa1afa38997a2866d91c81fdda8b36d436cd4ea7a82aed07d13c83c56eb3460e--readline-8.2.13.bottle_manifest.json
==> Pouring readline--8.2.13.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/readline/8.2.13: 51 files, 2MB
==> Installing llama.cpp dependency: libxcrypt
==> Downloading https://ghcr.io/v2/homebrew/core/libxcrypt/manifests/4.4.36
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/950e230307625f9e57d74f0076caab42b6c67a325c70b83efa2c9cc84be1f839--libxcrypt-4.4.36.bottle_manifest.json
==> Pouring libxcrypt--4.4.36.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/libxcrypt/4.4.36: 25 files, 371.8KB
==> Installing llama.cpp dependency: sqlite
==> Downloading https://ghcr.io/v2/homebrew/core/sqlite/manifests/3.46.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/b34f1b968c8d3adc8763b7151754bc4f348c94372d89d1ae23cd11b643b7bdb2--sqlite-3.46.1.bottle_manifest.json
==> Pouring sqlite--3.46.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/sqlite/3.46.1: 13 files, 6.2MB
==> Installing llama.cpp dependency: util-linux
==> Downloading https://ghcr.io/v2/homebrew/core/util-linux/manifests/2.40.2-1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/635c3a4838ab534b3fed900de735d2527a71bca53917fcc61cc683223c724e8a--util-linux-2.40.2-1.bottle_manifest.json
==> Pouring util-linux--2.40.2.x86_64_linux.bottle.1.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/util-linux/2.40.2: 443 files, 25.7MB
==> Installing llama.cpp dependency: openldap
==> Downloading https://ghcr.io/v2/homebrew/core/openldap/manifests/2.6.8
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/521e6d28774a296c05aaf8bccd4a248f5bbfe6ad48d824592473a62c51c48a0d--openldap-2.6.8.bottle_manifest.json
==> Pouring openldap--2.6.8.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/openldap/2.6.8: 344 files, 8.8MB
==> Installing llama.cpp dependency: curl
==> Downloading https://ghcr.io/v2/homebrew/core/curl/manifests/8.10.1
Already downloaded: /home/wsluser/.cache/Homebrew/downloads/af1e83d2a4319eea26a3b24697e09962fe4f9553fea673669e24cb5494696e5a--curl-8.10.1.bottle_manifest.json
==> Pouring curl--8.10.1.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/curl/8.10.1: 531 files, 4.9MB
==> Installing llama.cpp
==> Pouring llama.cpp--3950.x86_64_linux.bottle.tar.gz
🍺  /home/linuxbrew/.linuxbrew/Cellar/llama.cpp/3950: 115 files, 38.6MB
==> Running `brew cleanup llama.cpp`...
Disable this behaviour by setting HOMEBREW_NO_INSTALL_CLEANUP.
Hide these hints with HOMEBREW_NO_ENV_HINTS (see `man brew`).

wsluser@DESKTOP-QQS4GT7:~$ 

After build, you could find all command line scrips at /home/linuxbrew/.linuxbrew/bin/ directory:

wsluser@DESKTOP-QQS4GT7:~$ ls -l /home/linuxbrew/.linuxbrew/bin/llama-*
lrwxrwxrwx 1 wsluser wsluser 45 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-baby-llama -> ../Cellar/llama.cpp/3950/bin/llama-baby-llama
lrwxrwxrwx 1 wsluser wsluser 42 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-batched -> ../Cellar/llama.cpp/3950/bin/llama-batched
lrwxrwxrwx 1 wsluser wsluser 48 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-batched-bench -> ../Cellar/llama.cpp/3950/bin/llama-batched-bench
lrwxrwxrwx 1 wsluser wsluser 40 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-bench -> ../Cellar/llama.cpp/3950/bin/llama-bench
lrwxrwxrwx 1 wsluser wsluser 38 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-cli -> ../Cellar/llama.cpp/3950/bin/llama-cli
lrwxrwxrwx 1 wsluser wsluser 58 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-convert-llama2c-to-ggml -> ../Cellar/llama.cpp/3950/bin/llama-convert-llama2c-to-ggml
lrwxrwxrwx 1 wsluser wsluser 52 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-cvector-generator -> ../Cellar/llama.cpp/3950/bin/llama-cvector-generator
lrwxrwxrwx 1 wsluser wsluser 44 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-embedding -> ../Cellar/llama.cpp/3950/bin/llama-embedding
lrwxrwxrwx 1 wsluser wsluser 48 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-eval-callback -> ../Cellar/llama.cpp/3950/bin/llama-eval-callback
lrwxrwxrwx 1 wsluser wsluser 46 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-export-lora -> ../Cellar/llama.cpp/3950/bin/llama-export-lora
lrwxrwxrwx 1 wsluser wsluser 49 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-gbnf-validator -> ../Cellar/llama.cpp/3950/bin/llama-gbnf-validator
lrwxrwxrwx 1 wsluser wsluser 39 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-gguf -> ../Cellar/llama.cpp/3950/bin/llama-gguf
lrwxrwxrwx 1 wsluser wsluser 44 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-gguf-hash -> ../Cellar/llama.cpp/3950/bin/llama-gguf-hash
lrwxrwxrwx 1 wsluser wsluser 45 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-gguf-split -> ../Cellar/llama.cpp/3950/bin/llama-gguf-split
lrwxrwxrwx 1 wsluser wsluser 41 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-gritlm -> ../Cellar/llama.cpp/3950/bin/llama-gritlm
lrwxrwxrwx 1 wsluser wsluser 42 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-imatrix -> ../Cellar/llama.cpp/3950/bin/llama-imatrix
lrwxrwxrwx 1 wsluser wsluser 41 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-infill -> ../Cellar/llama.cpp/3950/bin/llama-infill
lrwxrwxrwx 1 wsluser wsluser 44 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-llava-cli -> ../Cellar/llama.cpp/3950/bin/llama-llava-cli
lrwxrwxrwx 1 wsluser wsluser 44 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-lookahead -> ../Cellar/llama.cpp/3950/bin/llama-lookahead
lrwxrwxrwx 1 wsluser wsluser 41 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-lookup -> ../Cellar/llama.cpp/3950/bin/llama-lookup
lrwxrwxrwx 1 wsluser wsluser 48 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-lookup-create -> ../Cellar/llama.cpp/3950/bin/llama-lookup-create
lrwxrwxrwx 1 wsluser wsluser 47 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-lookup-merge -> ../Cellar/llama.cpp/3950/bin/llama-lookup-merge
lrwxrwxrwx 1 wsluser wsluser 47 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-lookup-stats -> ../Cellar/llama.cpp/3950/bin/llama-lookup-stats
lrwxrwxrwx 1 wsluser wsluser 47 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-minicpmv-cli -> ../Cellar/llama.cpp/3950/bin/llama-minicpmv-cli
lrwxrwxrwx 1 wsluser wsluser 43 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-parallel -> ../Cellar/llama.cpp/3950/bin/llama-parallel
lrwxrwxrwx 1 wsluser wsluser 42 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-passkey -> ../Cellar/llama.cpp/3950/bin/llama-passkey
lrwxrwxrwx 1 wsluser wsluser 45 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-perplexity -> ../Cellar/llama.cpp/3950/bin/llama-perplexity
lrwxrwxrwx 1 wsluser wsluser 43 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-quantize -> ../Cellar/llama.cpp/3950/bin/llama-quantize
lrwxrwxrwx 1 wsluser wsluser 49 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-quantize-stats -> ../Cellar/llama.cpp/3950/bin/llama-quantize-stats
lrwxrwxrwx 1 wsluser wsluser 44 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-retrieval -> ../Cellar/llama.cpp/3950/bin/llama-retrieval
lrwxrwxrwx 1 wsluser wsluser 50 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-save-load-state -> ../Cellar/llama.cpp/3950/bin/llama-save-load-state
lrwxrwxrwx 1 wsluser wsluser 41 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-server -> ../Cellar/llama.cpp/3950/bin/llama-server
lrwxrwxrwx 1 wsluser wsluser 41 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-simple -> ../Cellar/llama.cpp/3950/bin/llama-simple
lrwxrwxrwx 1 wsluser wsluser 46 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-speculative -> ../Cellar/llama.cpp/3950/bin/llama-speculative
lrwxrwxrwx 1 wsluser wsluser 43 Oct 21 17:16 /home/linuxbrew/.linuxbrew/bin/llama-tokenize -> ../Cellar/llama.cpp/3950/bin/llama-tokenize

wsluser@DESKTOP-QQS4GT7:~$

What’s next?

Typically the next step is to validate the installation. Below link provides you not only the hello world use case, but most of modern common use cases.

If you are interested in building and installing Llama.cpp for different environment, check out below links:

Llama 3
Llama Cpp
Installation
AI
Generative Ai Tools
Recommended from ReadMedium