LLM By Examples: Build Llama.cpp for CPU only
In the evolving landscape of artificial intelligence, Llama.cpp stands out as an efficient tool for working with large language models. This article focuses on guiding users through the simplest installation process for Llama.cpp on CPU-only environments, ensuring that enthusiasts and developers can seamlessly integrate and utilize this powerful resource.
Setting up Llama.cpp on a CPU-only environment is a straightforward process, suitable for users who may not have access to powerful GPUs but still wish to explore the capabilities of large language models. This approach is particularly beneficial for experimentation, local development, or educational purposes. Throughout this guide, we will focus on the necessary prerequisites, including software dependencies and system requirements. We’ll provide step-by-step instructions on obtaining the source code, configuring the build environment, and compiling the binaries for optimal performance on CPUs. By the end of this installation guide, readers will be equipped to run Llama.cpp effectively, paving the way for further exploration and experimentation with AI-driven applications.
If you don’t familiar with core concepts of Llama.cpp, take a look below link first.
Prepare for installation
To demonstrate the process of build and installation, we will use Windows WSL2 environment with Ubuntu 24.04 LTS. You could do the same in any ubuntu installation.
In case if you are interested in how to setup such environment, take a look below links:
Below is what we have for the installation:
Welcome to Ubuntu 24.04.1 LTS (GNU/Linux 5.15.153.1-microsoft-standard-WSL2 x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/pro
System information as of Mon Oct 21 16:43:57 EDT 2024
System load: 0.24 Processes: 41
Usage of /: 0.1% of 1006.85GB Users logged in: 0
Memory usage: 2% IPv4 address for eth0: 172.29.11.145
Swap usage: 0%
This message is shown once a day. To disable it please create the
/home/wsluser/.hushlogin file.
wsluser@DESKTOP-QQS4GT7:~$ ~Build and Installation
To build Llama.cpp, we will need:
- cmake and support libraries
- git, we will need clone the llama.cpp git repo
Now, let’s get started.
wsluser@DESKTOP-QQS4GT7:~$ sudo apt install build-essential git cmake libopenblas-dev libatlas-base-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
git is already the newest version (1:2.43.0-1ubuntu7.1).
git set to manually installed.
The following additional packages will be installed:
bzip2 cmake-data cpp cpp-13 cpp-13-x86-64-linux-gnu cpp-x86-64-linux-gnu dpkg-dev fakeroot g++ g++-13
g++-13-x86-64-linux-gnu g++-x86-64-linux-gnu gcc gcc-13 gcc-13-base gcc-13-x86-64-linux-gnu gcc-x86-64-linux-gnu
libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libaom3 libarchive13t64 libasan8
libatlas3-base libatomic1 libc-dev-bin libc-devtools libc6-dev libcc1-0 libcrypt-dev libde265-0 libdpkg-perl
libfakeroot libfile-fcntllock-perl libgcc-13-dev libgd3 libgfortran5 libgomp1 libheif-plugin-aomdec
libheif-plugin-aomenc libheif-plugin-libde265 libheif1 libhwasan0 libisl23 libitm1 libjsoncpp25 liblsan0 libmpc3
libopenblas-pthread-dev libopenblas0 libopenblas0-pthread libquadmath0 librhash0 libstdc++-13-dev libtsan2 libubsan1
libuv1t64 libxpm4 linux-libc-dev lto-disabled-list make manpages-dev rpcsvc-proto
Suggested packages:
bzip2-doc cmake-doc cmake-format elpa-cmake-mode ninja-build cpp-doc gcc-13-locales cpp-13-doc debian-keyring
g++-multilib g++-13-multilib gcc-13-doc gcc-multilib autoconf automake libtool flex bison gdb gcc-doc
gcc-13-multilib gdb-x86-64-linux-gnu lrzip libatlas-doc liblapack-doc glibc-doc bzr libgd-tools libheif-plugin-x265
libheif-plugin-ffmpegdec libheif-plugin-jpegdec libheif-plugin-jpegenc libheif-plugin-j2kdec libheif-plugin-j2kenc
libheif-plugin-rav1e libheif-plugin-svtenc libstdc++-13-doc make-doc
The following NEW packages will be installed:
build-essential bzip2 cmake cmake-data cpp cpp-13 cpp-13-x86-64-linux-gnu cpp-x86-64-linux-gnu dpkg-dev fakeroot g++
g++-13 g++-13-x86-64-linux-gnu g++-x86-64-linux-gnu gcc gcc-13 gcc-13-base gcc-13-x86-64-linux-gnu
gcc-x86-64-linux-gnu libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libaom3
libarchive13t64 libasan8 libatlas-base-dev libatlas3-base libatomic1 libc-dev-bin libc-devtools libc6-dev libcc1-0
libcrypt-dev libde265-0 libdpkg-perl libfakeroot libfile-fcntllock-perl libgcc-13-dev libgd3 libgfortran5 libgomp1
libheif-plugin-aomdec libheif-plugin-aomenc libheif-plugin-libde265 libheif1 libhwasan0 libisl23 libitm1
libjsoncpp25 liblsan0 libmpc3 libopenblas-dev libopenblas-pthread-dev libopenblas0 libopenblas0-pthread libquadmath0
librhash0 libstdc++-13-dev libtsan2 libubsan1 libuv1t64 libxpm4 linux-libc-dev lto-disabled-list make manpages-dev
rpcsvc-proto
0 upgraded, 67 newly installed, 0 to remove and 0 not upgraded.
Need to get 107 MB of archives.
After this operation, 457 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://archive.ubuntu.com/ubuntu noble/main amd64 libuv1t64 amd64 1.48.0-1.1build1 [97.3 kB]
Get:2 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libc-dev-bin amd64 2.39-0ubuntu8.3 [60.8 kB]
Get:3 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 linux-libc-dev amd64 6.8.0-47.47 [1698 kB]
Get:4 http://archive.ubuntu.com/ubuntu noble/main amd64 libcrypt-dev amd64 1:4.4.36-4build1 [112 kB]
Get:5 http://archive.ubuntu.com/ubuntu noble/main amd64 rpcsvc-proto amd64 1.4.2-0ubuntu7 [67.4 kB]
Get:6 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libc6-dev amd64 2.39-0ubuntu8.3 [2164 kB]
Get:7 http://archive.ubuntu.com/ubuntu noble/main amd64 gcc-13-base amd64 13.2.0-23ubuntu4 [49.0 kB]
Get:8 http://archive.ubuntu.com/ubuntu noble/main amd64 libisl23 amd64 0.26-3build1 [680 kB]
Get:9 http://archive.ubuntu.com/ubuntu noble/main amd64 libmpc3 amd64 1.3.1-1build1 [54.5 kB]
Get:10 http://archive.ubuntu.com/ubuntu noble/main amd64 cpp-13-x86-64-linux-gnu amd64 13.2.0-23ubuntu4 [11.2 MB]
Get:11 http://archive.ubuntu.com/ubuntu noble/main amd64 cpp-13 amd64 13.2.0-23ubuntu4 [1032 B]
Get:12 http://archive.ubuntu.com/ubuntu noble/main amd64 cpp-x86-64-linux-gnu amd64 4:13.2.0-7ubuntu1 [5326 B]
Get:13 http://archive.ubuntu.com/ubuntu noble/main amd64 cpp amd64 4:13.2.0-7ubuntu1 [22.4 kB]
Get:14 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libcc1-0 amd64 14.2.0-4ubuntu2~24.04 [48.0 kB]
Get:15 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libgomp1 amd64 14.2.0-4ubuntu2~24.04 [148 kB]
Get:16 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libitm1 amd64 14.2.0-4ubuntu2~24.04 [29.7 kB]
Get:17 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libatomic1 amd64 14.2.0-4ubuntu2~24.04 [10.5 kB]
Get:18 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libasan8 amd64 14.2.0-4ubuntu2~24.04 [3031 kB]
Get:19 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 liblsan0 amd64 14.2.0-4ubuntu2~24.04 [1322 kB]
Get:20 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libtsan2 amd64 14.2.0-4ubuntu2~24.04 [2772 kB]
Get:21 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libubsan1 amd64 14.2.0-4ubuntu2~24.04 [1184 kB]
Get:22 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libhwasan0 amd64 14.2.0-4ubuntu2~24.04 [1641 kB]
Get:23 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libquadmath0 amd64 14.2.0-4ubuntu2~24.04 [153 kB]
Get:24 http://archive.ubuntu.com/ubuntu noble/main amd64 libgcc-13-dev amd64 13.2.0-23ubuntu4 [2688 kB]
Get:25 http://archive.ubuntu.com/ubuntu noble/main amd64 gcc-13-x86-64-linux-gnu amd64 13.2.0-23ubuntu4 [21.9 MB]
Get:26 http://archive.ubuntu.com/ubuntu noble/main amd64 gcc-13 amd64 13.2.0-23ubuntu4 [482 kB]
Get:27 http://archive.ubuntu.com/ubuntu noble/main amd64 gcc-x86-64-linux-gnu amd64 4:13.2.0-7ubuntu1 [1212 B]
Get:28 http://archive.ubuntu.com/ubuntu noble/main amd64 gcc amd64 4:13.2.0-7ubuntu1 [5018 B]
Get:29 http://archive.ubuntu.com/ubuntu noble/main amd64 libstdc++-13-dev amd64 13.2.0-23ubuntu4 [2399 kB]
Get:30 http://archive.ubuntu.com/ubuntu noble/main amd64 g++-13-x86-64-linux-gnu amd64 13.2.0-23ubuntu4 [12.5 MB]
Get:31 http://archive.ubuntu.com/ubuntu noble/main amd64 g++-13 amd64 13.2.0-23ubuntu4 [14.5 kB]
Get:32 http://archive.ubuntu.com/ubuntu noble/main amd64 g++-x86-64-linux-gnu amd64 4:13.2.0-7ubuntu1 [964 B]
Get:33 http://archive.ubuntu.com/ubuntu noble/main amd64 g++ amd64 4:13.2.0-7ubuntu1 [1100 B]
Get:34 http://archive.ubuntu.com/ubuntu noble/main amd64 make amd64 4.3-4.1build2 [180 kB]
Get:35 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libdpkg-perl all 1.22.6ubuntu6.1 [269 kB]
Get:36 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 bzip2 amd64 1.0.8-5.1build0.1 [34.5 kB]
Get:37 http://archive.ubuntu.com/ubuntu noble/main amd64 lto-disabled-list all 47 [12.4 kB]
Get:38 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 dpkg-dev all 1.22.6ubuntu6.1 [1074 kB]
Get:39 http://archive.ubuntu.com/ubuntu noble/main amd64 build-essential amd64 12.10ubuntu1 [4928 B]
Get:40 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libarchive13t64 amd64 3.7.2-2ubuntu0.2 [381 kB]
Get:41 http://archive.ubuntu.com/ubuntu noble/main amd64 libjsoncpp25 amd64 1.9.5-6build1 [82.8 kB]
Get:42 http://archive.ubuntu.com/ubuntu noble/main amd64 librhash0 amd64 1.4.3-3build1 [129 kB]
Get:43 http://archive.ubuntu.com/ubuntu noble/main amd64 cmake-data all 3.28.3-1build7 [2155 kB]
Get:44 http://archive.ubuntu.com/ubuntu noble/main amd64 cmake amd64 3.28.3-1build7 [11.2 MB]
Get:45 http://archive.ubuntu.com/ubuntu noble/main amd64 libfakeroot amd64 1.33-1 [32.4 kB]
Get:46 http://archive.ubuntu.com/ubuntu noble/main amd64 fakeroot amd64 1.33-1 [67.2 kB]
Get:47 http://archive.ubuntu.com/ubuntu noble/main amd64 libalgorithm-diff-perl all 1.201-1 [41.8 kB]
Get:48 http://archive.ubuntu.com/ubuntu noble/main amd64 libalgorithm-diff-xs-perl amd64 0.04-8build3 [11.2 kB]
Get:49 http://archive.ubuntu.com/ubuntu noble/main amd64 libalgorithm-merge-perl all 0.08-5 [11.4 kB]
Get:50 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libaom3 amd64 3.8.2-2ubuntu0.1 [1941 kB]
Get:51 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libgfortran5 amd64 14.2.0-4ubuntu2~24.04 [916 kB]
Get:52 http://archive.ubuntu.com/ubuntu noble/universe amd64 libatlas3-base amd64 3.10.3-13ubuntu1 [3544 kB]
Get:53 http://archive.ubuntu.com/ubuntu noble/universe amd64 libatlas-base-dev amd64 3.10.3-13ubuntu1 [3827 kB]
Get:54 http://archive.ubuntu.com/ubuntu noble/main amd64 libheif-plugin-aomdec amd64 1.17.6-1ubuntu4 [10.3 kB]
Get:55 http://archive.ubuntu.com/ubuntu noble/main amd64 libde265-0 amd64 1.0.15-1build3 [166 kB]
Get:56 http://archive.ubuntu.com/ubuntu noble/main amd64 libheif-plugin-libde265 amd64 1.17.6-1ubuntu4 [8158 B]
Get:57 http://archive.ubuntu.com/ubuntu noble/main amd64 libheif1 amd64 1.17.6-1ubuntu4 [276 kB]
Get:58 http://archive.ubuntu.com/ubuntu noble/main amd64 libxpm4 amd64 1:3.5.17-1build2 [36.5 kB]
Get:59 http://archive.ubuntu.com/ubuntu noble/main amd64 libgd3 amd64 2.3.3-9ubuntu5 [128 kB]
Get:60 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libc-devtools amd64 2.39-0ubuntu8.3 [69.7 kB]
Get:61 http://archive.ubuntu.com/ubuntu noble/main amd64 libfile-fcntllock-perl amd64 0.22-4ubuntu5 [30.7 kB]
Get:62 http://archive.ubuntu.com/ubuntu noble/main amd64 libheif-plugin-aomenc amd64 1.17.6-1ubuntu4 [14.7 kB]
Get:63 http://archive.ubuntu.com/ubuntu noble/universe amd64 libopenblas0-pthread amd64 0.3.26+ds-1 [7175 kB]
Get:64 http://archive.ubuntu.com/ubuntu noble/universe amd64 libopenblas0 amd64 0.3.26+ds-1 [6168 B]
Get:65 http://archive.ubuntu.com/ubuntu noble/universe amd64 libopenblas-pthread-dev amd64 0.3.26+ds-1 [4900 kB]
Get:66 http://archive.ubuntu.com/ubuntu noble/universe amd64 libopenblas-dev amd64 0.3.26+ds-1 [19.6 kB]
Get:67 http://archive.ubuntu.com/ubuntu noble/main amd64 manpages-dev all 6.7-2 [2013 kB]
Fetched 107 MB in 3s (40.5 MB/s)
Extracting templates from packages: 100%
Selecting previously unselected package libuv1t64:amd64.
(Reading database ... 40787 files and directories currently installed.)
Preparing to unpack .../00-libuv1t64_1.48.0-1.1build1_amd64.deb ...
Unpacking libuv1t64:amd64 (1.48.0-1.1build1) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../01-libc-dev-bin_2.39-0ubuntu8.3_amd64.deb ...
Unpacking libc-dev-bin (2.39-0ubuntu8.3) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../02-linux-libc-dev_6.8.0-47.47_amd64.deb ...
Unpacking linux-libc-dev:amd64 (6.8.0-47.47) ...
Selecting previously unselected package libcrypt-dev:amd64.
Preparing to unpack .../03-libcrypt-dev_1%3a4.4.36-4build1_amd64.deb ...
Unpacking libcrypt-dev:amd64 (1:4.4.36-4build1) ...
Selecting previously unselected package rpcsvc-proto.
Preparing to unpack .../04-rpcsvc-proto_1.4.2-0ubuntu7_amd64.deb ...
Unpacking rpcsvc-proto (1.4.2-0ubuntu7) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../05-libc6-dev_2.39-0ubuntu8.3_amd64.deb ...
Unpacking libc6-dev:amd64 (2.39-0ubuntu8.3) ...
Selecting previously unselected package gcc-13-base:amd64.
Preparing to unpack .../06-gcc-13-base_13.2.0-23ubuntu4_amd64.deb ...
Unpacking gcc-13-base:amd64 (13.2.0-23ubuntu4) ...
Selecting previously unselected package libisl23:amd64.
Preparing to unpack .../07-libisl23_0.26-3build1_amd64.deb ...
Unpacking libisl23:amd64 (0.26-3build1) ...
Selecting previously unselected package libmpc3:amd64.
Preparing to unpack .../08-libmpc3_1.3.1-1build1_amd64.deb ...
Unpacking libmpc3:amd64 (1.3.1-1build1) ...
Selecting previously unselected package cpp-13-x86-64-linux-gnu.
Preparing to unpack .../09-cpp-13-x86-64-linux-gnu_13.2.0-23ubuntu4_amd64.deb ...
Unpacking cpp-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Selecting previously unselected package cpp-13.
Preparing to unpack .../10-cpp-13_13.2.0-23ubuntu4_amd64.deb ...
Unpacking cpp-13 (13.2.0-23ubuntu4) ...
Selecting previously unselected package cpp-x86-64-linux-gnu.
Preparing to unpack .../11-cpp-x86-64-linux-gnu_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking cpp-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package cpp.
Preparing to unpack .../12-cpp_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking cpp (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package libcc1-0:amd64.
Preparing to unpack .../13-libcc1-0_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libcc1-0:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libgomp1:amd64.
Preparing to unpack .../14-libgomp1_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libgomp1:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../15-libitm1_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libitm1:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libatomic1:amd64.
Preparing to unpack .../16-libatomic1_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libatomic1:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libasan8:amd64.
Preparing to unpack .../17-libasan8_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libasan8:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../18-liblsan0_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking liblsan0:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libtsan2:amd64.
Preparing to unpack .../19-libtsan2_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libtsan2:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../20-libubsan1_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libubsan1:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libhwasan0:amd64.
Preparing to unpack .../21-libhwasan0_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libhwasan0:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libquadmath0:amd64.
Preparing to unpack .../22-libquadmath0_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libquadmath0:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libgcc-13-dev:amd64.
Preparing to unpack .../23-libgcc-13-dev_13.2.0-23ubuntu4_amd64.deb ...
Unpacking libgcc-13-dev:amd64 (13.2.0-23ubuntu4) ...
Selecting previously unselected package gcc-13-x86-64-linux-gnu.
Preparing to unpack .../24-gcc-13-x86-64-linux-gnu_13.2.0-23ubuntu4_amd64.deb ...
Unpacking gcc-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Selecting previously unselected package gcc-13.
Preparing to unpack .../25-gcc-13_13.2.0-23ubuntu4_amd64.deb ...
Unpacking gcc-13 (13.2.0-23ubuntu4) ...
Selecting previously unselected package gcc-x86-64-linux-gnu.
Preparing to unpack .../26-gcc-x86-64-linux-gnu_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking gcc-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package gcc.
Preparing to unpack .../27-gcc_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking gcc (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package libstdc++-13-dev:amd64.
Preparing to unpack .../28-libstdc++-13-dev_13.2.0-23ubuntu4_amd64.deb ...
Unpacking libstdc++-13-dev:amd64 (13.2.0-23ubuntu4) ...
Selecting previously unselected package g++-13-x86-64-linux-gnu.
Preparing to unpack .../29-g++-13-x86-64-linux-gnu_13.2.0-23ubuntu4_amd64.deb ...
Unpacking g++-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Selecting previously unselected package g++-13.
Preparing to unpack .../30-g++-13_13.2.0-23ubuntu4_amd64.deb ...
Unpacking g++-13 (13.2.0-23ubuntu4) ...
Selecting previously unselected package g++-x86-64-linux-gnu.
Preparing to unpack .../31-g++-x86-64-linux-gnu_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking g++-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package g++.
Preparing to unpack .../32-g++_4%3a13.2.0-7ubuntu1_amd64.deb ...
Unpacking g++ (4:13.2.0-7ubuntu1) ...
Selecting previously unselected package make.
Preparing to unpack .../33-make_4.3-4.1build2_amd64.deb ...
Unpacking make (4.3-4.1build2) ...
Selecting previously unselected package libdpkg-perl.
Preparing to unpack .../34-libdpkg-perl_1.22.6ubuntu6.1_all.deb ...
Unpacking libdpkg-perl (1.22.6ubuntu6.1) ...
Selecting previously unselected package bzip2.
Preparing to unpack .../35-bzip2_1.0.8-5.1build0.1_amd64.deb ...
Unpacking bzip2 (1.0.8-5.1build0.1) ...
Selecting previously unselected package lto-disabled-list.
Preparing to unpack .../36-lto-disabled-list_47_all.deb ...
Unpacking lto-disabled-list (47) ...
Selecting previously unselected package dpkg-dev.
Preparing to unpack .../37-dpkg-dev_1.22.6ubuntu6.1_all.deb ...
Unpacking dpkg-dev (1.22.6ubuntu6.1) ...
Selecting previously unselected package build-essential.
Preparing to unpack .../38-build-essential_12.10ubuntu1_amd64.deb ...
Unpacking build-essential (12.10ubuntu1) ...
Selecting previously unselected package libarchive13t64:amd64.
Preparing to unpack .../39-libarchive13t64_3.7.2-2ubuntu0.2_amd64.deb ...
Unpacking libarchive13t64:amd64 (3.7.2-2ubuntu0.2) ...
Selecting previously unselected package libjsoncpp25:amd64.
Preparing to unpack .../40-libjsoncpp25_1.9.5-6build1_amd64.deb ...
Unpacking libjsoncpp25:amd64 (1.9.5-6build1) ...
Selecting previously unselected package librhash0:amd64.
Preparing to unpack .../41-librhash0_1.4.3-3build1_amd64.deb ...
Unpacking librhash0:amd64 (1.4.3-3build1) ...
Selecting previously unselected package cmake-data.
Preparing to unpack .../42-cmake-data_3.28.3-1build7_all.deb ...
Unpacking cmake-data (3.28.3-1build7) ...
Selecting previously unselected package cmake.
Preparing to unpack .../43-cmake_3.28.3-1build7_amd64.deb ...
Unpacking cmake (3.28.3-1build7) ...
Selecting previously unselected package libfakeroot:amd64.
Preparing to unpack .../44-libfakeroot_1.33-1_amd64.deb ...
Unpacking libfakeroot:amd64 (1.33-1) ...
Selecting previously unselected package fakeroot.
Preparing to unpack .../45-fakeroot_1.33-1_amd64.deb ...
Unpacking fakeroot (1.33-1) ...
Selecting previously unselected package libalgorithm-diff-perl.
Preparing to unpack .../46-libalgorithm-diff-perl_1.201-1_all.deb ...
Unpacking libalgorithm-diff-perl (1.201-1) ...
Selecting previously unselected package libalgorithm-diff-xs-perl:amd64.
Preparing to unpack .../47-libalgorithm-diff-xs-perl_0.04-8build3_amd64.deb ...
Unpacking libalgorithm-diff-xs-perl:amd64 (0.04-8build3) ...
Selecting previously unselected package libalgorithm-merge-perl.
Preparing to unpack .../48-libalgorithm-merge-perl_0.08-5_all.deb ...
Unpacking libalgorithm-merge-perl (0.08-5) ...
Selecting previously unselected package libaom3:amd64.
Preparing to unpack .../49-libaom3_3.8.2-2ubuntu0.1_amd64.deb ...
Unpacking libaom3:amd64 (3.8.2-2ubuntu0.1) ...
Selecting previously unselected package libgfortran5:amd64.
Preparing to unpack .../50-libgfortran5_14.2.0-4ubuntu2~24.04_amd64.deb ...
Unpacking libgfortran5:amd64 (14.2.0-4ubuntu2~24.04) ...
Selecting previously unselected package libatlas3-base:amd64.
Preparing to unpack .../51-libatlas3-base_3.10.3-13ubuntu1_amd64.deb ...
Unpacking libatlas3-base:amd64 (3.10.3-13ubuntu1) ...
Selecting previously unselected package libatlas-base-dev:amd64.
Preparing to unpack .../52-libatlas-base-dev_3.10.3-13ubuntu1_amd64.deb ...
Unpacking libatlas-base-dev:amd64 (3.10.3-13ubuntu1) ...
Selecting previously unselected package libheif-plugin-aomdec:amd64.
Preparing to unpack .../53-libheif-plugin-aomdec_1.17.6-1ubuntu4_amd64.deb ...
Unpacking libheif-plugin-aomdec:amd64 (1.17.6-1ubuntu4) ...
Selecting previously unselected package libde265-0:amd64.
Preparing to unpack .../54-libde265-0_1.0.15-1build3_amd64.deb ...
Unpacking libde265-0:amd64 (1.0.15-1build3) ...
Selecting previously unselected package libheif-plugin-libde265:amd64.
Preparing to unpack .../55-libheif-plugin-libde265_1.17.6-1ubuntu4_amd64.deb ...
Unpacking libheif-plugin-libde265:amd64 (1.17.6-1ubuntu4) ...
Selecting previously unselected package libheif1:amd64.
Preparing to unpack .../56-libheif1_1.17.6-1ubuntu4_amd64.deb ...
Unpacking libheif1:amd64 (1.17.6-1ubuntu4) ...
Selecting previously unselected package libxpm4:amd64.
Preparing to unpack .../57-libxpm4_1%3a3.5.17-1build2_amd64.deb ...
Unpacking libxpm4:amd64 (1:3.5.17-1build2) ...
Selecting previously unselected package libgd3:amd64.
Preparing to unpack .../58-libgd3_2.3.3-9ubuntu5_amd64.deb ...
Unpacking libgd3:amd64 (2.3.3-9ubuntu5) ...
Selecting previously unselected package libc-devtools.
Preparing to unpack .../59-libc-devtools_2.39-0ubuntu8.3_amd64.deb ...
Unpacking libc-devtools (2.39-0ubuntu8.3) ...
Selecting previously unselected package libfile-fcntllock-perl.
Preparing to unpack .../60-libfile-fcntllock-perl_0.22-4ubuntu5_amd64.deb ...
Unpacking libfile-fcntllock-perl (0.22-4ubuntu5) ...
Selecting previously unselected package libheif-plugin-aomenc:amd64.
Preparing to unpack .../61-libheif-plugin-aomenc_1.17.6-1ubuntu4_amd64.deb ...
Unpacking libheif-plugin-aomenc:amd64 (1.17.6-1ubuntu4) ...
Selecting previously unselected package libopenblas0-pthread:amd64.
Preparing to unpack .../62-libopenblas0-pthread_0.3.26+ds-1_amd64.deb ...
Unpacking libopenblas0-pthread:amd64 (0.3.26+ds-1) ...
Selecting previously unselected package libopenblas0:amd64.
Preparing to unpack .../63-libopenblas0_0.3.26+ds-1_amd64.deb ...
Unpacking libopenblas0:amd64 (0.3.26+ds-1) ...
Selecting previously unselected package libopenblas-pthread-dev:amd64.
Preparing to unpack .../64-libopenblas-pthread-dev_0.3.26+ds-1_amd64.deb ...
Unpacking libopenblas-pthread-dev:amd64 (0.3.26+ds-1) ...
Selecting previously unselected package libopenblas-dev:amd64.
Preparing to unpack .../65-libopenblas-dev_0.3.26+ds-1_amd64.deb ...
Unpacking libopenblas-dev:amd64 (0.3.26+ds-1) ...
Selecting previously unselected package manpages-dev.
Preparing to unpack .../66-manpages-dev_6.7-2_all.deb ...
Unpacking manpages-dev (6.7-2) ...
Setting up libaom3:amd64 (3.8.2-2ubuntu0.1) ...
Setting up manpages-dev (6.7-2) ...
Setting up lto-disabled-list (47) ...
Setting up libxpm4:amd64 (1:3.5.17-1build2) ...
Setting up libfile-fcntllock-perl (0.22-4ubuntu5) ...
Setting up libalgorithm-diff-perl (1.201-1) ...
Setting up libuv1t64:amd64 (1.48.0-1.1build1) ...
Setting up linux-libc-dev:amd64 (6.8.0-47.47) ...
Setting up libgomp1:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up bzip2 (1.0.8-5.1build0.1) ...
Setting up libfakeroot:amd64 (1.33-1) ...
Setting up fakeroot (1.33-1) ...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
Setting up rpcsvc-proto (1.4.2-0ubuntu7) ...
Setting up gcc-13-base:amd64 (13.2.0-23ubuntu4) ...
Setting up make (4.3-4.1build2) ...
Setting up libquadmath0:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libmpc3:amd64 (1.3.1-1build1) ...
Setting up libatomic1:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libjsoncpp25:amd64 (1.9.5-6build1) ...
Setting up libdpkg-perl (1.22.6ubuntu6.1) ...
Setting up libgfortran5:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libubsan1:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libhwasan0:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up librhash0:amd64 (1.4.3-3build1) ...
Setting up libcrypt-dev:amd64 (1:4.4.36-4build1) ...
Setting up libasan8:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up cmake-data (3.28.3-1build7) ...
Setting up libtsan2:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libisl23:amd64 (0.26-3build1) ...
Setting up libde265-0:amd64 (1.0.15-1build3) ...
Setting up libc-dev-bin (2.39-0ubuntu8.3) ...
Setting up libarchive13t64:amd64 (3.7.2-2ubuntu0.2) ...
Setting up libalgorithm-diff-xs-perl:amd64 (0.04-8build3) ...
Setting up libcc1-0:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up liblsan0:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libitm1:amd64 (14.2.0-4ubuntu2~24.04) ...
Setting up libalgorithm-merge-perl (0.08-5) ...
Setting up libopenblas0-pthread:amd64 (0.3.26+ds-1) ...
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in auto mode
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in auto mode
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so.0 to provide /usr/lib/x86_64-linux-gnu/libopenblas.so.0 (libopenblas.so.0-x86_64-linux-gnu) in auto mode
Setting up libatlas3-base:amd64 (3.10.3-13ubuntu1) ...
Setting up cpp-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Setting up libatlas-base-dev:amd64 (3.10.3-13ubuntu1) ...
update-alternatives: using /usr/lib/x86_64-linux-gnu/atlas/libblas.so to provide /usr/lib/x86_64-linux-gnu/libblas.so (libblas.so-x86_64-linux-gnu) in auto mode
update-alternatives: using /usr/lib/x86_64-linux-gnu/atlas/liblapack.so to provide /usr/lib/x86_64-linux-gnu/liblapack.so (liblapack.so-x86_64-linux-gnu) in auto mode
Setting up dpkg-dev (1.22.6ubuntu6.1) ...
Setting up libgcc-13-dev:amd64 (13.2.0-23ubuntu4) ...
Setting up libopenblas0:amd64 (0.3.26+ds-1) ...
Setting up cmake (3.28.3-1build7) ...
Setting up libc6-dev:amd64 (2.39-0ubuntu8.3) ...
Setting up libstdc++-13-dev:amd64 (13.2.0-23ubuntu4) ...
Setting up cpp-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Setting up libopenblas-pthread-dev:amd64 (0.3.26+ds-1) ...
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so to provide /usr/lib/x86_64-linux-gnu/libblas.so (libblas.so-x86_64-linux-gnu) in auto mode
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so to provide /usr/lib/x86_64-linux-gnu/liblapack.so (liblapack.so-x86_64-linux-gnu) in auto mode
update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblas.so to provide /usr/lib/x86_64-linux-gn
u/libopenblas.so (libopenblas.so-x86_64-linux-gnu) in auto mode
Setting up cpp-13 (13.2.0-23ubuntu4) ...
Setting up gcc-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Setting up libopenblas-dev:amd64 (0.3.26+ds-1) ...
Setting up gcc-13 (13.2.0-23ubuntu4) ...
Setting up cpp (4:13.2.0-7ubuntu1) ...
Setting up g++-13-x86-64-linux-gnu (13.2.0-23ubuntu4) ...
Setting up gcc-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Setting up gcc (4:13.2.0-7ubuntu1) ...
Setting up g++-x86-64-linux-gnu (4:13.2.0-7ubuntu1) ...
Setting up g++-13 (13.2.0-23ubuntu4) ...
Setting up g++ (4:13.2.0-7ubuntu1) ...
update-alternatives: using /usr/bin/g++ to provide /usr/bin/c++ (c++) in auto mode
Setting up build-essential (12.10ubuntu1) ...
Setting up libheif1:amd64 (1.17.6-1ubuntu4) ...
Setting up libgd3:amd64 (2.3.3-9ubuntu5) ...
Setting up libc-devtools (2.39-0ubuntu8.3) ...
Setting up libheif-plugin-aomdec:amd64 (1.17.6-1ubuntu4) ...
Setting up libheif-plugin-libde265:amd64 (1.17.6-1ubuntu4) ...
Setting up libheif-plugin-aomenc:amd64 (1.17.6-1ubuntu4) ...
Processing triggers for libc-bin (2.39-0ubuntu8.3) ...
Processing triggers for man-db (2.12.0-4build2) ...
wsluser@DESKTOP-QQS4GT7:~$ sudo apt-get install libcurl4-openssl-dev
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Suggested packages:
libcurl4-doc libidn11-dev libkrb5-dev libldap2-dev librtmp-dev libssh2-1-dev libssl-dev
The following NEW packages will be installed:
libcurl4-openssl-dev
0 upgraded, 1 newly installed, 0 to remove and 27 not upgraded.
Need to get 386 kB of archives.
After this operation, 1698 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libcurl4-openssl-dev amd64 7.81.0-1ubuntu1.18 [386 kB]
Fetched 386 kB in 1s (468 kB/s)
Selecting previously unselected package libcurl4-openssl-dev:amd64.
(Reading database ... 37472 files and directories currently installed.)
Preparing to unpack .../libcurl4-openssl-dev_7.81.0-1ubuntu1.18_amd64.deb ...
Unpacking libcurl4-openssl-dev:amd64 (7.81.0-1ubuntu1.18) ...
Setting up libcurl4-openssl-dev:amd64 (7.81.0-1ubuntu1.18) ...
Processing triggers for man-db (2.10.2-1) ...
wsluser@DESKTOP-QQS4GT7:~$ git clone https://github.com/ggerganov/llama.cpp
Cloning into 'llama.cpp'...
remote: Enumerating objects: 35869, done.
remote: Counting objects: 100% (8839/8839), done.
remote: Compressing objects: 100% (269/269), done.
remote: Total 35869 (delta 8710), reused 8570 (delta 8570), pack-reused 27030 (from 1)
Receiving objects: 100% (35869/35869), 58.89 MiB | 18.83 MiB/s, done.
Resolving deltas: 100% (26204/26204), done.
wsluser@DESKTOP-QQS4GT7:~$
wsluser@DESKTOP-QQS4GT7:~$ cd llama.cpp
wsluser@DESKTOP-QQS4GT7:~/llama.cpp$ cmake -B build
-- The C compiler identification is GNU 13.2.0
-- The CXX compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.43.0")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- OpenMP found
-- Using llamafile
-- Using AMX
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done (5.1s)
-- Generating done (0.2s)
-- Build files have been written to: /home/wsluser/llama.cpp/build
wsluser@DESKTOP-QQS4GT7:~/llama.cpp$ cmake --build build --config Release
[ 0%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml.c.o
[ 1%] Building CXX object ggml/src/CMakeFiles/ggml.dir/ggml-backend.cpp.o
[ 2%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-quants.c.o
[ 2%] Building CXX object ggml/src/CMakeFiles/ggml.dir/llamafile/sgemm.cpp.o
[ 3%] Building CXX object ggml/src/CMakeFiles/ggml.dir/ggml-amx/mmq.cpp.o
[ 4%] Building CXX object ggml/src/CMakeFiles/ggml.dir/ggml-amx.cpp.o
[ 4%] Building C object ggml/src/CMakeFiles/ggml.dir/ggml-aarch64.c.o
[ 5%] Linking CXX shared library libggml.so
[ 5%] Built target ggml
[ 5%] Building CXX object src/CMakeFiles/llama.dir/llama.cpp.o
[ 6%] Building CXX object src/CMakeFiles/llama.dir/llama-vocab.cpp.o
[ 6%] Building CXX object src/CMakeFiles/llama.dir/llama-grammar.cpp.o
[ 7%] Building CXX object src/CMakeFiles/llama.dir/llama-sampling.cpp.o
[ 7%] Building CXX object src/CMakeFiles/llama.dir/unicode.cpp.o
[ 8%] Building CXX object src/CMakeFiles/llama.dir/unicode-data.cpp.o
[ 9%] Linking CXX shared library libllama.so
[ 9%] Built target llama
[ 9%] Generating build details from Git
-- Found Git: /usr/bin/git (found version "2.43.0")
[ 10%] Building CXX object common/CMakeFiles/build_info.dir/build-info.cpp.o
[ 10%] Built target build_info
[ 10%] Building CXX object common/CMakeFiles/common.dir/arg.cpp.o
[ 11%] Building CXX object common/CMakeFiles/common.dir/common.cpp.o
[ 11%] Building CXX object common/CMakeFiles/common.dir/console.cpp.o
[ 12%] Building CXX object common/CMakeFiles/common.dir/json-schema-to-grammar.cpp.o
[ 12%] Building CXX object common/CMakeFiles/common.dir/log.cpp.o
[ 13%] Building CXX object common/CMakeFiles/common.dir/ngram-cache.cpp.o
[ 14%] Building CXX object common/CMakeFiles/common.dir/sampling.cpp.o
[ 14%] Building CXX object common/CMakeFiles/common.dir/train.cpp.o
[ 15%] Linking CXX static library libcommon.a
[ 15%] Built target common
[ 15%] Building CXX object tests/CMakeFiles/test-tokenizer-0.dir/test-tokenizer-0.cpp.o
[ 16%] Linking CXX executable ../bin/test-tokenizer-0
[ 16%] Built target test-tokenizer-0
[ 16%] Building CXX object tests/CMakeFiles/test-tokenizer-1-bpe.dir/test-tokenizer-1-bpe.cpp.o
[ 17%] Linking CXX executable ../bin/test-tokenizer-1-bpe
[ 17%] Built target test-tokenizer-1-bpe
[ 17%] Building CXX object tests/CMakeFiles/test-tokenizer-1-spm.dir/test-tokenizer-1-spm.cpp.o
[ 18%] Linking CXX executable ../bin/test-tokenizer-1-spm
[ 18%] Built target test-tokenizer-1-spm
[ 18%] Building CXX object tests/CMakeFiles/test-log.dir/test-log.cpp.o
[ 19%] Building CXX object tests/CMakeFiles/test-log.dir/get-model.cpp.o
[ 19%] Linking CXX executable ../bin/test-log
[ 19%] Built target test-log
[ 19%] Building CXX object tests/CMakeFiles/test-arg-parser.dir/test-arg-parser.cpp.o
[ 20%] Building CXX object tests/CMakeFiles/test-arg-parser.dir/get-model.cpp.o
[ 21%] Linking CXX executable ../bin/test-arg-parser
[ 21%] Built target test-arg-parser
[ 22%] Building CXX object tests/CMakeFiles/test-quantize-fns.dir/test-quantize-fns.cpp.o
[ 22%] Building CXX object tests/CMakeFiles/test-quantize-fns.dir/get-model.cpp.o
[ 23%] Linking CXX executable ../bin/test-quantize-fns
[ 23%] Built target test-quantize-fns
[ 23%] Building CXX object tests/CMakeFiles/test-quantize-perf.dir/test-quantize-perf.cpp.o
[ 24%] Building CXX object tests/CMakeFiles/test-quantize-perf.dir/get-model.cpp.o
[ 24%] Linking CXX executable ../bin/test-quantize-perf
[ 24%] Built target test-quantize-perf
[ 25%] Building CXX object tests/CMakeFiles/test-sampling.dir/test-sampling.cpp.o
[ 25%] Building CXX object tests/CMakeFiles/test-sampling.dir/get-model.cpp.o
[ 26%] Linking CXX executable ../bin/test-sampling
[ 26%] Built target test-sampling
[ 26%] Building CXX object tests/CMakeFiles/test-chat-template.dir/test-chat-template.cpp.o
[ 27%] Building CXX object tests/CMakeFiles/test-chat-template.dir/get-model.cpp.o
[ 27%] Linking CXX executable ../bin/test-chat-template
[ 27%] Built target test-chat-template
[ 27%] Building CXX object tests/CMakeFiles/test-grammar-parser.dir/test-grammar-parser.cpp.o
[ 28%] Building CXX object tests/CMakeFiles/test-grammar-parser.dir/get-model.cpp.o
[ 28%] Linking CXX executable ../bin/test-grammar-parser
[ 28%] Built target test-grammar-parser
[ 29%] Building CXX object tests/CMakeFiles/test-llama-grammar.dir/test-llama-grammar.cpp.o
[ 29%] Building CXX object tests/CMakeFiles/test-llama-grammar.dir/get-model.cpp.o
[ 30%] Linking CXX executable ../bin/test-llama-grammar
[ 30%] Built target test-llama-grammar
[ 31%] Building CXX object tests/CMakeFiles/test-grammar-integration.dir/test-grammar-integration.cpp.o
[ 31%] Building CXX object tests/CMakeFiles/test-grammar-integration.dir/get-model.cpp.o
[ 32%] Linking CXX executable ../bin/test-grammar-integration
[ 32%] Built target test-grammar-integration
[ 33%] Building CXX object tests/CMakeFiles/test-grad0.dir/test-grad0.cpp.o
[ 33%] Building CXX object tests/CMakeFiles/test-grad0.dir/get-model.cpp.o
[ 34%] Linking CXX executable ../bin/test-grad0
[ 34%] Built target test-grad0
[ 34%] Building CXX object tests/CMakeFiles/test-barrier.dir/test-barrier.cpp.o
[ 35%] Building CXX object tests/CMakeFiles/test-barrier.dir/get-model.cpp.o
[ 36%] Linking CXX executable ../bin/test-barrier
[ 36%] Built target test-barrier
[ 37%] Building CXX object tests/CMakeFiles/test-backend-ops.dir/test-backend-ops.cpp.o
[ 37%] Building CXX object tests/CMakeFiles/test-backend-ops.dir/get-model.cpp.o
[ 38%] Linking CXX executable ../bin/test-backend-ops
[ 38%] Built target test-backend-ops
[ 39%] Building CXX object tests/CMakeFiles/test-rope.dir/test-rope.cpp.o
[ 39%] Building CXX object tests/CMakeFiles/test-rope.dir/get-model.cpp.o
[ 40%] Linking CXX executable ../bin/test-rope
[ 40%] Built target test-rope
[ 41%] Building CXX object tests/CMakeFiles/test-model-load-cancel.dir/test-model-load-cancel.cpp.o
[ 41%] Building CXX object tests/CMakeFiles/test-model-load-cancel.dir/get-model.cpp.o
[ 42%] Linking CXX executable ../bin/test-model-load-cancel
[ 42%] Built target test-model-load-cancel
[ 42%] Building CXX object tests/CMakeFiles/test-autorelease.dir/test-autorelease.cpp.o
[ 43%] Building CXX object tests/CMakeFiles/test-autorelease.dir/get-model.cpp.o
[ 43%] Linking CXX executable ../bin/test-autorelease
[ 43%] Built target test-autorelease
[ 44%] Building CXX object tests/CMakeFiles/test-json-schema-to-grammar.dir/test-json-schema-to-grammar.cpp.o
[ 44%] Building CXX object tests/CMakeFiles/test-json-schema-to-grammar.dir/get-model.cpp.o
[ 45%] Linking CXX executable ../bin/test-json-schema-to-grammar
[ 45%] Built target test-json-schema-to-grammar
[ 45%] Building C object tests/CMakeFiles/test-c.dir/test-c.c.o
[ 46%] Linking C executable ../bin/test-c
[ 46%] Built target test-c
[ 47%] Building CXX object examples/cvector-generator/CMakeFiles/llama-cvector-generator.dir/cvector-generator.cpp.o
[ 47%] Linking CXX executable ../../bin/llama-cvector-generator
[ 47%] Built target llama-cvector-generator
[ 47%] Building CXX object examples/baby-llama/CMakeFiles/llama-baby-llama.dir/baby-llama.cpp.o
[ 48%] Linking CXX executable ../../bin/llama-baby-llama
[ 48%] Built target llama-baby-llama
[ 48%] Building CXX object examples/batched-bench/CMakeFiles/llama-batched-bench.dir/batched-bench.cpp.o
[ 49%] Linking CXX executable ../../bin/llama-batched-bench
[ 49%] Built target llama-batched-bench
[ 49%] Building CXX object examples/batched/CMakeFiles/llama-batched.dir/batched.cpp.o
[ 50%] Linking CXX executable ../../bin/llama-batched
[ 50%] Built target llama-batched
[ 51%] Building CXX object examples/convert-llama2c-to-ggml/CMakeFiles/llama-convert-llama2c-to-ggml.dir/convert-llama2c-to-ggml.cpp.o
[ 51%] Linking CXX executable ../../bin/llama-convert-llama2c-to-ggml
[ 51%] Built target llama-convert-llama2c-to-ggml
[ 52%] Building CXX object examples/embedding/CMakeFiles/llama-embedding.dir/embedding.cpp.o
[ 52%] Linking CXX executable ../../bin/llama-embedding
[ 52%] Built target llama-embedding
[ 53%] Building CXX object examples/eval-callback/CMakeFiles/llama-eval-callback.dir/eval-callback.cpp.o
[ 54%] Linking CXX executable ../../bin/llama-eval-callback
[ 54%] Built target llama-eval-callback
[ 54%] Building CXX object examples/export-lora/CMakeFiles/llama-export-lora.dir/export-lora.cpp.o
[ 55%] Linking CXX executable ../../bin/llama-export-lora
[ 55%] Built target llama-export-lora
[ 55%] Building CXX object examples/gbnf-validator/CMakeFiles/llama-gbnf-validator.dir/gbnf-validator.cpp.o
[ 56%] Linking CXX executable ../../bin/llama-gbnf-validator
[ 56%] Built target llama-gbnf-validator
[ 57%] Building C object examples/gguf-hash/CMakeFiles/sha256.dir/deps/sha256/sha256.c.o
[ 57%] Built target sha256
[ 58%] Building C object examples/gguf-hash/CMakeFiles/xxhash.dir/deps/xxhash/xxhash.c.o
[ 58%] Built target xxhash
[ 58%] Building C object examples/gguf-hash/CMakeFiles/sha1.dir/deps/sha1/sha1.c.o
[ 58%] Building CXX object examples/gguf-hash/CMakeFiles/llama-gguf-hash.dir/gguf-hash.cpp.o
[ 59%] Linking CXX executable ../../bin/llama-gguf-hash
[ 59%] Built target llama-gguf-hash
[ 60%] Building CXX object examples/gguf-split/CMakeFiles/llama-gguf-split.dir/gguf-split.cpp.o
[ 60%] Linking CXX executable ../../bin/llama-gguf-split
[ 60%] Built target llama-gguf-split
[ 60%] Building CXX object examples/gguf/CMakeFiles/llama-gguf.dir/gguf.cpp.o
[ 61%] Linking CXX executable ../../bin/llama-gguf
[ 61%] Built target llama-gguf
[ 62%] Building CXX object examples/gritlm/CMakeFiles/llama-gritlm.dir/gritlm.cpp.o
[ 62%] Linking CXX executable ../../bin/llama-gritlm
[ 62%] Built target llama-gritlm
[ 63%] Building CXX object examples/imatrix/CMakeFiles/llama-imatrix.dir/imatrix.cpp.o
[ 63%] Linking CXX executable ../../bin/llama-imatrix
[ 63%] Built target llama-imatrix
[ 64%] Building CXX object examples/infill/CMakeFiles/llama-infill.dir/infill.cpp.o
[ 64%] Linking CXX executable ../../bin/llama-infill
[ 64%] Built target llama-infill
[ 64%] Building CXX object examples/llama-bench/CMakeFiles/llama-bench.dir/llama-bench.cpp.o
[ 65%] Linking CXX executable ../../bin/llama-bench
[ 65%] Built target llama-bench
[ 65%] Building CXX object examples/llava/CMakeFiles/llava.dir/llava.cpp.o
[ 66%] Building CXX object examples/llava/CMakeFiles/llava.dir/clip.cpp.o
[ 66%] Built target llava
[ 67%] Linking CXX static library libllava_static.a
[ 67%] Built target llava_static
[ 67%] Linking CXX shared library libllava_shared.so
[ 67%] Built target llava_shared
[ 68%] Building CXX object examples/llava/CMakeFiles/llama-llava-cli.dir/llava-cli.cpp.o
[ 69%] Linking CXX executable ../../bin/llama-llava-cli
[ 69%] Built target llama-llava-cli
[ 70%] Building CXX object examples/llava/CMakeFiles/llama-minicpmv-cli.dir/minicpmv-cli.cpp.o
[ 70%] Linking CXX executable ../../bin/llama-minicpmv-cli
[ 70%] Built target llama-minicpmv-cli
[ 70%] Building CXX object examples/lookahead/CMakeFiles/llama-lookahead.dir/lookahead.cpp.o
[ 71%] Linking CXX executable ../../bin/llama-lookahead
[ 71%] Built target llama-lookahead
[ 71%] Building CXX object examples/lookup/CMakeFiles/llama-lookup.dir/lookup.cpp.o
[ 72%] Linking CXX executable ../../bin/llama-lookup
[ 72%] Built target llama-lookup
[ 72%] Building CXX object examples/lookup/CMakeFiles/llama-lookup-create.dir/lookup-create.cpp.o
[ 73%] Linking CXX executable ../../bin/llama-lookup-create
[ 73%] Built target llama-lookup-create
[ 73%] Building CXX object examples/lookup/CMakeFiles/llama-lookup-merge.dir/lookup-merge.cpp.o
[ 74%] Linking CXX executable ../../bin/llama-lookup-merge
[ 74%] Built target llama-lookup-merge
[ 75%] Building CXX object examples/lookup/CMakeFiles/llama-lookup-stats.dir/lookup-stats.cpp.o
[ 75%] Linking CXX executable ../../bin/llama-lookup-stats
[ 75%] Built target llama-lookup-stats
[ 76%] Building CXX object examples/main/CMakeFiles/llama-cli.dir/main.cpp.o
[ 76%] Linking CXX executable ../../bin/llama-cli
[ 76%] Built target llama-cli
[ 77%] Building CXX object examples/parallel/CMakeFiles/llama-parallel.dir/parallel.cpp.o
[ 77%] Linking CXX executable ../../bin/llama-parallel
[ 77%] Built target llama-parallel
[ 78%] Building CXX object examples/passkey/CMakeFiles/llama-passkey.dir/passkey.cpp.o
[ 78%] Linking CXX executable ../../bin/llama-passkey
[ 78%] Built target llama-passkey
[ 79%] Building CXX object examples/perplexity/CMakeFiles/llama-perplexity.dir/perplexity.cpp.o
[ 80%] Linking CXX executable ../../bin/llama-perplexity
[ 80%] Built target llama-perplexity
[ 80%] Building CXX object examples/quantize-stats/CMakeFiles/llama-quantize-stats.dir/quantize-stats.cpp.o
[ 81%] Linking CXX executable ../../bin/llama-quantize-stats
[ 81%] Built target llama-quantize-stats
[ 81%] Building CXX object examples/quantize/CMakeFiles/llama-quantize.dir/quantize.cpp.o
[ 82%] Linking CXX executable ../../bin/llama-quantize
[ 82%] Built target llama-quantize
[ 82%] Building CXX object examples/retrieval/CMakeFiles/llama-retrieval.dir/retrieval.cpp.o
[ 83%] Linking CXX executable ../../bin/llama-retrieval
[ 83%] Built target llama-retrieval
[ 84%] Generating theme-snowstorm.css.hpp
[ 85%] Generating colorthemes.css.hpp
[ 86%] Generating completion.js.hpp
[ 87%] Generating index-new.html.hpp
[ 87%] Generating index.html.hpp
[ 87%] Generating index.js.hpp
[ 87%] Generating json-schema-to-grammar.mjs.hpp
[ 88%] Generating loading.html.hpp
[ 89%] Generating prompt-formats.js.hpp
[ 89%] Generating style.css.hpp
[ 89%] Generating system-prompts.js.hpp
[ 90%] Generating theme-beeninorder.css.hpp
[ 90%] Generating theme-ketivah.css.hpp
[ 91%] Generating theme-mangotango.css.hpp
[ 91%] Generating theme-playground.css.hpp
[ 92%] Generating theme-polarnight.css.hpp
[ 93%] Building CXX object examples/server/CMakeFiles/llama-server.dir/server.cpp.o
[ 93%] Linking CXX executable ../../bin/llama-server
[ 93%] Built target llama-server
[ 94%] Building CXX object examples/save-load-state/CMakeFiles/llama-save-load-state.dir/save-load-state.cpp.o
[ 94%] Linking CXX executable ../../bin/llama-save-load-state
[ 94%] Built target llama-save-load-state
[ 95%] Building CXX object examples/simple/CMakeFiles/llama-simple.dir/simple.cpp.o
[ 95%] Linking CXX executable ../../bin/llama-simple
[ 95%] Built target llama-simple
[ 96%] Building CXX object examples/speculative/CMakeFiles/llama-speculative.dir/speculative.cpp.o
[ 96%] Linking CXX executable ../../bin/llama-speculative
[ 96%] Built target llama-speculative
[ 97%] Building CXX object examples/tokenize/CMakeFiles/llama-tokenize.dir/tokenize.cpp.o
[ 97%] Linking CXX executable ../../bin/llama-tokenize
[ 97%] Built target llama-tokenize
[ 98%] Building CXX object pocs/vdot/CMakeFiles/llama-vdot.dir/vdot.cpp.o
[ 99%] Linking CXX executable ../../bin/llama-vdot
[ 99%] Built target llama-vdot
[ 99%] Building CXX object pocs/vdot/CMakeFiles/llama-q8dot.dir/q8dot.cpp.o
[100%] Linking CXX executable ../../bin/llama-q8dot
[100%] Built target llama-q8dot
wsluser@DESKTOP-QQS4GT7:~/llama.cpp$After build, you could find all command line scrips at llama.cpp/build/bin directory:
wsluser@DESKTOP-QQS4GT7:~/llama.cpp$ ls -l build/bin
total 29768
-rwxr-xr-x 1 wsluser wsluser 426280 Oct 21 16:55 llama-baby-llama
-rwxr-xr-x 1 wsluser wsluser 937920 Oct 21 16:56 llama-batched
-rwxr-xr-x 1 wsluser wsluser 937968 Oct 21 16:56 llama-batched-bench
-rwxr-xr-x 1 wsluser wsluser 485384 Oct 21 16:56 llama-bench
-rwxr-xr-x 1 wsluser wsluser 974792 Oct 21 16:57 llama-cli
-rwxr-xr-x 1 wsluser wsluser 368080 Oct 21 16:56 llama-convert-llama2c-to-ggml
-rwxr-xr-x 1 wsluser wsluser 967344 Oct 21 16:55 llama-cvector-generator
-rwxr-xr-x 1 wsluser wsluser 946544 Oct 21 16:56 llama-embedding
-rwxr-xr-x 1 wsluser wsluser 942528 Oct 21 16:56 llama-eval-callback
-rwxr-xr-x 1 wsluser wsluser 972208 Oct 21 16:56 llama-export-lora
-rwxr-xr-x 1 wsluser wsluser 28688 Oct 21 16:56 llama-gbnf-validator
-rwxr-xr-x 1 wsluser wsluser 28496 Oct 21 16:56 llama-gguf
-rwxr-xr-x 1 wsluser wsluser 98952 Oct 21 16:56 llama-gguf-hash
-rwxr-xr-x 1 wsluser wsluser 48576 Oct 21 16:56 llama-gguf-split
-rwxr-xr-x 1 wsluser wsluser 943008 Oct 21 16:56 llama-gritlm
-rwxr-xr-x 1 wsluser wsluser 985240 Oct 21 16:56 llama-imatrix
-rwxr-xr-x 1 wsluser wsluser 957184 Oct 21 16:56 llama-infill
-rwxr-xr-x 1 wsluser wsluser 1234472 Oct 21 16:57 llama-llava-cli
-rwxr-xr-x 1 wsluser wsluser 946968 Oct 21 16:57 llama-lookahead
-rwxr-xr-x 1 wsluser wsluser 967728 Oct 21 16:57 llama-lookup
-rwxr-xr-x 1 wsluser wsluser 962824 Oct 21 16:57 llama-lookup-create
-rwxr-xr-x 1 wsluser wsluser 73536 Oct 21 16:57 llama-lookup-merge
-rwxr-xr-x 1 wsluser wsluser 967464 Oct 21 16:57 llama-lookup-stats
-rwxr-xr-x 1 wsluser wsluser 1229800 Oct 21 16:57 llama-minicpmv-cli
-rwxr-xr-x 1 wsluser wsluser 946928 Oct 21 16:57 llama-parallel
-rwxr-xr-x 1 wsluser wsluser 942296 Oct 21 16:57 llama-passkey
-rwxr-xr-x 1 wsluser wsluser 1032136 Oct 21 16:57 llama-perplexity
-rwxr-xr-x 1 wsluser wsluser 21520 Oct 21 16:58 llama-q8dot
-rwxr-xr-x 1 wsluser wsluser 369416 Oct 21 16:57 llama-quantize
-rwxr-xr-x 1 wsluser wsluser 228288 Oct 21 16:57 llama-quantize-stats
-rwxr-xr-x 1 wsluser wsluser 956008 Oct 21 16:57 llama-retrieval
-rwxr-xr-x 1 wsluser wsluser 942680 Oct 21 16:58 llama-save-load-state
-rwxr-xr-x 1 wsluser wsluser 1944592 Oct 21 16:58 llama-server
-rwxr-xr-x 1 wsluser wsluser 27024 Oct 21 16:58 llama-simple
-rwxr-xr-x 1 wsluser wsluser 962456 Oct 21 16:58 llama-speculative
-rwxr-xr-x 1 wsluser wsluser 339232 Oct 21 16:58 llama-tokenize
-rwxr-xr-x 1 wsluser wsluser 22240 Oct 21 16:58 llama-vdot
-rwxr-xr-x 1 wsluser wsluser 943248 Oct 21 16:55 test-arg-parser
-rwxr-xr-x 1 wsluser wsluser 18208 Oct 21 16:55 test-autorelease
-rwxr-xr-x 1 wsluser wsluser 339920 Oct 21 16:55 test-backend-ops
-rwxr-xr-x 1 wsluser wsluser 22024 Oct 21 16:55 test-barrier
-rwxr-xr-x 1 wsluser wsluser 15768 Oct 21 16:55 test-c
-rwxr-xr-x 1 wsluser wsluser 351832 Oct 21 16:55 test-chat-template
-rwxr-xr-x 1 wsluser wsluser 57032 Oct 21 16:55 test-grad0
-rwxr-xr-x 1 wsluser wsluser 570680 Oct 21 16:55 test-grammar-integration
-rwxr-xr-x 1 wsluser wsluser 41104 Oct 21 16:55 test-grammar-parser
-rwxr-xr-x 1 wsluser wsluser 592984 Oct 21 16:55 test-json-schema-to-grammar
-rwxr-xr-x 1 wsluser wsluser 46984 Oct 21 16:55 test-llama-grammar
-rwxr-xr-x 1 wsluser wsluser 38272 Oct 21 16:54 test-log
-rwxr-xr-x 1 wsluser wsluser 16504 Oct 21 16:55 test-model-load-cancel
-rwxr-xr-x 1 wsluser wsluser 17736 Oct 21 16:55 test-quantize-fns
-rwxr-xr-x 1 wsluser wsluser 42072 Oct 21 16:55 test-quantize-perf
-rwxr-xr-x 1 wsluser wsluser 17424 Oct 21 16:55 test-rope
-rwxr-xr-x 1 wsluser wsluser 45648 Oct 21 16:55 test-sampling
-rwxr-xr-x 1 wsluser wsluser 354536 Oct 21 16:54 test-tokenizer-0
-rwxr-xr-x 1 wsluser wsluser 331656 Oct 21 16:54 test-tokenizer-1-bpe
-rwxr-xr-x 1 wsluser wsluser 331336 Oct 21 16:54 test-tokenizer-1-spm
wsluser@DESKTOP-QQS4GT7:~/llama.cpp$ If you see some errors during the compile and build process, write down the file name. Most of time, the error only impacts the file or model related, not whole installation.
What’s next?
Typically the next step is to validate the installation. Below link provides you not only the hello world use case, but most of modern common use cases.
If you are interested in building and installing Llama.cpp for different environment, check out below links:
