System Specifications

Nvidia Driver: NVIDIA-Linux-aarch64-570.86.16.run
Host: Ampere Altra + ALTRAD8UD
Host OS: Ubuntu 22.04 with HWE kernel (6.8)
Guest OS: Ubuntu 22.04 (ubuntu-22.04-live-server-arm64.iso)
GPU: Nvidia RTX 4080 16GB

Assumptions

1. You are familiar with Ubuntu and its basic commands.
2. You have experience using Virt-manager.
3. All commands are executed as the root user.

If anything is unclear, you can refer to external resources for additional guidance.

Host Configuration

Enable IOMMU

To enable IOMMU, you need to enable the SR-IOV option in the BIOS and verify whether the Linux kernel has IOMMU enabled by default.

You can check if IOMMU is enabled by running:

$ dmesg | grep -i iommu

Example output:

[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic root=UUID=6b78fa89-a575-432d-a445-1497c3467214 ro iommu=on
[    0.000000] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic iommu=on", will be passed to user space.
[   11.561684] iommu: Default domain type: Translated
[   11.566470] iommu: DMA domain TLB invalidation policy: strict mode

If IOMMU is not enabled, add iommu=on to the Linux kernel boot parameters:

$ vim /etc/default/grub 

Modify the line:

GRUB_CMDLINE_LINUX_DEFAULT="iommu=on"

Then update GRUB and reboot:

$ update-grub2  
$ reboot  

Additionally, enable SR-IOV in the BIOS. The exact location of this setting varies depending on the BIOS, but it is typically found under the PCIe subsystem or related options.

Upgrade Host to HWE Kernel

I recommend using the Hardware Enablement (HWE) kernel on the host. While I’m unsure if the regular kernel works, the HWE kernel has been reliable in my experience. Install it with:

sudo apt install linux-generic-hwe-22.04

Configure VFIO on Host

The VM relies on the VFIO driver for GPU passthrough. To configure VFIO, you need to pass the PCIe device information to the VFIO driver.

First, identify the GPU’s PCIe device IDs:

$ lspci -nn

Example output:

0005:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2704] (rev a1)
0005:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bb] (rev a1)

Here, 10de:2704 is the GPU’s PCIe device ID, and 10de:22bb is the audio device ID. At a minimum, you need to pass through the GPU device.

Next, edit the VFIO configuration file to include these IDs:

$ vim /etc/modprobe.d/vfio.conf

Add the following line:

options vfio-pci ids=10de:2704,10de:22bb

Disable Nvidia Driver on HOST

To prevent the host from loading the Nvidia driver, add the Nvidia modules to the kernel’s blocklist:

$ vim /etc/modprobe.d/blacklist.conf

Add the following lines:

blacklist nvidia
blacklist nvidia_drm
blacklist nvidia_modeset

Update the initramfs and reboot:

$ update-initramfs -u
$ reboot

Configure VM

Install Virt-manager

In this article, we used virt-manager as VM manager, first step is install virt-manager, suppose Ubuntu will install all relative packages.

$ apt install virt-manager 

If you’re using SSH with X11 forwarding (e.g., ssh -X host) or MobaXTerm on Windows, Virt-manager will display the remote X window. If neither method works, consider installing a KDE desktop on the host and accessing it via the BMC remote console.

(Optional) Install KDE Plasma Desktop:

(option)
$ apt install kde-plasma-desktop

Create VM image

Virt-manager creates fixed-size VM images by default. If you prefer dynamic allocation, create the image manually:

$ qemu-img create -f qcow2 ubuntu2204.qcow2 200G

Add Nvidia device to VM

If the host is configured correctly, Virt-manager will list all PCIe devices, including the Nvidia GPU. Add the GPU and its audio device (e.g., 0005:01:00.0 and 0005:01:00.1) to the VM’s hardware list.

After adding the devices, proceed with the Ubuntu 22.04 installation.

After add hardware

Now, it can run begin install to install ubuntu 22.04

Disable secure Boot in UEFI

By default, Virt-manager enables Secure Boot. However, Nvidia drivers may not work with Secure Boot enabled. Even though the Nvidia installer includes a driver signing feature, the driver may still fail to load. To avoid issues, disable Secure Boot in the VM’s UEFI settings.

During the VM’s boot process, press the DEL key to enter UEFI settings and uncheck the Secure Boot option.

Before installing the Nvidia driver, ensure the necessary development packages are installed:

$ apt install build-essential

Then, install the Nvidia driver and reboot the VM.

GPU Passthrough Test

If everything is set up correctly, running nvidia-smi should display the GPU’s status.

For testing, you can use Ollama with the Deepseek-R1 model. Install Ollama with:

curl -fsSL https://ollama.com/install.sh | sh

Pull the Deepseek-R1 model. Since the GPU has 16GB of memory, the 14B model is a good choice (it requires ~10GB):

ollama run deepseek-r1:14b

Ask a question like, “Why is the sky blue?” This will trigger the model’s Chain-of-Thought (CoT) reasoning.

Monitor the GPU’s status using nvidia-smi to ensure it’s functioning correctly.

The post GPU Passthrough on ARM64 with Libvirt/Virt-manager appeared first on richliu's blog.

Fine-Tune LLM 是指在現在的模型架構之上，再加上一個小的模型，就使用讓原來的模型支援我們要的結果。比較常見的應該是在 Civitai 上可以看到很多不同風格的 Stable Diffusion Lora model 。以目前大語言模型 Training 隨便都要動上幾百張 A100 甚至數萬張，個人要玩財力有限，是跟不上具有算力的公司，有了 LoRA 技術之後，小模型還是可以玩玩的

本篇使用的硬體跟這篇　LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform　一樣，許多前置的設定也可以參考這篇，本篇只解釋我是怎麼做完整個流程的 。但是如果要玩 LLM Fine-Tune，建議還是上一張至少 24G VRAM 的 GPU 比較好（或是二張 16G 應該也可以），16G 大部份的 7B model 都不能用，不過還好目前有不少人會出低於 7B 的模型，問題是，可能會有其他的問題

資料清洗 (Data Cleaning)

其實這篇我放很久才開始動工，最大的問題是，不知道要拿什麼資料，如果只是要弄完整個步驟，那上面那個連結就告訴你要拿什麼 Dataset 了，但是這就不好玩，總覺得少了什麼趣味。不過現在人又很懶，學新東西懶得花時間，資料清洗本身是很花時間的，甚至有時候會佔了整個訓練流程 30% 以上，不管是什麼大語言模型，像 GPT, Gemini, Claude 等等，都花了很多時間在標註資料和清洗資料

不過最近 Claude.ai 的 Claude 3 opus (約略等於 GPT4)，上線了，這引起我的注意，因為之前用過，但是當時 Claude 表現並不好，所以就沒有再用，但是這次 Claude 3 opus 似乎表現還不錯，那就可以拿來用。Claude 最大的好處就是可能會去存取外部網頁，我知道 GPT4 也可以，但是我們下次一定黨一定會想先用個免費版的。在測試過後，Claude 3 sonnet 的表現還可以，那就可以拿來當我們資料清洗的工具。

工具有了，內容永遠是最麻煩的，但是這個內容比較麻煩的是，一定要有一定的資料量和鑑別性（我沒學過資料清洗講錯請見諒），要不然在測試的時候無法確定這就是我們輸入的資料。這也是我一直沒動工的原因，好的資料集難找，前幾天 3/11日，是核能流言終結者協會成立十週年吧。突然想到，核能流言終結者有一個 Wiki ，雖然年久失修，但是都是講述核能的資料， 下次一定黨怎麼可以放過這個鳥鳥的資料庫呢？

資料有了，工具有了，接下來就可以開幹了

MariaDB+phpMyAdmin

我找了一陣子，沒有合適的資料編輯工具，要不然就是 CSV ，但是也不方便編輯。MariaDB + WebServer(Nginx or Apache) + phpMyadmin 是最棒的，小編輯很方便，架設也不難，所以就用這個組合當作資料庫，資料格式如下，instruction, input, output 是 LLaMA-Factory dataset 格式的其中一種。我就照弄就好了，以下這是 TABLE 名，至於 database 取名就大家開心了

CREATE TABLE `nuclear` (
  `id` int(11) NOT NULL,
  `instruction` text DEFAULT NULL,
  `input` text DEFAULT NULL,
  `output` text DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE `nuclear`
  ADD PRIMARY KEY (`id`);
ALTER TABLE `nuclear`
  MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=0;
COMMIT;

如果有問題直接按編輯就可以小修一下，非常方便

請 Claude 清洗資料

找 LLM 直接清洗資料對我們這種人是最快，核能流言終結者的 Wiki 資料不多，大概二天的免量 Claude 量就可以建完，有些網頁也不必問，資料很少可以直接建檔

請 Claude 吐資料的詠唱方式是這樣的　

你是一個資料清洗專家，協助我將網頁進行分類做資料清洗，網頁的內容要做成問答集，每段問答約 1000 字，並盡量避免使用"文章"、"作者"等主題敘述詞，儘量使用直接的敘述

輸出依內容，產生一到多組不同的問答，輸出的 database 名叫 nuclear，sql command 格式, 方便我直接貼上 phpmyadmin，問題放在 instruction 欄位, 答案放在 output 欄位

網頁如下 https://nuke.fandom.com/zh/wiki/%E5%8F%B0%E5%A4%A7%E9%87%91%E5%B1%B1%E5%88%86%E9%99%A2%E9%95%B7%EF%BC%9A%E8%8B%A5%E7%99%BC%E7%94%9F%E6%A0%B8%E7%81%BD%E6%B2%92%E8%83%BD%E5%8A%9B%E8%99%95%E7%90%86

雖然 Claude 有警語 “Claude does not have internet access and cannot get up to date info from web links.” 但是我給的網頁都沒問題，如果不能向外存取，應該貼上內容也是可以的（可以取得當然比較方便）。以下就是部份 Claude 的回答

好的,我將根據網頁內容,分成多組問答,每組約1000字,並避免使用"文章"、"作者"等主題敘述詞。問題將放在instruction欄位,答案將放在output欄位,使用SQL命令插入nuclear資料表。

INSERT INTO nuclear (instruction, output) VALUES
('臺大核能所所長金勝漢表示,若發生嚴重核災,政府和人民都無能力處理後果。請簡述這個說法的背景和重點。', '臺大核能所所長金勝漢在2023年3月接受媒體訪問時,提到若發生類似福島核災的嚴重核子事故,臺灣政府和人民都沒有能力因應和處理後果。他的這番話,引發社會高度關注核能安全議題。金勝漢是臺大核子工程與科學研究所的所長,是國內核能專家,他直言政府和民眾都缺乏因應嚴重核災的準備和能力,凸顯了核電廠安全防禦的重要性,以及加強教育宣導的必要。')

將 SQL 這一段直接貼到 phpMyAdmin 就可以了

接下來就是不停的產生，貼上，在貼上前可以看看資料，有時需要修改一下。因為我只是測試，所以有些錯誤我也就算了，偶爾才會修改一下文章和用詞。所以我建了約 85 筆問答的資料，雖然不多，但是應該可以做些測試了

輸出 json file

接下來就是寫一隻程式輸出到 json file ，當然這種小程式就詠唱叫 Claude 輸出就好了，品質還不錯。記得將參數換成要連線的資料庫

import mysql.connector
import json

# 連接到 MySQL 資料庫
mydb = mysql.connector.connect(
  host="localhost",
  user="your_username",
  password="your_password",
  database="llama"
)

# 獲取遊標
mycursor = mydb.cursor()

# 查詢資料表
mycursor.execute("SELECT instruction, input, output FROM nuclear")

# 獲取查詢結果
rows = mycursor.fetchall()

# 將查詢結果轉換為 JSON 格式
data = []
for row in rows:
    data.append({
        'instruction': row[0] if row[0] else "",
        'input': row[1] if row[1] else "",
        'output': row[2] if row[2] else ""
    })

# 將 JSON 資料寫入檔案
with open('nuclear.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, ensure_ascii=False, indent=4)

# 關閉資料庫連線
mydb.close()

這樣就會輸出 nuclear.json ，放到 LLaMA-Factory 的 data 目錄，修改 dataset_info.json 加上這行，不需要產生 sha1 ，不設看起來就不檢查

  "nuclear_train": {
    "file_name": "nuclear.json"
  },

設完之後，reload dataset 就好

調整參數

首先是選 model ，我本來要試 Mistral-7B，但是一用就 Out of Memory 了。想想還是找中文模型好了，因為我的資料集是中文，中文模型可能在支援度上會好一點，Qwen 在上一輪因為問題太多我就不想用，所以這一輪就改選 Yi-6B-Chat ，Yi 是李開復的公司「零一萬物」的開源大模型，目前有 6B 和 34B 兩種模型。

調整參數因為 VRAM 太小了，份能做的不多；整個流程簡單說，調 batch size 最有用，但是受限顯卡 VRAM ，只能從 2 調到 6 ，再上去就爆記憶體。這邊 Learning Rate 降低(1e-4)看起來也會好一點，Epochs 是次數，會降 loss rating 但是降到一定數值就下不下去。調整完我的輸出就看起來比較正常，其他還有很多參數，但是沒深入研究，不確定影響為何

接下來的步驟都和上一篇一樣，Chat 和 Export 都是

測試

最後結果我們會輸出到 /nvme/newmodel/Yi-6B-Chat，和上一篇不一樣，這邊要用 convert.py 轉檔，不是convert-hf-to-gguf.py [ref]

python3 /nvme/llama.cpp/convert.py Yi-6B-Chat
<略>
Wrote Yi-6B-Chat/ggml-model-f16.gguf

# 量化，可以變 q4/q5 等等
/nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q4_0.bin q4_K_M
 /nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q5_0.bin q5_K_M

測試 model

# 測試原始版本的輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
I understand that there may have been concerns and misunderstandings regarding the establishment of a nuclear waste storage faci
lity on Taiwan's Yi-Lan Island. It is important to address these issues with honesty and transparency. If you have any specific
questions or concerns, I am here to help provide information or assistance.<|im_end|>

# Q5 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q5_0.bin -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
我們已經對此事件進行了詳細的調查和審查，確認原能會在當時的工作中沒有任何欺騙或不當行為。事實上，原能會在規劃這項建設時已經與當地
居民進行了充分的溝通和協商，並根據《核子設施法》的要求徵得了必要的手續。在此過程中，原能會也確認了該地區能夠承受相關設施的運轉和
風險。因此，您所指的欺騙或未經同意的說法是不符合事實的。<|im_end|>

> 核能流言終結者是什麼?
 **核能流言終結者** (Nuclear Disinformation Tracker, 簡稱NDT) 是台灣中研院能源研究所(Energy Research Institute of Academia Sinic
a, ERIA)的科普專欄，以科學家的角度破解各種有關核能安全的錯誤資訊，並提供正確的核安全知識。

# Q4 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q4_0.bin -ins
> 核能流言終結者是什麼？
 核能流言終結者是一個專門用來解釋有關核電站運作和核廢料處理等方面的知識的網站。該網站由香港科技大學公共衞生及基層醫療學院團隊創
立，旨在消除大眾對核電站運作的各種疑慮和誤解。<|im_end|>


# F16 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
 這個說法是錯誤的。原能會於1982年在蘭嶼設立核廢料貯存場時，確實有經過多次和居民的溝通協商，包括了說明會、座談會等等。居民在了解
相關資訊後才同意興建。因此，說原能會欺騙居民是沒有根據的。

> 核能流言終結者是什麼?
 "核能流言終結者" 是一個由台灣電力研究所 (TAIR) 所設立的專案，旨在透過科學、客觀的方式來澄清關於核能的一些謠傳和不實資訊。這些資
訊包括了對核能發電的誤解、不實的風險宣傳等等。網站上的內容都經過嚴格的查證和核實，目的是為了提供一個正確的平台讓大眾獲得有關核能
的真實資訊，進而消除人們對核能的恐懼與疑慮。

從以上的結果可以看到

* 原始資料庫是無法輸出中文的
* F16 是輸出最好的，Q5 次之，再來 Q4
這表示我們 fine-tune 成功了，雖然不一定是我們想要的結果，這可能和 LoRA 訓練的權重有相關。對於核能流言終結者的說明都是錯的，可能要針對原始資料庫修改增加說明，或是要加 Batch size

Ollama

Ollama 有點奇怪，因為我直接下命令都會出些很奇怪的結果，跟 llama.cpp 直接執行的結果不同。最後調整是用 F16 才可以正常輸出，以下是 modelfile

FROM /nvme/newmodel/Yi-6B-Chat/ggml-model-f16.gguf


PARAMETER temperature 0.8
PARAMETER num_ctx 512


TEMPLATE """[INST] {{ if .System }}<<SYS>>{{ .System }}<</SYS>>

{{ end }}{{ .Prompt }} [/INST] """
SYSTEM """"""
PARAMETER stop [INST]
PARAMETER stop [/INST]
PARAMETER stop <<SYS>>
PARAMETER stop <</SYS>>

建立 ollama database ，並且 reload

ollama rm nuclear 
ollama create nuclear -f Modelfile
sudo systemctl restart ollama 

我要下 parameter 才會得到正確的輸出結果，ex:

這樣應該算簡單吧

The post Easy to Fine-Tune Large Language Model with LLaMA-Factory appeared first on richliu's blog.

Qualcomm’s new desktop SoCs might change this landscape. However, we can still leverage the Ampere Altra platform to run LLMs.

Since this is a personal blog, most articles here are my personal notes. I’ll keep them updated as my understanding evolves.

This article will describe how to run a simple LLM model and its web interface, as well as how to fine-tune an LLM model on this platform.

Before running the commands, you’ll need to install the Nvidia driver. Refer to this article, “[How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform]”, for instructions on installing the Nvidia driver and Docker driver.

Ollama and Open-Webui don’t require a GPU. However, having a GPU is beneficial. Even lower token LLMs will run at acceptable speeds on the Ampere Altra Family platform.

This article’s hardware setup is as follows:

• CPU : Ampere Altra Family
• Board: AsRock ALTRAD8UD
• GPU: Nvidia RTX 4080

Use current LLM model with GUI

At this point in the process, using Ollama and Open-Webui is a straightforward approach for running LLMs. Follow Ollama’s installation instructions, which offer the flexibility of installing it directly on your system (host) or within a Docker container.

curl -fsSL https://ollama.com/install.sh | sh

By default, the Ollama service listens on localhost (127.0.0.1). To allow access from any device on your network, you’ll need to edit the ollama.service file.

In the [Service] section, add the following line:

Environment="OLLAMA_HOST=0.0.0.0:11434"

For Open-Webui, we recommend using Docker for a simpler solution. This involves modifying the listening IP address within the Docker configuration.

Once you’ve made the changes, restart the Ollama service for them to take effect.

systemctl daemon-reload
systemctl restart ollama

Run the ollama command and try it

# ollama run llama2
>>> who are you

I'm LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner.
I'm here to help you with any questions or topics you'd like to discuss! Is there something specific you'd like to talk
about or ask?

>>> 

If doesn’t work, try to restart ollama and try again.

Big LLM

If choose some big LLM like LLaMA2-70b or Qwen

Open-Webui

Following command is to run the Open-Webui docker image on host, 3000 is host port for Open-Webui. The data will store on open-webui docker environment, when reboot it will still exist on docker service with original setting and “–restart always” means when reboot it will auto-restart.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

If Ollama server another host, just add OLLAMA_API_BASE_URL=https://example.com/api to the docker command,

docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=https://example.com/api -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Upon login, you’ll need to provide an email and password. Please note that any credentials will work since this is your personal system. Once logged in, you can also edit the OLLAMA_API_BASE on the settings page.

Now, it connected from Open-Webui to ollmam backend, and we have a running sytsem on our machine.
Run sample chat here and choice llama2:latest as LLM model.

LLM Fine-Turning – LLaMA-Factory

Fine-tuning LLMs ideally requires a GPU card for better performance. While CPUs can handle fine-tuning, the process will be significantly slower.

Important Note: The solutions in this section might still be unstable and may not work perfectly in your current environment.

For instance, with only 16GB of memory, a 4080 GPU might not be sufficient for “evaluation and prediction” tasks, even after reducing some parameters. This still can lead to CUDA out-of-memory errors.

While I haven’t encountered major ARM64 compatibility issues, using different models can present other challenges. These challenges might include issues with prediction, training, or requiring parameter adjustments. Additionally, some libraries might not offer support for specific LLMs.

Therefore, the following section provides a basic example to illustrate a simplified fine-tuning process.

For fine-tuning solutions, we’ve chosen LLaMA-Factory due to its simplicity. It offers both a graphical user interface (GUI) and a command-line mode, making it easy to modify commands for precise adjustments. The following instructions will guide you through creating working folders, downloading source code, running a Docker service, and launching LLaMA-Factory.

# Create work folder
mkdir -p /nvme/model 
cd /nvme 
git clone https://github.com/hiyouga/LLaMA-Factory.git

# Run docker, it will use host GPU and map the folder into docker.
sudo docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -it --rm \
     -v /nvme:/nvme \
     -p 7860:7860 \
     nvcr.io/nvidia/pytorch:24.01-py3

# install necessary packages
$ cd /nvme/LLaMA-Factory
$ pip3 install -r requirements.txt
$ pip3 install tiktoken transformers_stream_generator

# run the LLaMA-Factory
$ CUDA_VISIBLE_DEVICES=0 python src/train_web.py

Now, it can acces the webgui via http://host IP:7860.

It can create another Docker window to run command on the LLaMA-Factory docker image.

docker exec -it <docker name> /bin/bash 

for download LLMs, you need to use git download from huggingface, ex:

# LLama 2 need username and token(not passowrd) to download it. 
git clone https://huggingface.co/meta-llama/Llama-2-7b-hf

# Qwen
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B

Pre-Training

Model name, choice the LLMs name , like Qwen1.5-1.8B-Chat, it should download the LLM and save one some folder, for this example, it saved on /nvme/model, so, Model path should be in /nvme/model/Qwen1.5-1.8B-Chat.

First time, I suggest to use small LLM for fine-tuning, Qweb1.5-0.8B might have issue, will suggest from 1.5-1.8B to start.

Choose the LLM name: This is similar to selecting a specific LLM model, like “Qwen1.5-1.8B-Chat”. The instructions will indicate your LLM folder. For this example, the model would be saved in /nvme/model/Qwen1.5-1.8B-Chat.

Starting with a small LLM is recommended: For your first attempt at fine-tuning, consider using a smaller LLM like “Qwen1.5-1.8B”. It’s possible that “Qweb1.5-0.8B” might cause issues, so we recommend starting with models in the 1.5-1.8B range.

Here are the fine-tuning options you can adjust in LLaMA-Factory:

Dataset: You can add datasets to this mode. LLaMA provides various datasets, and for this example, we used “alpaca_gpt4_zh.”
Gradient accumulation: This is currently set to 4.
Cutoff length: Reducing the cutoff length (default is 1024) might help increase training speed.
Compute type: The compute type depends on your environment. In this case, fp16 works, but bp16 doesn’t sometimes.
Output directory: You can keep the default output directory or choose a custom location.
Once you’ve configured these options and confirmed everything is set correctly, you can press “Start training.” Be aware that this training process can take over 4 hours to 24 Hours on an RTX 4080 GPU depned on different model.

Evaluate & Predict

Next step is evaluate and predict, Adpater path just fill previous steps’ output dir. Dataset is the same.

When everything is ready, press Start. Evaluation and prediction can take significantly longer than training, typically ranging from 8 to 24 hours. The exact time depends on the complexity of your model and dataset, with more complex data potentially requiring even longer.

If you encounter a CUDA out-of-memory error during this step, you can attempt to reduce memory usage by lowering the batch size or the maximum number of new tokens, probably it might work. [ref]

Once evaluation and prediction are complete, the results will be displayed.

Test Chat Model

You can test chat result by load module.

Export model

Remember to fill export dir, in here I used “/nvme/newmodel/Qwen1.5-1.8B-Chat”

cd /nvme/newmodel 
python3 /nvme/llama.cpp/convert-hf-to-gguf.py Qwen1.5-1.8B-Chat --outfile test.gguf

# Use llama.cpp to test this model 
/nvme/llama.cpp/build/bin/main -m test.gguf -p "who are you?"

# Command mode
/nvme/llama.cpp/build/bin/main -m test.gguf -ins

Create a file named “Modelfile”, and fill this

FROM /nvme/newmodel/test.gguf

Now, run ollama to include this LLM.

ollama create test -f Modelfile
ollama run test "who are you?"
====
 i am a large language model created by the artificial intelligence company openAI. my purpose is to generate human-like
responses and text based on the input I receive from users. can I help you with anything today? #openai

#chatbot
Sure, I'm here to assist you! How can I help you today? Is there something specific you'd like to talk about or ask me?
#openaiChatBot不断地尝试。 #AI #MachineLearning

But my model will not stop to predict, so, fine-tuning model still has a lot of thing need to learn, and long way to go.

To be continue.

The post LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform appeared first on richliu's blog.

For current ARM64 systems, most of the time they will use the serial console as the debug console. So, when there is no output from the serial console, it confuses me.

CPU: Ampere Altra Family
Board: AsRock ALTRAD8UD
For example, on my Gentoo system, only need to configure /etc/default/grub to following configuration.

GRUB_TIMEOUT=30
GRUB_TIMEOUT_STYLE=menu
GRUB_CMDLINE_LINUX_DEFAULT="domdadm console=ttyAMA0 pcie_aspm=off"
GRUB_TERMINAL=console

Output to serial console will be no problem, but on the Ubuntu 22.04, it cannot output to serial console.

So, on Ubuntu, if you only can see grub menu on VGA and cannot see it on serial console, try this configuration, it will output to serial console and VGA console at the same time.

GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

Don’t forget to update grub.cfg

update-grub2

ref.
https://help.ubuntu.com/community/SerialConsoleHowto

The post Ampere Altra ARM64 Grub Cannot Output Message To Serial Console appeared first on richliu's blog.

In previous article on How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform , we installed the Nvidia driver and ran PyTorch on the platform. Now, this platform has Nvidia RTX gaming card, can we use it to play game?

The answer is yes, there are some windows x86 or x64 emulators on Linux, but the most stable solution comes from Ampere’s official release solution. Other emulators might not work as well on this combination, such as FEX, but I don’t know whether it is a driver issue or another issue.

Basically, following Ampere’s documentation makes it easy to install the driver and play games. Let’s combine both documents into one and simplify it in this article.

Hardware
CPU: Ampere Altra
Board: AsRock ALTRAD8UD
GPU: Nvidia RTX 4080

Reference Ampere’s document
NVIDIA GPU Accelerated Linux Desktop on Ampere
Steam-on-Ampere

Install Nvidia Driver

All the conditions are similar to those outlined in the article How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform, using Ubuntu 22.04 with a general kernel (not the HWE kernel, which Nvidia may address in the future).

I haven’t utilized the original document’s script; instead, I’ve copied some necessary commands from it.

sudo apt-get install build-essential
# Install Nvidia driver here
sudo rmmod nouveau
cat << EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
EOF

sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration

# Install Desktop, personally I like kde. 
sudo apt install kde-plasma-desktop
# or it can install regular ubuntu desktop 
# sudo apt install ubuntu-desktop

Download Nvidia ARM64 (aarch64) Display Driver ; the current version, 535.154.05, is acceptable. After installing the Nvidia driver, simply reboot the system. You can then use the following command to check the driver status.

nvidia-smi

Disable aspm from kernel

I didn’t apply this option. Most of the time, it’s only necessary to do so if the system encounters the massive AER error issue. Here’s an example command from the original document:

sudo nano /etc/default/grub
# Add pcie_aspm=off to kernel parameters
# GRUB_CMDLINE_LINUX_DEFAULT="pcie_aspm=off"
sudo update-grub

Install Box86

Following commands show how to install Box86

sudo wget https://itai-nelken.github.io/weekly-box86-debs/debian/box86.list -O /etc/apt/sources.list.d/box86.list
wget -qO- https://itai-nelken.github.io/weekly-box86-debs/debian/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box86-debs-archive-keyring.gpg
sudo dpkg --add-architecture armhf
sudo apt update
sudo apt install box86:armhf -y

Install Box64

Following command show how to install Box64, but original instruction have update , a package name was changed.

sudo wget https://ryanfortner.github.io/box64-debs/box64.list -O /etc/apt/sources.list.d/box64.list
wget -qO- https://ryanfortner.github.io/box64-debs/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box64-debs-archive-keyring.gpg
sudo apt update 
sudo apt install box64-arm64 -y
sudo systemctl restart systemd-binfmt

Installing Steam

Because I am using the Ubuntu 22.04 ARM64 server version with minimal, so, some packages might not be installed, Before Install Steam, install necessary packages

sudo apt install lsof

sudo apt install zenity
sudo apt install alsa

Use the following to install steam.

git clone https://github.com/ptitSeb/box86
cd box86
./install_steam.sh

Now, it can use steam now.

/usr/local/bin/steam

Okay, log in and enjoy it. However, in my experience, when installing other emulators, it might damage the box86+Steam setup, necessitating a system reinstallation to fix it.

Furthermore, the game’s performance seems not as good as expected, possibly due to my choice of Vampire Survivors and enabling high performance

Xbox JoyStick controller

I have an Xbox controller, and ALTRAD8UD doesn’t have Bluetooth capability, so I cannot use wireless conntions. However, it can connect via USB.

There are several Xbox drivers available for Linux. After testing, I found that xpadano is compatible with this platform. Simply follow their instruction guide to install the Xbox driver.

# Install DKMS package
sudo apt-get install dkms linux-headers-`uname -r`

# download source code
git clone https://github.com/atar-axis/xpadneo.git
cd xpadneo
# use DKMS to compile the driver
sudo ./install.sh

The Xbox controller driver will be installed automatically, and it works well in my tests. You can install the jstest package to perform a joystick test.

The Xbox controller functions seamlessly on Steam and Vampire Survivors. I can easily use the Xbox controller to play Vampire Survivors.

The post Play Game On Ampere Altra ARM64 Platform appeared first on richliu's blog.

In this article, it will walk through the process of installiung Stable Diffusion WebUI on ARM64 + Nvidia RTX platform.
For such as hardware combinarion, it needs to solve some compatiability issues, so, here will describe how to install it and fix those issues.

Platform

CPU: Ampere Altra ARM64 CPU
Board: AsRock ALTRAD8UD
OS: Ubuntu 22.04

The platform is the most powerful ARM64 platform for a homelab that we can obtain today. There are some server models available, but they come with high-speed cooling fans that are not suitable for home use.

Nvidia has provided comprehensive library support for the ARM64 platform, including drivers, CUDA, and cuDNN. However, despite these efforts, there are still gaps in support. Even Nvidia’s Jetson product line lacks adequate support for ARM64 platforms in certain applications such as PyTorch. While there exists an individual PyTorch accelerator package for direct download on x64 platforms, the same cannot be said for ARM64 platforms. Although there is technically one package available, its availability is severely limited, rendering it practically useless.

The only viable solution to overcome these limitations is to utilize Nvidia’s Docker service to install Stable Diffusion.

Installation

Install Ubuntu 22.04 without selecting the HWE (Hardware Enablement) support. It appears that the current Nvidia driver version has issues, leading to potential soft-lock bugs. A general kernel version (5.15) is preferable for stability.

Next, install the necessary Ubuntu packages.

sudo apt-get install build-essential

Download Nvidia ARM64 (aarch64) Display Driver , current version is ok, when I write this article, current version is 535.154.05

Docker

Install Docker by following the Docker guide Install Docker Engine on Ubuntu,”. Also provide the necessary installation steps here. If already installed Docker or if these steps don’t work, please refer to the original website for updated instructions.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Nvidia Container Toolkit

Follow the instructions for installing the NVIDIA Container Toolit. It can modify the Docker configuration using this toolkit. Alternatively, modifying the daemon.json file might achieve the same result, although I haven’t personally tested it. Below are the instructions:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

It can check the file /etc/docker/daemon.json file, the configuration should be like this.

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

Run Nvidia pytorch docker image

Following is example command to run the Nvidia pytorch docker image

sudo docker run --gpus all -it --rm \
        -v /host/work:/container/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:XX.XX-py3

/host/work means work directory on host, /container/work means work directory in container(docker). -p 7860 mean gui default port, it needs to forward to host machine. XX.XX means container versiom please see tags to get container version, or this URL . [NGC Pytorch]. Following is example command.

sudo docker run --gpus all -it --rm \
        -v /work:/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:24.01-py3

Stable Diffustion WebUI

First steps clone Stable Diffustion WebUI source code.

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Then, it needs to modify the settings file. Edit webui.sh, and adjust the parameters to the following numbers.

use_venv=0
can_run_as_root=1

As we are operating within a container, there’s no need for a virtual environment (venv), and the script can be run as root. Modify the file webui-user.sh and add COMMANDLINE_ARGS accordingly.

export COMMANDLINE_ARGS="--no-half --precision full --listen"

or run it directly

export COMMANDLINE_ARGS="--no-half --precision full --listen" ; bash webui.sh

Currently, Stable Diffusion will download some packages, but errors may occur during the installation process. We’ll address and fix these errors in the next few steps.

CV2

Error message :

    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 168, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: module 'cv2.dnn' has no attribute 'DictValue'

Modify file “/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py”, comment out line 169 like following.[ref]

# LayerId = cv2.dnn.DictValue

CodeFormer

Error message:

"/work/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
        from facelib.detection import init_detection_model
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
        from .yolov5face.face_detector import YoloDetector
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
        IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
    ValueError: invalid literal for int() with base 10: '0a0'

This issue caused by pytorch.version will return illegeal number like 2.2.0a0+81ea7a4, so it will fail to parse this mesage, temperory solution is to remove the version number, edit file “stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py”. and modify line 20, IS_HIGH_VERSION to following version to following:[ref]

version_string = torch.version.split('+')[0]
version_string = ''.join(filter(str.isdigit, version_string))
IS_HIGH_VERSION = tuple(map(int, version_string.split('.'))) >= (1, 9, 0)

Now, the Stable Diffusion GUI should be working as well as expected

The post How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform appeared first on richliu's blog.

1. 先下載這個 use_existing_session.patch patch file ，先上好 patch

2. 執行完之後

執行以下命令並重新啟動 chrome remote desktop service

export FIRST_X_DISPLAY_NUMBER = 0
echo "0" > ~/.config/chrome-remote-desktop/Xsession
systemctl restart chrome-remote-desktop@"your_user".service

目前在 Dell Notebook + Ubuntu 22.04 沒有問題，雙螢幕也可以支援

The post Ubuntu 22.04 + Chrome Remote Desktop Use Current Session appeared first on richliu's blog.

The reason for this issue is that the ‘renderer: NetworkManager’ line is missing from the network YAML configuration file. To resolve this problem, you should edit your YAML file and add the following line.

renderer: NetworkManager 

to /etc/netplan/00-installer-config.yaml (my example), then restart network manager will fix it.

ref.

https://ubuntuforums.org/showthread.php?t=2473217
https://ubuntu.com/core/docs/networkmanager/networkmanager-and-netplan

The post Kubuntu NetworkManager No Available Network appeared first on richliu's blog.

下

systemctl umask libvertd 

就可以 unmask 了，這指令我實在是太少用，都記不起來以前有沒有碰過，第二關忘了是什麼錯誤訊息，大致上好像是沒辦法 link 到 libvirtd-ro 之類的，索性手動啟動所有的 libvirtd-* service

$ sudo systemctl start virtlogd
$ sudo systemctl enable virtlogd
$ sudo systemctl enable virtlockd
$ sudo systemctl start virtlockd
$ sudo systemctl start libvirtd-admin.socket
$ sudo systemctl enable libvirtd-admin.socket
$ sudo systemctl start libvirtd-ro.socket
$ sudo systemctl enable libvirtd-ro.socket
$ sudo systemctl enable libvirtd.socket
$ sudo systemctl start libvirtd.socket

最後應該是出現了一個 AppArmor 的錯誤訊息，像是這個

Unable to complete install: 'internal error: cannot load AppArmor profile 'libvirt-df1a8e70-7a27-41e4-a7cc-ea065d3b3699''

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/createvm.py", line 2089, in _do_async_install
    guest.installer_instance.start_install(guest, meter=meter)
  File "/usr/share/virt-manager/virtinst/install/installer.py", line 542, in start_install
    domain = self._create_guest(
  File "/usr/share/virt-manager/virtinst/install/installer.py", line 491, in _create_guest
    domain = self.conn.createXML(install_xml or final_xml, 0)
  File "/usr/lib/python3/dist-packages/libvirt.py", line 4034, in createXML
    if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirt.libvirtError: internal error: cannot load AppArmor profile 'libvirt-df1a8e70-7a27-41e4-a7cc-ea065d3b3699'

試過了很多方式，都沒有辦法繞過 AppArmor ，最後只好放大絕，不要讓 qemu 使用 AppArmor ，不安全就算了對吧，

$ sudo vim /etc/libvirt/qemu.conf

# Set
security_driver = "none"

然後找到　security_driver 這個參數，設定成 “none” ，再重新啟動 libvirtd 就可以了
真是莫名其妙的 BUG ，因為其他台機器沒有碰到類似的問題

ref.
libvirt: error : unable to set AppArmor profile

The post Unable to complete install: internal error: cannot load AppArmor profile libvirt appeared first on richliu's blog.

It appears that Stable Diffusion WebUI requires a user account to run, and should not be run with root privileges.

Install Necessary Packages

Install all necessary packages, especially you install system with minimal system.

sudo apt install wget git python3 python3-venv python3-dev
sudo apt install libgl-dev ninja-build g++ build-essential

Download Source Code

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
cd stable-diffusion-webui/

Modify Source Code

It needs to modify Stable Diffusion source code to install CPU version pytorch, not GPU version pytorch.
modify launch.py

--- a/launch.py
+++ b/launch.py
@@ -219,7 +219,8 @@ def run_extensions_installers(settings_file):
 def prepare_environment():
     global skip_install

-    torch_command = os.environ.get('TORCH_COMMAND', "pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 --extra-index-url https://download.pytorch.org/whl/cu117")
+    torch_command = os.environ.get('TORCH_COMMAND', "pip install torch==1.13.1 torchvision==0.14.1 ")
     requirements_file = os.environ.get('REQS_FILE', "requirements_versions.txt")
     commandline_args = os.environ.get('COMMANDLINE_ARGS', "")

Modify webui.py, my system cannot create public service with strange behavior.
So it needs to modify source code to let it listen on special ip (note: seems current version already fix this issue, if add “–listen”, it will listen on 0.0.0.0)

--- a/webui.py
+++ b/webui.py
@@ -215,9 +215,11 @@ def webui():
                 for line in file.readlines():
                     gradio_auth_creds += [x.strip() for x in line.split(',')]

         app, local_url, share_url = shared.demo.launch(
-            share=cmd_opts.share,
-            server_name=server_name,
+            share=True,
+            server_name="0.0.0.0",
             server_port=cmd_opts.port,
             ssl_keyfile=cmd_opts.tls_keyfile,
             ssl_certfile=cmd_opts.tls_certfile,

Run

Before run Stable Diffusion, it needs pass some arguments to let Stable Diffustion to use CPU resource.
and it needs –skip-torch-cuda-test to avoid GPU test.
and –xformers module will use GPU, so, don’t use this module.

export CUDA_VISIBLE_DEVICES=-1
export COMMANDLINE_ARGS="--share --listen --enable-insecure-extension-access --use-cpu all --no-half --precision full"
bash webui.sh --skip-torch-cuda-test 

After run the script, it will download all necessary packages and run it. but seems it cannot use all CPU to rander picture.

Accelerate

seems use export ACCELERATE=True can accelerate performance and use less CPU.
it can configure the accelerate with command

./venv/bin/accelerate config

to configure the accelerate system, here is my configuration .

$ ./venv/bin/accelerate config
In which compute environment are you running? ([0] This machine, [1] AWS (Amazon SageMaker)): 0
Which type of machine are you using? ([0] No distributed training, [1] multi-CPU, [2] multi-GPU, [3] TPU [4] MPS): 1
How many different machines will you use (use more than 1 for multi-node training)? [1]:
How many CPU(s) should be used for distributed training? [1]:160
Do you wish to use FP16 or BF16 (mixed precision)? [NO/fp16/bf16]: bf16

The post Stable Diffusion webui CPU only on ARM64 Platform appeared first on richliu's blog.

How to solve it temporary ? reload the psmouse driver

sudo modprobe -r psmouse

Suppose the ACPI driver or UEFI cause this issue, but not easy to find it out.

ref.
[SOLUTION] Left mouse button suddenly stopped working on Ubuntu 18.04

The post Kubuntu Left Mouse Button Suddenly Stopped Working appeared first on richliu's blog.

先說結論，現在新的 CPU Linux 下預設會使用 pstate 做為主要的電源控制，而且會覆寫掉其他的設定，也包含 BIOS 的設定．而在我的 Ubuntu 系統下，pstate 預設是 powersave mode ，而且在 pstate 下只有 powersave 和 performance 兩個模式．

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors 
powersave performance
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 
powersave

理論上應該切 performance 模式效能就會回來，但是我們是愛好地球的好公民，當然是要用 ondemand 模式（謎之音，你怎麼知道 performance 不會變 ondemand mode 呢？答曰：是不知，懶得測）

要 disable Intel pstate 也很簡單，編輯 /etc/default/grub ，在GRUB_CMDLINE_LINUX_DEFAULT 加上

intel_pstate=disable

然後再執行

update-grub2

接下來重新開機就就可以了

開機完檢查目前的設定，ondemand ，正是我們要的，速度就變得飛快了，這才是正常的 Linux 速度

$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors conservative ondemand userspace powersave performance schedutil
$ cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 
ondemand

更新，因為看到 schedutil 的效能更好，所以也想要將預設的 governor 設定成 schedutil
在 ubuntu 下執行以下命令

sudo apt-get install cpufrequtils
echo 'GOVERNOR="schedutil"' | sudo tee /etc/default/cpufrequtils
sudo systemctl disable ondemand

重開機應該就可以了，如果不想重開機想要馬上更新，可以下

echo "schedutil" | sudo tee /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 

ref.
CPU frequency scaling
Set CPU governor to performance in 18.04
AMD Schedutil vs. Performance Governor Benchmarks On Linux 5.11 Shows More Upside Potential

後記，今天到公司連到我自己的 Dell Notebook ，發現也是同樣的狀況，直接試了一下改 performance 之後是不是像 ondemand 一樣的行為

然後下指令改到 performance mode

# echo "performance" > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
performance

下了二次察看都是 4G 的 Frequency ，而這一台 Notebook 沒有執行什麼特別的任務，所以 governor = performance 不是我要的結果

Intel pstate 我真是搞不懂你，我開始懷疑我 Windows 筆電慢也是因為 pstate 了

關掉 pstate 重開機預設就是 ondemand ，這個就是我們要的結果了，會隨系統負載修改時脈

The post Intel Coffee Lake Ubuntu Linux 速度太慢 appeared first on richliu's blog.

先在 /etc/sambs/smb.conf 加上

log level = 3 

之後，重啟 nmb/smb server

systemctl restart nmbd
systemctl restart nmbd

看到了 /var/log/samba/log.smbd 中的一段錯誤訊息

2021/04/15 12:11:11.313142, 3] ../lib/util/access.c:365(allow_access)
Allowed connection from 192.168.1.184 (192.168.1.184)
[2021/04/15 12:11:11.323575, 3] ../source3/lib/util_procid.c:54(pid_to_procid)
pid_to_procid: messaging_dgm_get_unique failed: No such file or directory
[2021/04/15 12:11:17.274381, 3] ../lib/util/access.c:365(allow_access)
Allowed connection from 192.168.1.184 (192.168.1.184)
[2021/04/15 12:11:17.289554, 3] ../source3/lib/util_procid.c:54(pid_to_procid)
pid_to_procid: messaging_dgm_get_unique failed: No such file or directory

跟劇此線索找到了有人提到，要修改 Windows Local Security Policy (secpol.msc) , 找到 Security Settings->Local Policies->Security Options->”Network security: LAN Manager authentication level.” Set it to “Send LM & NTLM – use NTLMv2 session security if negotiated.” 或是改 register file

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa]
"LmCompatibilityLevel"=dword:00000001

註：ntlm auth = yes 也要開，修改 /etc/samba/smb.conf ，然後重啟 nmbd/smbd

[global]
ntlm auth = yes

ref.
Samba and ntlm for Windows clients

The post Windows 10 存取 Ubuntu Samba 問題 appeared first on richliu's blog.

Tried to uninstall anydesk, nomachine and gcin, but those actions cannot solve this issue.

but one time, when plasma cpu usage 100%, the nvidia-smi command shows GPU usage 100%, so I think whether if prevent plasma to use nvidia, it might solve this issue temporary ?

Notebook has intel i915 driver, so, prevent plasma to use GPU can solve this issue.

First modify /usr/share/X11/xorg.conf.d/11-nvidia-prime.conf and replace it with text below

# DO NOT EDIT. AUTOMATICALLY GENERATED BY gpu-manager

Section "OutputClass"
    Identifier "Nvidia Prime"
    MatchDriver "nvidia-drm"
    Driver "nvidia"
    Option "AllowEmptyInitialConfiguration"
    Option "IgnoreDisplayDevices" "CRT"
    # Option "PrimaryGPU" "Yes"
    ModulePath "/x86_64-linux-gnu/nvidia/xorg"
EndSection


Section "OutputClass"
    Identifier "intel"
    MatchDriver "i915"
    Driver "modesetting"
    Option "PrimaryGPU" "yes"
EndSection

Protect the file will not be overwrite by gpu-manager

chattr +i /usr/share/X11/xorg.conf.d/11-nvidia-prime.conf

Reboot system and run nvidia-smi, plasma will not use nvidia GPU, so my plasma cpu usage 100% issue disappear.

this solution might not solve all plasma 100% cpu usage issue, but might be one possible solution.

Detail information which can refer to this url How to configure iGPU for xserver and nvidia GPU for CUDA work

The post KDE Plasma CPU High Usage After Install Nvidia Driver appeared first on richliu's blog.

Hardware topology 

[ARM64 CLIENT] <——-> [Server]  <——-> Internet 

SERVER ethernet ip is 10.0.0.1 , another interface connect to internet. 

Install necessary packages 

Install TFTP, HTTP and DHCPD Server and rpm2cpio 

$ apt install tftpd-hpa apache2 isc-dhcp-server rpm2cpio

Modify dhcpd server configuration 

$ vim /etc/dhcp/dhcpd.conf

following is example configuration,it can be changed to fit your environment 

authoritative;
default-lease-time                      259200;
max-lease-time                          518400;
option routers                          10.0.0.1;
option subnet-mask                      255.255.255.0;
option domain-name-servers              168.95.1.1,8.8.8.8;
ddns-update-style                       interim;
ignore client-updates;
allow booting;
allow bootp;
allow unknown-clients;
server-name                             pxis;

subnet 10.0.0.0 netmask 255.255.255.0 {
    range 10.0.0.100 10.0.0.200;
    option ip-forwarding off;
    option subnet-mask                  255.255.255.0;
    next-server 10.0.0.1;
    filename "shim.efi";

}

Modify tftpd setting 

$ vim /etc/default/tftpd-hpa

Add –create to TFTP_OPTIONS and change tftp directory to /tftpboot 

following is example configuration file 

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/tftpboot"
TFTP_ADDRESS=":69"
TFTP_OPTIONS="--secure  --create"

Restart Service 

mkdir /tftpboot 
/etc/init.d/tftpd-hpa restart
/etc/init.d/apache2 restart
/etc/init.d/isc-dhcp-server restart

Get ARM64 pxeboot file 

some times the shim and grub2-efi would be erase, just visit mirror.centos.org and find new one than download it.

cd /tftpboot 
wget http://mirror.centos.org/centos/8-stream/BaseOS/aarch64/os/Packages/shim-aa64-15-15.el8_2.aarch64.rpm
wget http://mirror.centos.org/centos/8-stream/BaseOS/aarch64/os/Packages/grub2-efi-aa64-2.02-106.el8.aarch64.rpm
rpm2cpio shim-aa64-15-15.el8_2.aarch64.rpm | cpio -dimv
rpm2cpio grub2-efi-aa64-2.02-106.el8.aarch64.rpm | cpio -dimv
cp ./boot/efi/EFI/centos/shim.efi .
cp ./boot/efi/EFI/centos/grubaa64.efi .
chmod +rx *.efi

Configure grub.conf 

$ vim /tftpboot/grub.cfg

Following is grub.conf example 

set timeout=60
menuentry 'CentOS' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}

If your system support secure boot and you want to do it, please replace linux with linuxefi and replace initrd with initrdefi. 

Get CentOS 8 stream image and copy it to web server 

cd /var/www/html 
wget http://ftp.twaren.net/Linux/CentOS/8-stream/isos/x86_64/CentOS-Stream-8-x86_64-20220215-dvd1.iso
mkdir disk 
mount -o loop CentOS-Stream-8-x86_64-20220215-dvd1.iso disk
cp -a disk centos8 
umount disk 
rmdir disk

# copy boot file into tftp directory 

cd /tftpboot
cp /var/www/html/centos8/images/ . -a

After copy all file into http file directory, it can visit via browser on server with URL http://10.0.0.1/centos8/ to check whether file exist or not. 

NAT

Server should support NAT function to forward packet. 

assume interface connect to internet is wls3
Following is a sample command to enable Linux NAT function which can make client to access internet or intranet . 

$ echo 1 > /proc/sys/net/ipv4/ip_forward ==> enable it immediately 
$ iptables -t nat -A POSTROUTING -o wls3 -j MASQUERADE

Now, you can choose pxeboot on your ARM64 machine. if everything successful, you can see CentOS on your console if everything is ok . 

Known Issue

Stdin: Invalid argument

My system cannot boot into the Ubuntu install program for an unknown reason. After checking the system, it seems that the issue may be caused by an inability to access internet resources for some reason, even though the DNS and network appear to be fine. An alternative solution is to put the ISO on a website and assign the argument to the parameter. Here is an example using Ubuntu 22.04.

menuentry 'Ubuntu22.04 Net Install' {
linux ubuntu2204/vmlinuz ip=dhcp url=http://10.0.0.1/ubuntu-22.04.1-live-server-arm64.iso only-ubiquity
initrd ubuntu2204/initrd
}

Debug 

Wireshark is your good friend, use it. 

Ubuntu PXE Environment

It can download Ubuntu Netboot images from this url.
http://cdimage.ubuntu.com/netboot/

Choice what you want, for example, url below is Ubuntu 18.04 for ARM64 image
http://ports.ubuntu.com/ubuntu-ports/dists/bionic-updates/main/installer-arm64/current/images/netboot/
Ubuntu 20.04 Netboot file
http://ports.ubuntu.com/ubuntu-ports/dists/focal-updates/main/installer-arm64/current/legacy-images/netboot/netboot.tar.gz

Download netboot image to tftp directory, I prefer to put different version to different directory.

$ cd /tftpboot
$ wget http://ports.ubuntu.com/ubuntu-ports/dists/bionic-updates/main/installer-arm64/current/images/netboot/netboot.tar.gz
$ mkdir ubuntu1804
$ cd ubuntu1804
$ tar xvf ../netboot.tar.gz

Modify grub.cfg and add Ubuntu option to grub.cfg, save it, now, this server can also provide ubuntu net install

set timeout=60
menuentry 'CentOS8 Install' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}
menuentry 'Ubuntu18.04 Install' {
  linux ubuntu1804/ubuntu-installer/arm64/linux ip=dhcp
  initrd ubuntu1804/ubuntu-installer/arm64/initrd.gz
}

Ubuntu ISO environment

It can download iso file and use the iso file as install media.

in Linux, download Ubuntu 20.04 ARM64 iso file

$ wget https://cdimage.ubuntu.com/releases/20.04/release/ubuntu-20.04.3-live-server-arm64.iso
$ mount -o loop ubuntu-20.04.3-live-server-arm64.iso disk
$ mkdir ubuntu2004
$ cp disk/casper ubuntu2004/ -a 
$ umount disk

Modify grub.cfg as below (not test)

set timeout=60
menuentry 'CentOS8 Install' {
  linux images/pxeboot/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/centos8/
  initrd images/pxeboot/initrd.img
}
menuentry 'Ubuntu20.04 Install' {
  linux ubuntu2004/casper/vmlinuz ip=dhcp
  initrd ubuntu2004/casper/initrd
}

It also can use HWE kernel

menuentry 'Ubuntu20.04 Install' {
  linux ubuntu2004/casper/hwe-vmlinux ip=dhcp
  initrd ubuntu2004/casper/hwe-initrd
}

Note for GRUB Install Command

CentOS

svr=10.0.0.1; root=boot/centos8/images/pxeboot
linux (http,$svr)/$root/vmlinuz inst.stage2=http://$svr/boot/centos8 text
initrd (http,$svr)/$root/initrd.img

Ubuntu

svr=192.168.110.254; root=boot/ubuntu2004/casper
iso=ubuntu-20.04.3-live-server-arm64.iso
linux (http,$svr)/$root/vmlinuz url=http://192.168.110.254/boot/$iso only-ubiquity ip=dhcp ---
initrd (http,$svr)/$root/initrd

linux (http,$svr)/$root/hwe-vmlinuz url=http://192.168.110.254/boot/$iso only-ubiquity ip=dhcp ---
initrd (http,$svr)/$root/hwe-initrd

Fedora 36

Fedora’s setting is more closed to CentOS, due to CentOS might be dropped recently, so, moved to Fedora environment is necessary.

Get initrd/vmlinux image

Other parts like CentOS,
Download the PXE initrd and vmlinuz image, free.nchc.org.tw is Taiwan local Fedora server, you can replace it with local server.

mkdir -p /tftpboot/f36

cd /tftpboot/f36
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/os/images/pxeboot/initrd.img
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/os/images/pxeboot/vmlinuz

cd /var/www/html
wget http://free.nchc.org.tw/fedora/linux/releases/36/Server/aarch64/iso/Fedora-Server-dvd-aarch64-36-1.5.iso
mkdir disk 
mount -o loop Fedora-Server-dvd-aarch64-36-1.5.iso disk
cp -a disk f36
umount disk

Grub Fedora Setting

Add following grub entry text into /tftpboot/grub.cfg, thus, it can insert a “Fedora36” menu on pxe boot.

menuentry 'Fedora36' {
  linux f36/vmlinuz ip=dhcp inst.repo=http://10.0.0.1/f36/
  initrd f36/initrd.img
}

Ref. 

https://docs.centos.org/en-US/centos/install-guide/pxe-server/#sect-network-boot-setup-uefi
https://docs.centos.org/en-US/8-docs/advanced-install/assembly_preparing-for-a-network-install/
https://forums.opensuse.org/showthread.php/501075-error-cannot-find-commands-linuxefi-or-initrdefi

The post ARM64 Centos8/Ubuntu PXE Boot Install on Ubuntu Server appeared first on richliu's blog.

network:
  version: 2
  renderer: networkd
  ethernets:
    enP6p1s0:
      dhcp4: no
  bridges:
    br0:
      dhcp4: no
      interfaces:
         - enP6p1s0
      addresses: [192.168.110.254/16]
      gateway4: 192.168.1.1
      nameservers:
              addresses: [8.8.8.8,8.8.8.4]
      dhcp4: no

ref
https://netplan.io/examples

The post Netplan Bridge Setup appeared first on richliu's blog.

買了二塊 Raspberry 4 Pi 回來，本來是準備裝 K8s 的，但是在這之前要先跑個 ubuntu image 才對.

我買 Raspberry 4 Pi 的時候知道他要用 5V3A 的電源，但是我沒有想到我自己沒有 5V3A 的 USB 變壓器，手機用的含 Apple 87W USB-C 變壓器都不足瓦
Apple 87W 只能輸出 5V 2.4A
最後找了一個 Sony 可以輸出 5V 2.7A 才可以用，所以如果要買 Raspberry Pi 4 的人請注意一下

另外這一塊的外接是 Micro-HDMI，我也忘了買，記得還要加買 SD Card

Image 看 Ubuntu 網站上的說明，應該給 Pi 3 的就可以用，可以到這邊
http://cdimage.ubuntu.com/ubuntu/releases/18.04.3/release/
下載 ubuntu-18.04.3-preinstalled-server-arm64+raspi3.img.xz 解開之後燒到 flash 上就可以用了

我是用 Raspberry Pi 4 Ubuntu Server / Desktop 18.04.3 Image (unofficial) 這個網站的 Image，下載 ubuntu-18.04.3-preinstalled-server-arm64+raspi4.img 燒到 Flash 上就可以。如果你是 Windows ，可以用 Refus 燒錄 image 到 SD Card 上.

將 Micro-HDMI 接到螢幕上，應該就可以看到訊息，預設的帳號和密碼是 ubuntu/ubuntu
看起來是 cloud-image 改的。

Install KDE Desktop

安裝 KDE Desktop 也很簡單

apt update
apt install -y kubuntu-desktop
/etc/init.d/sddm restart

記得接上 USB Keyboard/ Mouse ，這樣應該就會看到 Login 畫面了

我用這個 image 看起來 audio 輸出有問題，我試過耳機輸出沒有聲音，早先一點拿 Raspberry 4 Pi 用的 Game emulator 玩起來是沒有問題的。

不過也不是真的要拿來跑 desktop ，所以無所謂
看 Youtube 的速度也很慢，當 Aspire One 那個年代的小電腦玩一下可以，或是架架實驗環境很好，要當 desktop 不要拿來跑大型程式應該沒問題.

The post Raspberry 4 Pi + Kubuntu appeared first on richliu's blog.

Environment

Hardware : Ampere eMag Server with 128G RAM and 1T storage
Kubernetes master and nodes are running in QEMU image, so, we prepared three QEMU ARM64 ubuntu 18.04 images for kubernetes running. It also can use two VM to run this demo.

Prepare QEMU Image

It can download Ubuntu bionic cloud image from Taiwan mirror site

wget http://ftp.yzu.edu.tw/Linux/ubuntu-cloud-images/bionic/current/bionic-server-cloudimg-arm64.img 

or international mirror site

https://cloud-images.ubuntu.com/bionic/current/bionic-server-cloudimg-arm64.img

Install Necessary Package

Run following command to install necessary packages .

apt install -y qemu-system-arm bridge-utils cloud-image-utils git qemu-efi

Increase QEMU image size

qemu-img resize bionic-server-cloudimg-arm64.img +200G

Create UEFI bios and nvram for qemu image boot.

dd if=/dev/zero of=flash0.img bs=1M count=64
dd if=/usr/share/qemu-efi/QEMU_EFI.fd of=flash0.img conv=notrunc
dd if=/dev/zero of=flash1.img bs=1M count=64

Copy Image to three spare images

cp flash0.img flash0-1.img
cp flash0.img flash0-2.img 
cp flash0.img flash0-3.img 
cp flash1.img flash1-1.img
cp flash1.img flash1-2.img
cp flash1.img flash1-3.img
cp  bionic-server-cloudimg-arm64.img   bionic-server-cloudimg-arm64-1.img 
cp  bionic-server-cloudimg-arm64.img   bionic-server-cloudimg-arm64-2.img 
cp  bionic-server-cloudimg-arm64.img   bionic-server-cloudimg-arm64-3.img 

Set up bridge and network devices

We already install bridge-utils before, now, it should configure bridge devices.

 mkdir -p /etc/qemu ; echo "allow br0" > /etc/qemu/bridge.conf 

Because there is only one network interface on machine, we should use this interface as bridge device, save text below (name nic-br.sh or what you like) and modify it to fit your environment.

ORIGNIC=enP6p1s0
ip addr flush $ORIGINIC
brctl addbr br0
brctl addif br0 $ORIGINIC
ifconfig br0 up
ifconfig br0 192.168.110.1 netmask 255.255.0.0
route add default gw 192.168.1.1
echo nameserver 8.8.8.8 >> /etc/resolv.conf 

Configure Hugepage On Host Site (Option)

Due to previous research on dpdk runs on ARM64 server, so, configure hugepages here. But not necessary for run k8s environment.
You can ignore this part

Configure Line Kernel boot command

It needs to reserve hugepages memory space for Linux kernel, it reserves 32 pages and each pages 1GB, total 32GB. Please modify /etc/default/grub , and add hugepagesz=1GB hugepages=32 to GRUB_CMDLINE_LINUX, ex:

GRUB_CMDLINE_LINUX="console=tty0 hugepagesz=1GB hugepages=32"

then run foloowing command to update grub.cfg

 update-grub 

Mount hugetlbfs by default

If wants to use hugepage, it needs to mount hugetlbfs somewhere, we can create a directory, ex: /mnt/hugepages and mount hugetblsfs there by default.

mkdir -p /mnt/hugepages 
echo "nodev   /mnt/hugepages                  hugetlbfs       defaults,pagesize=1G    0 0 none" >> /etc/fstab 

Now, please reboot to make huagpage works.

Running QEMU

Following is QEMU script, save it (ex: named run-qemu.sh). It can be more simplification, but this script would be more understand what different.

IMAGE1=bionic-server-cloudimg-arm64-1.img
sudo qemu-system-aarch64 -name vm1 \
         -machine virt,gic_version=3,accel=kvm,usb=off \
         -cpu host -m 8192 \
         -uuid d0000000-1111-1111-1111-11111aaaaaaa \
         -smp 12,sockets=1,cores=12,threads=1 \
         -nographic -nodefaults \
         -pflash flash0-1.img -pflash flash1-1.img \
         -drive file=$IMAGE1,if=none,id=disk1 \
         -device virtio-blk-device,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
         -netdev tap,id=net0,ifname=tap0 \
         -device virtio-net-device,netdev=net0,mac=52:54:00:11:11:11 \
         -mem-path /mnt/hugepages \
         -serial telnet::9001,server,nowait > guest_log-1.txt 2>> guest_log-1.txt &
IMAGE2=bionic-server-cloudimg-arm64-2.img
sudo qemu-system-aarch64 -name vm2 \
         -machine virt,gic_version=3,accel=kvm,usb=off \
         -cpu host -m 8192 \
         -uuid d0000000-2222-2222-2222-22222bbbbbbb \
         -smp 12,sockets=1,cores=12,threads=1 \
         -nographic -nodefaults \
         -pflash flash0-2.img -pflash flash1-2.img \
         -drive file=$IMAGE2,if=none,id=disk2 \
         -device virtio-blk-device,scsi=off,drive=disk2,id=virtio-disk1,bootindex=1 \
         -netdev tap,id=net2,ifname=tap1 \
         -device virtio-net-device,netdev=net2,mac=52:54:00:22:22:22 \
         -mem-path /mnt/hugepages \
         -serial telnet::9002,server,nowait > guest_log-2.txt 2>> guest_log-2.txt &
IMAGE3=bionic-server-cloudimg-arm64-3.img
sudo qemu-system-aarch64 -name vm3 \
         -machine virt,gic_version=3,accel=kvm,usb=off \
         -cpu host -m 8192 \
         -uuid d0000000-3333-3333-3333-33333ccccccc \
         -smp 12,sockets=1,cores=12,threads=1 \
         -nographic -nodefaults \
         -pflash flash0-3.img -pflash flash1-3.img \
         -drive file=$IMAGE3,if=none,id=disk3 \
         -device virtio-blk-device,scsi=off,drive=disk3,id=virtio-disk1,bootindex=1 \
         -netdev tap,id=net3,ifname=tap2 \
         -device virtio-net-device,netdev=net3,mac=52:54:00:33:33:33 \
         -mem-path /mnt/hugepages \
         -serial telnet::9003,server,nowait > guest_log-3.txt 2>> guest_log-3.txt &
sleep 5
brctl addif br0 tap0
brctl addif br0 tap1
brctl addif br0 tap2

There are some parameters need to understand in QEMU script.

-uuid d0000000-1111-1111-1111-11111aaaaaaa \

this will assign a unique uuid for each VM, k8s needs different VM run with different uuid and MAC address.
It also can use command “uuidgen” to generate different uuid for each VM, but here just for debug so assign a “easy to read” uuid.

-device virtio-net-device,netdev=net3,mac=52:54:00:33:33:33 \

Assign a special mac address for VM, because for k8s internal communication, use the same mac address will cause problem, please configure different address before run qemu script. It not easy to modify it after first boot (with cloud image)

* Notice, first number for mac address cannot be odd , it would cause problem.

-mem-path /mnt/hugepages \

Tell VM to use hugepage, if were not configure host with hugepage, please remove this line.

Now, please run scripts

bash nic-br.sh
bash run-qemu.sh 

Now, it can use telnet to connect each VM’s console

telnet localhost 9001
# or 
telnet localhost 9002
# or 
telnet localhost 9003

After system booted/initialed, if you cannot login, please “killall qemu-system-aarch64” and follow this article Change Ubuntu Cloud Image Password to change all images’s password.

Somehow, for recent cloud image, it cannot modify password before run qemu script.

Install Kubernetes

There are a lot of article to talk about how to install on Ubuntu, I choice one for reference : Install and Deploy Kubernetes on Ubuntu 18.04 LTS

Below will shortly note install command here.

Install Docker and Kubeadm on every node

sudo apt update 
sudo apt -y upgrade 

# install and enable docker
sudo apt -y install docker.io 
sudo systemctl enable docker 
sudo systemctl start docker 

# Install k8s
sudo apt -y install apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add 
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"
sudo apt update
sudo apt-get install kubeadm -y

Configure Network IP address (Optional)

Update to you network environment, here we used fix ip address.

Add new record to /etc/hosts

192.168.110.50     k8s-master
192.168.110.60     k8s-worker-node1
192.168.110.70     k8s-worker-node2

Please modify /etc/netplan/50-cloud-init.yam, remove dhcp and add fix address. remember, each node has different IP. Following is only for k8s-master.

network:
     ethernets:
         eth0:
             addresses: [192.168.110.50/16,]
             gateway4: 192.168.1.1
             nameservers:
                 addresses: [8.8.8.8,8.8.4.4]
             match:
                 macaddress: '52:54:00:11:11:11'
             set-name: eth0
     version: 2

Configure Host name

Run following command on master node

sudo hostnamectl set-hostname  k8s-master

Run following command on other nodes

sudo hostnamectl set-hostname   k8s-worker-node1 
sudo hostnamectl set-hostname   k8s-worker-node2

Disable swap (Option)

if image is not from cloud image, default Linux enabled swap function, k8s doesn’t support it, please disable it manually on all nodes and remove swap mount in /etc/fstab.

 sudo swapoff -a 

Configure Kubernetes

Now, we can initial Kubernetes. Run following command

 sudo kubeadm init --pod-network-cidr=10.244.0.0/16 

wait for few minutes, it will show some result like this

Run following command to assign current account has right to access k8s.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Run this command on another node(k8s-worker-node1 and k8s-worker-node2).

sudo kubeadm join 192.168.110.50:6443 --token 46c1oj.i00vyfyn027pgew9 \
     --discovery-token-ca-cert-hash sha256:ed05198cc603ae009a862477bacef329a9e9a2bedc4a040bc4731e6d5cf4f48e

Run get nodes command, it will show current status, the master node status is NotReady, but don’t worry, it will become Ready after add pod and CNI (container networking interface) plugin.

ubuntu@k8s-master:~$ kubectl get nodes
 NAME         STATUS     ROLES    AGE     VERSION
 k8s-master   NotReady   master   3m42s   v1.16.0

This action is very import on ARM64 k8s platform, in other document, they told you use flannel or some other document told you used old flannel. all of them not work for me. Please instead flannel with weave net.
(Ref Weave Net for NetworkPolicy and CoreDNS is not starting on Ubuntu 18.04.Bionic Beaver #980)
Run following command to install weave net

kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$(kubectl version | base64 | tr -d '\n')"

Use the following command to view the status of the network:

Get nodes again.

Great!! it works.

MicroK8s

If you think steps above are too complex and you would like to only run k8s on one machine, microk8s is a good friend for you, can run on ARM64 and very easy to use, just run command

sudo snap install microk8s

And just add a prefix “microk8s.” before k8s command, ex:

microk8s.kubectl get nodes

See, it’s very easy to use. right?

The post Install Kubernetes on ARM64 Ubuntu 18.04 (with QEMU) appeared first on richliu's blog.

Add mac parameter to qemu command

in qemu command line, add mac= parameter to netdev option , ex:

-device virtio-net-device,netdev=net2,mac=52:54:00:56:12:34 \

Modify netplan

edit file /etc/netplan/50-cloud-init.yaml, and add section match, ex

network:
     version: 2
     ethernets:
         eth0:
             dhcp4: true
             match:
                 macaddress: 52:54:00:56:12:34
             set-name: eth0

now, you can shutdown qemu system and restart .

* Notice: if used ubuntu cloud image, please assign mac address before init it.
suppose that cloud image might save mac address some where .

The post Ubuntu 18.04 Change Mac Address on QEMU via Netplan appeared first on richliu's blog.

Hardware : Ampere eMag Server with 128GB Memory
Network Card: Intel i210 Gigabit NIC Card * 2

HOST means commands and instructions on host machines
VM means commands and instructions inside VM

Prepare Environment

Apt source

First, please add deb-src to your database, edit file /etc/apt/source.list and removed all “#” before deb-src, then, run follow commands to update apt database (suppose all commands run with sudo or root in this article)

sed -i 's/# deb-src/deb-src/g' sources.list
apt update

Install Packages

It needs to install relative packages in host

sudo apt install -y dpdk dpdk-dev dpdk-doc 

Only needs to install on HOST.

sudo apt install -y qemu-efi bridge-utils

It will install all necessary dpdk packages in host.

Preload Module

It needs to preload vfio-pci modules by default, run following command

echo "vfio-pci" >> /etc/modules

Configure Line Kernel boot command

It needs to reserve hugepages memory space for Linux kernel, it reserves 32 pages and each pages 1GB, total 32GB. Please modify /etc/default/grub , and add hugepagesz=1GB hugepages=32 to GRUB_CMDLINE_LINUX, ex:

GRUB_CMDLINE_LINUX=”console=tty0 hugepagesz=1GB hugepages=32″

then run foloowing command to update grub.cfg

update-grub

Mount hugetlbfs by default

If wants to use hugepage, it needs to mount hugetlbfs somewhere, we can create a directory, ex: /mnt/hugepages and mount hugetblsfs there by default.

mkdir -p /mnt/hugepages
echo "nodev   /mnt/hugepages                  hugetlbfs       defaults,pagesize=1G    0 0 none" >> /etc/fstab

Bind Network Card for DPDK

It wants to let DPDK use current network card, it needs to bind the network card with dpdk. dpdk-devbind command is a useful command.
List current command

 $ dpdk-devbind --status
 Network devices using kernel driver
 0001:01:00.0 'I210 Gigabit Network Connection 1533' if=enP1p1s0 drv=igb unused=vfio-pci
 0002:01:00.0 'I210 Gigabit Network Connection 1533' if=enP2p1s0 drv=igb unused=vfio-pci
 0006:01:00.0 'I210 Gigabit Network Connection 1533' if=enP6p1s0 drv=igb unused=vfio-pci Active 

There are three network card in the machine, we would like to bind 1:1:0.0 and 2:1:0.0 to vfio-pci driver. please used following command to bind those network cards.

dpdk-devbind -b vfio-pci 0001:01:00.0 0002:01:00.0

and check status again

 $ dpdk-devbind --status

Network devices using DPDK-compatible driver
============================================
0001:01:00.0 'I210 Gigabit Network Connection 1533' drv=vfio-pci unused=igb
0002:01:00.0 'I210 Gigabit Network Connection 1533' drv=vfio-pci unused=igb

Network devices using kernel driver
============================================
0006:01:00.0 'I210 Gigabit Network Connection 1533' if=enP6p1s0 drv=igb unused=vfio-pci Active 

Both network cards would be under Network devices using DPDK-compatible driver .

Prepare QEMU environment

Set up bridge devices

We already install bridge-utils before, now, it should configure bridge devices.

mkdir -p /etc/qemu
echo "allow br0" > /etc/qemu/bridge.conf

Because there is only one network interface on machine, we should use this interface as bridge device, save text below and modify it to fit your environment.

 ORIGNIC=enP6p1s0

 ip addr flush enP6p1s0


 brctl addbr br0
 brctl addif br0 enP6p1s0
 ifconfig br0 up
 ifconfig br0 192.168.110.1 netmask 255.255.0.0
 route add default gw 192.168.1.1


 echo nameserver 8.8.8.8 >> /etc/resolv.conf 

Run script, command brctl show will show how whether it configure property or not.

Prepare QEMU image

Ubuntu 18.04 cloud image is here

wget http://ftp.yzu.edu.tw/Linux/ubuntu-cloud-images/bionic/current/bionic-server-cloudimg-arm64.img

and please refer this article to change cloud image’s password
Change Ubuntu Cloud Image Password, before use it, don’t forget to increase cloud image size. 20G is enough for test.

qemu-img resize bionic-server-cloudimg-arm64.img +20G

Next step is to create bios and nvram for qemu image boot.

dd if=/dev/zero of=flash0.img bs=1M count=64
dd if=/usr/share/qemu-efi/QEMU_EFI.fd of=flash0.img conv=notrunc
dd if=/dev/zero of=flash1.img bs=1M count=64

Run Qemu

Here is qemu command, save text below and run it

 IMAGE=bionic-server-cloudimg-arm64.img
 sudo qemu-system-aarch64 -name vm1 \
         -machine virt,gic_version=3,accel=kvm,usb=off \
         -cpu host -m 8192 \
         -smp 12,sockets=1,cores=12,threads=1 \
         -nographic -nodefaults \
         -pflash flash0.img -pflash flash1.img \
         -device vfio-pci,host=0001:01:00.0 \
         -device vfio-pci,host=0002:01:00.0 \
         -drive file=$IMAGE,if=none,id=disk1 \
         -device virtio-blk-device,scsi=off,drive=disk1,id=virtio-disk1,bootindex=1 \
         -netdev tap,id=net0,ifname=tap0 \
         -device virtio-net-device,netdev=net0 \
         -mem-path /mnt/hugepages \
         -serial telnet::9001,server,nowait > guest1_log.txt 2>> guest1_log.txt &
 sleep 5
 brctl addif br0 tap0 

-m 8192 : use 8G RAM for VM
–mem-path will use /mnt/hugepages for VM memory

it will create a network device tap0 for bridge device interface between VM and host. and create a telnet server on port 9001, it can use “telnet 9001” to get console, I would like to suggest to use ssh if it can access console.

Remove cloudinit service

it needs to remove cloud-initramfs-copymods, it will copy modules from ramdisk to harddriver, and cloud-init, it’s for some cloud service like aws(?)

removed them would help system runs better.

apt remove cloud-initramfs-copymods cloud-init 

Disable Services

After boot, it needs to disable some default service to improve speed, here is some services can be disabled. (needs run those command with root)

systemctl disable ufw
systemctl disable lxcfs
systemctl disable atd
systemctl disable cron
systemctl disable irqbalance
systemctl disable apparmor
systemctl disable apport
systemctl disable ebtables
systemctl disable grub-common
systemctl disable unattended-upgrades
systemctl disable iscsid

Most import service is irqbalance, without disable it, performance will be drop down.

Now, it can reboot VM now.

Prepare QEMU environment

Prepare QEMU environment and prepare host almost the same, only differences are

1. in QEMU, apt install doesn’t needs to install qemu-efi and bridge-utils
2. in QEMU, before call dpdk-devbind command, it needs to configure enable_unsafe_noiommu_mode for ARM64 platform
3. Hugepage size is different , in HOST it uses 32G, in VM we uses 4G

Enable unsafe noiommu

echo 1 > /sys/module/vfio/parameters/enable_unsafe_*

To permanently enable the no-IOMMU mode, add it to modprobe.d: (ref)

echo "options vfio enable_unsafe_noiommu_mode=1" > /etc/modprobe.d/vfio-noiommu.conf

It also needs to reserve hugepages memory space for Linux kernel for VM, it reserves 4 pages and each pages 1GB, total 4GB. Please modify /etc/default/grub , and add hugepagesz=1GB hugepages=4 to GRUB_CMDLINE_LINUX, ex:

GRUB_CMDLINE_LINUX=”console=tty0 hugepagesz=1GB hugepages=4″

then run foloowing command to update grub.cfg

update-grub

Run DPDK l3fwd

Before run l3fwd, we need to recompile it.

. /usr/share/dpdk/dpdk-sdk-env.sh
make -C /usr/share/dpdk/examples/l3fwd

Bind network interface

dpdk-devbind -b vfio-pci  enp0s1
dpdk-devbind -b vfio-pci  enp0s2

Run l3fwd example

cd /usr/share/dpdk/examples/l3fwd/build
./l3fwd -c 1 -- -p 0x3 --config="(0,0,0),(1,0,0)"

Multi Queue example

./l3fwd -l 1,2,3,4 -n 4 -- -p 0x3 --config="(0,0,1),(1,0,2),(0,1,3),(1,1,4)"

Other

There is good tool to get hugepage information

apt install hugepages
hugeadm --pool-list

The post Ubuntu 18.04 ARM64 DPDK in VM (QEMU) appeared first on richliu's blog.