OS: ubuntu 24.04

Install necessary packages

sudo apt update
sudo apt-get install -y build-essential chrpath cpio debianutils diffstat gawk git iputils-ping libdata-dumper-simple-perl liblz4-tool libsdl1.2-dev libthread-queue-any-perl locales python3 socat subversion texinfo wget zstd
sudo apt-get install -y uuid-dev acpica-tools

If want to compile it on x86 platform, it needs to install cross compiler gcc

sudo apt install gcc-aarch64-linux-gnu
export GCC_AARCH64_PREFIX=aarch64-linux-gnu-

Download git source code

export WORKSPACE=/nvme/tianocore
mkdir -p $WORKSPACE
cd $WORKSPACE
git clone https://github.com/tianocore/edk2.git
cd edk2
git submodule update --init
cd ..

git clone https://github.com/tianocore/edk2-platforms.git
cd edk2-platforms
git submodule update --init
cd ..
git clone https://github.com/tianocore/edk2-non-osi.git

git clone https://github.com/AmpereComputing/edk2-ampere-tools.git

git clone --depth 1 https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git

Setup the environment, first time, it needs to run the script in the edk2 directory

make -C edk2/BaseTools
cd edk2
. edksetup.sh

then it should run it again on <WORKSPACE>

cd $WORKSPACE
. edk2/edksetup.sh

Build ARM ATF

cd $WORKSPACE
cd trusted-firmware-a                                                                        
make fiptool  
make certtool  

add those parameters to your environment file and source it.

export WORKSPACE=/nvme/tianocore
export PACKAGES_PATH=$PWD/edk2:$PWD/edk2-platforms:$PWD/edk2-non-osi:$PWD/edk2-platforms/Features
ATFPATH=$WORKSPACE
export MANUFACTURER=ASRockRack
export BOARD_NAME=Altra1L2T
export PATH=$PATH:$ATFPATH/AtfTools/tools/cert_create:$ATFPATH/AtfTools/tools/fiptool

Build the edk2

# build release version
./edk2-platforms/Platform/Ampere/buildfw.sh -m ASRockRack -p Altra1L2T -b RELEASE
# build debug version 
./edk2-platforms/Platform/Ampere/buildfw.sh -m ASRockRack -p Altra1L2T -b DEBUG
# Change default parameters
EDK2_X86_EMULATOR_ENABLE=FALSE EDK2_SHELL_ENABLE=TRUE ./edk2-platforms/Platform/Ampere/buildfw.sh -m ASRockRack -p Altra1L2T -b RELEASE

Detail parameres can be seen in the build_fw.sh file

./edk2-platforms/Platform/Ampere/buildfw.sh  --help
Usage:
  ./edk2-platforms/Platform/Ampere/buildfw.sh [options]

Options:
  -b <bldtype>, --build <bldtype>  Specify the build type: DEBUG or RELEASE
  -t <tc>, --toolchain <tc>        Specify the toolchain to use: GCC or CLANG
  -m <mfg>, --manufacturer <mfg>   Specify platform manufacturer (e.g. Ampere)
  -p <plat>, --platform <plat>     Specify platform to build (e.g. Jade)
  -l <kern>, --linuxboot <kern>    Build LinuxBoot firmware instead of full EDK2 with UEFI Shell, specifying path to flashkernel
  -f, --flash                      Copy firmware to BMC and flash firmware (keeping EFI variables and NVPARAMs) after building
  -F, --full-flash                 Copy firmware to BMC and flash full EEPROM (resetting EFI variables and NVPARAMs) after building

  Note: flash options require bmc.sh file with env vars BMC_HOST, BMC_USER and BMC_PASS defined

  Available manufacturers:
    ADLINK
    Ampere
    ASRockRack

  Available platforms:
    ADLINK     -> ComHpcAlt
    Ampere     -> Jade
    ASRockRack -> Altra1L2Q
    ASRockRack -> Altra1L2T

Environment Variables:
  SECUREBOOT_DIR       - directory to store SecureBoot keys, certs etc.
  USE_EXISTING_SB_KEYS - use existing Secure Boot Platform and Update keys
  DOWNLOAD_MS_SB_KEYS  - force re-download of Microsoft Secure Boot KEK and DB certificates
  CERT_PASSWORD        - password to use when generating Platform and Update Keys and certificates
                         defaults to "password" if not specified.

  EDK2_SECURE_BOOT_ENABLE             (TRUE)
  EDK2_NETWORK_ENABLE                 (TRUE)
  EDK2_INCLUDE_TFTP_COMMAND           (TRUE)
  EDK2_NETWORK_IP6_ENABLE             (TRUE)
  EDK2_NETWORK_ALLOW_HTTP_CONNECTIONS (FALSE)
  EDK2_NETWORK_TLS_ENABLE             (TRUE)
  EDK2_REDFISH_ENABLE                 (TRUE)
  EDK2_PERFORMANCE_MEASUREMENT_ENABLE (FALSE)
  EDK2_TPM2_ENABLE                    (TRUE)
  EDK2_HEAP_GUARD_ENABLE              (FALSE)
  EDK2_X86_EMULATOR_ENABLE            (TRUE)
  EDK2_SHELL_ENABLE                   (TRUE)

If build_fw.sh run’s without any problem, it will generate a 10MB size BIOS image, but the stock ASRR image size is 32MB. Therefore, you needs to copy the EDK2 image into the stock image at offset 0x600000. You can download the Altrad8ud stock firmware from the Altrad8ud firmware download site.

dd if=edk2-uefi.bin of=stock-bios.bin bs=1MB seek=6 conv=notrunc

Here is a sample script to package it again. It will generate fw-altrad8ud.tar, which can be upgraded via the WebGUI.

cp ampere/bios.bin .
cp ampere/MANIFEST .

dd if=Build/Altra1L2T/altra1l2t_uefi.bin of=bios.bin bs=1M seek=6 conv=notrunc

tar cf fw-altrad8ud.tar bios.bin MANIFEST

Or you also can copy bios.bin(modified) to the BMC and update it directly from the BMC.
because scp or sftp didn’t work for me, I used the http to download the image.

# cd /tmp
# curl <your web>/bios.bin
# ampere_flash_bios.sh bios.bin

ref. EDK2: new support for ASRock Rack ALTRAD8UD-1L2T / ALTRAD8UD2-1L2Q and build improvements for Mt Jade and COM-HPC-ALT

The post Build Master EDK2 for Altrad8ud appeared first on richliu's blog.

所以只能用之前的 Open Source 開發環境編 Verilog 程式。首先請參照Build Risc-V on Ubuntu Linux設定開發環境

最難的部份是找到 FPGA 的 clock input pin ，每一家 LLM 給的值都是錯的。實際上就是 User Manual 以下這段，不過我找到是從 litex-board 抓出來才知道原來是這一段，給個 table 啊，這文件不行，其它的 LED dip switch 都給了，clock 不給.

An on-board 100 MHz LVDS oscillator is provided for general purpose use. This clock source is connected to differential inputs P3 and P4 and must be used as LVDS inputs to the FPGA. This pin pair also provides optimal interface to the FPGA PLL for customized use.

P3 and P4 pin 利用 LVDS 當 input. 這樣就可以寫 lpf 檔案，只能能抓出 clk ，而且知道FPGA ball number 要跟 verilog 對應。這樣就可以將LED 也抓進來了
而且看起來目前 FPGA 主要就是靠這兩根 pin 做 clk 輸入，100Mhz，如果是當 PCIe device 時也可以從 PCIe 那邊接 PCIe 的 clock 進來

接下來我們就可以編 mapping file ，像是 ecp5evn.lpf 內容如下

LOCATE COMP "clk100" SITE "P3";
IOBUF PORT "clk100" IO_TYPE=LVDS;


LOCATE COMP "rst" SITE "H2";
IOBUF PORT "rst" IO_TYPE=LVCMOS33 PULLMODE=DOWN;

# 將 Verilog 中的 "led" port 定位到 FPGA 的 LED output
LOCATE COMP "led[0]" SITE "F16";
IOBUF PORT "led[0]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[1]" SITE "E17";
IOBUF PORT "led[1]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[2]" SITE "F18";
IOBUF PORT "led[2]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[3]" SITE "F17";
IOBUF PORT "led[3]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[4]" SITE "E18";
IOBUF PORT "led[4]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[5]" SITE "D18";
IOBUF PORT "led[5]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[6]" SITE "D17";
IOBUF PORT "led[6]" IO_TYPE=LVCMOS33;
LOCATE COMP "led[7]" SITE "E16";
IOBUF PORT "led[7]" IO_TYPE=LVCMOS33;

然後寫一個簡單的 verilog 程式驗證我們的 code 是不是對的，像是一顆簡單的 CPU，然後寫了一個簡單的程式計算 1+10

module aCPU (
    input  clk,   
    input  rst,
    output reg [7:0] led,    // LED output
    output reg [7:0] debug_led    // LED output
);
    // command
    parameter OP_NOP = 4'b0000   ;
    parameter OP_LI  = 4'b0001   ;
    parameter OP_ADD = 4'b0010   ;
    parameter OP_ADDI= 4'b0011   ; // copy s_reg + imm [1:0] reg to d_reg
    parameter OP_BNER0 = 4'b0100   ;
    parameter OP_OUT = 4'b0101   ;
    parameter OP_HALT = 4'b0110   ;
    reg [3:0] pc;
    reg [7:0] register [0:3];
    reg [15:0] ir;   // ir = 指令暫存

    wire [3:0] opcode /* verilator public */ = ir[15:12];
    wire [1:0] s_reg  = ir[11:10];
    wire [1:0] d_reg  = ir[9:8];
    wire [7:0] value  = ir[7:0];

    reg [15:0] program_memory [0:15];


    // 初始化程式記憶體
    initial begin
        // 程式: 計算 1 + 10 並輸出
        program_memory[0] = {OP_LI, 2'b00, 2'b00, 8'd1};    // LI only load to d_REG. s_reg no work
        program_memory[1] = {OP_LI, 2'b01, 2'b01, 8'd11};   //
        program_memory[2] = {OP_ADDI, 2'b10, 2'b10, 8'b0000}; // R3 = R2 + IMM
        program_memory[3] = {OP_ADD, 2'b00, 2'b00, 8'b0001}; // R2 = R0 + IMM => R0
        program_memory[4] = {OP_BNER0, 2'b00, 2'b01,8'b00000010}; // R0 跟 R1 比若不相同跳到 2
        program_memory[5] = {OP_OUT,  2'b10, 2'b00, 8'b00000000}; // 輸出 s_reg to serial
        program_memory[6] = {OP_HALT, 2'b10, 2'b01, 8'b00000000};     // 停止
    end
    always @(posedge clk or posedge rst) begin
        if (rst) begin
            pc <= 0;
            register[0] <= 0;
            register[1] <= 0;
            register[2] <= 0;
            register[3] <= 0;
            led <= 8'b11111111;
            debug_led <= 8'b11111111;
        end
        else begin
            pc <= pc + 1;
            ir <= program_memory[pc];
            // $display("pc[%d] ir:%x reg[0]:%d reg[1]:%d reg[2]:%d value: %x \n", pc, ir, register[0], register[1], register[2], value);
            case (opcode)
                 OP_NOP: begin
                 end
                 OP_LI: begin
                    register[s_reg] <= value;
                 end
                 OP_ADD: begin
                    register[d_reg] <= register[s_reg] + value;
                 end
                 OP_ADDI: begin
                    register[d_reg] <= register[s_reg] + register[value[1:0]];
                 end
                 OP_BNER0: begin
                    if (register[s_reg] != register[d_reg]) begin
                        pc <= value[3:0];
                    end
                 end
                 OP_OUT: begin
                    led <= register[s_reg];
                 end
                 OP_HALT: begin
                    pc <= pc - 1;
                 end
                 default: begin
                    // $display("Error No this command OP_CODE: %x ir: %x ", opcode, ir);
                 end
            endcase
        end
    end
endmodule

當然為了這個程式就需要寫一個 test bench 檔案，就叫 ${PRJNAME}_tb.cpp
這不僅僅是當 test bench ，後來在弄的過程才知道可以靠 $display 顯示電路內的狀態，比幻想簡單太多了

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include "VsCPU.h"
#include "verilated.h"

#include "verilated_vcd_c.h"

int main (int argc, char** argv, char** env){

    VerilatedContext* contextp = new VerilatedContext;
    contextp->commandArgs(argc, argv);
    VsCPU* top = new VsCPU{contextp};

    VerilatedVcdC* tfp = new VerilatedVcdC;
    contextp->traceEverOn(true); // open trace function
    top->trace(tfp, 99);
    tfp->open("wave.vcd");
    int clk = 0;

    int cycles = 0;
    while (!contextp->gotFinish() && (cycles < 100) ) {
      clk = !clk;
      top->clk100 = clk;
      top->eval();
      tfp->dump(contextp->time()); // dump wave
      contextp->timeInc(1); // move time to next clock

      if (cycles > 50) {
          // 檢查 LED 輸出是否為預期值 (1 + 10 = 11)
          if (top->led == (u_int8_t) ~0xc8) {
              printf("SUCCESS: 1 + 10 = %d\n", top->led);
              break;
          }
      }
      cycles++;
      if (cycles >= 99) {
          printf("TIMEOUT: Expected result not reached\n");
      }

    }
    //
    if (tfp) {
        tfp->close();
        delete tfp;
    }
    delete top;
    return 0;
}

然後撰寫一個 Makefile ，可以直接 Compile Code 變 bitstream 而且可以上傳到 FPGA 上.

DEVICE = um-45k
PACKAGE = CABGA381

PRJNAME=aCPU

all: ${PRJNAME}.bit

${PRJNAME}.json: ${PRJNAME}.v
        yosys -p "synth_ecp5 -json ${PRJNAME}.json" ${PRJNAME}.v

${PRJNAME}_out.config: ${PRJNAME}.json
        nextpnr-ecp5 --$(DEVICE) --package $(PACKAGE) --json ${PRJNAME}.json \
        --lpf ecp5evn.lpf --textcfg ${PRJNAME}_out.config

${PRJNAME}.bit: ${PRJNAME}_out.config
        ecppack --svf ${PRJNAME}.svf ${PRJNAME}_out.config ${PRJNAME}.bit

sim:
        verilator -Wall --cc --exe --build --trace ${PRJNAME}.v ${PRJNAME}_tb.cpp

program:
        openFPGALoader -b ecp5_evn ${PRJNAME}.bit

clean:
        rm -f *.json *.config *.bit *.svf

make sim 之後執行 ./obj_dir/VaCPU 就可以看到程式輸出的結果，果然還是印出來比較簡單 debug

同時也會產生 wave.vcd 這時可以用 gtkwave 看波形

make program 之後就可以燒到 FPGA 板子上啦，輸出的燈號是反向的

The post Run verilog Program on Lattice ECP5 Versa with OpenSource Tools appeared first on richliu's blog.

Lattice Versa EVB Board 晶片型號不同

這個是最主要的問題，因為文件並不會提到這件事，都會用 versa_ecp5 帶過，但是實際上 versa_ecp5 是使用 LFE5UM5G-85F 的晶片，晶片比較新，邏輯閘數也比較多。一般市面上賣的 versa_ecp5 是 LFE5UM5G-45F，一片約在一萬元左右。
我從淘寶看到一塊比較便宜的 LFE5UM5G-45F ，結果買回來後是 LFE5UM-45F ，沒有 5G。查詢網路上的資料，大概只差在 SEDERS 最高頻可以到 5G ，還有速度比較快。

但是 LiteX default versa_ecp5 是預設用 LFE5UM5G-85F ，在燒錄時就會失敗，ID 不一樣

此時我們就要重編一次 Linux-on-litex-vexriscv ，加上 device 的參數，這樣就可以了

./make.py --device LFE5UM --board=versa_ecp5 --cpu-count=1 --build

編出來的 bitstream 和 fpga hardware 不同的錯誤訊息，可以看到 hardware id 不一樣

$ openFPGALoader -b ecp5_evn versa_ecp5.bit
empty
Jtag frequency : requested 6.00MHz    -> real 6.00MHz
Open file: DONE
b3bdffff
Parse file: DONE
mismatch between target's idcode and bitstream idcode
        bitstream has 0x81112043 hardware requires 0x01112043
Error: Failed to program FPGA: std::exception

連接上 EVB Board

EVB Board 應該支援 ethernet load ，但是我還沒試出來，就當 TBD 吧，看設定檔內 ethernet 是可以用的

EVB Board 上的 FT2232H 支援 JTAG 和 Serial Port ，Linux kernel 的 image 可以從 serial port load，不過速度好慢啊

我之前用 Diamond programmer 要燒錄編出來的 bitstream file ，但是因為 id mismatch 的關係所以燒不上去，所以我最後是找了一台乾淨的 Linux 直接接 EVB Board 的 USB （要注意，這塊版子的 USB Port 是 mini-USB ，如果沒附線要有支援 mini-USB 的線），就這樣找到最後才發現根本型號不一樣，所以這篇就不會提到 Diamond programmer

記得接上之前要將 FTDI 的J50跳線改成下圖這樣，要跳過 iSPclock

（原文 Note: If you are using a Versa board, you will need to change J50 to bypass the iSPclock. Re-arrange the jumpers to connect pins 1-2 and 3-5 (leaving one jumper spare). See p19 of the Versa Board user guide.）

接上去之後，上電之後，Linux 下會出現 /dev/ttyUSB0 是 JTAG 的介面，/dev/ttyUSB1 是 serial port 的介面

在另一個視窗先執行 litex_term（記得在 venv 環境下執行）

litex_term --images=images/boot.json /dev/ttyUSB1

燒錄 bitstream 到板子上

最後一步就是燒錄上去了，我是用 OpenFPGAloader ，以下是安裝步驟

sudo apt install -y  libftdi1-dev

cd /nvme/cpu
git clone https://github.com/trabucayre/openFPGALoader
cd openFPGALoader
mkdir -p build
cd build
cmake ../
make -j `nproc`
sudo make install 

然後到 linux-on-litex-vexriscv 下用 OpenFPGAloader 暫時 load bitstream 上去，這速度很快，這樣就不用燒上去了

cd /nvme/cpu/linux-on-litex-vexriscv/build/versa_ecp5/gateware
$ openFPGALoader -b ecp5_evn versa_ecp5.bit
empty
Jtag frequency : requested 6.00MHz    -> real 6.00MHz
Open file: DONE
b3bdffff
Parse file: DONE
Enable configuration: DONE
SRAM erase: DONE
Loading: [==================================================] 100.00%
Done
Disable configuration: DONE

如果順利，可以在剛剛執行 litex_term 上看到 load image 訊息
如果用的是上一篇文章內提到的先下載好 RiscV 的 Linux image 和 rootfs ，就可以看到 Linux boot 了

這年代可以自己編一顆 CPU 跑起來也真的是很有趣

以下未完成，待續

設定 udev

這塊版子看起來要設定過 udev 才能被 LiteX 抓到，首先新增檔案 /etc/udev/rules.d/99-ftdi.rules
內容如下，記得 GROUP=換成 Linux login 帳號

SUBSYSTEMS=="usb", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6010", MODE="0660", GROUP="使用者名稱"

然後執行以下命令之後再重新上電即可

sudo udevadm control --reload-rules
sudo udevadm trigger

這個命令也可以像 OpenFPGAloader 一樣將 bitstream load 到 FPGA 上

./make.py --device LFE5UM --board=versa_ecp5 --cpu-count=1 --load 

45F Gate count 夠放二顆 CPU ，所以可以跑 SMP

./make.py --device LFE5UM --board=versa_ecp5 --cpu-count=2 --build
./make.py --device LFE5UM --board=versa_ecp5 --cpu-count=2 --load 

The post Running Risc-V Linux on Lattice Versa ECP5 EVB appeared first on richliu's blog.

OS: ubuntu 22.04 / 24.04

下載和 compile source code

理論上用 linux-on-litex-vexriscv 照著作就可以了，不過中間還是有很多邊邊角角的小問題要處理 ，如果對系統不夠熟的人會在這邊卡很久。

Install Ubuntu Packages

sudo apt-get install -y build-essential device-tree-compiler wget git python3-setuptools python3-dev
sudo apt-get install -y libssl-dev libncurses-dev libeigen3-dev
sudo apt-get install -y libevent-dev libjson-c-dev libboost-iostreams-dev
sudo apt-get install -y libreadline-dev tcl-dev libffi-dev bison flex gawk gtkwave

sudo apt-get install -y file bc unzip meson 
sudo apt-get install -y libboost-dev libboost-filesystem-dev libboost-thread-dev libboost-program-options-dev libboost-python-dev libboost-all-dev


# 24.04
sudo apt-get install -y python3-venv 

Install SBT

sudo apt-get update
sudo apt-get install apt-transport-https curl gnupg -yqq
echo "deb https://repo.scala-sbt.org/scalasbt/debian all main" | sudo tee /etc/apt/sources.list.d/sbt.list
echo "deb https://repo.scala-sbt.org/scalasbt/debian /" | sudo tee /etc/apt/sources.list.d/sbt_old.list
curl -sL "https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x2EE0EA64E40A89B84B2DF73499E82A75642AC823" | sudo -H gpg --no-default-keyring --keyring gnupg-ring:/etc/apt/trusted.gpg.d/scalasbt-release.gpg --import
sudo chmod 644 /etc/apt/trusted.gpg.d/scalasbt-release.gpg
sudo apt-get update
sudo apt-get -y install sbt

Install OpenJDK

這邊我們用 OpenJDK 17

sudo apt-get install -y openjdk-17-jdk openjdk-17-jre

重編 cmake

內建的 cmake 看起來版本太低不能用，所以要下載 cmake 手動編譯

wget https://github.com/Kitware/CMake/releases/download/v4.0.3/cmake-4.0.3.tar.gz
tar xvf cmake-4.0.3.tar.gz
cd cmake-4.0.3
./configure
make -j `nproc`
sudo make install

下載 linux-on-litex-vexriscv

編 buildroot 要指定 linux-on-litex-vexriscv ，所以先下載

git clone https://github.com/litex-hub/linux-on-litex-vexriscv

編 Buildroot 編出 Risc-V 的 Cross Compiler

我因為是在其他 CPU 的平台上跑，所以只能自己編 buildroot ，還好 buildroot 已經很成熟了，只是要花時間編出來而已，第一次有錯誤訊息沒關係，確認 output/host/bin/ 下有”riscv32-” 開頭的 toolchain 就可以了
等到編完 litex 之後再回來重編一次 buildroot 即可

git clone http://github.com/buildroot/buildroot
cd buildroot
make BR2_EXTERNAL=../linux-on-litex-vexriscv/buildroot/ litex_vexriscv_defconfig
make

iverilog 版本太舊，重編 iverilog

iverilog 和 yosys 都要注意不能用到 ubuntu 系統內建的版本，它們都太舊了
都要手動重裝新版的

sudo apt-get install -y autoconf gperf make gcc g++ bison flex
git clone https://github.com/steveicarus/iverilog.git
cd iverilog
autoconf
./configure 
sudo make -j `nproc` install 

下載 yosys/prjtrellis/nextpnr

主要是參照這篇 LiteX/VexRiscv 簡介與使用 (二) 始有晝夜 ，但是因為已經是幾年前了，有些東西有點小差異，不過不影響，目錄就用我的工作目錄

這邊要小心一點，最好是先清空相關的程式碼，像是 /nvme/cpu/sysroot/* 和 /usr/local/bin 下的某些檔案，我就吃了虧卡關了很久

如果是 Ubuntu 24.04 ，要先照下面一節的方式先切到 venv 的環境再安裝

# 如果還沒做就先將路徑加上去
# 編 buildroot 時不能加入 buildroot 的 path，要編 litex 時再加入即可export PATH=$PATH:/usr/local/bin:/nvme/cpu/buildroot/output/host/bin/:/nvme/cpu/sysroot/bin/:~/.local/bin:

mkdir -p /nvme/cpu/sysroot
cd /nvme/cpu
git clone --recursive https://github.com/YosysHQ/yosys
git clone --recursive https://github.com/YosysHQ/prjtrellis
git clone --recursive https://github.com/YosysHQ/nextpnr
cd yosys
make PREFIX="/nvme/cpu/sysroot" CONFIG=gcc ENABLE_TCL=1 ENABLE_ABC=1 ENABLE_GLOB=1 ENABLE_PLUGINS=1 ENABLE_LIBYOSYS=1 ENABLE_PROTOBUF=1 ENABLE_ZLIB=1 ENABLE_PYOSYS=1 test all install -j `nproc`
cd ..
cd prjtrellis/libtrellis
cmake -DCMAKE_INSTALL_PREFIX=/nvme/cpu/sysroot .
make install -j `nproc`
cd ../../
cd nextpnr
cmake . -DARCH=ecp5 -DTRELLIS_INSTALL_PREFIX=/nvme/cpu/sysroot -B build
cd build
sudo make install -j `nproc`

下載和編 litex

ubuntu 22.04 可以用 –user ，但是 24.04 就強制要用 venv　要先有環境

cd /nvme/cpu
python3 -m venv litex-venv
source litex-venv/bin/activate

Ubuntu 22.04

cd /nvme/cpu
wget https://raw.githubusercontent.com/enjoy-digital/litex/master/litex_setup.py
chmod +x litex_setup.py
./litex_setup.py --init --install --user

Ubuntu 24.04

./litex_setup.py --init --install

Compile Lite-X

cd /nvme/cpu/linux-on-litex-vexriscv
./make.py --board=versa_ecp5 --cpu-count=1 --build

跑 Simulation

順利的話這時環境應該 OK 了，可以跑 simulation 了

先下載 Linux images 到 images

cd /nvme/cpu/linux-on-litex-vexriscv/images
wget https://github.com/litex-hub/linux-on-litex-vexriscv/files/8331338/linux_2022_03_23.zip
unzip linux_2022_03_23.zip

跑 simulation

這時就可以跑 simulation 了，不過我跑了 64 Cores 好像更慢，看起來給愈多 code 會跑的更大，像是變 SMP 或是跑到 100Mhz （一核用 1Mhz）
可能要花時間設參數，–jobs 到是可以設大一點

./sim.py

# 或是
./sim.py 

# 或是
./sim.py  --jobs `nproc` --threads `nproc`

Build FPGA Bitstream Image

以下就是編 orange_crab FPGA 板子的 command

./make.py --board=orange_crab --cpu-count=1 --build

不過我編出來會 .dts failed
可以改編 versa_ecp5 ，這是 ecp5 的公板，目前應該只有 ecp5 可以靠Trellis/Yosys/nextpnr 這三個 Open Source 的軟體編出 FPGA 所需的 bitstream image，orange_crab 也是基於 ecp5 的 FPGA.

./make.py --board=versa_ecp5 --cpu-count=1 --build

不過我寫這篇的時候還沒買 FPGA ，所以燒到板子上測試要等下一篇了

Error Messages

Ram_1w_1rs_Generic.v:2:8: Timescale missing on this module as other modules have it

原始錯誤訊息

%Error-TIMESCALEMOD: /nvme/cpu/pythondata-cpu-vexriscv-smp/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v:2:8: Timescale missing on this module as other modules have it (IEEE 1800-2017 3.14.2.2)
                     /nvme/cpu/pythondata-cpu-vexriscv-smp/pythondata_cpu_vexriscv_smp/verilog/VexRiscvLitexSmpCluster_Cc1_Iw32Is4096Iy1_Dw32Ds4096Dy1_ITs4DTs4_Ldw32_Ood_Hb1.v:8:8: ... Location of module with timescale
    8 | module VexRiscvLitexSmpCluster_Cc1_Iw32Is4096Iy1_Dw32Ds4096Dy1_ITs4DTs4_Ldw32_Ood_Hb1 (
      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
%Error: Exiting due to 1 error(s)
        ... See the manual and https://verilator.org for more assistance.
make: *** [/nvme/cpu/litex/litex/build/sim/core/Makefile:44: sim] Error 1
make: Leaving directory '/nvme/cpu/linux-on-litex-vexriscv/build/sim/gateware'
Testing expectations for wide_thru_priority.v ..Test: code_verilog_tutorial_fsm_full -> ok

Test: code_verilog_tutorial_parity -> ok

修改檔案 /nvme/cpu/pythondata-cpu-vexriscv-smp/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v 加上一行

diff --git a/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v b/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v
index c6e6a96..6a57a2e 100644
--- a/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v
+++ b/pythondata_cpu_vexriscv_smp/verilog/Ram_1w_1rs_Generic.v
@@ -1,3 +1,5 @@
+`timescale 1ns/1ps
+

 module Ram_1w_1rs #(
         parameter integer wordCount = 0,

ERROR: Module `\ECLKBRIDGECS’ referenced in module `\orange_crab’ in cell `\ECLKBRIDGECS’ is not part of the design

Generating RTLIL representation for module `\VexRiscvLitexSmpCluster_Cc1_Iw32Is4096Iy1_Dw32Ds4096Dy1_ITs4DTs4_Ldw128_Ood_Hb1'.      ERROR: Module `\ECLKBRIDGECS' referenced in module `\orange_crab' in cell `\ECLKBRIDGECS' is not part of the design.   

yosys 太舊，我是裝到系統的 yosys ，移除掉系統的 yosys 再自己編 yosys 就可以了

Reference

LiteX/VexRiscv 簡介與使用 (二) 始有晝夜
LiteX
linux-on-litex-vexriscv
Prebuilt Bitstreams and Linux/OpenSBI images (抓 Linux Kernel 用)
Orange Crab 主網站
OSS CAD Suite
Buildroot

The post Build Risc-V on Ubuntu Linux appeared first on richliu's blog.

System Specifications

Nvidia Driver: NVIDIA-Linux-aarch64-570.86.16.run
Host: Ampere Altra + ALTRAD8UD
Host OS: Ubuntu 22.04 with HWE kernel (6.8)
Guest OS: Ubuntu 22.04 (ubuntu-22.04-live-server-arm64.iso)
GPU: Nvidia RTX 4080 16GB

Assumptions

1. You are familiar with Ubuntu and its basic commands.
2. You have experience using Virt-manager.
3. All commands are executed as the root user.

If anything is unclear, you can refer to external resources for additional guidance.

Host Configuration

Enable IOMMU

To enable IOMMU, you need to enable the SR-IOV option in the BIOS and verify whether the Linux kernel has IOMMU enabled by default.

You can check if IOMMU is enabled by running:

$ dmesg | grep -i iommu

Example output:

[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic root=UUID=6b78fa89-a575-432d-a445-1497c3467214 ro iommu=on
[    0.000000] Unknown kernel command line parameters "BOOT_IMAGE=/boot/vmlinuz-6.8.0-52-generic iommu=on", will be passed to user space.
[   11.561684] iommu: Default domain type: Translated
[   11.566470] iommu: DMA domain TLB invalidation policy: strict mode

If IOMMU is not enabled, add iommu=on to the Linux kernel boot parameters:

$ vim /etc/default/grub 

Modify the line:

GRUB_CMDLINE_LINUX_DEFAULT="iommu=on"

Then update GRUB and reboot:

$ update-grub2  
$ reboot  

Additionally, enable SR-IOV in the BIOS. The exact location of this setting varies depending on the BIOS, but it is typically found under the PCIe subsystem or related options.

Upgrade Host to HWE Kernel

I recommend using the Hardware Enablement (HWE) kernel on the host. While I’m unsure if the regular kernel works, the HWE kernel has been reliable in my experience. Install it with:

sudo apt install linux-generic-hwe-22.04

Configure VFIO on Host

The VM relies on the VFIO driver for GPU passthrough. To configure VFIO, you need to pass the PCIe device information to the VFIO driver.

First, identify the GPU’s PCIe device IDs:

$ lspci -nn

Example output:

0005:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2704] (rev a1)
0005:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bb] (rev a1)

Here, 10de:2704 is the GPU’s PCIe device ID, and 10de:22bb is the audio device ID. At a minimum, you need to pass through the GPU device.

Next, edit the VFIO configuration file to include these IDs:

$ vim /etc/modprobe.d/vfio.conf

Add the following line:

options vfio-pci ids=10de:2704,10de:22bb

Disable Nvidia Driver on HOST

To prevent the host from loading the Nvidia driver, add the Nvidia modules to the kernel’s blocklist:

$ vim /etc/modprobe.d/blacklist.conf

Add the following lines:

blacklist nvidia
blacklist nvidia_drm
blacklist nvidia_modeset

Update the initramfs and reboot:

$ update-initramfs -u
$ reboot

Configure VM

Install Virt-manager

In this article, we used virt-manager as VM manager, first step is install virt-manager, suppose Ubuntu will install all relative packages.

$ apt install virt-manager 

If you’re using SSH with X11 forwarding (e.g., ssh -X host) or MobaXTerm on Windows, Virt-manager will display the remote X window. If neither method works, consider installing a KDE desktop on the host and accessing it via the BMC remote console.

(Optional) Install KDE Plasma Desktop:

(option)
$ apt install kde-plasma-desktop

Create VM image

Virt-manager creates fixed-size VM images by default. If you prefer dynamic allocation, create the image manually:

$ qemu-img create -f qcow2 ubuntu2204.qcow2 200G

Add Nvidia device to VM

If the host is configured correctly, Virt-manager will list all PCIe devices, including the Nvidia GPU. Add the GPU and its audio device (e.g., 0005:01:00.0 and 0005:01:00.1) to the VM’s hardware list.

After adding the devices, proceed with the Ubuntu 22.04 installation.

After add hardware

Now, it can run begin install to install ubuntu 22.04

Disable secure Boot in UEFI

By default, Virt-manager enables Secure Boot. However, Nvidia drivers may not work with Secure Boot enabled. Even though the Nvidia installer includes a driver signing feature, the driver may still fail to load. To avoid issues, disable Secure Boot in the VM’s UEFI settings.

During the VM’s boot process, press the DEL key to enter UEFI settings and uncheck the Secure Boot option.

Before installing the Nvidia driver, ensure the necessary development packages are installed:

$ apt install build-essential

Then, install the Nvidia driver and reboot the VM.

GPU Passthrough Test

If everything is set up correctly, running nvidia-smi should display the GPU’s status.

For testing, you can use Ollama with the Deepseek-R1 model. Install Ollama with:

curl -fsSL https://ollama.com/install.sh | sh

Pull the Deepseek-R1 model. Since the GPU has 16GB of memory, the 14B model is a good choice (it requires ~10GB):

ollama run deepseek-r1:14b

Ask a question like, “Why is the sky blue?” This will trigger the model’s Chain-of-Thought (CoT) reasoning.

Monitor the GPU’s status using nvidia-smi to ensure it’s functioning correctly.

The post GPU Passthrough on ARM64 with Libvirt/Virt-manager appeared first on richliu's blog.

TCP 已經是一個非常陳舊的 Protocol ，其中很多特性已經不再適合目前的網路環境，像QUIC架在UDP上層，增加安全性和速度。SCTP主打多連結，可靠性和安全，SCTP在電信環境用的比較多。基於 UDP 大量傳送資料的有 UDT (UDP-Based Data Transfer protocol) 等等

TTPoE 看起來應該是 Telsa 降本增效（名詞好用就拿來用）的實作，因為如果是使用RDMA之類的解決方案應該都有更快的 NIC Card 可以用(目前最快是 800Gbps 2 Ports QSPF-DD), 但是 TTPoE 是用 100G NIC 卡但是重新實做了整個 NIC Card ，雖然這樣講，未來仍然有升級的空間

我認為 TTPoE 的主要設計目地就是 Low Latency and Low Cost，因為 TTPoE 看起來是要做 HBM2HBM 的資料同步，Low latency 應該還是主要考量，因為他們要實作在 Hardware 上

而 data center(DC) 下是 data loseless 的環境，相對的網路環境比較好，如果有開啟 ethernet congession control ，理論下會丟掉的封包比較少，頻寬大。基於這個特性，TTPoE 就是做刪去法，將 TCP 協定中多餘的 Latency 去掉，仍然使用 IP 協定，保有和現有網路的相容性，以下這張圖就可以清楚的知道，TTP 是取代 TCP 的位置，上層仍然能夠跟 RDMA 接，其他 socket 或 APP 應該也可以，不過主要應該還是給RDMA這類高速記憶體協定用

（註： 本圖來自 Tesla’s TTPoE at Hot Chips 2024: Replacing TCP for Low Latency Applications ，原圖應該是 Hot Chips 2024 Telsa 的 slide，以下不再重覆說明 ）

為了讓 Hardware 更容易設計，所以簡化了 TTP 的 Protocol State machine ，這是是目標也是結果，因為簡化 Protocol 也會簡化 state machine

將 TCP 的 Three way handshake 改成 Two way handshake，其實 internet 防火牆需要這個機制判斷是不是真實連線，但是 DC 內部不需要，拿掉合理，也可以減少很多 Latency
有些人要加速會改 TCP ACK 到下一筆資料送出的時間，不過這會造成某些硬體加速器的問題

這邊應該是要將如果有個 Packet 是 NACK ，後面的 packet 都重送，這樣雖然會傳比較多資料，但是 DC package loseless 環境，這樣的機率小，可以接受這樣的設計
TTP 也拿掉 tcp congestion control 這個功能，如果只在 DC 用，這功能完全不需要，因為路上幾乎都是大頻寬，如果頻寬不夠就要從別的地方觀測調整，不是 Protocol 的問題，packet 的 sliding window 就設一個固定值，符合系統大部份時間都可以以最高速運作即可

TTP Protocol 有些有趣的地方，Extension header 二個，第一個是指定 type 用的，第二個可能是未來用，這應該是固定大小，然後 Data payload 是 64 bytes 的倍數

有趣的是用 SRAM size 做 speculative transmission，這個設計我喜歡，好的演算法就是不需要演算法，讓系統自己去調整(self-adaptive)，不過我懷疑這個就是 ethernet 的 tx buffer 講成高大上而已

這意思就是前面還放傳統的 ethernet 架構，TTP mac controller 控制　ethernet 介面傳送資料，灰色部份都是現成的

TTP hardware micro-architecture 設計

下方的 Next packet linked list SRAM，雖然講是 linked list SRAM，不過為什麼不做 array 當 ring buffer 呢？還是指的是同一件事，做過 NIC 卡就知道這種地方設計就固定是那樣，尤其是做成 hardware 又要結構更簡單，所以我才懷疑前面講的 SRAM 做 flow control 就是講這邊，如果 ring buffer 滿了就表示外面滿了

一般這種高階通常都有數個 TX/RX buffer ，不知道是不是因為專供 RDMA 用所以這邊就只畫上一個 TX/RX buffer ？而且不確定是不是因為一個 Buffer 頻寬就滿了（或是 FPGA 只能規劃 1MB，都有可能）

80 microseconds 在高速網路世界已經算很久了，算 OK
slide 最後一行我覺得才是真正的目地，如果是在 AI server 上要跑這個，那不用更高階的卡跑 RDMA 就很合理了

“Mojo” 100GB dumb-NIC （話說我以前做過的某 project 我叫 SmartXXX 呢）
中間這麼大一顆可能是 SoC 也可能是 FPGA ，我覺得 FPGA 的機率大一點，畢竟要改 code 這階段還是用 FPGA 穩定

CPU 的是用 Gen3 x16 和 8GB DDR4，Gen3 比較有趣，表示這真的是降成本達到目地

Latencies ，竟然 TTPoE 比 Nvlink 還快，Nvlink 已經算是對傳了，要過 ethernet 還比較快讓我難以想像

總結

我覺得技術細節透露有限，但是這個方向的確可以大幅減少 latency ，在網路的世界內，減少 latency 就是增加速率和效率，在頻寬固定的狀況之下

雖然相容性不佳，只是給 RDMA 用，但是我覺得仍不失為 intranet 上有趣的應用，而且的確很適合 DC 使用

ref. Github ttpoe Linux kernel 的 software 實作和規格, 要先有 software 才能搞 hardware 啊.

The post TTPoE Tesla Transport Protocol (TTP) Over Ethernet 簡評 appeared first on richliu's blog.

no-dangling-assignment-gsl -std=c++20 -Wno-trigraphs -gsimple-template-names -fno-exceptio00:01:51 [472/19443]
nc++ -isystem../../third_party/libc++/src/include -isystem../../third_party/libc++abi/src/include -fvisibility
-inlines-hidden -O2 -pipe -march=armv8.2-a -Wno-unknown-warning-option -c ../../third_party/libyuv/source/comp
are_neon64.cc -o obj/third_party/libyuv/libyuv_neon/compare_neon64.o
../../third_party/libyuv/source/compare_neon64.cc:175:8: error: instruction requires: dotprod
  175 |       "udot        v4.4s, v0.16b, v6.16b         \n"
      |        ^
<inline asm>:14:1: note: instantiated into assembly here
   14 | udot        v4.4s, v0.16b, v6.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:176:8: error: instruction requires: dotprod
  176 |       "udot        v5.4s, v1.16b, v6.16b         \n"
      |        ^
<inline asm>:15:1: note: instantiated into assembly here
   15 | udot        v5.4s, v1.16b, v6.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:204:8: error: instruction requires: dotprod
  204 |       "udot        v4.4s, v0.16b, v0.16b         \n"
      |        ^
<inline asm>:10:1: note: instantiated into assembly here
   10 | udot        v4.4s, v0.16b, v0.16b
      | ^
../../third_party/libyuv/source/compare_neon64.cc:205:8: error: instruction requires: dotprod
  205 |       "udot        v5.4s, v1.16b, v1.16b         \n"
      |        ^
<inline asm>:11:1: note: instantiated into assembly here
   11 | udot        v5.4s, v1.16b, v1.16b
      | ^
4 errors generated.
[10940/63891] aarch64-unknown-linux-gnu-clang++-18 -MD -MF obj/third_party/libyuv/libyuv_neon/rotate_neon.o.d

Seems because some -march/-mcpu/-mtune drop ‘dotprod’ parameters, it needs to add the extension manuall. ‘+dotprod’.

Edit the file /etc/portage/make.conf, and change the COMMON_FLAGS to

COMMON_FLAGS="-mcpu=neoverse-n1+crc+crypto+ssbs+dotprod -O2 -pipe"

It can use the following command to get current CPU flag

gcc -v -E -x c /dev/null -o /dev/null -march=native -mcpu=native 2>&1 | grep /cc1

Thanks Gentoo user zukunf to provide those information.

The post Gentoo ARM64 Altra Compile chromium Issue appeared first on richliu's blog.

我的環境是 ARM64，所以以 ARM64 為主，編的版本是 15.0-Current

# Install git first
# pkg install git
# Install Source code from Git 
git clone https://git.freebsd.org/src.git /usr/src

cd /usr/src/sys/arm64/conf
cp GENERIC MYKERNEL
vim MYKERNEL

以下應該可以整理成一個 script

cd /usr/src

# Remove old file
mkdir /usr/obj/tmp
chflags -R noschg /usr/obj/*
rm -rf /usr/obj/*

# Build world and kernel 
# Cannot use 128 cores to build that because it might cause build problem. 
# reduce it to 16 might be better 
make -j 16 buildworld
# Clean kernel if necesssary 
# make -j 16 cleankernel KERNCONF=MYKERNEL
make -j 16 buildkernel KERNCONF=MYKERNEL

# Install world and kernel to a temperory directory
mkdir /usr/obj/tmp
make installworld DESTDIR=/usr/obj/tmp
make installkernel KERNCONF=MYKERNEL DESTDIR=/usr/obj/tmp

# copy release document to the directory
make distribution DESTDIR=/usr/obj/tmp

# build release
cd release
make -j 16 release KERNCONF=MYKERNEL NODOC=yes NOPORTS=yes NOSRC=yes

Release files (iso, image 等等) 會在
/ussr/obj/usr/src/arm64.aarch64/release 下

Clean kernel

make cleankernel KERNCONF=MYKERNEL

修改 EFI_STAGING_SIZE，修改 /usr/src/stand/efi/loader/copy.c , 找到

#define DEFAULT_EFI_STAGING_SIZE

統統改成要的值，ex: 100 ，這樣就好，暴力解決

Enable exec=”copy_staging auto” in /boot/loader.conf

這個功能因為要在編 world 放到 loader.conf 內，這樣 iso 和 memstick 檔案內才，所以我用了一個 hacker 的手法，編輯 release/Makefile ，約 Line 188，加上

echo exec=\"copy_staging disable\" >> ${.TARGET}/boot/loader.conf

Enable Early Printf In Kernel

編輯 MYKERNEL ，加上[ref]

options EARLY_PRINTF=pl011

The post FreeBSD Build Kernel appeared first on richliu's blog.

The Dynatron W3 package comes in a standard case.

includes a tower heatsink, one 120mm 4-pin fan, and thermal paste.

The base is not entirely copper, and its area seems somewhat small

other parts

While no documentation is provided, installation is straightforward for those familiar with DIY PC building

There is a gap between the DDR slots and the CPU fan, but larger DIMMs might require disassembling and reinstalling the tower fan.

In a preliminary test using stress-ng with 128 threads on an AltraMax 128-30 CPU and ASRock Rack ALTRAD8UD-1L2T.

Test command is

stress-ng --cpu 128 --vm 128 --vm-bytes 128M --timeout 180s --metrics-brief

the fan operated at 1400 RPM and maintained a quiet profile. The CPU temperature reached 64°C, staying below 70°C, with power consumption around 178W.

This performance suggests the tower fan is suitable for general purposes.

Notice: The CPU’s TDP is rated at 250W, this maximum power draw is only achievable under full system stress, such as utilizing all 8 DRAM controllers and all PCIe root controllers. Therefore, this figure should be considered as a reference rather than a precise measurement.

The post Dynatron W3 for Ampere AltraMax appeared first on richliu's blog.

Fine-Tune LLM 是指在現在的模型架構之上，再加上一個小的模型，就使用讓原來的模型支援我們要的結果。比較常見的應該是在 Civitai 上可以看到很多不同風格的 Stable Diffusion Lora model 。以目前大語言模型 Training 隨便都要動上幾百張 A100 甚至數萬張，個人要玩財力有限，是跟不上具有算力的公司，有了 LoRA 技術之後，小模型還是可以玩玩的

本篇使用的硬體跟這篇　LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform　一樣，許多前置的設定也可以參考這篇，本篇只解釋我是怎麼做完整個流程的 。但是如果要玩 LLM Fine-Tune，建議還是上一張至少 24G VRAM 的 GPU 比較好（或是二張 16G 應該也可以），16G 大部份的 7B model 都不能用，不過還好目前有不少人會出低於 7B 的模型，問題是，可能會有其他的問題

資料清洗 (Data Cleaning)

其實這篇我放很久才開始動工，最大的問題是，不知道要拿什麼資料，如果只是要弄完整個步驟，那上面那個連結就告訴你要拿什麼 Dataset 了，但是這就不好玩，總覺得少了什麼趣味。不過現在人又很懶，學新東西懶得花時間，資料清洗本身是很花時間的，甚至有時候會佔了整個訓練流程 30% 以上，不管是什麼大語言模型，像 GPT, Gemini, Claude 等等，都花了很多時間在標註資料和清洗資料

不過最近 Claude.ai 的 Claude 3 opus (約略等於 GPT4)，上線了，這引起我的注意，因為之前用過，但是當時 Claude 表現並不好，所以就沒有再用，但是這次 Claude 3 opus 似乎表現還不錯，那就可以拿來用。Claude 最大的好處就是可能會去存取外部網頁，我知道 GPT4 也可以，但是我們下次一定黨一定會想先用個免費版的。在測試過後，Claude 3 sonnet 的表現還可以，那就可以拿來當我們資料清洗的工具。

工具有了，內容永遠是最麻煩的，但是這個內容比較麻煩的是，一定要有一定的資料量和鑑別性（我沒學過資料清洗講錯請見諒），要不然在測試的時候無法確定這就是我們輸入的資料。這也是我一直沒動工的原因，好的資料集難找，前幾天 3/11日，是核能流言終結者協會成立十週年吧。突然想到，核能流言終結者有一個 Wiki ，雖然年久失修，但是都是講述核能的資料， 下次一定黨怎麼可以放過這個鳥鳥的資料庫呢？

資料有了，工具有了，接下來就可以開幹了

MariaDB+phpMyAdmin

我找了一陣子，沒有合適的資料編輯工具，要不然就是 CSV ，但是也不方便編輯。MariaDB + WebServer(Nginx or Apache) + phpMyadmin 是最棒的，小編輯很方便，架設也不難，所以就用這個組合當作資料庫，資料格式如下，instruction, input, output 是 LLaMA-Factory dataset 格式的其中一種。我就照弄就好了，以下這是 TABLE 名，至於 database 取名就大家開心了

CREATE TABLE `nuclear` (
  `id` int(11) NOT NULL,
  `instruction` text DEFAULT NULL,
  `input` text DEFAULT NULL,
  `output` text DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
ALTER TABLE `nuclear`
  ADD PRIMARY KEY (`id`);
ALTER TABLE `nuclear`
  MODIFY `id` int(11) NOT NULL AUTO_INCREMENT, AUTO_INCREMENT=0;
COMMIT;

如果有問題直接按編輯就可以小修一下，非常方便

請 Claude 清洗資料

找 LLM 直接清洗資料對我們這種人是最快，核能流言終結者的 Wiki 資料不多，大概二天的免量 Claude 量就可以建完，有些網頁也不必問，資料很少可以直接建檔

請 Claude 吐資料的詠唱方式是這樣的　

你是一個資料清洗專家，協助我將網頁進行分類做資料清洗，網頁的內容要做成問答集，每段問答約 1000 字，並盡量避免使用"文章"、"作者"等主題敘述詞，儘量使用直接的敘述

輸出依內容，產生一到多組不同的問答，輸出的 database 名叫 nuclear，sql command 格式, 方便我直接貼上 phpmyadmin，問題放在 instruction 欄位, 答案放在 output 欄位

網頁如下 https://nuke.fandom.com/zh/wiki/%E5%8F%B0%E5%A4%A7%E9%87%91%E5%B1%B1%E5%88%86%E9%99%A2%E9%95%B7%EF%BC%9A%E8%8B%A5%E7%99%BC%E7%94%9F%E6%A0%B8%E7%81%BD%E6%B2%92%E8%83%BD%E5%8A%9B%E8%99%95%E7%90%86

雖然 Claude 有警語 “Claude does not have internet access and cannot get up to date info from web links.” 但是我給的網頁都沒問題，如果不能向外存取，應該貼上內容也是可以的（可以取得當然比較方便）。以下就是部份 Claude 的回答

好的,我將根據網頁內容,分成多組問答,每組約1000字,並避免使用"文章"、"作者"等主題敘述詞。問題將放在instruction欄位,答案將放在output欄位,使用SQL命令插入nuclear資料表。

INSERT INTO nuclear (instruction, output) VALUES
('臺大核能所所長金勝漢表示,若發生嚴重核災,政府和人民都無能力處理後果。請簡述這個說法的背景和重點。', '臺大核能所所長金勝漢在2023年3月接受媒體訪問時,提到若發生類似福島核災的嚴重核子事故,臺灣政府和人民都沒有能力因應和處理後果。他的這番話,引發社會高度關注核能安全議題。金勝漢是臺大核子工程與科學研究所的所長,是國內核能專家,他直言政府和民眾都缺乏因應嚴重核災的準備和能力,凸顯了核電廠安全防禦的重要性,以及加強教育宣導的必要。')

將 SQL 這一段直接貼到 phpMyAdmin 就可以了

接下來就是不停的產生，貼上，在貼上前可以看看資料，有時需要修改一下。因為我只是測試，所以有些錯誤我也就算了，偶爾才會修改一下文章和用詞。所以我建了約 85 筆問答的資料，雖然不多，但是應該可以做些測試了

輸出 json file

接下來就是寫一隻程式輸出到 json file ，當然這種小程式就詠唱叫 Claude 輸出就好了，品質還不錯。記得將參數換成要連線的資料庫

import mysql.connector
import json

# 連接到 MySQL 資料庫
mydb = mysql.connector.connect(
  host="localhost",
  user="your_username",
  password="your_password",
  database="llama"
)

# 獲取遊標
mycursor = mydb.cursor()

# 查詢資料表
mycursor.execute("SELECT instruction, input, output FROM nuclear")

# 獲取查詢結果
rows = mycursor.fetchall()

# 將查詢結果轉換為 JSON 格式
data = []
for row in rows:
    data.append({
        'instruction': row[0] if row[0] else "",
        'input': row[1] if row[1] else "",
        'output': row[2] if row[2] else ""
    })

# 將 JSON 資料寫入檔案
with open('nuclear.json', 'w', encoding='utf-8') as f:
    json.dump(data, f, ensure_ascii=False, indent=4)

# 關閉資料庫連線
mydb.close()

這樣就會輸出 nuclear.json ，放到 LLaMA-Factory 的 data 目錄，修改 dataset_info.json 加上這行，不需要產生 sha1 ，不設看起來就不檢查

  "nuclear_train": {
    "file_name": "nuclear.json"
  },

設完之後，reload dataset 就好

調整參數

首先是選 model ，我本來要試 Mistral-7B，但是一用就 Out of Memory 了。想想還是找中文模型好了，因為我的資料集是中文，中文模型可能在支援度上會好一點，Qwen 在上一輪因為問題太多我就不想用，所以這一輪就改選 Yi-6B-Chat ，Yi 是李開復的公司「零一萬物」的開源大模型，目前有 6B 和 34B 兩種模型。

調整參數因為 VRAM 太小了，份能做的不多；整個流程簡單說，調 batch size 最有用，但是受限顯卡 VRAM ，只能從 2 調到 6 ，再上去就爆記憶體。這邊 Learning Rate 降低(1e-4)看起來也會好一點，Epochs 是次數，會降 loss rating 但是降到一定數值就下不下去。調整完我的輸出就看起來比較正常，其他還有很多參數，但是沒深入研究，不確定影響為何

接下來的步驟都和上一篇一樣，Chat 和 Export 都是

測試

最後結果我們會輸出到 /nvme/newmodel/Yi-6B-Chat，和上一篇不一樣，這邊要用 convert.py 轉檔，不是convert-hf-to-gguf.py [ref]

python3 /nvme/llama.cpp/convert.py Yi-6B-Chat
<略>
Wrote Yi-6B-Chat/ggml-model-f16.gguf

# 量化，可以變 q4/q5 等等
/nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q4_0.bin q4_K_M
 /nvme/llama.cpp/build/bin/quantize ./Yi-6B-Chat/ggml-model-f16.gguf ./Yi-6B-Chat/ggml-model-q5_0.bin q5_K_M

測試 model

# 測試原始版本的輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
I understand that there may have been concerns and misunderstandings regarding the establishment of a nuclear waste storage faci
lity on Taiwan's Yi-Lan Island. It is important to address these issues with honesty and transparency. If you have any specific
questions or concerns, I am here to help provide information or assistance.<|im_end|>

# Q5 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q5_0.bin -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
我們已經對此事件進行了詳細的調查和審查，確認原能會在當時的工作中沒有任何欺騙或不當行為。事實上，原能會在規劃這項建設時已經與當地
居民進行了充分的溝通和協商，並根據《核子設施法》的要求徵得了必要的手續。在此過程中，原能會也確認了該地區能夠承受相關設施的運轉和
風險。因此，您所指的欺騙或未經同意的說法是不符合事實的。<|im_end|>

> 核能流言終結者是什麼?
 **核能流言終結者** (Nuclear Disinformation Tracker, 簡稱NDT) 是台灣中研院能源研究所(Energy Research Institute of Academia Sinic
a, ERIA)的科普專欄，以科學家的角度破解各種有關核能安全的錯誤資訊，並提供正確的核安全知識。

# Q4 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-q4_0.bin -ins
> 核能流言終結者是什麼？
 核能流言終結者是一個專門用來解釋有關核電站運作和核廢料處理等方面的知識的網站。該網站由香港科技大學公共衞生及基層醫療學院團隊創
立，旨在消除大眾對核電站運作的各種疑慮和誤解。<|im_end|>


# F16 輸出
/nvme/llama.cpp/build/bin/main -m ./Yi-6B-Chat/ggml-model-f16.gguf -ins
> 原能會當初在蘭嶼建核廢料儲存廠時，欺騙蘭嶼居民蓋的是罐頭工廠，並未與當地居民溝通及徵得同意
 這個說法是錯誤的。原能會於1982年在蘭嶼設立核廢料貯存場時，確實有經過多次和居民的溝通協商，包括了說明會、座談會等等。居民在了解
相關資訊後才同意興建。因此，說原能會欺騙居民是沒有根據的。

> 核能流言終結者是什麼?
 "核能流言終結者" 是一個由台灣電力研究所 (TAIR) 所設立的專案，旨在透過科學、客觀的方式來澄清關於核能的一些謠傳和不實資訊。這些資
訊包括了對核能發電的誤解、不實的風險宣傳等等。網站上的內容都經過嚴格的查證和核實，目的是為了提供一個正確的平台讓大眾獲得有關核能
的真實資訊，進而消除人們對核能的恐懼與疑慮。

從以上的結果可以看到

* 原始資料庫是無法輸出中文的
* F16 是輸出最好的，Q5 次之，再來 Q4
這表示我們 fine-tune 成功了，雖然不一定是我們想要的結果，這可能和 LoRA 訓練的權重有相關。對於核能流言終結者的說明都是錯的，可能要針對原始資料庫修改增加說明，或是要加 Batch size

Ollama

Ollama 有點奇怪，因為我直接下命令都會出些很奇怪的結果，跟 llama.cpp 直接執行的結果不同。最後調整是用 F16 才可以正常輸出，以下是 modelfile

FROM /nvme/newmodel/Yi-6B-Chat/ggml-model-f16.gguf


PARAMETER temperature 0.8
PARAMETER num_ctx 512


TEMPLATE """[INST] {{ if .System }}<<SYS>>{{ .System }}<</SYS>>

{{ end }}{{ .Prompt }} [/INST] """
SYSTEM """"""
PARAMETER stop [INST]
PARAMETER stop [/INST]
PARAMETER stop <<SYS>>
PARAMETER stop <</SYS>>

建立 ollama database ，並且 reload

ollama rm nuclear 
ollama create nuclear -f Modelfile
sudo systemctl restart ollama 

我要下 parameter 才會得到正確的輸出結果，ex:

這樣應該算簡單吧

The post Easy to Fine-Tune Large Language Model with LLaMA-Factory appeared first on richliu's blog.

Qualcomm’s new desktop SoCs might change this landscape. However, we can still leverage the Ampere Altra platform to run LLMs.

Since this is a personal blog, most articles here are my personal notes. I’ll keep them updated as my understanding evolves.

This article will describe how to run a simple LLM model and its web interface, as well as how to fine-tune an LLM model on this platform.

Before running the commands, you’ll need to install the Nvidia driver. Refer to this article, “[How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform]”, for instructions on installing the Nvidia driver and Docker driver.

Ollama and Open-Webui don’t require a GPU. However, having a GPU is beneficial. Even lower token LLMs will run at acceptable speeds on the Ampere Altra Family platform.

This article’s hardware setup is as follows:

• CPU : Ampere Altra Family
• Board: AsRock ALTRAD8UD
• GPU: Nvidia RTX 4080

Use current LLM model with GUI

At this point in the process, using Ollama and Open-Webui is a straightforward approach for running LLMs. Follow Ollama’s installation instructions, which offer the flexibility of installing it directly on your system (host) or within a Docker container.

curl -fsSL https://ollama.com/install.sh | sh

By default, the Ollama service listens on localhost (127.0.0.1). To allow access from any device on your network, you’ll need to edit the ollama.service file.

In the [Service] section, add the following line:

Environment="OLLAMA_HOST=0.0.0.0:11434"

For Open-Webui, we recommend using Docker for a simpler solution. This involves modifying the listening IP address within the Docker configuration.

Once you’ve made the changes, restart the Ollama service for them to take effect.

systemctl daemon-reload
systemctl restart ollama

Run the ollama command and try it

# ollama run llama2
>>> who are you

I'm LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational manner.
I'm here to help you with any questions or topics you'd like to discuss! Is there something specific you'd like to talk
about or ask?

>>> 

If doesn’t work, try to restart ollama and try again.

Big LLM

If choose some big LLM like LLaMA2-70b or Qwen

Open-Webui

Following command is to run the Open-Webui docker image on host, 3000 is host port for Open-Webui. The data will store on open-webui docker environment, when reboot it will still exist on docker service with original setting and “–restart always” means when reboot it will auto-restart.

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

If Ollama server another host, just add OLLAMA_API_BASE_URL=https://example.com/api to the docker command,

docker run -d -p 3000:8080 -e OLLAMA_API_BASE_URL=https://example.com/api -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Upon login, you’ll need to provide an email and password. Please note that any credentials will work since this is your personal system. Once logged in, you can also edit the OLLAMA_API_BASE on the settings page.

Now, it connected from Open-Webui to ollmam backend, and we have a running sytsem on our machine.
Run sample chat here and choice llama2:latest as LLM model.

LLM Fine-Turning – LLaMA-Factory

Fine-tuning LLMs ideally requires a GPU card for better performance. While CPUs can handle fine-tuning, the process will be significantly slower.

Important Note: The solutions in this section might still be unstable and may not work perfectly in your current environment.

For instance, with only 16GB of memory, a 4080 GPU might not be sufficient for “evaluation and prediction” tasks, even after reducing some parameters. This still can lead to CUDA out-of-memory errors.

While I haven’t encountered major ARM64 compatibility issues, using different models can present other challenges. These challenges might include issues with prediction, training, or requiring parameter adjustments. Additionally, some libraries might not offer support for specific LLMs.

Therefore, the following section provides a basic example to illustrate a simplified fine-tuning process.

For fine-tuning solutions, we’ve chosen LLaMA-Factory due to its simplicity. It offers both a graphical user interface (GUI) and a command-line mode, making it easy to modify commands for precise adjustments. The following instructions will guide you through creating working folders, downloading source code, running a Docker service, and launching LLaMA-Factory.

# Create work folder
mkdir -p /nvme/model 
cd /nvme 
git clone https://github.com/hiyouga/LLaMA-Factory.git

# Run docker, it will use host GPU and map the folder into docker.
sudo docker run --gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -it --rm \
     -v /nvme:/nvme \
     -p 7860:7860 \
     nvcr.io/nvidia/pytorch:24.01-py3

# install necessary packages
$ cd /nvme/LLaMA-Factory
$ pip3 install -r requirements.txt
$ pip3 install tiktoken transformers_stream_generator

# run the LLaMA-Factory
$ CUDA_VISIBLE_DEVICES=0 python src/train_web.py

Now, it can acces the webgui via http://host IP:7860.

It can create another Docker window to run command on the LLaMA-Factory docker image.

docker exec -it <docker name> /bin/bash 

for download LLMs, you need to use git download from huggingface, ex:

# LLama 2 need username and token(not passowrd) to download it. 
git clone https://huggingface.co/meta-llama/Llama-2-7b-hf

# Qwen
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
git clone https://huggingface.co/Qwen/Qwen1.5-1.8B

Pre-Training

Model name, choice the LLMs name , like Qwen1.5-1.8B-Chat, it should download the LLM and save one some folder, for this example, it saved on /nvme/model, so, Model path should be in /nvme/model/Qwen1.5-1.8B-Chat.

First time, I suggest to use small LLM for fine-tuning, Qweb1.5-0.8B might have issue, will suggest from 1.5-1.8B to start.

Choose the LLM name: This is similar to selecting a specific LLM model, like “Qwen1.5-1.8B-Chat”. The instructions will indicate your LLM folder. For this example, the model would be saved in /nvme/model/Qwen1.5-1.8B-Chat.

Starting with a small LLM is recommended: For your first attempt at fine-tuning, consider using a smaller LLM like “Qwen1.5-1.8B”. It’s possible that “Qweb1.5-0.8B” might cause issues, so we recommend starting with models in the 1.5-1.8B range.

Here are the fine-tuning options you can adjust in LLaMA-Factory:

Dataset: You can add datasets to this mode. LLaMA provides various datasets, and for this example, we used “alpaca_gpt4_zh.”
Gradient accumulation: This is currently set to 4.
Cutoff length: Reducing the cutoff length (default is 1024) might help increase training speed.
Compute type: The compute type depends on your environment. In this case, fp16 works, but bp16 doesn’t sometimes.
Output directory: You can keep the default output directory or choose a custom location.
Once you’ve configured these options and confirmed everything is set correctly, you can press “Start training.” Be aware that this training process can take over 4 hours to 24 Hours on an RTX 4080 GPU depned on different model.

Evaluate & Predict

Next step is evaluate and predict, Adpater path just fill previous steps’ output dir. Dataset is the same.

When everything is ready, press Start. Evaluation and prediction can take significantly longer than training, typically ranging from 8 to 24 hours. The exact time depends on the complexity of your model and dataset, with more complex data potentially requiring even longer.

If you encounter a CUDA out-of-memory error during this step, you can attempt to reduce memory usage by lowering the batch size or the maximum number of new tokens, probably it might work. [ref]

Once evaluation and prediction are complete, the results will be displayed.

Test Chat Model

You can test chat result by load module.

Export model

Remember to fill export dir, in here I used “/nvme/newmodel/Qwen1.5-1.8B-Chat”

cd /nvme/newmodel 
python3 /nvme/llama.cpp/convert-hf-to-gguf.py Qwen1.5-1.8B-Chat --outfile test.gguf

# Use llama.cpp to test this model 
/nvme/llama.cpp/build/bin/main -m test.gguf -p "who are you?"

# Command mode
/nvme/llama.cpp/build/bin/main -m test.gguf -ins

Create a file named “Modelfile”, and fill this

FROM /nvme/newmodel/test.gguf

Now, run ollama to include this LLM.

ollama create test -f Modelfile
ollama run test "who are you?"
====
 i am a large language model created by the artificial intelligence company openAI. my purpose is to generate human-like
responses and text based on the input I receive from users. can I help you with anything today? #openai

#chatbot
Sure, I'm here to assist you! How can I help you today? Is there something specific you'd like to talk about or ask me?
#openaiChatBot不断地尝试。 #AI #MachineLearning

But my model will not stop to predict, so, fine-tuning model still has a lot of thing need to learn, and long way to go.

To be continue.

The post LLM Chat WebGUI and Fine-Turning on Ampere Altra ARM64 Platform appeared first on richliu's blog.

For current ARM64 systems, most of the time they will use the serial console as the debug console. So, when there is no output from the serial console, it confuses me.

CPU: Ampere Altra Family
Board: AsRock ALTRAD8UD
For example, on my Gentoo system, only need to configure /etc/default/grub to following configuration.

GRUB_TIMEOUT=30
GRUB_TIMEOUT_STYLE=menu
GRUB_CMDLINE_LINUX_DEFAULT="domdadm console=ttyAMA0 pcie_aspm=off"
GRUB_TERMINAL=console

Output to serial console will be no problem, but on the Ubuntu 22.04, it cannot output to serial console.

So, on Ubuntu, if you only can see grub menu on VGA and cannot see it on serial console, try this configuration, it will output to serial console and VGA console at the same time.

GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=10
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"

Don’t forget to update grub.cfg

update-grub2

ref.
https://help.ubuntu.com/community/SerialConsoleHowto

The post Ampere Altra ARM64 Grub Cannot Output Message To Serial Console appeared first on richliu's blog.

In previous article on How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform , we installed the Nvidia driver and ran PyTorch on the platform. Now, this platform has Nvidia RTX gaming card, can we use it to play game?

The answer is yes, there are some windows x86 or x64 emulators on Linux, but the most stable solution comes from Ampere’s official release solution. Other emulators might not work as well on this combination, such as FEX, but I don’t know whether it is a driver issue or another issue.

Basically, following Ampere’s documentation makes it easy to install the driver and play games. Let’s combine both documents into one and simplify it in this article.

Hardware
CPU: Ampere Altra
Board: AsRock ALTRAD8UD
GPU: Nvidia RTX 4080

Reference Ampere’s document
NVIDIA GPU Accelerated Linux Desktop on Ampere
Steam-on-Ampere

Install Nvidia Driver

All the conditions are similar to those outlined in the article How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform, using Ubuntu 22.04 with a general kernel (not the HWE kernel, which Nvidia may address in the future).

I haven’t utilized the original document’s script; instead, I’ve copied some necessary commands from it.

sudo apt-get install build-essential
# Install Nvidia driver here
sudo rmmod nouveau
cat << EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
blacklist nouveau
options nouveau modeset=0
EOF

sudo nvidia-xconfig -a --cool-bits=31 --allow-empty-initial-configuration

# Install Desktop, personally I like kde. 
sudo apt install kde-plasma-desktop
# or it can install regular ubuntu desktop 
# sudo apt install ubuntu-desktop

Download Nvidia ARM64 (aarch64) Display Driver ; the current version, 535.154.05, is acceptable. After installing the Nvidia driver, simply reboot the system. You can then use the following command to check the driver status.

nvidia-smi

Disable aspm from kernel

I didn’t apply this option. Most of the time, it’s only necessary to do so if the system encounters the massive AER error issue. Here’s an example command from the original document:

sudo nano /etc/default/grub
# Add pcie_aspm=off to kernel parameters
# GRUB_CMDLINE_LINUX_DEFAULT="pcie_aspm=off"
sudo update-grub

Install Box86

Following commands show how to install Box86

sudo wget https://itai-nelken.github.io/weekly-box86-debs/debian/box86.list -O /etc/apt/sources.list.d/box86.list
wget -qO- https://itai-nelken.github.io/weekly-box86-debs/debian/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box86-debs-archive-keyring.gpg
sudo dpkg --add-architecture armhf
sudo apt update
sudo apt install box86:armhf -y

Install Box64

Following command show how to install Box64, but original instruction have update , a package name was changed.

sudo wget https://ryanfortner.github.io/box64-debs/box64.list -O /etc/apt/sources.list.d/box64.list
wget -qO- https://ryanfortner.github.io/box64-debs/KEY.gpg | sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/box64-debs-archive-keyring.gpg
sudo apt update 
sudo apt install box64-arm64 -y
sudo systemctl restart systemd-binfmt

Installing Steam

Because I am using the Ubuntu 22.04 ARM64 server version with minimal, so, some packages might not be installed, Before Install Steam, install necessary packages

sudo apt install lsof

sudo apt install zenity
sudo apt install alsa

Use the following to install steam.

git clone https://github.com/ptitSeb/box86
cd box86
./install_steam.sh

Now, it can use steam now.

/usr/local/bin/steam

Okay, log in and enjoy it. However, in my experience, when installing other emulators, it might damage the box86+Steam setup, necessitating a system reinstallation to fix it.

Furthermore, the game’s performance seems not as good as expected, possibly due to my choice of Vampire Survivors and enabling high performance

Xbox JoyStick controller

I have an Xbox controller, and ALTRAD8UD doesn’t have Bluetooth capability, so I cannot use wireless conntions. However, it can connect via USB.

There are several Xbox drivers available for Linux. After testing, I found that xpadano is compatible with this platform. Simply follow their instruction guide to install the Xbox driver.

# Install DKMS package
sudo apt-get install dkms linux-headers-`uname -r`

# download source code
git clone https://github.com/atar-axis/xpadneo.git
cd xpadneo
# use DKMS to compile the driver
sudo ./install.sh

The Xbox controller driver will be installed automatically, and it works well in my tests. You can install the jstest package to perform a joystick test.

The Xbox controller functions seamlessly on Steam and Vampire Survivors. I can easily use the Xbox controller to play Vampire Survivors.

The post Play Game On Ampere Altra ARM64 Platform appeared first on richliu's blog.

In this article, it will walk through the process of installiung Stable Diffusion WebUI on ARM64 + Nvidia RTX platform.
For such as hardware combinarion, it needs to solve some compatiability issues, so, here will describe how to install it and fix those issues.

Platform

CPU: Ampere Altra ARM64 CPU
Board: AsRock ALTRAD8UD
OS: Ubuntu 22.04

The platform is the most powerful ARM64 platform for a homelab that we can obtain today. There are some server models available, but they come with high-speed cooling fans that are not suitable for home use.

Nvidia has provided comprehensive library support for the ARM64 platform, including drivers, CUDA, and cuDNN. However, despite these efforts, there are still gaps in support. Even Nvidia’s Jetson product line lacks adequate support for ARM64 platforms in certain applications such as PyTorch. While there exists an individual PyTorch accelerator package for direct download on x64 platforms, the same cannot be said for ARM64 platforms. Although there is technically one package available, its availability is severely limited, rendering it practically useless.

The only viable solution to overcome these limitations is to utilize Nvidia’s Docker service to install Stable Diffusion.

Installation

Install Ubuntu 22.04 without selecting the HWE (Hardware Enablement) support. It appears that the current Nvidia driver version has issues, leading to potential soft-lock bugs. A general kernel version (5.15) is preferable for stability.

Next, install the necessary Ubuntu packages.

sudo apt-get install build-essential

Download Nvidia ARM64 (aarch64) Display Driver , current version is ok, when I write this article, current version is 535.154.05

Docker

Install Docker by following the Docker guide Install Docker Engine on Ubuntu,”. Also provide the necessary installation steps here. If already installed Docker or if these steps don’t work, please refer to the original website for updated instructions.

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Nvidia Container Toolkit

Follow the instructions for installing the NVIDIA Container Toolit. It can modify the Docker configuration using this toolkit. Alternatively, modifying the daemon.json file might achieve the same result, although I haven’t personally tested it. Below are the instructions:

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

It can check the file /etc/docker/daemon.json file, the configuration should be like this.

{
    "runtimes": {
        "nvidia": {
            "args": [],
            "path": "nvidia-container-runtime"
        }
    }
}

Run Nvidia pytorch docker image

Following is example command to run the Nvidia pytorch docker image

sudo docker run --gpus all -it --rm \
        -v /host/work:/container/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:XX.XX-py3

/host/work means work directory on host, /container/work means work directory in container(docker). -p 7860 mean gui default port, it needs to forward to host machine. XX.XX means container versiom please see tags to get container version, or this URL . [NGC Pytorch]. Following is example command.

sudo docker run --gpus all -it --rm \
        -v /work:/work \
        -p 7860:7860 \
        nvcr.io/nvidia/pytorch:24.01-py3

Stable Diffustion WebUI

First steps clone Stable Diffustion WebUI source code.

git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git

Then, it needs to modify the settings file. Edit webui.sh, and adjust the parameters to the following numbers.

use_venv=0
can_run_as_root=1

As we are operating within a container, there’s no need for a virtual environment (venv), and the script can be run as root. Modify the file webui-user.sh and add COMMANDLINE_ARGS accordingly.

export COMMANDLINE_ARGS="--no-half --precision full --listen"

or run it directly

export COMMANDLINE_ARGS="--no-half --precision full --listen" ; bash webui.sh

Currently, Stable Diffusion will download some packages, but errors may occur during the installation process. We’ll address and fix these errors in the next few steps.

CV2

Error message :

    if __load_extra_py_code_for_module("cv2", submodule, DEBUG):
  File "/usr/local/lib/python3.10/dist-packages/cv2/__init__.py", line 28, in __load_extra_py_code_for_module
    py_module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py", line 168, in <module>
    LayerId = cv2.dnn.DictValue
AttributeError: module 'cv2.dnn' has no attribute 'DictValue'

Modify file “/usr/local/lib/python3.10/dist-packages/cv2/typing/__init__.py”, comment out line 169 like following.[ref]

# LayerId = cv2.dnn.DictValue

CodeFormer

Error message:

"/work/stable-diffusion-webui/repositories/CodeFormer/facelib/utils/face_restoration_helper.py", line 7, in <module>
        from facelib.detection import init_detection_model
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/__init__.py", line 11, in <module>
        from .yolov5face.face_detector import YoloDetector
      File "/work/stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py", line 20, in <module>
        IS_HIGH_VERSION = tuple(map(int, torch.__version__.split('+')[0].split('.'))) >= (1, 9, 0)
    ValueError: invalid literal for int() with base 10: '0a0'

This issue caused by pytorch.version will return illegeal number like 2.2.0a0+81ea7a4, so it will fail to parse this mesage, temperory solution is to remove the version number, edit file “stable-diffusion-webui/repositories/CodeFormer/facelib/detection/yolov5face/face_detector.py”. and modify line 20, IS_HIGH_VERSION to following version to following:[ref]

version_string = torch.version.split('+')[0]
version_string = ''.join(filter(str.isdigit, version_string))
IS_HIGH_VERSION = tuple(map(int, version_string.split('.'))) >= (1, 9, 0)

Now, the Stable Diffusion GUI should be working as well as expected

The post How to Install Stable Diffusion GUI on ARM64 Nvidia RTX platform appeared first on richliu's blog.

What truly impresses me is the board’s specifications. Not only does it boast a compact Micro-ATX form factor, but it also packs a powerful punch with four PCIe 4.0 x16 lanes, easily handling even demanding homelab tasks. The dual M.2 slots and dual OCULink port can enable software RAID for your NVMe disks. As a server board, it offers a BMC for remote control and system management. While the x64 platform remains popular, this board is an exceptional choice for those who prefer the non-x64 architecture, considering the limited choices with high speed fan options in server platforms.

Information

Important Spec

AsRock ALTRAD8UD-1L2T Spec web site

Where To Buy

US/Newegg

ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G Without CPU,
Mother board only
USD$849.-

Asrock Rack Bundle ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) with Ampere Altra Q64-22 64 cores & 2U passive cooler, Dual 10G – Integrated by Asrock Rack with CPU and heatsink
Mother board + Altra Q64-22 + 2U passive cooler(heatsink)
USD$1,500.-

UK/Server Factory Ltd

ASRock ALTRAD8UD-1L2T: Board only, Inc VAT: £870.00.

News

ASRock Rack ALTRAD8UD-1L2T Review This is the Ampere Arm Motherboard You Want

Cooling solution

Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926 (Preview tower fan system)
Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System (my homemake Liquid/Water cooling system)

Other reference URL

ASRock Rack ALTRAD8UD Getting Started : A good note to start up to use the board.

Tricks for ALTRAD8UD

This chapter aims to guide users through using the ALTRAD8UD board, addressing potential information gaps left by the official ALTRAD8UD manual.

For memory compatibility reference, AsRock offers its own DRAM AVL (Approved Vendor List). Alternatively, users can access the Ampere Altra family device document, freely available without registration.

OpenBMC default username password

User: root
Pass: 0penBmc (first word is zero, not O).
both BMC console and GUI’s default username password are the same.

Serial Console over LAN Port

The board support Serial Port Header (9-pin COM1), it needs to use this serial port or VGA port to enter BIOS and get BMC IP.
After get the BMC IP, it can get more information from serial console over LAN port
Port 2201: ATF console
Port 2202: SCP console
Port 2203: Host console
Port 22: ssh console

Here is my script to run the sol and make it more easy to use

# It can put following two lines to bashrc
alias oit='ipmitool -U root -P 0penBmc -I lanplus -C 17'
alias ossh="sshpass -p 0penBmc ssh -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no"

# get host console 
ossh root@<bmc ip> -p 2203 
# get in bmc ssh console 
ossh root@<bmc ip> 

Serial Console Output To VGA

If you cannot see the serial console output to console port, probably it bacuase this option was enabled[ref], just turn it off and add console=tty0 or console=ttyAMA0 to grub.conf, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0"

Ipmitool Fan Control [ref]

Manual Full speed
ipmitool raw 0x3a 0x01 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64

Manual Half speed
ipmitool raw 0x3a 0x01 0x32 0x32 0x32 0x32 0x32 0x32 0x32 0x32

Auto fan control
ipmitool raw 0x3a 0x01 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0

NC-SI Port

In the user manual, NC-SI port is typo, the NC-SI port location is 5, in the document which marked as IPMI LAN port. if use this port, both host and BMC can use the same network cable to connect to the same ethernet.

Known Issues

Intel X550 massive AER error messages[ref1][ref2]

Seems that Intel NICs have compatibility issues with non-Intel platform systems. This can cause some server safety systems, like Advanced Error Reporting (AER), to generate many alerts and experience interface problems. A temporary solution is to add aspi_aspm=off to the Linux kernel parameters, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0 pcie_aspm=off"

It will fix the AER issue.

The post AsRock ALTRAD8UD-1L2T Ampere Altra/AltraMax Note appeared first on richliu's blog.

這次升級要從 x64 平台升級到 ARM64 平台，雖然目前 Linux 下 ARM64 支援度還是不錯，但是兩個平台能用的東西還是有差異，主要是紀錄一下升級過程中有什麼需要注意的事情

ARM64 Hardware Prepareation

CPU Board: Ampere Altra CPU + AsRock ALTRAD8U : 這個組合現在已經能在 newegg 上買到了，也是目前最強大的 ARM64 平台，當 homelab 平台非常適合

DRAM: 買這種主機板用的都是 RDIMM ，有可能便宜有可能貴，可以查 Gigabyte 的 QVL List[Ref] 內的 DRAM list 購買，有朋友就是在日本買到DDR4 2933 32GB RDIMM 一隻5000羊的DDR，目前匯率 NT$1,100.- 非常便宜

Network: AsRock ALTRAD8U 有 1*Intel i210 Ethernet 和 2*10G Intel x550 Ethernet ，這部份到不是問題，我另外拿了舊的 Intel x540 4 Port ethernet 當小 switch 用

SATA: 這種板子預設都沒有 SATA interface 了，所以要買 SATA 卡，有 PCIe 轉 SATA ，這個要注意 PCIe bandwidth 和 SATA 數量，一分錢一份貨。
如果要追求速度，淘寶上也可以買到便宜 SAS 外接 SATA ，淘寶的 LSI SAS 8port internal 卡大部份 NT$2500 到 NT$6000 都有，記得買轉接線要看好規格，SAS 接頭的規格很多，一不小心就會弄錯

SSD: 板子上介面有 2* M.2 2280，如果覺得不夠，可以買 OCU 轉 U.2 轉接線，或是還有 PCIe slot ，可以買 PCIe 轉 M.2 都可以。但是 U.2 的硬碟不一定好買，而且常常價位偏高，但是相對上是比較便宜的選擇

Software Move To ARM64 Platform

QEMU/Libvirt

這部份應該是最難的部份，不過理論上也不是不可以，我原來是 VirtualBox ，但是 ARM64 沒有 VirtualBox ，只有 Proxmox 有，剛看了一下，還支援 riscv64,loongarch64 [ref]，不過我個人不太想再學一套，所以還是用 virt-manager/libvirtd 這套，一般用途 virt-manager 和 libvirtd 應該也很夠用了。

目前用到現在都可以用，Ubuntu 22.04，Windows 10 for x86，Windows 11 for ARM，安裝沒什麼問題。只有 Windows 10 for x86 非常慢，慢到我不想再開了，不知道為什麼。

Qemu bridge devices: 這個應該是問題，如果是以前用 VirtualBox ，可以直接選 NIC card interface ，但是 virt-manager 似乎沒這個選項，所以要直接設定到 bridge device，不過影響不是很大，就只是設定一個 bridge device，然後將 host internet ethernet port device 綁上去就好了

其他可能是如果 Libvirt 要用 graphic 輸出，不能用 qxl ，只能用 VGA or ramfb ，尤其是 windows 只能用 ramfb，如果 Guest 是 Linux 可以用 VGA ，顯示比較正常，如果連 GUI 都沒有，就直接看 console 輸出吧，問題更少

Qemu file sharing

QEMU 支援 virtiofs ，不過這個功能在 gentoo 內 ARM64 是不能用的，因為不能用，所以我也不知道 x64 那邊會有什麼限制
另外一個是 virtio-9p ，這個缺點是分享進 VM 之後，沒辦法使用 user 權限存取，用 root 身份限制也很多，像是存取 non-root user 檔案會有問題[ref]

有其他需求的人可以考慮改用 samba or nfs ，但是效能就會大大下降了，這點可能是有效能需求的人要考量的

Firewall

原來的 Firewall script 似乎沒辦法和 Libvirt 一起工作，我又不想修改，最後比較簡單的作法就是在我的 script 執行完之後，再重啟 libvirtd ，這樣就正常了。
正常後別忘了執行 /etc/init.d/iptables save 儲存，這樣重開機之後才會正常執行 firewall rule.

Docker

試了一下似乎沒什麼問題，但是目前沒什麼要跑的 APP ，所以應該測不出什麼
不過之前 arm64 k8s docker [ref]都跑過了，應該踩雷的機會低很多

Other Software

在 Linux 下轉移到 ARM64 平台問題不大，大部份的 software 都是安裝之後 copy 過去
就算是在系統上有資料的 mysql 和 apache 都是這樣

這次我用了一個方式轉移 configuration files
1. 將要轉移的程式設定檔 commit 到 git server
2. 從新機器將 git server 拉回來 configuration files
3. diff 之後將新的設定套上去

因為我是 gentoo 玩家，gentoo 升級 packages 常常需 merge configuration files 了，所以這個轉移的過程不難，只是很耗費時間而已。不過因此絕大部份的設定檔都沒有問題，反而是少數系統本身自帶的設定檔反而怪怪的

KDE

之前我都遠端跑 vnc + kde ，不過 Gentoo 有個 BUG 就是 plasmashell 的 CPU usage 會很高
這次換 virt-manager 之後可以用 ssh remote control VM，發現這個方式還不錯，反正我也只有用到這個功能，KDE 等有需求的時候再開 vnc 就好了，平時是不需要開的

搬移步驟

1. 備份系統碟，以備不時之需
2. 重新規劃電腦的網路卡，如果只有一張就直接設成 bridge interface
3. software raid 的部份可以參考自己之前的文章 Linux Software Raid 系統轉移
4. UEFI 的部份可以參考自己之前的文章 Gentoo ARM64 UEFI Install Guide
5. 搬移 APPs
5.1 重要的 APP 才要搬資料，ex: Web, mysql
5.2 不重要的 APP 只要搬設定檔，ex: apt-cache-ng, squid 等等
5.3 如果是切換平台，要確定 VM 沒問題才能切

Known Issues

IPv6 Default Router

不知道為什麼，在 wan 的 bridge interface 會增加一個 default route fe80::1 ，這個設定會讓 IPv6 失效，後來解決方式是在 LAN 端加上一個 IP address fe80::1 才解決

efi boot

因為我系統碟是做 software raid ，每次編完 kernel 再 update grub 時，記得 efi boot 時有時會 fail ，試著將兩顆 nvme 都 mount 上來加上再安裝一次 efi partition 似乎才會正常，找了一下沒找到原因，不管了，反正能動就好

grub-mkconfig -o /boot/grub/grub.cfg
umount /boot/efi
mount /dev/nvme0n1p1 /boot/efi
grub-install --target=arm64-efi --efi-directory=/boot/efi --bootloader-id=Gentoo
umount /boot/efi
mount /dev/nvme1n1p1 /boot/efi
grub-install --target=arm64-efi --efi-directory=/boot/efi --bootloader-id=Gentoo
umount /boot/efi

ref.
None

The post [Gentoo] Upgrade From x64 to ARM64 appeared first on richliu's blog.

Chinese article version: 在 ARM64 QEMU System 上安裝 ARM64 Windows 11

Nowadays, ARM64 systems have become relatively mature, and recently, Windows 11 for ARM64 can also be installed on these systems. Naturally, there are people who want to run it on QEMU. However, most articles on this topic are not well-organized.

Here, let’s document the things what I discoverd and need to be prepared.

Preparations Hardware and Software

1) System: Ampere Altra CPU
Small ARM64 dev boards like RPI4 or RPI5 should also work, but installing on them might not be necessary, because it only has limitation CPU power and Memory. A more cost-effective solution is to use Apple Mx series CPUs [ref], as they are also ARM64, and many people purchase them for installing Linux.

2) ARM64 ISO Image
While there are numerous options available on internet, for a relatively legal acquisition of the ISO image, it is recommended to use Windows-11-On-Ampere
This step can be a bit complex, but the key is to use UUP Dump to fetch the Windows ISO and then use Rufus to burn it onto a USB Flash drive. During the burning process, Rufus can override some of the restrictions of Windows 11, such as TPM Support, which may not be fully supported on ARM64 CPUs.

3) Windows Virtio Driver

Download the Windows Virtio driver and place it into the USB Disk that was created in the previous step.

Here has download URL can download the Windows virtio driver : Proxmox Windows VirtIO Drivers, alternative download website:  download the latest stable or  download the most recent.
After downloading, extract the ISO contents and place them onto the USB Disk.

4) Convert USB Dump to Image
Linux users are likely familiar with this step.

dd if=/dev/sda of=/storage/Win11_ARM64_INSTALL.img

Why go through these steps? The primary reason is that within QEMU, it seems Windows cannot simultaneously configure two ISO devices. I encountered a freeze when attempting to install using the straightforward method. Hence, I went through this elaborate process to make it install smoothly.

5) virt-manager
Most examples available to others involve pure QEMU commands. However, this time, I wanted to avoid typing a bunch of commands and preferred a more user-friendly approach. Therefore, the examples provided here utilize virt-manager. Nonetheless, I believe the underlying logic is the same.

New Virtual Machine on QEMU

Virt Type: KVM
Architecutre: aarch64
Machine Type: virt

Next step, choice the ISO image we just created Win11_ARM64_INSTALL.img, Choose “the operaion system you are installing: Microsoft Windows 11”

For the CPU and memory, I used 16 cores and 16GB, which should be sufficient. A smaller combination might work, but I haven’t tested it because I’m too lazy to do so.

It’s recommended to set up the Disk Image manually, virt-manager tends to allocate the entire specified disk space right from the start, which may not be space-efficient. Windows 11, with its minimum space requirement of 52GB, can be allocated around 60GB or more based on your needs. Here is example command to pre-create qemu image.

qemu-img create -f qcow2 MyImage.qcow2 60G

Before executing, choose “Customize configuration before install.” There are certain adjustments to make. Remember to enable the XML editing feature within virt-manager beforehand. Go to the main program, select Edit -> Preferences -> Enable XML editing, and ensure it is checked.

In the XML section related to CPUs, locate the segment pertaining to ‘hyperv,’ and modify it with the following settings:

    <hyperv mode="custom">
      <relaxed state="off"/>
      <vapic state="off"/>
      <spinlocks state="off"/>
    </hyperv>

In the CPU section, change the quantity from ‘sockets’ to ‘cores.’ Below is an example for 16 cores:

Add those hardware setting into current configuration:
Input -> USB Keyboard
Input -> EvTouch USB Graphics Tablet
Graphics -> Spice server
Video -> Ramfb

For the Video option, it is essential to choose Ramfb; otherwise, the graphics might not display properly. QXL or Virtio should not be used. Virtio-gpu-pci hasn’t been tested, but some suggest it might work[ref].

Lastly, in Storage, add a USB CDROM device and set it up with the Win11_ARM64_INSTALL.img file you created earlier.

In the Boot Options section, enable the option for USB CDROM 1, and then you should be ready to boot.

If the configuration is correct, you should be able to boot to this screen successfully.

If following the above steps results in a hang during the installation process, you can try adjusting the boot options by changing the boot order. Try setting “disk” first and “USB CD ROM” second.

Install Windows

Under normal circumstances, Windows might not detect the disk. In such cases, select “Load driver” -> “25982_1000_ARM64_EN-US(C:)” -> “Virtio-driver” -> “viostor” -> “win11” -> “ARM64” (Note: The labels and directories here may vary based on different configurations).

In the final step, for the NIC driver, when prompted to connect to the network upon the initial entry into Windows 11, select “Install driver” -> “25982_1000_ARM64_EN-US(D:)” -> “Virtio-driver” -> “NetKVM” -> “win11” -> “ARM64”.

With this, you should have successfully completed the installation of Windows 11.

Happy Playing Windows 11 on ARM64 Qemu

ref.
virtio-win / virtio-win-pkg-scripts

The post Install ARM64 Windows 11 on ARM64 QEMU System appeared first on richliu's blog.

English please see : Install ARM64 Windows 11 on ARM64 QEMU System

現在 ARM64 的系統相對成熟了，最近 Windows 11 for ARM64 也可以在系統上安裝，自然也會有人想弄到 QEMU 上去，不過看來看去，大部份都沒有整理完畢的文章

這邊記錄一下需要準備的東西

準備

1) 系統：Ampere Altra CPU
RPI4 or RPI5 這種開發板應該也可以，因為這些板子只有有限的 CPU Power 和記憶體，就沒有安裝 qemu 的必要性，便宜一點的解決方案就是 Apple Mx 系列的 CPU [ref]，因為也是 ARM64 ，而且很多人也是買來安裝 Linux

2) ARM64 ISO Image
網路上應該很多，不過如果要相對合法取得 ISO image ，推薦 Windows-11-On-Ampere
這步驟有點複雜，不過重點是用 UUP Dump 抓回 Windows ISO ，然後用 Rufus 燒到 USB Flash ，在燒錄的時候 Rufus 可以解除一些 Windows 11 的限制，像是 TPM Support 等等這些在 ARM64 CPU 還不完全支援的設定

3) Windows Virtio Driver

下載 Windows virtio driver，然後放進去剛剛那一步做到的 USB Disk

Proxmox Windows VirtIO Drivers 這邊有下載點， download the latest stable or  download the most recent ，都可以下載到 VirtIO ISO，下載完將 ISO 解開放到 USB Disk

4) 將 USB Dump 成 Image
Linux user 應該很熟這個步驟

dd if=/dev/sda of=/storage/Win11_ARM64_INSTALL.img

為什麼要這樣做？最大的原因就是 QEMU 內 Windows 現在似乎沒辦法同時設定二個 ISO 設備，我之前這樣裝會卡住。所以就繞了一大串讓它可以裝，還可以吧

5) virt-manager
其他人能找到的範例都是 qemu command ，不過我這次實在是不想打一堆指令，只想點點點，所以這次的範例都是用 virt-manager ，不過我相信背後的思路都是一樣的

New Virtual Machine

Virt Type: KVM
Architecutre: aarch64
Machine Type: virt

下一步，ISO image 選剛剛做好的 Win11_ARM64_INSTALL.img，Choose the operaion system you are installing: 選 Microsoft Windows 11

CPU 和記憶體我用 16 cores 和 16GB，小一點應該也可以，不過我懶得試了

如果空間不夠，Disk Image 建議自己設定，virt-manager 設定會一開始就 allocate 設定的 disk space ，不省空間，Windows 11 最小需求空間是 52G ，可以設 60G 或是更大，看需求
這邊是範例命令，可以預先建好硬碟空間

qemu-img create -f qcow2 MyImage.qcow2 60G

在執行之前選擇 Customize configuration before install，有些東西要修改，記得要先開啟 virt-manager 內的 XML 修改功能，在主程式下的 Edit -> Perferences -> Enable XML editing ，請勾選

在 CPUs 的 XML 這一段，找到 hyperv 這一段，改成如下的設定

    <hyperv mode="custom">
      <relaxed state="off"/>
      <vapic state="off"/>
      <spinlocks state="off"/>
    </hyperv>

在 CPU 數量，要從 sockets 改成 Cores，以下就是 16 Cores 的範例

Add hardware ，加入以下設定
Input -> USB Keyboard
Input -> EvTouch USB Graphics Tablet
Graphics -> Spice server
Video -> Ramfb

Video 選項一定要用 Ramfb ，否則 Graphics 會出不來，qxl or virtio 都不能用，virtio-gpu-pci 沒試過，有人說可以用[ref]
最後是 Storage 增加 USB CDROM device，並且將剛剛的 Win11_ARM64_INSTALL.img 設定上去

在 Boot Options 這邊 enable USB CDROM 1的選項，然後就可以開機了

如果設定正確，那就可以正常開機到這個畫面

如果照著以上步驟卻在安裝過程卡住了，可以試著調整 boot option 切換一下開機的順序
先 disk 後 USB CD ROM .

Install Windows

正常狀況下，Windows 是找不到 disk 的，要選 load driver -> 25982_1000_ARM64_EN-US(C:) -> Virtio-driver -> viostor -> win11 -> ARM64 (註：這部份的標籤和目錄可能因不同的設定而異）

最後一段，NIC driver ，在初次進入 Windows 11 時，要求要連線到網路時，選 Install driver -> 25982_1000_ARM64_EN-US(D:) -> Virtio-driver -> NetKVM-> win11 -> ARM64

這樣就可以完成安裝 Windows 11 了

Happy Playing Windows 11 on ARM64 Qemu

ref.
virtio-win / virtio-win-pkg-scripts

The post 在 ARM64 QEMU System 上安裝 ARM64 Windows 11 appeared first on richliu's blog.

Chinese version Article is here (Ampere Altra/AltraMax 加AsRock ALTRAD8UD-1L2T 水冷方案)
2024/01/31 Update: Ampere will have 2U passive cooler Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926

The Ampere Altra/AltraMax CPU features a maximum of 128 ARM64 cores, typically found in cloud server. However, cloud server Care not suitable for placement in offices or homes due to the high-speed fans used for heat dissipation. The noise generated by these fans is generally intolerable for regular users. Additionally, adapting server motherboards to standard consumer motherboards is inconvenient. If there is a desire to utilize such a CPU in a different setting, the ADLink Ampere Altra Dev Platform presents a viable solution, complete with the option for liquid cooling.

Another solution is the AsRock ALTRAD8UD-1L2T, currently available on Newegg. Paired with an Ampere Altra Q64-22 CPU (64 cores, 2.2GHz), it comes with a complimentary 2U passive heatsink, priced at US$1,500, which can be considered affordable. However, it’s worth noting that the included heatsink is passive. If you prefer liquid cooling over installing a fan on the passive heatsink, then this article is the right guide for you.

The author have adding a declaration at the beginning of articles that may pose potential risks:

• This article is not an official endorsement by Ampere, Newegg, or any other mentioned merchants or vendors. It is solely a personal record.
• The author has a business relationship related to the production of the mentioned motherboard.
• The author is a first-time to use liquid cooling computer system , and the content may not be entirely accurate. Please be aware of potential risks.
• This article is not an official document from Ampere or AsRock.
• The original article is in Chinese, and the English translation is done using machine translation. Please forgive any inaccuracies in wording.
• I am not responsible for any consequences resulting from this article; for example, losses incurred due to improper implementation of instructions provided in this article.

The Ampere system primarily focuses on Cloud-native CPUs, and as of now, there is no manufacturer producing general-purpose cooling systems for it. The included 2U passive heatsink in this system is designed for server use, utilizing high-speed server fans for heat dissipation. While the Q64-22 included by Newegg has lower power consumption, its only cooling method involves direct airflow from an electric fan. Alternatively, users may resort to attaching an 8-inch fan onto it. Such cooling solutions are deemed unacceptable for enthusiasts and professionals like myself.

If you intend to upgrade the cooling system, typical tower-type coolers require replacing the fixture on the back of the motherboard. Moreover, compatibility with the fastening mechanisms of this specific motherboard can be challenging to find. Considering that only front-facing screws can be used, the only viable cooling solution is a liquid cooling system. Therefore, I have been experimenting with a liquid cooling solution that works for this scenario. I’m sharing this information for those adventurous individuals who might encounter similar challenges, and perhaps this approach can inspire some creative solutions.

In the upcoming sections of this article, I will guide you on how to set up a liquid cooling system specifically tailored for the Ampere Altra on the ALTRAD8UD-1L2T motherboard. In theory, since the screw hole positions are the same, this method should be applicable to other Ampere motherboards as well. However, the focus of this content will primarily be on the ALTRAD8UD-1L2T. The article will also cover any pitfalls encountered during assembly and the items needed for the setup(Chinese website).

Preparation

ALTRAD8UD-1L2T

You can find the specifications for this motherboard online, and I won’t repeat them here. However, what might confuse some people is the term “Deep mATX.” This is a specification unique to AsRock, where they increase the depth dimension of the mATX form factor while maintaining the standard width. The purpose is to provide more space to accommodate additional components. This particular specification is commonly seen in AsRock’s server-grade motherboards.

Chassis

Generally, most computer cases should work. However, I made a mistake on my end by choosing a case that was too small. The case I purchased is the Antec P110 Silent. While it has liquid cooling mounting points, it is not suitable for adding a liquid cooling system. The internal space is too limited, lacking a designated area for securing the water pump. Additionally, the Antec P110 liquid cooling radiator cannot be placed at the top as it interferes with the motherboard. Consequently, the radiator had to be positioned at the front of the case.

When making a purchase, it’s crucial to choose a case that is suitable for water cooling. In more extreme cases, considering a caseless system might be an option (hopefully, this can be updated in the next release).

Cases designed for liquid cooling systems typically have more significant internal space to accommodate a water pump, and there is ample room for fans without obstruction. Some all-in-one water cooling designs may be more compact and might not encounter this issue.

Liquid Cooling System

Liquid cooling systems have some considerations to take into account:

• Tubing type: Choose between soft or hard tubing. Hard tubing is suitable for experienced users, offering a longer lifespan and a more aesthetic appearance. Soft tubing is easier for assembly, making it suitable for beginners.
• All-in-one (AIO) or custom loop: Decide whether to go for a pre-assembled AIO liquid cooling system or build a custom loop system.

Since the initial focus was on researching liquid cooling, the combination chosen was a custom loop with soft tubing. Building a custom loop may encounter some issues, with one of the main challenges being the uncertainty about specifications.

The first crucial aspect is determining the size of the liquid cooling tubing, as additional components will depend on this size, limiting the options for procurement. Currently, the mainstream sizes are two-eighths (2/8’), three-eighths (3/8’), and four-eighths (4/8’) liquid cooling tubing.

Another parameter is the thickness of the tubing, thick tubing and thin tubing.
* thick tubing refer to 10/16mm or 3/8”-5/8” tubing.
* thin thbing refer to 10/13mm or 3/8”-1/2″ tubing.

Most common the inner diameter specification is G 1/4, when find some liquid colling componens like pressure relief valve needs it.

Once you’ve confirmed your inner and outer diameter, you can proceed to purchase the necessary liquid cooling components.

At a minimum, you’ll need:

• CPU water block *1
• Water Pump *1
• 240mm radiator *1
• 12cm fans *2
• Liquid coolant 500cc ~ 1000cc
• Tubing 2M
• Thermal paste
• Pressure relief valve
• Fittings: Quick connectors *6, 90-degree rotary fittings *(1~2)
• Tools for filling and draining: Filling tool, draining tool (watering can, large syringe, drainage hose)

Optional components:

• Flow meter, temperature gauge (note: buying these requires purchasing two additional quick connectors).

In practical testing, if focusing solely on CPU cooling, it appears that a 240mm radiator is sufficient. The thin three-eighths (3/8”) tubing provides a large enough water flow diameter. Choosing a smaller tube diameter may not be as suitable for this purpose.

CPU water block

This turned out to be quite troublesome. My original plan was to purchase a 50mmx50mm heatsink and connect it to the heatsink on the right. Then, I intended to secure it to the CPU socket using the fixtures shown in the image. However, as expected, unexpected issues arose – the world isn’t always that kind.

1. There was no water block with a 50mmx50mm connector available. So, I ended up buying a 40mmx40mm connector, one size smaller. This resulted in poorer heat dissipation performance. Fortunately, I never had the chance to test the inferior heat dissipation, so this step failed unnoticed.
2. Here’s where I made a mistake. Custom water block connectors, like the one shown, are typically 2/8″ fittings. However, the main liquid cooling system I bought has 3/8″ fittings, making them completely incompatible. The options were either to change the fittings or change the entire system. Changing the fittings was the more cost-effective choice, so I opted for that.

After connecting , it looked like the image on the right – and just looking at it made me worry that water would leak out.

So, I bought a two-eighths(2/8″) tubing to connect to this connector. However, while it could connect, it couldn’t link up with my three-eighths(3/8″) tubing in the water cooling system. I had to reluctantly abandon the custom water block connector.

The final solution had nothing to do with this connector.

Communical CPU water block

During the installation, I discovered that the Bykski CPU-XPH-T8 CPU water block, along with the provided adapter bracket, could be securely attached to the Ampere Altra’s socket. This was fantastic news and saved me a great deal of effort.

Although it’s designed for Intel LGA1700/LGA2066 sockets, the flexibility in its hole configuration allowed for adjustments, providing me with the necessary adaptability. As for whether other CPU brackets might be compatible, it’s likely that some could work. However, I haven’t had the chance to try others yet. If you have attempted this and found success, please let me know.

The base of this water block is quite substantial, and my intuition tells me that the cooling performance should be excellent. Therefore, I didn’t explore other compliant CPU water blocks.

However, another issue arises. The water block is 56mmx56mm, whereas the width of the Ampere Altra CPU socket is 55mm. The water block does not fit. At this point, there are two possible solutions:

Upon observing the CPU’s structure, the first solution seemed unfeasible. Cutting down the thickness of this water block would compromise its structural integrity, not to mention finding the tools or a professional service for the task.

Thus, the only viable option is the second solution. The simplest and most intuitive way to implement this is by using a thermal pad made of silicone to transfer heat upwards from the CPU.

Silicone Thermal Pad

But things couldn’t be as simple as I, a fool, initially thought. In conclusion, regular silicone thermal pads are not suitable for power monsters like the Ampere Altra. This type of thermal pad may be effective for low-power devices such as routers, switches, NVMe disks, etc., but it’s not useful for high-power CPUs.

In the image, there are 2mm thick silicone thermal pads, and you need two layers because the height is just that much. Essentially, this provides only a slight improvement over having no heat dissipation at all, so it’s not worth considering.

As for the high-efficiency silicone thermal pads available in the market, I don’t know if they would work. If you have them on hand, you can give them a try. However, I believe the chances of success are quite low.

Purple Copper Thermal Pad

Nowadays, a common method for cooling is using purple copper heat pipes. This type of heat dissipation is quite effective, and the 2U passive heatsink included with Newegg utilizes these heat pipes. Therefore, if we can customize the heatsink, wouldn’t that solve the problem? Luckily, the country right next to my country is the world’s largest manufacturing factory, capable of customizing just about anything. This made the solution very affordable and instantly resolved my problem.

The dimensions of the purple copper thermal pad are: Length x Width x Height = 64x54x4mm, with a rounded R5 edge.

Alternatively, you can measure the dimensions of the piece under the 2U passive heatsink to get the specifications. Placing it on the original heatsink, I can say the dimensions match perfectly. The craftsmanship is quite impressive, and there are no issues when it’s actually placed.

Assembly

First, place the purple copper thermal pad, and remember to apply thermal paste before doing so.

Apply thermal paste before placing CPU water cooling block.

Assemble the components.

Actually, I assembled the tubing on the left side incorrectly. I’ll need to purchase materials and quick connectors to replace it when I have the time. But for now, let’s leave it as is and make do.

Overall view.

This way, it’s evident that the case isn’t ideal. It’s too cramped, affecting airflow, and it cannot accommodate the components properly.

Shopping

If you are interested in replicating my setup, all the components were purchased from Taobao. I spent extra for testing and experimenting, but in reality, you should be able to set up the entire cooling system for around US$100 or NTD$3,000. This is comparable to the cost of an efficient dual-fan tower cooler.

Since this is a Chinese website, I’ll keep the text in Chinese and provide English translations afterward.

Liquid Cooling System

watertiger定制台式机电脑分体式CPU显卡水冷软管套装冷散热器 (“WaterTiger Custom Desktop Computer Split-Type CPU GPU Liquid Cooling Soft Tube Kit Radiator”)
Enter the URL choice “單 CPU 水冷 240 冷排” (“Single CPU Liquid Cooling 240 Radiator”)

At first, I chose this brand because the photos looked appealing, but it turned out to be a fortunate coincidence that saved me a lot of exploration time. This brand integrates various components into a set for sale, and the components they use are of decent quality. For example, they use the Enermax NEOChanger as the water pump, which is also a good pump. If you are familiar enough, you might not need these kits.

This kit does not include a pressure relief valve. You’ll need to purchase a pressure relief valve separately, and depending on your needs, you might want to buy additional quick connectors

Clamping Fittings

All the components can be purchased from the same place, saving the hassle of searching everywhere. It’s a good idea to buy a few extra parts to avoid trouble later on. The time cost is much higher than the cost of the components.

电脑水冷头扣具卡具CPU显卡南北桥固定笔记本散热器压条五金夹具 (“Computer Water Cooling Head Fastening Fixture, CPU GPU North-South Bridge Fixing, Laptop Heatsink Clamp, Hardware Clamp”)

Choice 台式机CPU冷头扣具(“Desktop CPU Cooling Head Fastening Fixture”) *2

M2 M3 M4不锈钢平头盘头螺丝钉水冷头水冷排机箱风扇安装长螺丝 (“M2 M3 M4 Stainless Steel Flat Head Countersunk Screws for Water Cooling Head, Radiator, Case Fan Installation – Long Screws”)

Fastening fixtures for securing CPU coolers and water blocks. The components can be increased based on requirements. The minimum required quantity includes
M3手拧螺母 (M3 thumb nuts)*4，M3*30 (M3 screws) *4 ，M3垫片(M3 washers) *4 ，M3六角螺母(M3 hex nuts) *4

电脑水冷水流计 液冷散热器流量器 流体温度观测器透明水流指示计 (“Computer Liquid Cooling Flow Meter, Liquid Cooler Flow Indicator, Fluid Temperature Monitor Transparent Flow Meter”) (option)

Choice: 3孔水流观测器 + 温度计 (“3-Port Flow Meter + Thermometer for Water Flow Observation”)

Nice to have items, but it can make me feel better.

Test Result

This system feels relatively quiet and not noisy.

The following tests are only for verifying the liquid cooling setup and are not formal performance tests. The results are provided for liquid cooling reference and do not include testing power consumption, etc.

Room temperature: 25 degrees Celsius
CPU : Ampere AltraMax 128-30
Run command : stress-ng –cpu 128 –iomix 128 –vm 128

Gentoo Linux kernel 6.6 ，CPU temperature is approximately，Power consumption CPU 150W/ IO 19W
Ubuntu 22.04 kernel 6.2(?)，CPU temperature is approximately 72 degrees，Power consumption CPU 178W/ IO 20W

In conclusion, I hope this article has been helpful for those eager to try it out.

Afterword

Newegg has also listed another model, which is only the motherboard version, only US$849.-
ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Singel Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G

The post Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System appeared first on richliu's blog.

English version article is here (Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System)
2024/01/31 Update: Ampere will have 2U passive cooler Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926

Ampere Altra/AltraMax CPU 最高單顆 CPU 是 128 核 ARM64 的 CPU，一般都是在伺服器主機上，但是伺服器主機非常不適合放在公司內或是家用，因為一般伺服器主機都會用高轉速風扇散熱，一般人是不能忍受這樣的噪音。伺服器主機板也不方便改裝到一般家用主機板，如果真的想要用，ADLink Ampere Altra Dev Platform 是一個不錯的解決方案，也有水冷的選項。
另一個解決方案就是 AsRock ALTRAD8UD-1L2T ，目前在 newegg 上架了，配合一個 Ampere Altra Q64-22 (64 cores 2.2G) 的 CPU，附贈一個 2U Passive 的散熱片，價格 US$1,500.- ，應該算便宜。但是送的是被動散熱，如果不想要裝一顆風扇到被動散熱上，想要改成水冷散熱，那看我這篇文章就對了。

本人習慣，在有可能造成危險的文章，會在文章前面加上宣言
* 本篇文章並非 Ampere, Newegg 或是其他任何文章內提到的商城及店家業配文，純屬個人紀錄
* 本文作者和部份這塊板子的生產有利益關係，特此揭示
* 作者是第一次接觸電腦系統水冷，內容可能不盡正確，請自行注意風險
* 本文並非 Ampere 或是 AsRock 官方文件
* 本人不對本篇文章的造成的任何影響負責任；例如看本人文章操作不當造成的損失等等

Ampere 系統主要是專攻 Cloud nativate CPU，所以直至目前為止，並沒有廠商生產一般用途的散熱系統。這個系統附贈的 2U Passive 散熱片是 Server 專用的，使用的是 Server 高速風扇散熱；雖然 newegg 附的是 Q64-22 ，功耗比較低，但是唯一的散熱方式就是拿電風扇直接吹，要不然就是卡一個8吋風扇在上面，這種散熱方案像我這樣的專業玩家是無法接受的

如果要進階散熱，一般的塔型散熱系統需要將主機板背面的夾具換掉，而且也要能跟這個板子的扣具相容，找到相容的塔型散熱難度可能也不小。考量到只能在正面鎖螺絲，那唯一能用的散熱系統就是水冷式散熱。所以我這段時間摸索出了一條可以用的水冷散熱解決方案，在此分享給想要踩雷的朋友們，或許這個做法也可以激發各位的創意。

接下來會在這篇文章內介紹要如何在 ALTRAD8UD-1L2T 板子上，搭建屬於 Ampere Altra 的水冷系統，理論上因為螺絲孔位一樣，所以 Ampere 其他的板子也可以用，但是本篇內容主要以 ALTRAD8UD-1L2T 為主，而本文亦會提到踩過的地雷或是採購項目。

準備

ALTRAD8UD-1L2T

上網可以找到這塊板子的規格，這些就不再重覆，但是一般人會比較迷惑的是，什麼是 Deep mATX ？
這個是 AsRock 自己的規格，是增加 mATX 的深度尺寸，保持 mATX 的寬度規格，以增加更多空間容納更多元件。這個特別的規格常見於 AsRock server grade 的主機板上。

機殼

一般機殼應該都可以。不過我這邊犯了一個錯誤，機殼買太小，我買的機殼是 Antec P110 Silent ，雖然有水冷孔位，但是這個機殼不適合加裝水冷，內部空間太小，沒有固定水泵的地方，Antec P110 水冷排無法放在上方，會卡到主機板，所以最後水冷排是放機殼前方

採購時，請找適合水冷的機殼，更極端一點可以考慮無機殼系統（希望下次更新可以更新無機殼系統）。

水冷機殼內部空間需求更大，要有地方放水泵，還有風扇放下去不會卡住。
有些一體式水冷的設計會比較小，比較不會有這個問題

水冷設備

水冷散熱有一些需要考慮的
* 軟管或是硬管（硬管比較適合有經驗的，壽命長，較美觀；軟體組裝方便，適合新手）
* 一體式水冷（廠商已經組裝好）或是分體式水冷（自組水冷系統）

因為一開始是研究如何建置水冷，所以一定是分體式水冷加軟管的組合。自組水水冷又會碰到一些問題，最主要的是不知道規格為何？
第一件需要確定的是水冷管大小，因為加購設備就是看水冷管大小，這會限制採購品項
目前主流有 二分(2/8’)、三分(3/8’)、四分(4/8’)水冷管，但是實際上在找產品的時候，會標為像 6*10(二分) 9.5*12.7(三分，有些會標 10*13) 另外還有 8*12 不知道對應到什麼，數字小的是內徑，數字大的是外徑 。三分又有分三分薄和三分厚，三分薄就是 10/13 ，三分厚就是 10/16 ，如果不是老手很容易在這邊搞錯。本文用的全部都是三分薄的系統

以上有英制和公制，要特別小心注意規格，像洩氣閥內徑規格就要用 G 1/4 規格的（目前大都是此規格）
確定好自己的內外徑之後，就可以採購需要的水冷套件

最少需要的是
CPU水冷頭*1
水泵*1
240寬冷排*1
12CM風扇*2
水冷液 500cc ~ 1000cc
水管 2M
散熱膏
洩壓閥
接頭：快速接頭*6 ，90度旋轉彎頭*(1~2個)
注水工具，放水工具（加水壺，大針筒，放水管）
可選：
流量計，水溫計（註：買這個要多買二個快速接頭）

實測如果只針對 CPU 散熱，目前看起來 240 水冷排就夠，而三分薄提供了夠大的水流口徑。
如果管徑再小可以不太合適。

CPU水冷散熱頭

這個頗麻煩的，我原來的規劃是買一個 50mmx50mm 的散熱片接到像右邊的散熱頭上。然後用圖中的夾具固定在 CPU 座上面，但是不出意外的就出了意外，這世界上沒這麼好的事情

1 ) 沒有 50mmx50mm 的水冷頭接頭，所以買的是 40mmx40mm 的接頭，小了一圈，散熱性能較差，幸運的是我根本沒機會測到散熱性能較差這步就失敗了

2) 我在這邊犯了一個錯誤，這種像是定製的水冷接頭都是 2/8″ 頭，但是我買的主要水冷系統是 3/8″ 的系統，完全接不上去，要就是要換頭，或是換系統；當然換頭比較便宜，所以選擇換頭

接上去之後像是右邊這樣，這光看都覺得水會跑出來吧

所以我買了二分管想要接這個接頭，但是二分管是可以接上去沒錯，但是接不上我水冷的三分管系統。只能果斷放棄接定製水冷頭

最後的解法已經和這個接頭無關了

商用水冷頭

在裝的時候發現買的套件附的 Bykski CPU-XPH-T8 這顆 CPU 水冷頭加轉接夾具可以合上到 Ampere Altra 的底座，這簡直太棒了啊。省了非常多功夫

雖然是給 Intel LGA1700/LGA2066用的，所以它的孔位設計成非常彈性，這就給了我調整的空間。至於其他 CPU 的夾具有沒有可能可以用，應該也有可以用的，不過這邊我還沒機會試，如果你有試過可以告訴我。
而這個水冷頭的底座非常大，直覺告訴我散熱效果應該非常好，所以也並未去找其他合規的 CPU 水冷頭

但是接下來又有另一個問題，這個水冷頭是 56mmx56mm ，而 Ampere Altra CPU socket 寬度為 55mm，這個水冷頭放不下。這時有二個選擇方案
1) 切割水冷頭讓它變小（還沒考慮高度問題）
2) 將熱從 CPU 接到水冷頭

第一個方案觀察了一下 CPU 工藝，覺得不可行，因為這顆水冷頭的厚度切下去會影響結構，更別提還要想辦法找到工具或是店家切割。

那就只能用第二個方案了，這個方案最簡單並且非常直覺的方式就是用矽膠導熱墊將熱拉上來

矽膠導熱墊 (thermal pad silicone)

但是事情怎麼會像我這個笨蛋想的這麼簡單，先說結論，一般的矽膠導熱墊不適用於 Ampere Altra 這種 Power Monster ，所以完全不用考慮。這種矽膠導熱墊給小功率的 router/switch/nvme disk 等等或許有用，對大功率的 CPU 是沒有用的

圖中是 2mm 厚的矽膠導熱墊，而且要二層，因為高度就是這麼高，基本上這個只比不接散熱好一點點而已，所以別想了

至於市面上賣的高效率矽膠導熱墊有沒有用我就不知道了，如果手上有現成的可以試試，不過我認為成功機率很低

紫銅導熱片

那現在一般的散熱是怎麼散熱呢？一般都是紫銅導熱管，這個散熱效率還不錯，newegg 附贈的 2U passive heatsink 就是使用這種導熱管散熱。所以如果可以客製散熱片，這樣不就解決問題了嗎？幸好我們旁邊就是世界最大的製造工廠，任何怪東西都可以客製，這樣一片非常便宜，瞬間解決了我的問題

紫銅導熱片的尺寸規格是
長x寬x高 = 64x54x4mm ，另外要倒 R5 的角

也可以量 2U passive heatsink 下面那一塊得到規格，放在原廠的散熱片上只能說尺寸一樣
工藝還真不錯，實際放下去也沒有問題

組合

先放上紫銅導熱片，記得放之前塗散熱膏

放上去之前再塗散熱膏

合體

其實這張圖左邊的管子我裝錯了，有空要再去買材料和快速接頭來換，但是現在就先這樣吧，撐一下。

全貌

這樣就看得出來這機殼不好了，太擠太小影響風流，不能放在上面

購物

如果你是想學我的方案，我所有的零件都是從 Taobao 購入，我因為有測試還有多花錢
但是實際上應該 US$100, NTD$3,000 左右可以搞定整個散熱系統。
跟一個效率好的雙風扇塔扇差不多

水冷系統

watertiger定制台式机电脑分体式CPU显卡水冷软管套装冷散热器
進去後選單 CPU 水冷 240 冷排

當初是看照片比較好看所以買這家的，但是誤打誤撞節省我很多摸索的時間。
這家也是將各家設備整合弄一套出來賣，但是東西用的還不錯，像水泵用的是 Enermax NEOChanger ，也是還不錯的水泵。如果夠熟是可以不用這些套件。

這個套件沒有洩壓閥，另外要多採購洩壓閥，還有依需求可以多買快速接頭

夾具

以下都在同一家採購，不用到處找，零件可以多買一點，免得到時候要找麻煩，時間成本遠高於零件成本。

电脑水冷头扣具卡具CPU显卡南北桥固定笔记本散热器压条五金夹具

選台式机CPU冷头扣具 *2

M2 M3 M4不锈钢平头盘头螺丝钉水冷头水冷排机箱风扇安装长螺丝

固定 CPU 冷頭扣具和水冷頭的零件，這個可以依需求增加，以下是最低需求量
M3手拧螺母*4，M3*30 *4 ，M3垫片*4 ，M3六角螺母*4

电脑水冷水流计 液冷散热器流量器 流体温度观测器透明水流指示计 (option)

選: 3孔水流观测器 + 温度计

可有可無的東西，不過有比較好玩

測試結果

這套系統個人感覺相對安靜，不會吵

以下測試只是驗證水冷測試，並非正式的 performance test ，結果僅供建置水冷參考，並非測試 power consumption等結果

室溫 25 度
CPU : Ampere AltraMax 128-30
Run command : stress-ng –cpu 128 –iomix 128 –vm 128

Gentoo Linux kernel 6.6 ，CPU 溫度約在 67 度，Power consumption CPU 150W/ IO 19W
Ubuntu 22.04 kernel 6.2(?)，CPU 溫度約在 72 度，Power consumption CPU 178W/ IO 20W

最後希望這篇文章有幫到躍躍欲試的你

後記

Newegg 還有上架另一個型號，只有主機板的版本，只賣US$849.-
ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Singel Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G

The post Ampere Altra/AltraMax 加AsRock ALTRAD8UD-1L2T 水冷方案 appeared first on richliu's blog.