Phoronix 於 2026 年 5 月報導，ARM 正在為 Linux kernel 提交一組 RFC patch，準備支援 FEAT_D128——也就是 128-bit page table descriptor 的新翻譯系統 VMSAv9-128，這是 ARMv9.3 之後的選用特性。

目前 patch 還在 RFC 階段，KVM、KASAN 等子系統尚未支援。但這代表 Linux 已經為 ARM 下一個世代的虛擬記憶體架構鋪路，預計會在 ARMv9.3 silicon 上市前準備好 enablement。

二、為什麼要做 D128？兩個動機

ARM 之所以要把 page table descriptor 從 64 bits 加到 128 bits，不是「順便升級」，而是兩個實際撞牆問題：

動機 1：實體位址 (PA) 撞到 52-bit 上限

現有 ARMv8/v9 的 VMSAv8-64 翻譯系統，descriptor 是 64 bits，扣掉屬性位元，能拿來放 PA 的最多 52 bits，也就是 4 PB。

聽起來很多，但對於 CXL pooled memory、AI/HPC 大模型訓練、雲端超大記憶體機台這條路線，4 PB 是可預見的天花板。要再往上走，架構上必須換成 128-bit descriptor。

動機 2：Descriptor 的屬性 bits 已經塞滿

64-bit descriptor 裡的 AttrIdx、AP、SH、AF、nG、PXN/UXN、MTE、BTI、GP……幾乎沒有空位。每次要加新功能（更細的權限、虛擬化 metadata、安全/機密運算欄位）都要從別的欄位擠空間。

128 bits 一次把容器翻倍，把累積的「想加但沒地方加」的需求解決，也替未來十年的擴展留路。

關鍵釐清：FEAT_D128 是 descriptor 變成 128 bits，不是位址空間變成 128 bits。新規格實際支援的 PA 是 56 bits（架構可再擴展），不是 2¹²⁸ bytes 那種天文數字。

三、技術代價：page walk 從 4 層變 5 層

代價不是零：

為了避免 page table 體積爆炸，ARM 把每層解析的 bits 從 9 砍到 8，所以總層數從 4 變 5。代價就是 TLB miss 時的 page walk 要多走一次記憶體存取。

四、效能影響：拿 x86 LA57 類比

ARM D128 silicon 還沒出，沒有實機數字。但可以參考 x86 從 4-level paging 走到 5-level paging（LA57，Ice Lake-SP 2019 開始）的實測：

為什麼平均退化這麼小？ 因為 page walker 自己也有 cache：
– x86 叫 paging structure cache
– ARM 叫 intermediate translation cache

上層的 PGD/PUD entry 幾乎都會 hit，所以實際只多 0–1 次記憶體存取。

ARM D128 的情況可能比 x86 LA57 略糟，原因有兩個：

1. Descriptor 從 8 bytes → 16 bytes，walker 一次 cache line 帶回的 entries 數量減半（8→4），prefetch 效率變差。
2. 每層只解析 8 bits 而不是 9 bits，translation cache 的 locality 稍有不同。

但 ARM 通常 TLB / walker cache 設計比 x86 還積極，會抵銷一些。整體推估：平均退化 < 1%，最壞 case workload 3–5%。

五、ARM 位址擴展演進史

D128 不是憑空出現，是一條延續路線：

六、為什麼 ARM 不直接學 x86 做 5-level paging 就好？

這是和 x86 路線最關鍵的差別。

x86 LA57 的做法很單純：descriptor 還是 64 bits 不變，只是再多疊一層 PML5，把 VA/PA 從 48 bits 推到 57 bits。

ARM 不能這樣做，因為瓶頸不是「層數不夠」：
– ARM 用 FEAT_LPA2 已經把 PA 推到 52 bits（在 4-level 架構內就達到了）
– 真正的瓶頸是 64-bit descriptor 的 attribute bits 已經用完

所以 ARM 必須換大的容器（descriptor 64→128 bits），不是只加層級。多出來的 64 bits 不只用來放更長的 PA，更重要的是放新功能的 metadata：擴展權限、虛擬化欄位、機密運算識別碼等等。

換句話說：
– x86 LA57 ＝ 把瓶子疊更高
– ARM D128 ＝ 換更大的瓶子

兩條路線解決的是不同的問題。

七、為什麼 KVM 還沒支援？

RFC patch 明確標註 KVM 暫不支援。原因不是難度高，而是工作量大。

ARM 虛擬化是 two-stage translation：
– Stage 1：Guest VA → Guest PA（Guest 控制）
– Stage 2：Guest PA → Host PA（Hypervisor 控制）

D128 上線後，Stage 2 的 page table 也要跟著走 D128 格式，KVM 的 page table 操作、shadow page table、IPA fault handler 全部都要改。這是另一個 patch series 的規模，所以 RFC 先把 Stage 1 (host kernel) 做出來，KVM 之後另案處理。

八、結論

FEAT_D128 不是「ARM 突發奇想做大記憶體」，而是兩個工程現實的回應：
1. 4 PB 的 PA 上限在十年內可見
2. 64-bit descriptor 的 metadata 空間用完了

代僸昢 page walk 多一層，從 x86 LA57 的實測經驗推估，平均效能退化 < 1%，最壞 case 3–5%，業界普遍可接受。

Linux 提早動作的原因，是 ARM 一貫的節奏：kernel patch 要在 silicon 出貨前準備好，等晶片到位時 distro 已能跑。所以現在看到的是「鋪路」，不是「迫切」。

ARMv9.3 silicon 預計 2027 之後上市，到時就會有實機數字驗證這些推估。

參考資料

原始新聞

• Arm Preparing The Linux Kernel For 128-bit Page Table Entries “FEAT_D128” — Phoronix

ARM 官方文件

• The Armv9.3 architecture extension — Arm Developer
• Feature names in A-profile architecture — Arm Developer
• TTBR0_EL1, Translation Table Base Register 0 (EL1) — Arm Developer

效能對比與背景延伸閱讀

• 5-level vs 4-level Page Tables: Does It Matter? — JabPerf
• Intel 5-level paging — Wikipedia
• 5-level paging — LWN.net
• TLB and Pagewalk Performance in Multicore Architectures (arXiv 2002.01073)
• Victima: Drastically Increasing Address Translation Reach (arXiv 2310.04158)
• AArch64 memory and paging — Mike’s homepage
• ARM Paging — OSDev Wiki

The post ARM 為 Linux 準備 128-bit Page Table Entries：FEAT_D128 解讀 appeared first on richliu's blog.

大概幾個月前我就發現硬碟有聲音，但是因為使用很正常，所以一直很懶得去處理，大概一個月前才跑去檢查那顆發出聲音，並不是功能影響到我，而是聲音實在讓我太抓狂了。

研究了半天，想說最近 SSD 和 DRAM 都狂漲，剛好看到美亞有黑五特價 Seagate IronWolf Pro 12TB 大概一顆 7500 左右，比 8TB 牌價還便宜，也比我現在用的 Toshiba 大了一倍，再瞄了一下台灣的價格大概都在 11500 左右，想想不想研究了，踩雷就算了吧

訂購中間可以感覺到美亞的落後，光出貨就花了十二天，沒有運貨途中資訊，讓常常買淘寶的我覺得 Amazon 真的是不行，雖然是黑五特價週，但是也太慢了吧

更換的時候我會順便升級系統，就碰到一堆問題

Genkernel dmraid 編譯錯誤

跑 genkernel 會出現 dmraid 的錯誤訊息

misc/file.c: In function 'rw_file':
misc/file.c:64:19: error: initialization of 'ssize_t (*)(void)' {aka 'long int (*)(void)'} from incompatible pointer type 'ssize_t (*)(int,  void *, long unsigned int)' {aka 'long int (*)(int,  void *, long unsigned int)'} [-Wincompatible-pointer-types]
   64 |                 { read, "read"},
      |                   ^~~~
misc/file.c:64:19: note: (near initialization for 'rw_spec[0].func')
In file included from /usr/include/unistd.h:1217,
                 from ./internal.h:24,
                 from misc/file.c:8:
/usr/include/bits/unistd.h:26:1: note: 'read' declared here
   26 | read (int __fd, __fortify_clang_overload_arg0 (void *, ,__buf), size_t __nbytes)
      | ^~~~
misc/file.c:65:19: error: initialization of 'ssize_t (*)(void)' {aka 'long int (*)(void)'} from incompatible pointer type 'ssize_t (*)(int,  const void *, size_t)' {aka 'long int (*)(int,  const void *, long unsigned int)'} [-Wincompatible-pointer-types]
   65 |                 { write, "writ"},
      |                   ^~~~~
misc/file.c:65:19: note: (near initialization for 'rw_spec[1].func')
/usr/include/unistd.h:378:16: note: 'write' declared here
  378 | extern ssize_t write (int __fd, const void *__buf, size_t __n) __wur
      |                ^~~~~
misc/file.c:79:18: error: too many arguments to function 'rw->func'; expected 0, have 3
   79 |         else if (rw->func(fd, buffer, size) != size)
format/ddf/ddf1.c: In function 'check_rd':
format/ddf/ddf1.c:891:25: warning: comparison between 'enum type' and 'enum status' [-Wenum-compare]
  891 |         return rd->type != s_broken;
      |                         ^~
format/ataraid/hpt37x.c: In function 'hpt37x_log':
format/ataraid/hpt37x.c:353:19: warning: taking address of packed member of 'struct hpt37x' may result in an unaligned pointer value [-Waddress-of-packed-member]
  353 |         for (el = hpt->errorlog; el < hpt->errorlog + 32; el++) {
      |                   ^~~

問了 Gemini 和 DeepSeek 後的結果都是

簡單來說，dmraid 這段程式碼非常老舊，它的函數指標定義不嚴謹。在舊版編譯器中，它把函數定義為 (void)（不接受參數）但卻傳入了參數，以前編譯器可能睜一隻眼閉一隻眼，但現在的編譯器（如 GCC 14）預設將這種「指標類型不相容」視為 Error (錯誤) 而不是 Warning，導致編譯失敗。

以下是修復方案，你需要修改原始碼中的 misc/file.c 檔案。（下略)

總之就是花了一點時間找，但是 AI 不識 genkernel ，所以也無從改起，看了一下 genkernel 是採用自己設定 package 自己編的方式，而沒有從 portage 將 patch 拿過來用，所以會 compile 錯誤。不過查了一下 dmraid 最後的 package 是 2010 年 release ，現在應該沒有人用 dmraid ，都是用 mdadm ，之前放在設定檔內應該是相容性的問題，在 /etc/genkernel.conf mark 掉 dmraid 就可以了。

init[1]: Id “f0” respawning too fast: disabled for 5 minutes

這個並不是這次升級的錯誤訊息，而是一直出現在我系統內的，問 GPT 的結果是

這通常發生在 /etc/inittab 中定義的某個終端機（TTY）或服務（識別碼為 f0）嘗試啟動但立即失敗，導致 init 陷入不斷重啟的無窮迴圈。

1. 找出 f0 是什麼
開啟 /etc/inittab 檔案，搜尋 f0 這一行：
grep "f0" /etc/inittab
常見的內容可能如下： f0:12345:respawn:/sbin/getty 9600 ttyS0

2. 常見原因與解決方法
序列埠設備不存在 (Serial Port)： 如果 f0 指向的是 /dev/ttyS0（或 ttyUSB0），但該硬體沒插好、驅動沒掛載或在虛擬機中未配置，getty 就會噴錯並結束。

解決：若不需要該序列埠，在該行最前面加上 # 註解掉。

波特率 (Baud Rate) 錯誤： 設定了硬體不支援的速率。

解決：檢查設定是否為正確的 9600 或 115200。

執行檔路徑錯誤： /sbin/getty 或指定的程式路徑不存在。

解決：確認檔案路徑正確，或改用 agetty。

3. 套用變更
修改完 /etc/inittab 並存檔後，不需要重新開機，執行以下指令讓 init 重新讀取配置：

Bash

telinit q

查了一下果然如此，不過上一代的 GPT 就不知道是什麼，主要是我現在 f0 那一段的 ttyAMA0 不知道為什麼不見了，這個是 ARM 常見的 console ，可能要花時間再查一下問題

Linux kernel 6.18 不支援 iptables

這個我比較崩潰，原來的 kernel 是 6.12 ，換到 6.18 之後感覺天都蹋了，想到我學 iptables 的時候那時候還是 ipchains 的尾聲，想想都快 20 年了，整天與 iptables 為伍，都知道要怎麼下指令和 debug ，但是現在僅僅只有 nft 了，如果你的 iptables 僅有 firewall ，還是可以用 iptables-nft 做轉換，但是如果是有一些特別的指令，像 TCPMSS 之類的，那還是建議改用 nft

還好這年代有 GPT ，直接餵 Gemini 和 DeepSeek ，DeepSeek 吐出來的比較簡潔，所以我就用 DeepSeek 產出的，因為我的網路稍複雜，產生出來的還是有些小問題，不過看了一下就知道怎麼改，問題不大

問題比較大的像是這行，這個 nft 有自己的指令不吃，而且也很難整合進原來的 nft table 內，我碰到的是不知道為什麼 iptables 都會重新產生一次，或是完全無法用，最後還是放棄回去用 nft

$IPTBIN -i $i -A FORWARD -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1400:1536 -j TCPMSS --clamp-mss-to-pmtu

還好第二次就比較順了，但是有個奇怪的點是，我的 sysctl 內的 ip forward 設定跑掉了，不知道誰偷偷改了

沒了 iptables 好失落啊，感覺什麼東西離我而去
想當年做 Hardware NAT 的時候天天跟 iptables / Linux kernel tcp stack 為伍，我是有感情的

其它像什麼 virbr0 消失就要吃自己的狗食了 [Gentoo] virt-manager/libvirt virbr0 doesn’t exist

The post [Gentoo] 2025 小升級流水帳 appeared first on richliu's blog.

用 AI 產出來的 configuration files ，DeepSeek 看起來比較正確，只是 Dashboard 跟 Grox 一樣都會錯，到是 Grox 在設定 Gafana 的 influxdB data source 的地方講對了，查詢種類要選 flux 才可以查到資料

試過了 Prometheus，結果發現可用的 Dashboard 更少，感覺沒過幾年世界就變了

設定檔我覺得可以丟給 AI ，畢竟我跑在 VM 內部又用 docker 跑，相對上還是麻煩了一點

至於 Gafana 的設定檔我就丟在這邊，堪用還有些小問題但是懶得修了，我只想看 input output 狀態而已
https://github.com/richliu/cp_apc_dash

The post CyberPower UPS + Apcupsd + Gafana + InfluxDB appeared first on richliu's blog.

What truly impresses me is the board’s specifications. Not only does it boast a compact Micro-ATX form factor, but it also packs a powerful punch with four PCIe 4.0 x16 lanes, easily handling even demanding homelab tasks. The dual M.2 slots and dual OCULink port can enable software RAID for your NVMe disks. As a server board, it offers a BMC for remote control and system management. While the x64 platform remains popular, this board is an exceptional choice for those who prefer the non-x64 architecture, considering the limited choices with high speed fan options in server platforms.

Information

Important Spec

AsRock ALTRAD8UD-1L2T Spec web site

Where To Buy

US/Newegg

ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G Without CPU,
Mother board only
USD$849.-

Asrock Rack Bundle ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Single Socket (LGA 4926) with Ampere Altra Q64-22 64 cores & 2U passive cooler, Dual 10G – Integrated by Asrock Rack with CPU and heatsink
Mother board + Altra Q64-22 + 2U passive cooler(heatsink)
USD$1,500.-

UK/Server Factory Ltd

ASRock ALTRAD8UD-1L2T: Board only, Inc VAT: £870.00.

News

ASRock Rack ALTRAD8UD-1L2T Review This is the Ampere Arm Motherboard You Want

Cooling solution

Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926 (Preview tower fan system)
Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System (my homemake Liquid/Water cooling system)

Other reference URL

ASRock Rack ALTRAD8UD Getting Started : A good note to start up to use the board.

Tricks for ALTRAD8UD

This chapter aims to guide users through using the ALTRAD8UD board, addressing potential information gaps left by the official ALTRAD8UD manual.

For memory compatibility reference, AsRock offers its own DRAM AVL (Approved Vendor List). Alternatively, users can access the Ampere Altra family device document, freely available without registration.

OpenBMC default username password

User: root
Pass: 0penBmc (first word is zero, not O).
both BMC console and GUI’s default username password are the same.

Serial Console over LAN Port

The board support Serial Port Header (9-pin COM1), it needs to use this serial port or VGA port to enter BIOS and get BMC IP.
After get the BMC IP, it can get more information from serial console over LAN port
Port 2201: ATF console
Port 2202: SCP console
Port 2203: Host console
Port 22: ssh console

Here is my script to run the sol and make it more easy to use

# It can put following two lines to bashrc
alias oit='ipmitool -U root -P 0penBmc -I lanplus -C 17'
alias ossh="sshpass -p 0penBmc ssh -o GlobalKnownHostsFile=/dev/null -o UserKnownHostsFile=/dev/null -oStrictHostKeyChecking=no"

# get host console 
ossh root@<bmc ip> -p 2203 
# get in bmc ssh console 
ossh root@<bmc ip> 

Serial Console Output To VGA

If you cannot see the serial console output to console port, probably it bacuase this option was enabled[ref], just turn it off and add console=tty0 or console=ttyAMA0 to grub.conf, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0"

Ipmitool Fan Control [ref]

Manual Full speed
ipmitool raw 0x3a 0x01 0x64 0x64 0x64 0x64 0x64 0x64 0x64 0x64

Manual Half speed
ipmitool raw 0x3a 0x01 0x32 0x32 0x32 0x32 0x32 0x32 0x32 0x32

Auto fan control
ipmitool raw 0x3a 0x01 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0

NC-SI Port

In the user manual, NC-SI port is typo, the NC-SI port location is 5, in the document which marked as IPMI LAN port. if use this port, both host and BMC can use the same network cable to connect to the same ethernet.

Known Issues

Intel X550 massive AER error messages[ref1][ref2]

Seems that Intel NICs have compatibility issues with non-Intel platform systems. This can cause some server safety systems, like Advanced Error Reporting (AER), to generate many alerts and experience interface problems. A temporary solution is to add aspi_aspm=off to the Linux kernel parameters, ex:

GRUB_CMDLINE_LINUX_DEFAULT="console=ttyAMA0 pcie_aspm=off"

It will fix the AER issue.

The post AsRock ALTRAD8UD-1L2T Ampere Altra/AltraMax Note appeared first on richliu's blog.

E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/c/curl/libcurl3-gnutls_7.81.0-1ubu
ntu1.15_arm64.deb 503 Connection closed, check DlMaxRetries [IP: 192.168.1.254 3142]

The reason is the system file open limit, it needs to modify it, it needs to modify file /etc/security/limits.conf and add those lines

apt-cacher-ng soft nofile 102400
apt-cacher-ng hard nofile 102400

If your system is heavy loading server, it can increase those numbers, and not existed the system cat /proc/sys/fs/nr_open setting.

Write the limit nofile to /etc/init.d/apt-cacher-ng doesn’t work for me.

ref.
Unable to download anything from apt-cacher-ng proxy
Increase max open files for Ubuntu/Upstart (initctl)

The post apt-cacher-ng E: Failed to fetch 503 Connection closed, check DlMaxRetries appeared first on richliu's blog.

Chinese version Article is here (Ampere Altra/AltraMax 加AsRock ALTRAD8UD-1L2T 水冷方案)
2024/01/31 Update: Ampere will have 2U passive cooler Making Arm Desktops Viable Ampere Altra Noctua NH-D9 AMP-4926 4U and NH-U14S AMP-4926

The Ampere Altra/AltraMax CPU features a maximum of 128 ARM64 cores, typically found in cloud server. However, cloud server Care not suitable for placement in offices or homes due to the high-speed fans used for heat dissipation. The noise generated by these fans is generally intolerable for regular users. Additionally, adapting server motherboards to standard consumer motherboards is inconvenient. If there is a desire to utilize such a CPU in a different setting, the ADLink Ampere Altra Dev Platform presents a viable solution, complete with the option for liquid cooling.

Another solution is the AsRock ALTRAD8UD-1L2T, currently available on Newegg. Paired with an Ampere Altra Q64-22 CPU (64 cores, 2.2GHz), it comes with a complimentary 2U passive heatsink, priced at US$1,500, which can be considered affordable. However, it’s worth noting that the included heatsink is passive. If you prefer liquid cooling over installing a fan on the passive heatsink, then this article is the right guide for you.

The author have adding a declaration at the beginning of articles that may pose potential risks:

• This article is not an official endorsement by Ampere, Newegg, or any other mentioned merchants or vendors. It is solely a personal record.
• The author has a business relationship related to the production of the mentioned motherboard.
• The author is a first-time to use liquid cooling computer system , and the content may not be entirely accurate. Please be aware of potential risks.
• This article is not an official document from Ampere or AsRock.
• The original article is in Chinese, and the English translation is done using machine translation. Please forgive any inaccuracies in wording.
• I am not responsible for any consequences resulting from this article; for example, losses incurred due to improper implementation of instructions provided in this article.

The Ampere system primarily focuses on Cloud-native CPUs, and as of now, there is no manufacturer producing general-purpose cooling systems for it. The included 2U passive heatsink in this system is designed for server use, utilizing high-speed server fans for heat dissipation. While the Q64-22 included by Newegg has lower power consumption, its only cooling method involves direct airflow from an electric fan. Alternatively, users may resort to attaching an 8-inch fan onto it. Such cooling solutions are deemed unacceptable for enthusiasts and professionals like myself.

If you intend to upgrade the cooling system, typical tower-type coolers require replacing the fixture on the back of the motherboard. Moreover, compatibility with the fastening mechanisms of this specific motherboard can be challenging to find. Considering that only front-facing screws can be used, the only viable cooling solution is a liquid cooling system. Therefore, I have been experimenting with a liquid cooling solution that works for this scenario. I’m sharing this information for those adventurous individuals who might encounter similar challenges, and perhaps this approach can inspire some creative solutions.

In the upcoming sections of this article, I will guide you on how to set up a liquid cooling system specifically tailored for the Ampere Altra on the ALTRAD8UD-1L2T motherboard. In theory, since the screw hole positions are the same, this method should be applicable to other Ampere motherboards as well. However, the focus of this content will primarily be on the ALTRAD8UD-1L2T. The article will also cover any pitfalls encountered during assembly and the items needed for the setup(Chinese website).

Preparation

ALTRAD8UD-1L2T

You can find the specifications for this motherboard online, and I won’t repeat them here. However, what might confuse some people is the term “Deep mATX.” This is a specification unique to AsRock, where they increase the depth dimension of the mATX form factor while maintaining the standard width. The purpose is to provide more space to accommodate additional components. This particular specification is commonly seen in AsRock’s server-grade motherboards.

Chassis

Generally, most computer cases should work. However, I made a mistake on my end by choosing a case that was too small. The case I purchased is the Antec P110 Silent. While it has liquid cooling mounting points, it is not suitable for adding a liquid cooling system. The internal space is too limited, lacking a designated area for securing the water pump. Additionally, the Antec P110 liquid cooling radiator cannot be placed at the top as it interferes with the motherboard. Consequently, the radiator had to be positioned at the front of the case.

When making a purchase, it’s crucial to choose a case that is suitable for water cooling. In more extreme cases, considering a caseless system might be an option (hopefully, this can be updated in the next release).

Cases designed for liquid cooling systems typically have more significant internal space to accommodate a water pump, and there is ample room for fans without obstruction. Some all-in-one water cooling designs may be more compact and might not encounter this issue.

Liquid Cooling System

Liquid cooling systems have some considerations to take into account:

• Tubing type: Choose between soft or hard tubing. Hard tubing is suitable for experienced users, offering a longer lifespan and a more aesthetic appearance. Soft tubing is easier for assembly, making it suitable for beginners.
• All-in-one (AIO) or custom loop: Decide whether to go for a pre-assembled AIO liquid cooling system or build a custom loop system.

Since the initial focus was on researching liquid cooling, the combination chosen was a custom loop with soft tubing. Building a custom loop may encounter some issues, with one of the main challenges being the uncertainty about specifications.

The first crucial aspect is determining the size of the liquid cooling tubing, as additional components will depend on this size, limiting the options for procurement. Currently, the mainstream sizes are two-eighths (2/8’), three-eighths (3/8’), and four-eighths (4/8’) liquid cooling tubing.

Another parameter is the thickness of the tubing, thick tubing and thin tubing.
* thick tubing refer to 10/16mm or 3/8”-5/8” tubing.
* thin thbing refer to 10/13mm or 3/8”-1/2″ tubing.

Most common the inner diameter specification is G 1/4, when find some liquid colling componens like pressure relief valve needs it.

Once you’ve confirmed your inner and outer diameter, you can proceed to purchase the necessary liquid cooling components.

At a minimum, you’ll need:

• CPU water block *1
• Water Pump *1
• 240mm radiator *1
• 12cm fans *2
• Liquid coolant 500cc ~ 1000cc
• Tubing 2M
• Thermal paste
• Pressure relief valve
• Fittings: Quick connectors *6, 90-degree rotary fittings *(1~2)
• Tools for filling and draining: Filling tool, draining tool (watering can, large syringe, drainage hose)

Optional components:

• Flow meter, temperature gauge (note: buying these requires purchasing two additional quick connectors).

In practical testing, if focusing solely on CPU cooling, it appears that a 240mm radiator is sufficient. The thin three-eighths (3/8”) tubing provides a large enough water flow diameter. Choosing a smaller tube diameter may not be as suitable for this purpose.

CPU water block

This turned out to be quite troublesome. My original plan was to purchase a 50mmx50mm heatsink and connect it to the heatsink on the right. Then, I intended to secure it to the CPU socket using the fixtures shown in the image. However, as expected, unexpected issues arose – the world isn’t always that kind.

1. There was no water block with a 50mmx50mm connector available. So, I ended up buying a 40mmx40mm connector, one size smaller. This resulted in poorer heat dissipation performance. Fortunately, I never had the chance to test the inferior heat dissipation, so this step failed unnoticed.
2. Here’s where I made a mistake. Custom water block connectors, like the one shown, are typically 2/8″ fittings. However, the main liquid cooling system I bought has 3/8″ fittings, making them completely incompatible. The options were either to change the fittings or change the entire system. Changing the fittings was the more cost-effective choice, so I opted for that.

After connecting , it looked like the image on the right – and just looking at it made me worry that water would leak out.

So, I bought a two-eighths(2/8″) tubing to connect to this connector. However, while it could connect, it couldn’t link up with my three-eighths(3/8″) tubing in the water cooling system. I had to reluctantly abandon the custom water block connector.

The final solution had nothing to do with this connector.

Communical CPU water block

During the installation, I discovered that the Bykski CPU-XPH-T8 CPU water block, along with the provided adapter bracket, could be securely attached to the Ampere Altra’s socket. This was fantastic news and saved me a great deal of effort.

Although it’s designed for Intel LGA1700/LGA2066 sockets, the flexibility in its hole configuration allowed for adjustments, providing me with the necessary adaptability. As for whether other CPU brackets might be compatible, it’s likely that some could work. However, I haven’t had the chance to try others yet. If you have attempted this and found success, please let me know.

The base of this water block is quite substantial, and my intuition tells me that the cooling performance should be excellent. Therefore, I didn’t explore other compliant CPU water blocks.

However, another issue arises. The water block is 56mmx56mm, whereas the width of the Ampere Altra CPU socket is 55mm. The water block does not fit. At this point, there are two possible solutions:

Upon observing the CPU’s structure, the first solution seemed unfeasible. Cutting down the thickness of this water block would compromise its structural integrity, not to mention finding the tools or a professional service for the task.

Thus, the only viable option is the second solution. The simplest and most intuitive way to implement this is by using a thermal pad made of silicone to transfer heat upwards from the CPU.

Silicone Thermal Pad

But things couldn’t be as simple as I, a fool, initially thought. In conclusion, regular silicone thermal pads are not suitable for power monsters like the Ampere Altra. This type of thermal pad may be effective for low-power devices such as routers, switches, NVMe disks, etc., but it’s not useful for high-power CPUs.

In the image, there are 2mm thick silicone thermal pads, and you need two layers because the height is just that much. Essentially, this provides only a slight improvement over having no heat dissipation at all, so it’s not worth considering.

As for the high-efficiency silicone thermal pads available in the market, I don’t know if they would work. If you have them on hand, you can give them a try. However, I believe the chances of success are quite low.

Purple Copper Thermal Pad

Nowadays, a common method for cooling is using purple copper heat pipes. This type of heat dissipation is quite effective, and the 2U passive heatsink included with Newegg utilizes these heat pipes. Therefore, if we can customize the heatsink, wouldn’t that solve the problem? Luckily, the country right next to my country is the world’s largest manufacturing factory, capable of customizing just about anything. This made the solution very affordable and instantly resolved my problem.

The dimensions of the purple copper thermal pad are: Length x Width x Height = 64x54x4mm, with a rounded R5 edge.

Alternatively, you can measure the dimensions of the piece under the 2U passive heatsink to get the specifications. Placing it on the original heatsink, I can say the dimensions match perfectly. The craftsmanship is quite impressive, and there are no issues when it’s actually placed.

Assembly

First, place the purple copper thermal pad, and remember to apply thermal paste before doing so.

Apply thermal paste before placing CPU water cooling block.

Assemble the components.

Actually, I assembled the tubing on the left side incorrectly. I’ll need to purchase materials and quick connectors to replace it when I have the time. But for now, let’s leave it as is and make do.

Overall view.

This way, it’s evident that the case isn’t ideal. It’s too cramped, affecting airflow, and it cannot accommodate the components properly.

Shopping

If you are interested in replicating my setup, all the components were purchased from Taobao. I spent extra for testing and experimenting, but in reality, you should be able to set up the entire cooling system for around US$100 or NTD$3,000. This is comparable to the cost of an efficient dual-fan tower cooler.

Since this is a Chinese website, I’ll keep the text in Chinese and provide English translations afterward.

Liquid Cooling System

watertiger定制台式机电脑分体式CPU显卡水冷软管套装冷散热器 (“WaterTiger Custom Desktop Computer Split-Type CPU GPU Liquid Cooling Soft Tube Kit Radiator”)
Enter the URL choice “單 CPU 水冷 240 冷排” (“Single CPU Liquid Cooling 240 Radiator”)

At first, I chose this brand because the photos looked appealing, but it turned out to be a fortunate coincidence that saved me a lot of exploration time. This brand integrates various components into a set for sale, and the components they use are of decent quality. For example, they use the Enermax NEOChanger as the water pump, which is also a good pump. If you are familiar enough, you might not need these kits.

This kit does not include a pressure relief valve. You’ll need to purchase a pressure relief valve separately, and depending on your needs, you might want to buy additional quick connectors

Clamping Fittings

All the components can be purchased from the same place, saving the hassle of searching everywhere. It’s a good idea to buy a few extra parts to avoid trouble later on. The time cost is much higher than the cost of the components.

电脑水冷头扣具卡具CPU显卡南北桥固定笔记本散热器压条五金夹具 (“Computer Water Cooling Head Fastening Fixture, CPU GPU North-South Bridge Fixing, Laptop Heatsink Clamp, Hardware Clamp”)

Choice 台式机CPU冷头扣具(“Desktop CPU Cooling Head Fastening Fixture”) *2

M2 M3 M4不锈钢平头盘头螺丝钉水冷头水冷排机箱风扇安装长螺丝 (“M2 M3 M4 Stainless Steel Flat Head Countersunk Screws for Water Cooling Head, Radiator, Case Fan Installation – Long Screws”)

Fastening fixtures for securing CPU coolers and water blocks. The components can be increased based on requirements. The minimum required quantity includes
M3手拧螺母 (M3 thumb nuts)*4，M3*30 (M3 screws) *4 ，M3垫片(M3 washers) *4 ，M3六角螺母(M3 hex nuts) *4

电脑水冷水流计 液冷散热器流量器 流体温度观测器透明水流指示计 (“Computer Liquid Cooling Flow Meter, Liquid Cooler Flow Indicator, Fluid Temperature Monitor Transparent Flow Meter”) (option)

Choice: 3孔水流观测器 + 温度计 (“3-Port Flow Meter + Thermometer for Water Flow Observation”)

Nice to have items, but it can make me feel better.

Test Result

This system feels relatively quiet and not noisy.

The following tests are only for verifying the liquid cooling setup and are not formal performance tests. The results are provided for liquid cooling reference and do not include testing power consumption, etc.

Room temperature: 25 degrees Celsius
CPU : Ampere AltraMax 128-30
Run command : stress-ng –cpu 128 –iomix 128 –vm 128

Gentoo Linux kernel 6.6 ，CPU temperature is approximately，Power consumption CPU 150W/ IO 19W
Ubuntu 22.04 kernel 6.2(?)，CPU temperature is approximately 72 degrees，Power consumption CPU 178W/ IO 20W

In conclusion, I hope this article has been helpful for those eager to try it out.

Afterword

Newegg has also listed another model, which is only the motherboard version, only US$849.-
ASRock Rack ALTRAD8UD-1L2T Deep Micro-ATX Server Motherboard Singel Socket (LGA 4926) Ampere Altra Max/Ampere Altra processors Dual 10G & 1G

The post Ampere Altra/AltraMax AsRock ALTRAD8UD-1L2T Liquid Cooling System appeared first on richliu's blog.

Just disable it in your BIOS and add “console=tty0” to /etc/default/grub, then update grub.cfg. it will work.

In AMI BIOS, it should be in the Advanced->Serial Port Console Redirection->Console Redirection

Set it to [Disabled]

Some Linux distrubtions might not use GRUB_CMDLINE_LINUX in the file /etc/default/grub with grub paramater GRUB_ENABLE_BLSCFG=true, if you this parameter in th e grub.cfg file, please disable it.

ref.
[RFC PATCH v2 0/3] Prefer working VT console over SPCR and device-tree chosen stdout-path

The post ARM64 cannot output Linux kernel message to VGA appeared first on richliu's blog.

• * The Ubuntu/Fedora rootfs on tftp/nfs server:
  It can refer to the my article MIPS/ARM64 Debian/Ubuntu debootstrap for building the Ubuntu rootfs.
  If want to build the Fedora rootfs, it can install a new system on qemu and copy it from the qcow2 image.

• * PXE Server:
  You can refer my article ARM64 Centos8/Ubuntu PXE Boot Install on Ubuntu Server to build a PXE server.

• NFS Server:
  Assume the rootfs for Ubuntu is on /nfs/ubuntu2204 and Fedora 36 is on /nfs/fedora36.

Ubuntu PXE Boot

Copy vmlinux and initramfs to tftp server. Copy your current system’s vmlinux to your tftpboot. For example, create a directory for Ubuntu 22.04 in /tftpboot/nfs/ubuntu2204, and copy the vmlinuz to the folder using the following command:

$ cp /boot/vmlinuz-$(uname -r) /tftpboot/nfs/ubuntu2204/vmlinuz

Copy the initrd.img to the folder using the following command:

$ cp /boot/initrd.img-$(uname -r) /tftpboot/nfs/ubuntu2204/initrd.img

Setup your PXE server’s grub.cfg and add a record. Use the following example for the record:

menuentry 'Ubuntu22.04 nfsboot' { 
    linux nfs/ubuntu2204/vmlinuz ip=dhcp root=/dev/nfs nfsroot=<NFS server IP>:/nfs/ubuntu2204,nolock rw netboot=nfs 
    initrd nfs/ubuntu2204/initrd.img 
}

Fedora PXE Boot

The procedure is similar to Ubuntu, but the default Fedora initramfs doesn’t support nfs boot.
1. Rebuild the initramfs with dracut. You need to add nfs and network support to your Fedora initramfs using the following command:

$ dracut --add nfs --add network --force

It will generate the same name vmlinuz and initramfs. Copy it to your directory just like Ubuntu.
2. Disable selinux.
Default Fedora selinux is enabled, but it cannot enable on nfs rootfs (or it can but needs to set up something). You can disable it by adding selinux=0 on boot kernel parameter. Use the following example for the record:

linux nfs/fedora36/vmlinuz boot=nfs root=/dev/nfs nfsroot=<NFS server IP>:/nfs/fedora36,nolock rw netboot=nfs ip=dhcp selitarnux=0

The post Ubuntu/Fedora PXE Boot appeared first on richliu's blog.

Install Necessary Packages

On the document, it missed some packages if you want to build CentOS kernel.

# follow original guide to install necessary packages
yum groupinstall "Development Tools"
yum install ncurses-devel
yum install hmaccalc zlib-devel binutils-devel elfutils-libelf-devel
# Install EPEL packages
sudo yum install epel-release
# Enable powertools packages
sudo dnf config-manager --set-enabled powertools

Fetch CentOS Linux Kernel Source

This steps will fetch current version CentOS kernel source code.
It will download current kernel version of kernel.src.rpm.

cd ~
mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
echo '%_topdir %(echo $HOME)/rpmbuild' > ~/.rpmmacros

KVER=`uname -r | sed 's/\.aarch64//'`
rpm -i https://vault.centos.org/8-stream/BaseOS/Source/SPackages/kernel-$KVER.src.rpm

Install Kernel Necessary Package

After get the kernel source code, it still needs some extra software packages to build the CentOS Linux kernel. If it doesn’t enable powertool on previous step, this step will fail.

cd ~/rpmbuild/SPECS

for pkg in `rpmbuild -bp --target=$(uname -m) kernel.spec 2>&1 | grep 'is needed by' |awk '{print $1}' | tr '\n' ' '`;do sudo yum install $pkg -y;done

Build CentOS Linux kernel

# Executes  the  "%prep" stage from the spec file, will not compile it
rpmbuild -bp --target=$(uname -m) kernel.spec
# compile kernel source code
rpmbuild -bb --target=`uname -m` kernel.spec 2> build-err.log | tee build-out.log

if everything is ok, the file will output on ~/rpmbuild/RPMS/aarch64/

Install rpm command

# Install necessary kernel files (examples)
yum localinstall  kernel-4.18.0-408.el8.aarch64.rpm kernel-core-4.18.0-408.el8.aarch64.rpm kernel-headers-4.18.0-408.el8.aarch64.rpm kernel-modules-4.18.0-408.el8.aarch64.rpm kernel-modules-extra-4.18.0-408.el8.aarch64.rpm
# Install all relative rpms 
rpm -ivh kernel-*.rpm 
yum localinstall kernel-*.rpm
# second time install all relative rpms 
rpm -ivh --force kernel-*.rpm

Apply patch for CentOS kernel

Another important but less instruction talk about how to apply it.

1. Edit the kernel.spec, add new line above linux-kernel-test.patch

## Patches needed for building this package

Patch000001: new_patch_4.18.patch

# empty final patch to facilitate testing of kernel patches
Patch999999: linux-kernel-test.patch

2. find “ApplyOptionalPatch linux-kernel-test.patch” and add new one before it.

ApplyOptionalPatch new_patch_4.18.patch
ApplyOptionalPatch linux-kernel-test.patch

3. Put file “new_patch_4.18.patch” to ~/rpmbuild/SOURCES/

4. Rebuild kernel

ref.
CentOS wiki : I Need to Build a Custom Kernel
CentOS wiki : I Need the Kernel Source
How to compile kernel module for CentOS8

The post Build CentOS 8 Kernel from Source on ARM64 Platform appeared first on richliu's blog.

猜測原理應該藉由中華電信內部設定會讓小封包先通行的設定，達到穩定 ping 值的目地

Windows 下就是用系統管理員執行 CMD 之後，下 command

netsh interface ipv4 show subinterfaces
# 取得目前上網的 interface
netsh interface ipv4 set subinterface "需修改的連線名" mtu=576 store=persistent

上面那個方法會讓所有的網站都使用同一組 MTU ，實務上可能連到某些網站就會有問題，例如我在測試中間就無法從 github 下載檔案，猜測可能是對方的防火牆偵測到惡意行為
不過嘛，我們用的是 Linux ，而且自已架 Router，所以可以更精準的修改 MTU

每個 DC server 的網段不一樣，我是在日區的 DC Elemental 伺服器，歐洲，北美和其他伺服器可能位址不一樣，這一段就是教大家如何抓出 ff14 DC 所使用的網段
首先在 Windows 下執行 resmon (資源監視器)，先選網路，再執行 FF14 ，然後選中 ffxiv_dx11.exe
此時在下面就會出現遠端位置和延遲時間，中華電信大多數時間都是在 60ms 以內，但是在某些狀況最高我看過 700ms 的延遲，然而 packet lost 是 0 ，遠端位址是 124.150.157.26

然後用這個查 AS number ，得到　124.150.152.0/21 整段都是 SquireEnix 的位置，此時再將這一段插入 iptables 的 rules 就好

iptables rules example

iptables  -i br0 -I FORWARD -p tcp -m tcp -d 124.150.152.0/21 --tcp-flags SYN,RST SYN -m tcpmss --mss 1400:1536 -j TCPMSS --set-mss 576

改完後試打一場開花神無延遲，會再觀察個幾天看看是不是都是無延遲，本來都要考慮買 VPN 了，如果可以用就不用買了

世界計畫日本服 Linux Command

台灣 hinet 連世界計畫日本服似乎也有類似的狀況，iPad 似乎也有支援修改 MTU 的功能，用這個值也可以用

至於 Linux 下，查對方的 IP 似乎有點麻煩，要將 connection table dump 出來一個一個改，因為世界計劃會連到很多 server ，有 google 甚至還有 tencent
最後找到可能是某組 Amazon-02 的 IP 段在 13.35.24.0/23，將該段的 MTU 改小一點，似乎就變順暢了
這樣似乎可以賣 HINET 遊戲加速器了？

# 清除某個 IP 的 conntrack table 
conntrack -D conntrack --orig-src 192.168.1.63
# Dump 該 IP 的 conntrack table 
conntrack -L |grep 192.168.1.63 | grep -v 208.67| less
# 找到之後再加入 iptables 
iptables -i br0 -I FORWARD -d 13.35.24.0/23 -p tcp -m tcp --tcp-flags SYN,RST SYN -m tcpmss --mss 1400:1536 -j TCPMSS --set-mss 576

The post FF14 中華電信 LAG 問題 appeared first on richliu's blog.

root cause is this package needs static lib support, but regular CentOS doesn’t have it.

# make                                                                             cc -g -O3 -static -pthread  multichase.o permutation.o arena.o util.o  -lrt -lm -o multichase
/usr/bin/ld: cannot find -lrt                                                                                 /usr/bin/ld: cannot find -lm                                                                                  /usr/bin/ld: cannot find -lpthread
/usr/bin/ld: cannot find -lc                                                                                  collect2: error: ld returned 1 exit status                                                                    make: *** [<builtin>: multichase] Error 1

Install necessary package and download the source code

# Install necessary package
yum install git
yum groupinstall "Development Tools"

# Download source code
git clone https://github.com/google/multichase

# Install epel
dnf config-manager --set-enabled powertools
dnf install epel-release epel-next-release

# Install static library 
yum install glibc-static

ref.
Extra Packages for Enterprise Linux (EPEL)

The post multichase on Centos 8 Compile Issue appeared first on richliu's blog.

使用 ipmitool 取得 watchdog information

# 未啟動的機器
$ sudo ipmitool mc watchdog get
Watchdog Timer Use:     Reserved (0x00)
Watchdog Timer Is:      Stopped
Watchdog Timer Actions: Hard Reset (0x01)
Pre-timeout interval:   0 seconds
Timer Expiration Flags: 0x00
Initial Countdown:      30 sec
Present Countdown:      30 sec

# 已啟動的機器
$ sudo ipmitool mc watchdog get
Watchdog Timer Use:     SMS/OS (0x44)
Watchdog Timer Is:      Started/Running
Watchdog Timer Actions: Hard Reset (0x01)
Pre-timeout interval:   0 seconds
Timer Expiration Flags: 0x00
Initial Countdown:      900 sec
Present Countdown:      850 sec

Install bmc-watchdog

Debian 和 Fedora 都是叫 freeipmi-bmc-watchdog，其它 distrubtion 應該也相去不遠

# Debian 
$ sudo apt install freeipmi-bmc-watchdog

# Fedora 
$ sudo yum install freeipmi-bmc-watchdog

bmc-watchdog 基本參數

bmc-watchdog 啟動參數

#
$ sudo bmc-watchdog -d -u 4 -p 0 -a 1 -i 900

# Start bmc-watchdog 
$ sudo bmc-watchdog -t 

Debian 的 bmc-watchdog 設定放在 /etc/default/bmc-watchdog，不確定是不是 default configure ，但是還是放在這邊

OPTIONS="-d -u 4 -p 0 -a 1 -F -P -L -S -O -i 900 -e 60"

設成預設啟動

$ sudo systemctl enable bmc-watchdog
$ sudo systemctl start bmc-watchdog

最後別忘了 enable watchdog in bmc
OS Watchdog Timer 那一項，如果沒事就不要開了，會一直重開

The post BMC Watchdog appeared first on richliu's blog.

[root@localhost linux-4.18.0-80.11.2.el8_0]# make install                                            [55/2101]
/bin/sh ./arch/arm64/boot/install.sh 4.18.0 \
arch/arm64/boot/Image System.map "/boot"
dkms: running auto installation service for kernel 4.18.0
Deprecated feature: REMAKE_INITRD
Deprecated feature: REMAKE_INITRD
Deprecated feature: REMAKE_INITRD
Sign command: /lib/modules/4.18.0/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub                                                               Deprecated feature: REMAKE_INITRD                                                                             
Building module:
Cleaning build area...(bad exit status: 2)
make -j160 KERNELRELEASE=4.18.0 -C /lib/modules/4.18.0/build M=/var/lib/dkms/bnxt_en/1.10.2.221.0.114.0/build
KVER=4.18.0...(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.18.0 (aarch64)
Consult /var/lib/dkms/bnxt_en/1.10.2.221.0.114.0/build/make.log for more information.
Sign command: /lib/modules/4.18.0/build/scripts/sign-file
Signing key: /var/lib/dkms/mok.key
Public certificate (MOK): /var/lib/dkms/mok.pub
Deprecated feature: REMAKE_INITRD

Building module:
Cleaning build area...(bad exit status: 2)
make -j160 KERNELRELEASE=4.18.0 -C /lib/modules/4.18.0/build M=/var/lib/dkms/bnxt_re/221.0.114.0/build KVER=4$
18.0 BNXT_EN_INC=/var/lib/dkms/bnxt_re/221.0.114.0/source BNXT_QPLIB_INC=/var/lib/dkms/bnxt_re/221.0.114.0/so$
rce...(bad exit status: 2)
Error! Bad return status for module build on kernel: 4.18.0 (aarch64)
Consult /var/lib/dkms/bnxt_re/221.0.114.0/build/make.log for more information.
 Done.

And check the log file “/var/lib/dkms/bnxt_re/221.0.114.0/build/make.log”

It shows

DKMS make.log for bnxt_en-1.10.2.221.0.114.0 for kernel 4.18.0 (aarch64)
Tue Sep 27 15:41:04 CST 2022
Makefile:603: arch/aarch64/Makefile: No such file or directory
make[2]: *** No rule to make target 'arch/aarch64/Makefile'.  Stop.

Seems cannot find the Makefile, that because the drvier cannot locate right Makefile, because the include architecture is arm64, not aarch64 in Linux kernel .

By the way, basicly arm64 and aarch64 are the same thing, I don’t know why it shows aarch64 on kernel architecture and use arm64 as fold name. probably some traditional history.

solution is easy, go to the include folder and link arm64 to aarch, DONE! ex:

cd /lib/modules/4.18.0/build/arch
ln -sf arm64 aarch64

The post Linux Kernel Install Fail Because Broadcom NetXtreme 100G DKMS Driver Build Failed. appeared first on richliu's blog.

Get NIC Device UUID

nmcli connection show eth0 |grep connection.uuid

ref.
[SOLVED] Finding UUID of a network card

Update CentOS 8.x to stream

CentOS 如果不升級的 stream 有很多套件會抓不到，像是 Ansible

升級方式 Ref. How to migrate from CentOS 8 to CentOS stream conversion

# Step 1 
# check the CentOS version 
more /etc/centos-release
# Step 2 
# Upgrade to CentOS Stream
sudo dnf in centos-release-stream
# Step 3 
# Migrate to CentOS Stream 
sudo dnf swap centos-linux-repos centos-stream-repos
# Step 4
# reboot
sudo reboot

Install bpftrace

Ref.
How To Install bpftrace on CentOS 8

sudo yum makecache --refresh
sudo yum -y install bpftrace

The post CentOS 其他 appeared first on richliu's blog.

Before Build Source Code

This procedure runs on ARM64 platform, it should be also run on x86 platform, but I have not try it.

Build environment : Ampere eMAG CPU (32 cores) with 256G DRAM and uses 64G/128G RAM to build code, sometime if it cannot finsih compile tasks, probably your DRAM is not sufficient to build the code, I tried 16G and it doesn;t work.

Because host is running Ubuntu 22.04, and OpenBMC and EDKII only can be built on Ubuntu 18.04, so, it needs to use either virtual machine or contianer technologies to have Ubuntu 18.04 environment.

This note uses qemu to prepare build environment, tried docker but it doesn’t work, probably I will re-try it someday.

If use NFS or qemu overlayers filesystem to bridge folders ourside into VM, it might cause build problem, so, please increase disk size for you virtual machine virtual disk size, use nfs or share folder for copy images or source code only.

If want to build a workable EDKII code and run it on Ampere Mt. Jade platform, it needs to have Ampere fimware ATF and SCP binary, it might register Ampere customer connect to download those image files, without it, it only can build OpenBMC binary.

Prepare Build Environment

Get the Ubuntu 18.04 cloud image

$ wget https://cloud-images.ubuntu.com/releases/bionic/release/ubuntu-18.04-server-cloudimg-arm64.img
$ sudo apt install qemu-efi

Prepare hostname and userdata for VM

it needs to prepare two files, meta-file and user-data, following is example for meta-file

instance-id: Image-openbmc
local-hostname: openbmc

user-data file, this example username is ubuntu, password is ubuntu

#cloud-config
password: ubuntu
chpasswd: { expire: False }
ssh_pwauth: True
ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+Y/6m4XFeEGKqR2H7wbsUQUVWvB0gC/+CeJqfyvOX++SETKapFtI5CiidkyW/B1GQM4XUlmE9zPlCAcolPfMpXB0CNQQ5l0F5Z+FW6b4RVjFV/KEcQ7R3XkKRa0XhC6hKkNc5P95fNhXr8Do1bSEmd9mPIiOrFaUEZ9EReO+ZTsGsytFIuQV8rOZfQjClktXrqk3F6jJ+TV3THcVNpKvnLfps56Hc/j9jdiadLv91yuifz4lCMbv/D/0hMtf4KhN5tV//QRgUxEUeC7aPW6sKD2GMYMxc9UsSyRRQ9IFWjJMkgzKyuxgNKe7u4WjERFoxEUrmFK01UsrX+lRw9e9n user@hostname

Than, it can create a configure file for QEMU

$ cloud-localds -v cloudinit.img user-data meta-file

Increase Ubuntu cloud image disk size, bigger is better, suggest more than 200G, minimal should be around 100G.

$ qemu-img resize ubuntu-18.04-server-cloudimg-arm64.img +200G

Prepare UEFI BIOS and storage for booting

dd if=/dev/zero of=flash0.img bs=1M count=64
dd if=/usr/share/qemu-efi/QEMU_EFI.fd of=flash0.img conv=notrunc
dd if=/dev/zero of=flash1.img bs=1M count=64

Running the VM script, it will also create a QEMU virtfs for file sharing, but do not use that folder to build code. How to access share folder, it can reference this article QEMU Share Directory with Host Without Networking

IMAGE1=ubuntu-18.04-server-cloudimg-arm64.img
sudo qemu-system-aarch64 -name vm1 \
         -machine virt,gic_version=3,accel=kvm,usb=off \
         -cpu host -m 128G \
         -smp 32,sockets=1,cores=32,threads=1 \
         -nographic -nodefaults \
         -pflash flash0.img -pflash flash1.img \
         -drive file=$IMAGE1,if=none,id=disk1 \
         -device virtio-blk-device,drive=disk1 \
         -drive file=cloudinit.img,if=virtio,format=raw \
         -virtfs local,path=/nvme/obmc,mount_tag=host0,security_model=passthrough,id=host0 \
         -netdev tap,id=net0,ifname=tap0 \
         -device virtio-net-device,netdev=net0,mac=52:54:00:11:11:12 \
         -serial telnet::9001,server,nowait > guest_log-1.txt 2>> guest_log-1.txt

Build Mt. Jade EDKII

Prepare VM environment

After login into VM, it needs to install software to compile EDKII.
Command to install basic packages.

sudo apt update 
sudo apt install -y git vim tmux lftp net-tools ethtool bc
sudo apt install -y \
  build-essential uuid-dev acpica-tools nasm \
  python3 python3-distutils python3-pip \
  gawk
sudo apt install -y libssl-dev gnu-efi
sudo apt install bison flex
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 1

Next step is follow this document to download Ampere EDKII source code and setup enviroment.
https://github.com/AmpereComputing/edk2.git

First, prepare build environment parameters, if VM reboot, it needs to re-run it.

export WORKSPACE=/nvme/tianocore
mkdir -p $WORKSPACE
cd $WORKSPACE
export PACKAGES_PATH=$PWD/edk2:$PWD/edk2-platforms
# PACKAGES_PATH is default include path, sometimes miss package can include more directories here. 
# if still cannot find some files, try to run submodules command on next step again. 
export PACKAGES_PATH=$PWD/edk2:$PWD/edk2-platforms:$PWD/edk2-non-osi/:$PWD/edk2-platforms/Features/

Chang into that folder, clone Ampere Computiner edk2 source code.

cd $WORKSPACE
git clone https://github.com/AmpereComputing/edk2.git
cd edk2
git submodule init && git submodule update
cd ..
git clone https://github.com/AmpereComputing/edk2-platforms.git

Get build environment

This step is to build the basic tools.

cd $WORKSPACE
. edk2/edksetup.sh

# Build BaseTool, only need to run once 
make -C edk2/BaseTools 

After compiled the EDK2 basic tool, next step is to prepare edk2-platforms and edk2-ampere-tools

edk2-ampere-tools has a very good build script,
it can help to build the EDK2 and also package ATF into together.
we will introduct it on following steps.

cd edk2-platforms
git checkout -b test remotes/origin/ampere

# make a link to library to fix some problem
ln -sf /nvme/tianocore/edk2-platforms/Features/Intel/OutOfBandManagement .
cd ..
git clone https://github.com/AmpereComputing/edk2-ampere-tools

Build and Sign the image

Because current tree Atftool migrate to OpenSSL 3.0, and Ubuntu 18.04 still use OpenSSL 1.0x,
I choice to roll back to old version to support OpenSSL 1.0x, support it can build Atftool on Ubuntu 22.04 or compile OpenSSL 3.0 manually, I might try it later.

cd /nvme/tianocore 
mkdir -p AtfTools
cd AtfTools
git init
git remote add origin -f https://github.com/ARM-software/arm-trusted-firmware.git
git config core.sparseCheckout true

echo -ne "include/tools_share\nmake_helpers\ntools/cert_create\ntools/fiptool" > .git/info/sparse-checkout

git -C . checkout --track origin/master
# Due to current AtfTool only support OpenSSL 3.0 , switch to previous version 
git checkout -b test b94bf967e62b23a376a5026de69d06a0e8d6bf78
make -C tools/cert_create CRTTOOL=cert_create
$ make -C tools/fiptool FIPTOOL=fiptool

configure PATH to fiptool and cert_create

cd /nvme/tianocore
export PATH=$PATH:$PWD/AtfTools/tools/cert_create:$PWD/AtfTools/tools/fiptool

This step is parepare Mt. Jade ATF and SCP binary for compile, it needs to get both binary file from Ampere customer connect. and also suggest to use the same version as OpenBMC/EDKII

cd $WORKSPACE
mv <path to>/altra_atf_signed_2.06.20220426.slim .
mv <path to>/altra_scp_signed_2.06.20220426.slim .
cd edk2-ampere-tools
./edk2-build.sh -b RELEASE Jade --atf-image $WORKSPACE/altra_atf_signed_2.06.20220426.slim --scp-image $WORKSPACE/altra_scp_signed_2.06.20220426.slim

If everything is perfect, the binary code would be looks like below.

Results: /nvme/tianocore/edk2-ampere-tools/BUILDS/jade_tianocore_atf_2.06.100
total 45888
-rw------- 1 root   root       3343 Jun 15 12:40 dbukey.auth
-rw------- 1 root   root       2002 Jun 15 12:40 del_dbukey.auth
-rw-r--r-- 1 root   root       2008 Jun 15 12:38 jade_board_setting.bin
-rw-rw-r-- 1 ubuntu ubuntu    14532 Jun 15 04:08 jade_board_setting.txt
-rw-r--r-- 1 root   root     670845 Jun 15 12:40 jade_scp_2.06.100.cap
-rw-r--r-- 1 root   root     262144 Jun 15 12:40 jade_scp_2.06.100.dbu.sig.img
-rw-r--r-- 1 root   root    8126464 Jun 15 12:38 jade_tianocore_2.06.100.fd
-rw-r--r-- 1 root   root   14040229 Jun 15 12:40 jade_tianocore_atf_2.06.100.cap
-rw-r--r-- 1 root   root   13631488 Jun 15 12:40 jade_tianocore_atf_2.06.100.dbu.sig.img
-rw-r--r-- 1 root   root   10225352 Jun 15 12:38 jade_tianocore_atf_2.06.100.img
------------------------------------------------------------
                          Ampere Jade (AARCH64) RELEASE pass
------------------------------------------------------------
pass    1
fail    0

It can use the jade_tianocore_atf_2.06.100.img for firmware update.

Note: When latest time I compile the edkII binary, latest version will fail to build, it need to roll back to previous version.

Manual Build EDKII

If still want to build EDKII manually, here is some command can help you.

cd edk2-platform
build -a AARCH64 -t GCC5 -b RELEASE -D SECURE_BOOT_ENABLE -p Platform/Ampere/JadePkg/Jade.dsc
or to use all CPU to compile the EDKII.
build -n `nproc` -a AARCH64 -t GCC5 -b RELEASE -D SECURE_BOOT_ENABLE -p Platform/Ampere/JadePkg/Jade.dsc

the generated image on $WORKSPACE/Build/Jade/RELEASE_GCC5/FV/BL33_JADE_UEFI.fd,
for this case, on the /nvme/tianocore/Build/Jade/RELEASE_GCC5/FV/BL33_JADE_UEFI.fd

then, it needs to use ampere special tool to generate EDKII image with Ampere ATF image.

Build Mt. Jade OpenBMC

Prepare build environment

Before build OpenBMC, it needs to install necessary packages,
it can build both OpenBMC and EDKII on the same VM, or it can build it on different VM.

Following is necessary packages for OpenBMC.

$ sudo apt-get install -yy \
     build-essential \
     chrpath \
     cpio \
     debianutils \
     diffstat \
     gawk \
     git \
     iputils-ping \
     libdata-dumper-simple-perl \
     liblz4-tool \
     libsdl1.2-dev \
     libthread-queue-any-perl \
     locales \
     python \
     python3 \
     socat \
     subversion \
     texinfo \
     wget \
     zstd

Download Open BMC source code and OpenBMC build script.

$ cd /nvme
$ git clone https://github.com/ampere-openbmc/openbmc.git
$ git clone https://github.com/openbmc/openbmc-build-scripts.git

modify openbmc-build-scripts/build-setup.sh around line #140,
replace “Clone in openbmc master to ${obmc_dir}” to https://github.com/ampere-openbmc/openbmc.git

it can download right source code.

git clone ${branch} https://github.com/ampere-openbmc/openbmc.git

If you use more than 32 cores to build the source code, it might cause compile fail because some compile tool might have problem on build code on ARM64 massive code. A easy way to check that is run dmesg to check whether Linux kernel has OOM killer on some package compile fail message like nodejs.

Limited CPU number to 32 , modify openbmc-build-scripts/build-setup.sh, on line 84.

 84 # num_cpu=${num_cpu:-$(nproc)}
 85 num_cpu=32

Build OpenBMC

This step is to prepare test build environment

$ cd /nvme
$ target=mtjade WORKSPACE=/nvme/openbmc branch="--branch ampere --single-branch" ./openbmc-build-scripts/build-setup.sh

Setup openbmc machine type to mt. Jade and compile OpenBMC.

$ export TEMPLATECONF=meta-ampere/meta-jade
$ export TEMPLATECONF=meta-ampere/meta-jade/conf/templates/default/
$ export TMPDIR=/nvme/temp
$ export LC_ALL=en_US.UTF-8

$ cd /nvme/openbmc
$ . openbmc-env
$ cd ..
$ source ./setup mtjade
$ bitbake obmc-phosphor-image

if system it mt.mitchell

$ export TEMPLATECONF=meta-ampere/meta-mitchell/conf/templates/default/
$ export TMPDIR=/nvme/temp
$ export LC_ALL=en_US.UTF-8

$ cd /nvme/openbmc
$ . openbmc-env
$ cd ..
$ source ./setup mtmitchell-dcscm
$ bitbake obmc-phosphor-image

When compile, there is an openocd compile error, it cannot access to original website probably caused by the website was down. Temperory solutoin is to remove the package.
modify file meta-ampere/meta-common/recipes-ac01/packagegroups/packagegroup-ampere-apps.bb, find openocd and delete it, down.

When finish compile, the binary will locate on tmp/deploy/images/mtjade/, absoult path is /nvme/openbmc/build/mtjade/tmp/deploy/images/mtjade/

ex: (only list partical files)

obmc-phosphor-image-mtjade-20220616040212.rootfs.manifest
obmc-phosphor-image-mtjade-20220616040212.rootfs.squashfs-xz
obmc-phosphor-image-mtjade-20220616040212.static.mtd
obmc-phosphor-image-mtjade-20220616040212.static.mtd.all.tar
obmc-phosphor-image-mtjade-20220616040212.static.mtd.tar
obmc-phosphor-image-mtjade-20220616040212.testdata.json
obmc-phosphor-image-mtjade.jffs2
obmc-phosphor-initramfs-mtjade-20220616004506.rootfs.cpio.xz
obmc-phosphor-initramfs-mtjade-20220616004506.rootfs.manifest
obmc-phosphor-initramfs-mtjade-20220616004506.testdata.json

obmc-phosphor-image-mtjade-20220616040212.static.mtd.tar is firmware file, it contain both obmc-phosphor-image-mtjade-20220616040212.static.mtd and obmc-phosphor-image-mtjade-20220616040212.rootfs.manifest files, and can be used for next step : firmware update.

Upgrade Firmware

*Notice: Update flash will take your own risk, especially physical damage your hardware..

If Mt. Jade is using AMI bios, suggest to use flash burner to update the OpenBMC firmware.
What I am using is GZU EZP XPro , it’s very cheap, only needs around US$30.- (please also purchase the SPI flash socket).

This step need to un-plug BMC flash from Mt. Jade motherboard, please remeber the flash pin 1 location, it needs to follow the original position and put it back to socket. If you miss you will broken it.

Update OpenBMC firmware

ref. OpenBMC Code Update

Update via URL

Suppose that it already has OpenBMC firmware, it can use software to upgrade firmware ,
first step is to get web token, then use this token on next step to upgrade firmware.

Following is a simple script to upgrade flash, it can be integrated into your script.

export BMC_IP=192.168.1.190
export FWFILE=obmc-phosphor-image-mtjade-20220616040212.static.mtd.tar

export token=`curl -k -H "Content-Type: application/json" -X POST https://${BMC_IP}/login -d '{"username" :  "root", "password" :  "0penBmc"}' | grep token | awk '{print $2;}' | tr -d '"'`

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    84  100    37  100    47    116    147 --:--:-- --:--:-- --:--:--   264


curl -k -H "X-Auth-Token: $token" \
       -H "Content-Type: application/octet-stream" \
       -X POST -T ${FWFILE} \
        https://${BMC_IP}/redfish/v1/UpdateService

{
  "@odata.id": "/redfish/v1/TaskService/Tasks/1",
  "@odata.type": "#Task.v1_4_3.Task",
  "Id": "1",
  "TaskState": "Running",
  "TaskStatus": "OK"
}

If conosle doesn’t not update and reboot, please use console login the OpenBMC and run “reboot” manually.
It will show message like below on OpenBMC console.

Updating kernel...
Erasing block: 69/69 (100%)
Writing kb: 4397/4397 (100%)
Verifying kb: 4397/4397 (100%)
Updating rofs...
Erasing block: 366/366 (100%)
Writing kb: 11580/23376 (49%)

Update via SSH

Just update firmware (ex: obmc-phosphor-image-mtjade-20220616040212.static.mtd.tar) to BMC’s directory /tmp/images , it will shows a hash number like /tmp/images/d6718c8f, then run the following command

busctl set-property xyz.openbmc_project.Software.BMC.Updater \
  /xyz/openbmc_project/software/d6718c8f \
  xyz.openbmc_project.Software.Activation RequestedActivation s \
  xyz.openbmc_project.Software.Activation.RequestedActivations.Active

Update UEFI firmware

Upgrade UEFI firmware
it needs to pack the UEFI image first , and it need to create MANIFEST file, looks like this

VER_purpose=xyz.openbmc_project.Software.Version.VersionPurpose.Host
version=jade_tianocore_atf_2.06.100
KeyType=OpenBMC
HashType=RSA-SHA256
MachineName=mtjade

make image file and MINIFEST file into the same file

$ tar -cvf jade_tianocore_atf_2.06.100.tar jade_tianocore_atf_2.06.100.img MANIFEST 

Run command below to upgrade UEFI firmware

export BMC_IP=192.168.1.190

export FWFILE=jade_tianocore_atf_2.06.100.tar

export token=`curl -k -H "Content-Type: application/json" -X POST https://${BMC_IP}/login -d '{"username" :  "root", "password" :  "0penBmc"}' | grep token | awk '{print $2;}' | tr -d '"'`

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    84  100    37  100    47    116    147 --:--:-- --:--:-- --:--:--   264


curl -k -H "X-Auth-Token: $token" \
       -H "Content-Type: application/octet-stream" \
       -X POST -T ${FWFILE} \
        https://${BMC_IP}/redfish/v1/UpdateService

Like OpenBMC, if it doesn’t update, just reboot to update it.
if stiall not work, try to upgrade it manually, yes, when I wrote this article, update via redfish doesn’t work, so, just copy it from github.

# on HOST
$ scp jade_tianocore_atf_2.06.100.tar root@192.168.1.190:/tmp

# on Target 
$ cd /tmp/images
$ mkdir fw
$ cd fw
$ tar xvf /tmp/jade_tianocore_atf_2.06.100.tar
$ firmware_update.sh /tmp/images/fw
--- Current Chassis State: On
--- Turning the Chassis off
--- Switch the host SPI bus to BMC.
Run update Primary Host SPI-NOR
--- Bind the ASpeed SMC driver
--- Locking power control
--- Flashing firmware to @/dev/mtd13
Erasing block: 157/157 (100%)
Writing kb: 9985/9985 (100%)
Verifying kb: 9985/9985 (100%)
--- Unlocking power control
--- Switch back to the primary Host SPI-NOR.
Turn on the Host

Update SCP firmware

It needs to use sftp to transfer SCP firmware to target.
Following is example to upgrade SCP firmware.

On host
$ scp altra_scp_signed_2.06.20220426.slim root@192.168.1.190:/tmp

On OpenBMC side
$ ampere_firmware_upgrade.sh smpmpro /tmp/altra_scp_signed_2.06.20220426.slim

After upgrade BIOS and SCP, if got any problem, suggest to reset bmc

$ ipmitool -U root -P 0penBmc -I lanplus -C 17 -H 192.168.2.18 mc reset cold

Serial Console

it can use command to get system CPU console.

# CPU console 
$ ssh root@192.168.190 -p 2200 

Reference

Ampere OpenBMC github
EDK2 Ampere Tool
Ampere EDK2
Ampere Computing Github

Error Message

Error: TEMPLATECONF value (which is /nvme/openbmc/meta-ampere/meta-mitchell) must point to meta-some-layer/conf/templates/template-name

Error message

Initializing OE build env

Error: TEMPLATECONF value (which is /nvme/openbmc/meta-ampere/meta-mitchell) must point to meta-some-layer/conf/templates/template-name

This is because followed the old configuration, it needs to point to right folder like

export TEMPLATECONF=meta-ampere/meta-jade/conf/templates/default/
. openbmc-env

.

The post Ampere Altra Mt. Jade OpenBMC and EDKII build guide on ARM64 platform appeared first on richliu's blog.

Install Necessary Packages

Install packages, it should have to install other packages like openssl-dev.

get package ampere-8.3.0-20191025-dynamic-nosysroot-nativetools.tar.xz from Ampere customer connect or ampere development source web.

yum groupinstall "Development Tools"

yum -y install ncurses-devel
yum -y install hmaccalc zlib-devel binutils-devel elfutils-devel libelf-devel newt-devel python-devel
yum -y install audit-libs-devel numactl-devel pciutils-devel openssl-devel
yum -y install bc bison perl-ExtUtils-Embed.noarch xmlto asciidoc
rpm –ivh kernel-4.18.0-80.11.2.el8.<version>.src.rpm


mkdir -p /opt/amp
tar -xf ampere-8.3.0-20191025-dynamic-nosysroot-nativetools.tar.xz -C /opt/amp
export PATH=/opt/amp/ampere-8.3.0-20191025-dynamic-nosysroot/bin:$PATH;
which gcc
/opt/amp/ampere-8.3.0-20191025/bin/gcc

if gcc is not in /opt/amp, please check your directory in /opt/amy

Build Kernel

command to build kernel

cd /root/rpmbuild/SOURCES/
tar xvf linux-4.18.0-80.11.2.el8_0.tar.xz
cd linux-4.18.0-80.11.2.el8_0
cp ../kernel-aarch64-emag.config .config
make olddefconfig

Try build once and make sure everything is ok

make -j `nproc`

If buiild OK, generate RPM files.

make -j `nproc` rpm-pkg

if everything is ok, the RPM files will be on /root/rpmbuild/RPMS/aarch64/

Install RPM packages

/root/rpmbuild/RPMS/aarch64
yum localinstall kernel-4.18.0-1.aarch64.rpm kernel-headers-4.18.0-1.aarch64.rpm

The post Build Ampere CentOS Kernel appeared first on richliu's blog.

Install CentOS 7 on Ampere Altra CPU

Earlier CentOS 7 default OS is too old to support ARM64 CPU and SSIF driver,
if you have problem to install on Ampre Altra CPU, it needs to block SSIF and change PCI parameter during booting. Add parameters below on Linux kernel booting parameter when install CentOS7.

modprobe.blacklist=ipmi_si pci=nomsi

Prepare Build Environment

Before download and build Linux kernel 5.x, it needs to run those command to prepare build environment

yum install -y centos-release-scl-rh
yum install -y devtoolset-9*
yum groupinstall -y "Development Tools"
yum install -y ncurses-devel
yum install -y hmaccalc zlib-devel binutils-devel elfutils-libelf-devel
yum install -y python3

Download Linux kernel

Ampere LTS kernel can be download from github, here is the command to checkout Ampere LTS kernel and switch version to 5.10

cd /root
git clone https://github.com/AmpereComputing/ampere-lts-kernel.git
cd /ampere-lte-kernel
git checkout -b origin/linux-5.10.y 5.10.y

On later Linux kernel, it needs to use git switch to switch to 5.10.y

git switch 5.10.y

Build Linux kernel

It’s simple to build Linux kernel

scl enable devtoolset-9 bash
cd /root/ampere-lte-kernel
cp arch/arm64/configs/altra_5.10_defconfig .config
make olddefconfig
make -j `nproc` && make -j `nproc` modules  && make modules_install && make install

GRUB Setting

Suggest to change default GRUB timeout to choice right kernel to boot.

sed -i 's/GRUB_TIMEOUT=5/GRUB_TIMEOUT=30/g' /etc/default/grub
grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg

OK, right now it can reboot your system to have try.

Ref:
Good mirror site for CentOS7 package.
https://mirrors.bfsu.edu.cn/centos-altarch/7.9.2009/os/aarch64/

The post Build Linux kernel 5.x on CentOS 7 appeared first on richliu's blog.

NVME 4K Write Performance

Situation:
Run fio 4K write on 24 NVME drivers, but some drivers might not get higher IOPS performance

Solution:
It should be caused by enable SMMU, so, if doesn’t use VM, just disable iommu to get better performance
Add iommu.passthrough=1

General IO performance Tuning

For high bandwidth card, most time it needs to disable irq_balancing
$ systemctl stop irqbalance.service

Some Performance Tuning Parameters

ulimit -s unlimited 
ulimit -l 2097152 
echo 8 | sudo tee /proc/sys/vm/dirty_ratio 
echo 1 | sudo tee /proc/sys/vm/swappiness 
echo 1 | sudo tee /proc/sys/vm/zone_reclaim_mode 
echo 3 | sudo tee /proc/sys/vm/drop_caches 
echo 0 | sudo tee /proc/sys/kernel/randomize_va_space 
echo 0 | sudo tee /proc/sys/kernel/numa_balancing 
echo never > /sys/kernel/mm/transparent_hugepage/enabled 
cpupower frequency-set -g performance 
export MALLOC_CONF=thp:always
 

The post ARM64 Linux Performance Tuning appeared first on richliu's blog.

htoprc , put it on /etc/htoprc or ~/.config/htop/htoprc

# Beware! This file is rewritten by htop when settings are changed in the interface.
# The parser is also very primitive, and not human-friendly.
fields=0 48 17 18 38 39 40 2 46 47 49 1
sort_key=1
sort_direction=1
hide_threads=0
hide_kernel_threads=1
hide_userland_threads=0
shadow_other_users=0
show_thread_names=0
show_program_path=1
highlight_base_name=0
highlight_megabytes=1
highlight_threads=1
tree_view=0
header_margin=1
detailed_cpu_time=0
cpu_count_from_zero=0
show_cpu_usage=1
show_cpu_frequency=0
update_process_names=0
account_guest_in_cpu_meter=0
color_scheme=0
enable_mouse=1
delay=15
left_meters=LeftCPUs4 Memory Swap
left_meter_modes=1 1 1
right_meters=RightCPUs4 Tasks LoadAverage Uptime
right_meter_modes=1 2 2 2

The post htop more column for mass CPUs appeared first on richliu's blog.

[ 775.510780] igb: Intel(R) Gigabit Ethernet Network Driver – version 5.4.0-k
[ 775.510782] igb: Copyright (c) 2007-2014 Intel Corporation.
[ 775.526676] igb 0000:01:00.0: The NVM Checksum Is Not Valid
[ 775.602407] igb: probe of 0000:01:00.0 failed with error -5
[ 775.617632] igb 0000:01:00.1: The NVM Checksum Is Not Valid
[ 775.693405] igb: probe of 0000:01:00.1 failed with error -5
[ 775.708632] igb 0000:01:00.2: The NVM Checksum Is Not Valid
[ 775.784404] igb: probe of 0000:01:00.2 failed with error -5
[ 775.799632] igb 0000:01:00.3: The NVM Checksum Is Not Valid
[ 775.875410] igb: probe of 0000:01:00.3 failed with error -5

解決方法也很簡單，下載 Intel® Ethernet連接啟動公用程式、預啟動映射和 EFI 驅動程式 在 Linux 下解開之後，

$ cd APPS/BootUtil/Linux_x64
$ chmod +x bootutil64e
$ sudo ./bootutil64e
Connection to QV driver failed – please reinstall it!
Intel(R) Ethernet Flash Firmware Utility
BootUtil version 1.7.26.3
Copyright (C) 2003-2020 Intel Corporation

ERROR: The adapter (location 0:25.0) cannot be initialized due to inaccessible device memory.
Update the device driver and reboot the system before running this utility again.
Consult the utility documentation for more information.
Type BootUtil -? for help
Port Network Address Location Series WOL Flash Firmware Version
==== =============== ======== ======= === ============================= =======
2 A0369FA1FB70 1:00.0 Gigabit NO FLASH Unknown
3 A0369FA1FB71 1:00.1 Gigabit N/A FLASH Disabled
4 A0369FA1FB72 1:00.2 Gigabit N/A FLASH Disabled
5 A0369FA1FB73 1:00.3 Gigabit N/A FLASH Disabled

2,3,4,5 就是我們要處理的網路卡，只要下以下指令之後再重開機就會恢復了

$ sudo ./bootutil64e -NIC 2 -defcfg
$ sudo ./bootutil64e -NIC 3 -defcfg
$ sudo ./bootutil64e -NIC 4 -defcfg
$ sudo ./bootutil64e -NIC 5 -defcfg

ref. How to repair the checksum of the non-volatile memory (NVM) of Intel Ethernet Controller I219-V of an ASUS laptop?

The post The NVM Checksum Is Not Valid appeared first on richliu's blog.