配置 SSH 免密：

```shell
ssh-copy-id -i ~/.ssh/id_ed25519.pub <user>@<host>
```

在目标端配置 sudo 免密：

https://serverfault.com/questions/160581/how-to-setup-passwordless-sudo-on-linux

```
%wheel ALL=(ALL:ALL) NOPASSWD: ALL
```

The **key** is to add it after the last line which says

```
#includedir /etc/sudoers.d
```

创建快照：

```shell
# sudo btrfs subvolume snapshot -r <fs-path> <snapshot-path>
sudo btrfs subvolume snapshot -r / /sysfs_ro
```

:::note{title="NOTE"}
这里创建的只读快照，是只有单个 subvolume 的。如果需要迁移多个 subvolume，需要执行多次快照和发送。
:::



发送：

```shell
sudo pacman -S pv
```

```shell
# sudo btrfs send <snapshot-path> | pv | ssh <target-machine> "sudo btrfs receive <target-fs-path>"
sudo btrfs send /sysfs_ro | pv | ssh vm-arch "sudo btrfs receive /mnt/arc"

# 发送完成后，路径是 /mnt/arc/sysfs_ro
```

通过 SSH 迁移 btrfs volumes

需要安装好：

- pv
- zstd

把命令中的 `ssh <user>@<host>` 替换成目标端：

```shell
docker images --format "{{.Repository}}:{{.Tag}}" | grep -v 'none' \
| xargs -I{} bash -c \
"docker save {} | zstd -T0 --fast | pv | ssh <user>@<host> 'zstd -d | docker load'"
```

通过 SSH 迁移 docker images

ArchLinux （2025-09-06）启动 GPU 容器失败：

```bash
-> % docker run --rm --gpus=all nvidia/cuda:12.1.1-base-ubuntu22.04 nvidia-smi
Unable to find image 'nvidia/cuda:12.1.1-base-ubuntu22.04' locally
12.1.1-base-ubuntu22.04: Pulling from nvidia/cuda
aece8493d397: Already exists
dd4939a04761: Pull complete
b0d7cc89b769: Pull complete
1532d9024b9c: Pull complete
04fc8a31fa53: Pull complete
Digest: sha256:457a4076c56025f51217bff647ca631c7880ad3dbf546b03728ba98297ebbc22
Status: Downloaded newer image for nvidia/cuda:12.1.1-base-ubuntu22.04
docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: ldcache error: process /sbin/ldconfig terminated with signal 9
```



解决办法：

[System Update in Arch Linux, broked NVIDIA container. Open Source drivers](https://github.com/NVIDIA/nvidia-container-toolkit/issues/1246#issuecomment-3194219487)

```bash
sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
sudo nvidia-ctk config --in-place --set nvidia-container-runtime.mode=cdi
systemctl restart docker
```

```bash
# /etd/docker/daemon.json
  "default-runtime": "nvidia",
  "runtimes": {
    "nvidia": {
      "path": "/usr/bin/nvidia-container-runtime",
      "runtimeArgs": []
    }
  }
```

ArchLinux NVIDIA GPU 容器启动失败

[Intel® Memory Latency Checker v3.11b](https://www.intel.com/content/www/us/en/developer/articles/tool/intelr-memory-latency-checker.html)



## AMD Ryzen 7 9700X

```bash
-> % sudo lshw -c memory -short
H/W path                  Device          Class          Description
====================================================================
/0/0                                      memory         64KiB BIOS
/0/d                                      memory         640KiB L1 cache
/0/e                                      memory         8MiB L2 cache
/0/f                                      memory         32MiB L3 cache
/0/13                                     memory         48GiB System Memory
/0/13/0                                   memory         [empty]
/0/13/1                                   memory         24GiB DIMM Synchronous Unbuffered (Unregistered) 5600 MHz (0.2 ns)
/0/13/2                                   memory         [empty]
/0/13/3                                   memory         24GiB DIMM Synchronous Unbuffered (Unregistered) 5600 MHz (0.2 ns)
```

```bash
-> % sudo ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0          80.3

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      58929.8
3:1 Reads-Writes :      66580.4
2:1 Reads-Writes :      66096.6
1:1 Reads-Writes :      55213.5
Stream-triad like:      67975.4

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        58812.5

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  178.12    58921.6
 00002  175.72    59148.9
 00008  173.88    58971.0
 00015  172.73    58965.9
 00050  185.50    59092.4
 00100  193.09    58822.9
 00200  185.97    58934.0
 00300  130.40    47925.9
 00400  111.70    37966.8
 00500  105.12    31714.0
 00700  102.12    23952.2
 01000  100.80    17469.9
 01300   99.64    13890.6
 01700   96.24    10925.5
 02500   94.55     7768.6
 03500   93.30     5820.1
 05000   93.43     4298.0
 09000   93.51     2705.6
 20000   92.08     1611.9

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        15.7
Local Socket L2->L2 HITM latency        15.8
```

## AMD AI MAX 395+

```bash
-> % sudo lshw -c memory -short
H/W path          Device          Class          Description
============================================================
/0/0                              memory         64KiB BIOS
/0/b                              memory         1280KiB L1 cache
/0/c                              memory         16MiB L2 cache
/0/d                              memory         64MiB L3 cache
/0/11                             memory         128GiB System Memory
/0/11/0                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/1                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/2                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/3                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/4                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/5                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/6                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
/0/11/7                           memory         16GiB Synchronous Unbuffered (Unregistered) 8532 MHz (0.1 ns)
```

```bash
-> % sudo ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         154.4

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      117081.5
3:1 Reads-Writes :      151323.4
2:1 Reads-Writes :      160109.6
1:1 Reads-Writes :      179796.4
Stream-triad like:      134033.9

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        117020.3

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  206.43   116980.0
 00002  205.88   116874.9
 00008  203.66   115473.1
 00015  204.44   115052.1
 00050  207.31   118860.5
 00100  203.07   115144.3
 00200  217.25    95711.7
 00300  196.43    89988.4
 00400  180.71    71684.1
 00500  173.23    59571.4
 00700  169.98    44459.9
 01000  163.40    32452.3
 01300  161.46    25534.4
 01700  160.37    19887.2
 02500  159.66    13889.4
 03500  159.96    10142.6
 05000  158.44     7281.4
 09000  160.18     4242.9
 20000  159.33     2144.0

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        17.5
Local Socket L2->L2 HITM latency        17.5
```

## Intel Ultra 9 285H

```bash
-> % sudo lshw -c memory -short
H/W path          Device          Class          Description
============================================================
/0/0                              memory         64KiB BIOS
/0/12                             memory         64GiB System Memory
/0/12/0                           memory         32GiB SODIMM Synchronous 5600 MHz (0.2 ns)
/0/12/1                           memory         32GiB SODIMM Synchronous 5600 MHz (0.2 ns)
/0/1d                             memory         288KiB L1 cache
/0/1e                             memory         384KiB L1 cache
/0/1f                             memory         18MiB L2 cache
/0/20                             memory         24MiB L3 cache
/0/21                             memory         320KiB L1 cache
/0/22                             memory         640KiB L1 cache
/0/23                             memory         10MiB L2 cache
/0/24                             memory         24MiB L3 cache
/0/100/14.2                       memory         RAM memory
```

```bash
-> % sudo ./Linux/mlc
[sudo] password for fanmi:
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for sequential access (in ns)...
                Numa node
Numa node            0
       0         270.1

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      78440.3
3:1 Reads-Writes :      76891.5
2:1 Reads-Writes :      77205.5
1:1 Reads-Writes :      77033.2
Stream-triad like:      77061.9

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        78578.3

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  251.56    76195.1
 00002  249.19    75876.3
 00008  288.68    80638.9
 00015  275.13    79408.2
 00050  246.77    75827.7
 00100  209.39    61399.6
 00200  193.59    46888.4
 00300  193.61    34877.7
 00400  186.20    27353.6
 00500  183.14    22485.5
 00700  181.26    17103.6
 01000  179.27    12473.8
 01300  178.00     9898.2
 01700  177.66     7782.8
 02500  177.11     5550.5
 03500  177.62     4089.2
 05000  177.50     3024.4
 09000  178.82     1829.0
 20000  180.87     1019.9

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        91.0
Local Socket L2->L2 HITM latency        92.1
```

## AMD EPYC 7K62

```bash
-> % sudo lshw -c memory -short
[sudo] password for fanmi:
H/W path              Device           Class          Description
=================================================================
/0/0                                   memory         64KiB BIOS
/0/21                                  memory         224GiB System Memory
/0/21/0                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/1                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/2                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/3                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/4                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/5                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/6                                memory         32GiB DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz (0.3 ns)
/0/21/7                                memory         [empty]
/0/24                                  memory         3MiB L1 cache
/0/25                                  memory         24MiB L2 cache
/0/26                                  memory         192MiB L3 cache
```

```bash
-> % ./mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         110.7

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      17503.5
3:1 Reads-Writes :      16796.5
2:1 Reads-Writes :      16591.0
1:1 Reads-Writes :      16305.2
Stream-triad like:      16432.0

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        17127.6

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  501.66    17519.5
 00002  504.00    17527.5
 00008  532.26    17561.6
 00015  528.77    17552.4
 00050  538.20    17536.7
 00100  531.86    17565.1
 00200  532.29    17566.0
 00300  532.23    17560.9
 00400  532.54    17570.5
 00500  533.94    17542.5
 00700  526.42    17573.1
 01000  519.98    17563.5
 01300  498.84    17526.6
 01700  386.58    17527.4
 02500  174.14    15569.8
 03500  145.14    11328.5
 05000  135.52     8108.2
 09000  124.54     4806.6
 20000  119.60     2478.4

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        22.4
Local Socket L2->L2 HITM latency        22.7
```

```
[Sat Aug 30 11:21:54 2025] [Hardware Error]: event severity: fatal
[Sat Aug 30 11:21:54 2025] [Hardware Error]:  Error 0, type: fatal
[Sat Aug 30 11:21:54 2025] [Hardware Error]:  fru_text: DIMM Locate:  P0H0
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   section_type: memory error
[Sat Aug 30 11:21:54 2025] [Hardware Error]:    error_status: Storage error in DRAM memory (0x0000000000040400)
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   node:0 card:4 module:0
[Sat Aug 30 11:21:54 2025] [Hardware Error]:   error_type: 15, physical memory map-out event
```

真奇怪，又 g 了，明明修好了才是

重启了一下：

```
[Sat Aug 30 14:17:19 2025] BERT: Error records from previous boot:
[Sat Aug 30 14:17:19 2025] [Hardware Error]: It has been corrected by h/w and requires no further action
[Sat Aug 30 14:17:19 2025] [Hardware Error]: event severity: corrected
[Sat Aug 30 14:17:19 2025] [Hardware Error]:  Error 0, type: corrected
[Sat Aug 30 14:17:19 2025] [Hardware Error]:  fru_text: DIMM Locate:  P0H0
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   section_type: memory error
[Sat Aug 30 14:17:19 2025] [Hardware Error]:    error_status: Storage error in DRAM memory (0x0000000000040400)
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   physical_address: 0x0000003f56549180
[Sat Aug 30 14:17:19 2025] [Hardware Error]:   node:0 card:4 module:0 rank:0 bank:6 row:63538 column:528
[Sat Aug 30 14:17:19 2025] BERT: Total records found: 1
```

```
-> % free -mh
               total        used        free      shared  buff/cache   available
Mem:           251Gi       4.2Gi       248Gi       5.0Mi       645Mi       247Gi
Swap:          4.0Gi          0B       4.0Gi
```

居然自己好了。

```
-> % sudo mlc
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0         119.2

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      136823.2
3:1 Reads-Writes :      134545.9
2:1 Reads-Writes :      136703.3
1:1 Reads-Writes :      135919.7
Stream-triad like:      136468.7

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        136846.2

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  680.01   136402.3
 00002  680.09   136451.1
 00008  603.64   130970.7
 00015  380.77   105365.8
 00050  383.47   105430.4
 00100  390.13   105452.3
 00200  400.41   105823.8
 00300  216.26   107479.2
 00400  148.23    88973.3
 00500  139.61    73057.0
 00700  133.20    53527.2
 01000  124.36    38585.8
 01300  117.81    29940.8
 01700  116.19    23132.7
 02500  114.84    15983.0
 03500  114.62    11608.9
 05000  114.74     8304.9
 09000  115.69     4871.7
 20000  117.22     2491.8

Measuring cache-to-cache transfer latency (in ns)...
Local Socket L2->L2 HIT  latency        22.4
Local Socket L2->L2 HITM latency        22.7
```

![image.png](/static/img/8b333d5c47b3da1d7aeb36ae6eeee1ef.image.png)

## AMD Ryzen 7 9800X3D

16GB * 2, DDR5 6000MT/s

```bash
(0) > .\Windows\mlc.exe
Intel(R) Memory Latency Checker - v3.11b
Measuring idle latencies for random access (in ns)...
                Numa node
Numa node            0
       0          78.6

Measuring Peak Injection Memory Bandwidths for the system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using traffic with the following read-write ratios
ALL Reads        :      56582.8
3:1 Reads-Writes :      62128.6
2:1 Reads-Writes :      61323.4
1:1 Reads-Writes :      52797.4
Stream-triad like:      64569.7

Measuring Memory Bandwidths between nodes within system
Bandwidths are in MB/sec (1 MB/sec = 1,000,000 Bytes/sec)
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
                Numa node
Numa node            0
       0        55632.8

Measuring Loaded Latencies for the system
Using all the threads from each core if Hyper-threading is enabled
Using Read-only traffic type
Inject  Latency Bandwidth
Delay   (ns)    MB/sec
==========================
 00000  220.10    56594.2
 00002  248.74    56311.8
 00008  337.96    56143.3
 00015  286.18    55749.9
 00050  246.07    56395.8
 00100  219.09    56282.8
 00200  177.87    53891.6
 00300  113.36    41023.7
 00400  123.68    31897.4
 00500  117.87    25665.6
 00700  117.57    18838.9
 01000  110.93    13795.8
 01300  114.69    10782.7
 01700  106.25     8418.5
 02500  108.61     5935.5
 03500  134.22     4178.1
 05000  120.32     3221.0
 09000   99.62     2164.5
 20000  101.26     1314.9

Measuring cache-to-cache transfer latency (in ns)...
Unable to enable large page allocation
Using small pages for allocating buffers
Local Socket L2->L2 HIT  latency        19.1
Local Socket L2->L2 HITM latency        19.5
```

内存性能大比较！

机箱上有一个不可拆卸的螺丝柱顶到了内存插槽的第八槽，导致硬件错误



![image.png](/static/img/692ee5e5c1be1632c60283d5bdeac209.image.webp)

## Patch

用透明胶粘上了

![d8b90fddaba6719057055cdee3286f7f.jpg](/static/img/84845656c00b826e514bb800b6a46ba3.d8b90fddaba6719057055cdee3286f7f.webp)