PVE 6.8内核 17.0版本Nvidia虚拟化VGPU--基于P106-100
从闲鱼下单了一张99元包邮的矿卡P106-100,目前该卡应该是最有性价比了吧? (垃圾佬狂喜)6G显存还有着1060的性能还要啥自行车。网上已经有很多大神评测了该款显卡以及通过核显调动这张P106-100。本期我打算用它来做虚拟化,该卡支持VGPU虚拟化,可惜不支持视频解码被阉割掉了,用来打打游戏还是不错的。
关于拆箱
1)我买到的是七彩虹牌子的P106-100,整体外观看上去还是挺新的,背面贴了一堆标签,到我这已经不知道是第几手了。好在风扇正常没有噪音。
2)拆开内部看到还是有很多灰尘的,这显存都给干黑了,不知道服役了多少年
3)好在驱动、烤鸡这类测试都为正常,鲁大师得分13.5万
分,3DMARK压力测试通过率99.1%
不敢相信
进入正题
目前我的pve是8.22版本 6.8的内核。打算用N卡VGPU的17.0版本给P106-100做虚拟化
硬件配置
配置 | 型号 |
---|---|
系统 | PVE 8.2.2 |
内核 | Linux 6.8.4-3-pve |
CPU | Intel Core i7-8700 @ 3.2GHz 6核12线程 |
显卡 | 七彩虹 P106-100 6144MB |
VGPU驱动 | 550.54.14 |
点我展开
消费级显卡支持10系列20系列,但不支持30系列40系列
设备ID | 显卡型号 | VGPU模拟型号 |
---|---|---|
21c4 | TU116 GeForce GTX 1660 SUPER | Quadro RTX 6000 |
21d1 | TU116BM GeForce GTX 1660 Ti Mobile | Quadro RTX 6000 |
21c2 | TU116 | Quadro RTX 6000 |
2182 | TU116 GeForce GTX 1660 Ti | Quadro RTX 6000 |
2183 | TU116 | Quadro RTX 6000 |
2184 | TU116 GeForce GTX 1660 | Quadro RTX 6000 |
2187 | TU116 GeForce GTX 1650 SUPER | Quadro RTX 6000 |
2188 | TU116 GeForce GTX 1650 | Quadro RTX 6000 |
2191 | TU116M GeForce GTX 1660 Ti Mobile | Quadro RTX 6000 |
2192 | TU116M GeForce GTX 1650 Ti Mobile | Quadro RTX 6000 |
21ae | TU116GL | Quadro RTX 6000 |
21bf | TU116GL | Quadro RTX 6000 |
2189 | TU116 CMP 30HX | Quadro RTX 6000 |
1fbf | TU117GL | Quadro RTX 6000 |
1fbb | TU117GLM Quadro T500 Mobile | Quadro RTX 6000 |
1fd9 | TU117BM GeForce GTX 1650 Mobile Refresh | Quadro RTX 6000 |
1ff9 | TU117GLM Quadro T1000 Mobile | Quadro RTX 6000 |
1fdd | TU117BM GeForce GTX 1650 Mobile Refresh | Quadro RTX 6000 |
1f96 | TU117M GeForce GTX 1650 Mobile / Max-Q | Quadro RTX 6000 |
1f99 | TU117M | Quadro RTX 6000 |
1fae | TU117GL | Quadro RTX 6000 |
1fb8 | TU117GLM Quadro T2000 Mobile / Max-Q | Quadro RTX 6000 |
1fb9 | TU117GLM Quadro T1000 Mobile | Quadro RTX 6000 |
1f97 | TU117M GeForce MX450 | Quadro RTX 6000 |
1f98 | TU117M GeForce MX450 | Quadro RTX 6000 |
1f9c | TU117M GeForce MX450 | Quadro RTX 6000 |
1f9d | TU117M GeForce GTX 1650 Mobile / Max-Q | Quadro RTX 6000 |
1fb0 | TU117GLM Quadro T1000 Mobile | Quadro RTX 6000 |
1fb1 | TU117GL T600 | Quadro RTX 6000 |
1fb2 | TU117GLM Quadro T400 Mobile | Quadro RTX 6000 |
1fba | TU117GLM T600 Mobile | Quadro RTX 6000 |
1f42 | TU106 GeForce RTX 2060 SUPER | Quadro RTX 6000 |
1f47 | TU106 GeForce RTX 2060 SUPER | Quadro RTX 6000 |
1f50 | TU106BM GeForce RTX 2070 Mobile / Max-Q | Quadro RTX 6000 |
1f51 | TU106BM GeForce RTX 2060 Mobile | Quadro RTX 6000 |
1f54 | TU106BM GeForce RTX 2070 Mobile | Quadro RTX 6000 |
1f55 | TU106BM GeForce RTX 2060 Mobile | Quadro RTX 6000 |
1f81 | TU117 | Quadro RTX 6000 |
1f82 | TU117 GeForce GTX 1650 | Quadro RTX 6000 |
1f91 | TU117M GeForce GTX 1650 Mobile / Max-Q | Quadro RTX 6000 |
1f92 | TU117M GeForce GTX 1650 Mobile | Quadro RTX 6000 |
1f94 | TU117M GeForce GTX 1650 Mobile | Quadro RTX 6000 |
1f95 | TU117M GeForce GTX 1650 Ti Mobile | Quadro RTX 6000 |
1f76 | TU106GLM Quadro RTX 3000 Mobile Refresh | Quadro RTX 6000 |
1f07 | TU106 GeForce RTX 2070 Rev. A | Quadro RTX 6000 |
1f08 | TU106 GeForce RTX 2060 Rev. A | Quadro RTX 6000 |
1f09 | TU106 GeForce GTX 1660 SUPER | Quadro RTX 6000 |
1f0a | TU106 GeForce GTX 1650 | Quadro RTX 6000 |
1f10 | TU106M GeForce RTX 2070 Mobile | Quadro RTX 6000 |
1f11 | TU106M GeForce RTX 2060 Mobile | Quadro RTX 6000 |
1f12 | TU106M GeForce RTX 2060 Max-Q | Quadro RTX 6000 |
1f14 | TU106M GeForce RTX 2070 Mobile / Max-Q Refresh | Quadro RTX 6000 |
1f15 | TU106M GeForce RTX 2060 Mobile | Quadro RTX 6000 |
1f2e | TU106M | Quadro RTX 6000 |
1f36 | TU106GLM Quadro RTX 3000 Mobile / Max-Q | Quadro RTX 6000 |
1f0b | TU106 CMP 40HX | Quadro RTX 6000 |
1eb5 | TU104GLM Quadro RTX 5000 Mobile / Max-Q | Quadro RTX 6000 |
1eb6 | TU104GLM Quadro RTX 4000 Mobile / Max-Q | Quadro RTX 6000 |
1eb8 | TU104GL Tesla T4 | Quadro RTX 6000 |
1eb9 | TU104GL | Quadro RTX 6000 |
1ebe | TU104GL | Quadro RTX 6000 |
1ec2 | TU104 GeForce RTX 2070 SUPER | Quadro RTX 6000 |
1ec7 | TU104 GeForce RTX 2070 SUPER | Quadro RTX 6000 |
1ed0 | TU104BM GeForce RTX 2080 Mobile | Quadro RTX 6000 |
1ed1 | TU104BM GeForce RTX 2070 SUPER Mobile / Max-Q | Quadro RTX 6000 |
1ed3 | TU104BM GeForce RTX 2080 SUPER Mobile / Max-Q | Quadro RTX 6000 |
1f02 | TU106 GeForce RTX 2070 | Quadro RTX 6000 |
1f04 | TU106 | Quadro RTX 6000 |
1f06 | TU106 GeForce RTX 2060 SUPER | Quadro RTX 6000 |
1ef5 | TU104GLM Quadro RTX 5000 Mobile Refresh | Quadro RTX 6000 |
1e81 | TU104 GeForce RTX 2080 SUPER | Quadro RTX 6000 |
1e82 | TU104 GeForce RTX 2080 | Quadro RTX 6000 |
1e84 | TU104 GeForce RTX 2070 SUPER | Quadro RTX 6000 |
1e87 | TU104 GeForce RTX 2080 Rev. A | Quadro RTX 6000 |
1e89 | TU104 GeForce RTX 2060 | Quadro RTX 6000 |
1e90 | TU104M GeForce RTX 2080 Mobile | Quadro RTX 6000 |
1e91 | TU104M GeForce RTX 2070 SUPER Mobile / Max-Q | Quadro RTX 6000 |
1e93 | TU104M GeForce RTX 2080 SUPER Mobile / Max-Q | Quadro RTX 6000 |
1eab | TU104M | Quadro RTX 6000 |
1eae | TU104M | Quadro RTX 6000 |
1eb0 | TU104GL Quadro RTX 5000 | Quadro RTX 6000 |
1eb1 | TU104GL Quadro RTX 4000 | Quadro RTX 6000 |
1eb4 | TU104GL T4G | Quadro RTX 6000 |
1e04 | TU102 GeForce RTX 2080 Ti | Quadro RTX 6000 |
1e07 | TU102 GeForce RTX 2080 Ti Rev. A | Quadro RTX 6000 |
1e2d | TU102 GeForce RTX 2080 Ti Engineering Sample | Quadro RTX 6000 |
1e2e | TU102 GeForce RTX 2080 Ti 12GB Engineering Sample | Quadro RTX 6000 |
1e30 | TU102GL Quadro RTX 6000/8000 | Quadro RTX 6000 |
1e36 | TU102GL Quadro RTX 6000 | Quadro RTX 6000 |
1e37 | TU102GL GRID RTX T10-4/T10-8/T10-16 | Quadro RTX 6000 |
1e38 | TU102GL | Quadro RTX 6000 |
1e3c | TU102GL | Quadro RTX 6000 |
1e3d | TU102GL | Quadro RTX 6000 |
1e3e | TU102GL | Quadro RTX 6000 |
1e78 | TU102GL Quadro RTX 6000/8000 | Quadro RTX 6000 |
1e09 | TU102 CMP 50HX | Quadro RTX 6000 |
1dba | GV100GL Quadro GV100 | Tesla V100 32GB PCIE |
1e02 | TU102 TITAN RTX | Quadro RTX 6000 |
1cfa | GP107GL Quadro P2000 | Tesla P40 |
1cfb | GP107GL Quadro P1000 | Tesla P40 |
1d01 | GP108 GeForce GT 1030 | Tesla P40 |
1d10 | GP108M GeForce MX150 | Tesla P40 |
1d11 | GP108M GeForce MX230 | Tesla P40 |
1d12 | GP108M GeForce MX150 | Tesla P40 |
1d13 | GP108M GeForce MX250 | Tesla P40 |
1d16 | GP108M GeForce MX330 | Tesla P40 |
1d33 | GP108GLM Quadro P500 Mobile | Tesla P40 |
1d34 | GP108GLM Quadro P520 | Tesla P40 |
1d52 | GP108BM GeForce MX250 | Tesla P40 |
1d56 | GP108BM GeForce MX330 | Tesla P40 |
1d81 | GV100 TITAN V | Tesla V100 32GB PCIE |
1cb6 | GP107GL Quadro P620 | Tesla P40 |
1cba | GP107GLM Quadro P2000 Mobile | Tesla P40 |
1cbb | GP107GLM Quadro P1000 Mobile | Tesla P40 |
1cbc | GP107GLM Quadro P600 Mobile | Tesla P40 |
1cbd | GP107GLM Quadro P620 | Tesla P40 |
1ccc | GP107BM GeForce GTX 1050 Ti Mobile | Tesla P40 |
1ccd | GP107BM GeForce GTX 1050 Mobile | Tesla P40 |
1ca8 | GP107GL | Tesla P40 |
1caa | GP107GL | Tesla P40 |
1cb1 | GP107GL Quadro P1000 | Tesla P40 |
1cb2 | GP107GL Quadro P600 | Tesla P40 |
1cb3 | GP107GL Quadro P400 | Tesla P40 |
1c70 | GP106GL | Tesla P40 |
1c81 | GP107 GeForce GTX 1050 | Tesla P40 |
1c82 | GP107 GeForce GTX 1050 Ti | Tesla P40 |
1c83 | GP107 GeForce GTX 1050 3GB | Tesla P40 |
1c8c | GP107M GeForce GTX 1050 Ti Mobile | Tesla P40 |
1c8d | GP107M GeForce GTX 1050 Mobile | Tesla P40 |
1c8e | GP107M | Tesla P40 |
1c8f | GP107M GeForce GTX 1050 Ti Max-Q | Tesla P40 |
1c90 | GP107M GeForce MX150 | Tesla P40 |
1c91 | GP107M GeForce GTX 1050 3 GB Max-Q | Tesla P40 |
1c92 | GP107M GeForce GTX 1050 Mobile | Tesla P40 |
1c94 | GP107M GeForce MX350 | Tesla P40 |
1c96 | GP107M GeForce MX350 | Tesla P40 |
1ca7 | GP107GL | Tesla P40 |
1c36 | GP106 P106M | Tesla P40 |
1c07 | GP106 P106-100 | Tesla P40 |
1c09 | GP106 P106-090 | Tesla P40 |
1c20 | GP106M GeForce GTX 1060 Mobile | Tesla P40 |
1c21 | GP106M GeForce GTX 1050 Ti Mobile | Tesla P40 |
1c22 | GP106M GeForce GTX 1050 Mobile | Tesla P40 |
1c23 | GP106M GeForce GTX 1060 Mobile Rev. 2 | Tesla P40 |
1c2d | GP106M | Tesla P40 |
1c30 | GP106GL Quadro P2000 | Tesla P40 |
1c31 | GP106GL Quadro P2200 | Tesla P40 |
1c35 | GP106M Quadro P2000 Mobile | Tesla P40 |
1c60 | GP106BM GeForce GTX 1060 Mobile 6GB | Tesla P40 |
1c61 | GP106BM GeForce GTX 1050 Ti Mobile | Tesla P40 |
1c62 | GP106BM GeForce GTX 1050 Mobile | Tesla P40 |
1bb8 | GP104GLM Quadro P3000 Mobile | Tesla P40 |
1bb9 | GP104GLM Quadro P4200 Mobile | Tesla P40 |
1bbb | GP104GLM Quadro P3200 Mobile | Tesla P40 |
1bc7 | GP104 P104-101 | Tesla P40 |
1be0 | GP104BM GeForce GTX 1080 Mobile | Tesla P40 |
1be1 | GP104BM GeForce GTX 1070 Mobile | Tesla P40 |
1c00 | GP106 | Tesla P40 |
1c01 | GP106 | Tesla P40 |
1c02 | GP106 GeForce GTX 1060 3GB | Tesla P40 |
1c03 | GP106 GeForce GTX 1060 6GB | Tesla P40 |
1c04 | GP106 GeForce GTX 1060 5GB | Tesla P40 |
1c06 | GP106 GeForce GTX 1060 6GB Rev. 2 | Tesla P40 |
1b87 | GP104 P104-100 | Tesla P40 |
1ba0 | GP104M GeForce GTX 1080 Mobile | Tesla P40 |
1ba1 | GP104M GeForce GTX 1070 Mobile | Tesla P40 |
1ba2 | GP104M GeForce GTX 1070 Mobile | Tesla P40 |
1ba9 | GP104M | Tesla P40 |
1baa | GP104M | Tesla P40 |
1bad | GP104 GeForce GTX 1070 Engineering Sample | Tesla P40 |
1bb0 | GP104GL Quadro P5000 | Tesla P40 |
1bb1 | GP104GL Quadro P4000 | Tesla P40 |
1bb3 | GP104GL Tesla P4 | Tesla P40 |
1bb4 | GP104GL Tesla P6 | Tesla P40 |
1bb5 | GP104GLM Quadro P5200 Mobile | Tesla P40 |
1bb6 | GP104GLM Quadro P5000 Mobile | Tesla P40 |
1bb7 | GP104GLM Quadro P4000 Mobile | Tesla P40 |
1b06 | GP102 GeForce GTX 1080 Ti | Tesla P40 |
1b07 | GP102 P102-100 | Tesla P40 |
1b30 | GP102GL Quadro P6000 | Tesla P40 |
1b38 | GP102GL Tesla P40 | Tesla P40 |
1b70 | GP102GL | Tesla P40 |
1b78 | GP102GL | Tesla P40 |
1b80 | GP104 GeForce GTX 1080 | Tesla P40 |
1b81 | GP104 GeForce GTX 1070 | Tesla P40 |
1b82 | GP104 GeForce GTX 1070 Ti | Tesla P40 |
1b83 | GP104 GeForce GTX 1060 6GB | Tesla P40 |
1b84 | GP104 GeForce GTX 1060 3GB | Tesla P40 |
1b39 | GP102GL Tesla P10 | Tesla P40 |
1b00 | GP102 TITAN X | Tesla P40 |
1b01 | GP102 GeForce GTX 1080 Ti 10GB | Tesla P40 |
1b02 | GP102 TITAN Xp | Tesla P40 |
1b04 | GP102 | Tesla P40 |
179c | GM107 GeForce 940MX | Tesla M10 |
17c2 | GM200 GeForce GTX TITAN X | Tesla M60 |
17c8 | GM200 GeForce GTX 980 Ti | Tesla M60 |
17f0 | GM200GL Quadro M6000 | Tesla M60 |
17f1 | GM200GL Quadro M6000 24GB | Tesla M60 |
17fd | GM200GL Tesla M40 | Tesla M60 |
1617 | GM204M GeForce GTX 980M | Tesla M60 |
1618 | GM204M GeForce GTX 970M | Tesla M60 |
1619 | GM204M GeForce GTX 965M | Tesla M60 |
161a | GM204M GeForce GTX 980 Mobile | Tesla M60 |
1667 | GM204M GeForce GTX 965M | Tesla M60 |
1725 | GP100 | Tesla P40 |
172e | GP100 | Tesla P40 |
172f | GP100 | Tesla P40 |
174d | GM108M GeForce MX130 | Tesla M10 |
174e | GM108M GeForce MX110 | Tesla M10 |
1789 | GM107GL GRID M3-3020 | Tesla M10 |
1402 | GM206 GeForce GTX 950 | Tesla M60 |
1406 | GM206 GeForce GTX 960 OEM | Tesla M60 |
1407 | GM206 GeForce GTX 750 v2 | Tesla M60 |
1427 | GM206M GeForce GTX 965M | Tesla M60 |
1430 | GM206GL Quadro M2000 | Tesla M60 |
1431 | GM206GL Tesla M4 | Tesla M60 |
1436 | GM206GLM Quadro M2200 Mobile | Tesla M60 |
15f0 | GP100GL Quadro GP100 | Tesla P40 |
15f1 | GP100GL | Tesla P40 |
1404 | GM206 GeForce GTX 960 FAKE | Tesla M60 |
13d8 | GM204M GeForce GTX 970M | Tesla M60 |
13d9 | GM204M GeForce GTX 965M | Tesla M60 |
13da | GM204M GeForce GTX 980 Mobile | Tesla M60 |
13e7 | GM204GL GeForce GTX 980 Engineering Sample | Tesla M60 |
13f0 | GM204GL Quadro M5000 | Tesla M60 |
13f1 | GM204GL Quadro M4000 | Tesla M60 |
13f2 | GM204GL Tesla M60 | Tesla M60 |
13f3 | GM204GL Tesla M6 | Tesla M60 |
13f8 | GM204GLM Quadro M5000M / M5000 SE | Tesla M60 |
13f9 | GM204GLM Quadro M4000M | Tesla M60 |
13fa | GM204GLM Quadro M3000M | Tesla M60 |
13fb | GM204GLM Quadro M5500 | Tesla M60 |
1401 | GM206 GeForce GTX 960 | Tesla M60 |
13b3 | GM107GLM Quadro K2200M | Tesla M10 |
13b4 | GM107GLM Quadro M620 Mobile | Tesla M10 |
13b6 | GM107GLM Quadro M1200 Mobile | Tesla M10 |
13b9 | GM107GL NVS 810 | Tesla M10 |
13ba | GM107GL Quadro K2200 | Tesla M10 |
13bb | GM107GL Quadro K620 | Tesla M10 |
13bc | GM107GL Quadro K1200 | Tesla M10 |
13bd | GM107GL Tesla M10 | Tesla M10 |
13c0 | GM204 GeForce GTX 980 | Tesla M60 |
13c1 | GM204 | Tesla M60 |
13c2 | GM204 GeForce GTX 970 | Tesla M60 |
13c3 | GM204 | Tesla M60 |
13d7 | GM204M GeForce GTX 980M | Tesla M60 |
1389 | GM107GL GRID M30 | Tesla M10 |
1390 | GM107M GeForce 845M | Tesla M10 |
1391 | GM107M GeForce GTX 850M | Tesla M10 |
1392 | GM107M GeForce GTX 860M | Tesla M10 |
1393 | GM107M GeForce 840M | Tesla M10 |
1398 | GM107M GeForce 845M | Tesla M10 |
1399 | GM107M GeForce 945M | Tesla M10 |
139a | GM107M GeForce GTX 950M | Tesla M10 |
139b | GM107M GeForce GTX 960M | Tesla M10 |
139c | GM107M GeForce 940M | Tesla M10 |
139d | GM107M GeForce GTX 750 Ti | Tesla M10 |
13b0 | GM107GLM Quadro M2000M | Tesla M10 |
13b1 | GM107GLM Quadro M1000M | Tesla M10 |
13b2 | GM107GLM Quadro M600M | Tesla M10 |
1347 | GM108M GeForce 940M | Tesla M10 |
1348 | GM108M GeForce 945M / 945A | Tesla M10 |
1349 | GM108M GeForce 930M | Tesla M10 |
134b | GM108M GeForce 940MX | Tesla M10 |
134d | GM108M GeForce 940MX | Tesla M10 |
134e | GM108M GeForce 930MX | Tesla M10 |
134f | GM108M GeForce 920MX | Tesla M10 |
137a | GM108GLM Quadro K620M / Quadro M500M | Tesla M10 |
137b | GM108GLM Quadro M520 Mobile | Tesla M10 |
137d | GM108M GeForce 940A | Tesla M10 |
1380 | GM107 GeForce GTX 750 Ti | Tesla M10 |
1381 | GM107 GeForce GTX 750 | Tesla M10 |
1382 | GM107 GeForce GTX 745 | Tesla M10 |
1340 | GM108M GeForce 830M | Tesla M10 |
1341 | GM108M GeForce 840M | Tesla M10 |
1344 | GM108M GeForce 845M | Tesla M10 |
1346 | GM108M GeForce 930M | Tesla M10 |
PVE基本设置
安装PVE过程略过,提前将pve安装好,在安装VGPU前先将pve底层设置优化一下
本篇文章将大量使用nano
文本编辑命令,至于怎么使用自行百度,这里不重复造轮子了。 知道如何保存就行Ctrl +X
输入“Y
”回车保存
BIOS设置
提前在BIOS开启以下参数
- 开启VT-d --硬件直通必须开启
- 开启SRIOV --如有
- 开启Above 4G --如有
- 关闭安全启动 --在security菜单 secure boot 改disabled
更换系统源
国内清华源
# 编辑sources.list
nano /etc/apt/sources.list
# 将原有的源链接在句首加 # 注释掉,更换以下清华源信息
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm main contrib non-free non-free-firmware
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm-updates main contrib non-free non-free-firmware
deb https://mirrors.tuna.tsinghua.edu.cn/debian/ bookworm-backports main contrib non-free non-free-firmware
deb https://mirrors.tuna.tsinghua.edu.cn/debian-security bookworm-security main contrib non-free non-free-firmware
更换企业源
国内清华源
# 编辑pve-enterprise.list
nano /etc/apt/sources.list.d/pve-enterprise.list
# 将原有的源链接在句首加 # 注释掉,更换以下清华源信息
deb https://mirrors.tuna.tsinghua.edu.cn/proxmox/debian bookworm pve-no-subscription
修复源401错误
# 编辑ceph.list
nano /etc/apt/sources.list.d/ceph.list
# 将原有的源链接在句首加 # 注释掉,添加中科大ceph源
deb https://mirrors.ustc.edu.cn/proxmox/debian/ceph-quincy bookworm no-subscription
执行更新源
apt update
LXC容器更源
国内清华源
# 备份APLInfo.pm
cp /usr/share/perl5/PVE/APLInfo.pm /usr/share/perl5/PVE/APLInfo.pm_back
# 替换为清华源:
sed -i 's|http://download.proxmox.com|https://mirrors.tuna.tsinghua.edu.cn/proxmox|g' /usr/share/perl5/PVE/APLInfo.pm
# 重启服务后生效
systemctl restart pvedaemon.service
PVE常用优化脚本
可以做下以下优化1)去掉登录订阅提示;2)合并local-lvm以最大化利用硬盘空间;3)添加CPU频率硬盘温度;4)删掉不用的内核等信息;
# 下载pve_source二进制文件到/root目录
wget https://yangwenqing.com/files/pve/pve_source && chmod +x pve_source && ./pve_source
开启硬件直通
需要提前在主板BIOS开启虚拟化功能,才能开启硬件直通。在BIOS开启vt-d
,AMD平台是iommu
,并且开启SRIOV
和Above 4G
选项
使用nano
命令编辑/etc/default/grub
Intel CPU
nano /etc/default/grub
# 开启iommu分组,在里面找到:GRUB_CMDLINE_LINUX_DEFAULT="quiet"项将其修改为
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction"
# 更新grub
update-grub
加载内核模块
编辑nano /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
屏蔽设备
添加设备黑名单,编辑pve-blacklist.conf
# 编辑pve-blacklist.conf
nano /etc/modprobe.d/pve-blacklist.conf
# 直通AMD显卡,请使用下面命令
blacklist radeon
blacklist amdgpu
# 直通NVIDIA显卡,请使用下面命令
blacklist nouveau
blacklist nvidia
blacklist nvidiafb
# 直通INTEL核显,请使用下面命令
blacklist i915
blacklist snd_hda_intel
blacklist snd_hda_codec_hdmi
# 允许不安全的设备中断
options vfio_iommu_type1 allow_unsafe_interrupts=1
执行更新initramfs
# 更新initramfs
update-initramfs -u -k all
# 重启
reboot
验证是否开启直通
# 验证是否开启iommu
dmesg | grep iommu
或者
dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
出现如下例子。则代表成功
[ 1.341100] pci 0000:00:00.0: Adding to iommu group 0
[ 1.341116] pci 0000:00:01.0: Adding to iommu group 1
[ 1.341126] pci 0000:00:02.0: Adding to iommu group 2
[ 1.341137] pci 0000:00:14.0: Adding to iommu group 3
[ 1.341146] pci 0000:00:17.0: Adding to iommu group 4
此时输入命令
find /sys/kernel/iommu_groups/ -type l
#出现很多直通组,就代表成功了。如果没有任何东西,就是没有开启
lsmod | grep vfio
# 检测模块是否加载
vfio_pci 16384 4
vfio_pci_core 94208 1 vfio_pci
vfio_iommu_type1 49152 2
vfio 57344 17 vfio_pci_core,vfio_iommu_type1,vfio_pci
iommufd 73728 1 vfio
irqbypass 16384 41 vfio_pci_core,kvm
#出现这类信息,就代表成功了。
配置VGPU_Unlock
# 创建vgpu_unlock文件夹
mkdir /etc/vgpu_unlock
# 创建profile_override.toml文件
touch /etc/vgpu_unlock/profile_override.toml
# 创建nvidia-vgpud.service.d,nvidia-vgpu-mgr.service.d启动服务
mkdir /etc/systemd/system/{nvidia-vgpud.service.d,nvidia-vgpu-mgr.service.d}
# 写入路径信息
echo -e "[Service]\nEnvironment=LD_PRELOAD=/opt/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so" > /etc/systemd/system/nvidia-vgpud.service.d/vgpu_unlock.conf
echo -e "[Service]\nEnvironment=LD_PRELOAD=/opt/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so" > /etc/systemd/system/nvidia-vgpu-mgr.service.d/vgpu_unlock.conf
# 重新加载服务
systemctl daemon-reload
执行完成后,cat下查看服务配置是否与下边一致
cat /etc/systemd/system/{nvidia-vgpud.service.d,nvidia-vgpu-mgr.service.d}/*
[Service]
Environment=LD_PRELOAD=/opt/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so
[Service]
Environment=LD_PRELOAD=/opt/vgpu_unlock-rs/target/release/libvgpu_unlock_rs.so
下载预编译好的libvgpu_unlock_rs.so文件
mkdir -p /opt/vgpu_unlock-rs/target/release
cd /opt/vgpu_unlock-rs/target/release
wget -O libvgpu_unlock_rs.so https://yun.yangwenqing.com/ESXI_PVE/vGPU/NVIDIA/17.0/libvgpu_unlock_rs_only_17.0.so
安装VGPU驱动
安装显卡驱动需要用到的依赖
apt install build-essential dkms mdevctl pve-headers-$(uname -r)
# 下载显卡驱动
wget "https://yun.yangwenqing.com/ESXI_PVE/vGPU/NVIDIA/17.0/Host_Drivers/NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run"
# 赋予执行权限
chmod +x NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run
# 安装驱动(默认回车直至安装完成即可)
./NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run -m=kernel
# 重启
reboot
其他补充提示:
1) 如之前安装过了显卡驱动,则需要先卸载,再安装
# 卸载显卡驱动
./NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run --uninstall
# 移除显卡相关程序
sudo apt-get remove --purge nvidia-*
# 安装驱动(默认回车直至安装完成即可)
./NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run -m=kernel
2) 下载慢?试试aria2吧
# 在pve安装aria2
apt install aria2
# 4线程下载文件
aria2c -s 4 -x 4 -j 10 'https://yun.yangwenqing.com/ESXI_PVE/vGPU/NVIDIA/17.0/Host_Drivers/NVIDIA-Linux-x86_64-550.54.10-vgpu-kvm-patched-kernel6.8-OA5500.run'
重启完成后查看相关服务状态
# 查看相关服务状态
systemctl status {nvidia-vgpud.service,nvidia-vgpu-mgr.service}
# 重新启动相关服务
systemctl restart {nvidia-vgpud.service,nvidia-vgpu-mgr.service}
# 停止相关服务
systemctl stop {nvidia-vgpud.service,nvidia-vgpu-mgr.service}
随后使用nvidia-smi
root@pve8:~# nvidia-smi
Sun Jul 21 19:09:09 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.10 Driver Version: 550.54.10 CUDA Version: N/A |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA P106-100 Off | 00000000:01:00.0 Off | N/A |
| 23% 44C P0 28W / 120W | 27MiB / 6144MiB | 1% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| No running processes found |
+-----------------------------------------------------------------------------------------+
以及mdevctl types
查看
root@pve8:~# mdevctl types
0000:01:00.0
nvidia-522
Available instances: 0
Device API: vfio-pci
Name: NVIDIA RTXA6000-1B
Description: num_heads=4, frl_config=45, framebuffer=1024M, max_resolution=5120x2880, max_instance=32
nvidia-523
Available instances: 0
Device API: vfio-pci
Name: NVIDIA RTXA6000-2B
Description: num_heads=4, frl_config=45, framebuffer=2048M, max_resolution=5120x2880, max_instance=24
nvidia-524
Available instances: 0
Device API: vfio-pci
Name: NVIDIA RTXA6000-1Q
Description: num_heads=4, frl_config=60, framebuffer=1024M, max_resolution=5120x2880, max_instance=32
nvidia-525
Available instances: 0
Device API: vfio-pci
Name: NVIDIA RTXA6000-2Q
Description: num_heads=4, frl_config=60, framebuffer=2048M, max_resolution=7680x4320, max_instance=24
搭建fastapi-dls授权服务
你可以在内网或者外网部署好Docker环境,然后搭建fastapi-dls授权服务,我这里提供一个pve lxc的Docker容器,部署到内网进行授权。
# 进入pve备份文件夹
cd /var/lib/vz/dump/
# 使用wget命令下载lxc docker 容器备份包
wget https://yun.yangwenqing.com/ESXI_PVE/PVE/Docker/vzdump_lxc_docker_root_123123.tar.zst
# 或者使用aria2c命令多线程下载lxc docker 容器备份包
aria2c -s 4 -x 4 -j 10 'https://yun.yangwenqing.com/ESXI_PVE/PVE/Docker/vzdump_lxc_docker_root_123123.tar.zst'
# 重命名为vzdump-lxc-100-2023_11_14-15_docker.tar.zst
mv vzdump_lxc_docker_root_123123.tar.zst vzdump-lxc-100-2023_11_14-15_docker.tar.zst
LXC容器信息:
- 默认IP地址:192.168.3.74
- 账号:root
- 密码:123123
1)将下载下来的LXC容器进行还原
2)并将原来的IP改为自己内网的IP,我这里用的就是3网段就不改了。
3)登录LXC容器(账号:root
密码:123123
)并创建授权服务:
# 删除旧的容器
docker rm af06b7705582
# 创建授权服务,注意下边的IP(192.168.3.74)改为刚刚自己修改好的
docker run --restart always -d -e DLS_URL=192.168.3.74 -e DLS_PORT=443 -p 443:443 makedie/fastapi-dls
创建虚拟机(Win10为例)
创建win10虚拟机(21H2)
名称:自定义
光驱挂载:Win安装ISO镜像
系统中显卡:默认(
安装并且能远程后,建议改成无
)机型:q35
BIOS:OVMF
EFI分区:UEFI(OVMF)需要
磁盘:SATA 大小100G(按需设置,或硬盘直通)
CPU:host 核心数量8(按需设置)
内存:4G及以上
网络:virtIO(半虚拟化或网卡直通)网卡
建立好虚拟机后
添加CD/DVD设备1个,挂载virtIO驱动ISO镜像
添加PCI设备:nvidia显卡(勾选ROM-Bar和PCIE在Mdev类型中选择vgpu设备524)
音频设备:ich9-intel-hda/后端驱动器:无
![创建win10虚拟机 创建win10虚拟机](https://yangwenqing.com/usr/uploads/2024/07/2398888164.png)
其中 NVIDIA RTXA6000-1Q 是 mdev 的名字,RTXA6000--显卡名,1--1G 显存,Q 代表 vWS
关于最后一位字母,如下
A = Virtual Applications (vApps)
B = Virtual Desktops (vPC)
C = AI/Machine Learning/Training (vCS or vWS)
Q = Virtual Workstations (vWS)(性能最佳)
在配置vgpu的时候,选择正确的型号型号即可。
资源下载
# Windows 10 Enterprise LTSC 2021 (x64) - DVD (Chinese-Simplified).ISO
https://yun.yangwenqing.com/Windows/MSDN/Windows_Desktop/Windows10/Windows%2010%20Enterprise%20LTSC%202021%20(x64)%20-%20DVD%20(Chinese-Simplified).ISO
# virtIO驱动
https://yun.yangwenqing.com/ESXI_PVE/PVE/virtio-win-0.1.240.iso
# NVIDIA Guest驱动 551.61
https://yun.yangwenqing.com/ESXI_PVE/vGPU/NVIDIA/17.0/Guest_Drivers/551.61_grid_win10_win11_server2022_dch_64bit_international.exe
安装NVIDIA Guest驱动
创建好系统之后,在系统中开启远程功能。如远程桌面,todesk,vnc,向日葵,parsec等然后安装 NVIDIA Guest驱动
启用授权
1)在浏览器访问fastapi-dls授权服务:https://192.168.3.74/-/client-token
会自动下载好授权文件
2)下载后放入C:\Program Files\NVIDIA Corporation\vGPU Licensing\ClientConfigToken\
这个目录下
3) 重启NVIDIA Display Container Ls服务,即可获得授权
也可以在cmd窗口使用nvidia-smi -q
命令查看授权情况如授权失败,需要检查电脑时间是否对得上授权服务器时间
4)使用Todesk远程工具连接上虚拟机跑个鲁大师测试一下,显卡是否正常工作
想解除60帧?
编辑
nano /etc/vgpu_unlock/profile_override.toml
文件,加入以下信息关闭虚拟机重新开机生效。[profile.nvidia-524]
num_displays = 1
display_width = 1920
display_height = 1080
max_pixels = 2073600
cuda_enabled = 1
frl_enabled = 0
framebuffer = 939524096
参数说明:
[profile.nvidia-524]nvidia-524 为vgpu型号的参数。如选的不是524则改成自己选的那个型号。
num_displays 最大显示器数量
display_width = 1920
display_height = 1080
max_pixels = 2073600 这3个是虚拟显示器的分辨率,max_pixels是长宽的乘积
cuda_enabled = 1是否开启cuda
frl_enabled = 0 是否限制帧数,0为不限制
framebuffer = 939524096 1GB显存,设定vgpu的显存
![突破60帧限制 突破60帧限制](https://yangwenqing.com/usr/uploads/2024/03/404333501.jpg)
自定义显卡显存大小
如要自定义修改显卡的显存大小,只需要修改framebuffer的值就好了,显存不建议低于1GB,官方说低于1GB将会禁用NVENC视频编码器
注意:vgpu会默认占用128M,所以如果要改显存,请将结果减去128M再去换算
例如,你期望显存为1GB,所以就用1024M-128M=896M
在线大小换算平台:https://www.bejson.com/convert/filesize/
![换算结果 换算结果](https://yangwenqing.com/usr/uploads/2024/07/3192000523.png)
5)在PVE后台查看是否调用成功
root@pve8:~# nvidia-smi
Sun Jul 21 19:43:31 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.10 Driver Version: 550.54.10 CUDA Version: N/A |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA P106-100 On | 00000000:01:00.0 Off | N/A |
| 23% 42C P8 7W / 120W | 1045MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+
+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 5675 C+G vgpu 1018MiB |
+-----------------------------------------------------------------------------------------+
虚拟机去虚拟化
有游戏需求的话,可以对虚拟机做下去虚拟化操作
# 查看当前安装的pve-qemu-kvm版本
dpkg -l|grep kvm
# 我这里是8.2.2-1
ii pve-qemu-kvm 8.2.2-1 amd64 Full virtualization on x86 hardware
安装去虚拟化修改后的版本
# 李晓流作者停留在8.1.5-3版本,我这里是8.2.2-1测试也是可以使用的,直接强制安装即可
wget https://yun.yangwenqing.com/ESXI_PVE/PVE/pve%E5%8F%8D%E8%99%9A%E6%8B%9F%E6%9C%BA%E6%A3%80%E6%B5%8B%E7%8E%A9%E6%B8%B8%E6%88%8F%E5%8C%85/pve-qemu-kvm_8.1.5-3_amd64_anti_detection20240302.deb
dpkg -i pve-qemu-kvm_8.1.5-3_amd64_anti_detection20240302.deb
AMD核显直通,Intel、AMD、NVIDIA三家独显直通,在虚拟机配置文件加入以下参数nano /etc/pve/qemu-server/虚拟机ID.conf
args: -cpu host,hypervisor=off,vmware-cpuid-freq=false,enforce=false,host-phys-bits=true, -smbios type=0,version=UX305UA.201 -smbios type=1,manufacturer=ASUS,product=UX305UA,version=2021.1 -smbios type=2,manufacturer=Intel,version=2021.5,product='Intel i7-12700' -smbios type=3,manufacturer=XBZJ -smbios type=17,manufacturer=KINGSTON,loc_pfx=DDR4,speed=3200,serial=114514,part=FF63 -smbios type=4,manufacturer=Intel,max-speed=3200,current-speed=3200
如要还原回原来的官方包pve-qemu-kvm可以到清华源去下载安装
# pve8到这个链接下载
https://mirrors.tuna.tsinghua.edu.cn/proxmox/debian/dists/bookworm/pve-no-subscription/binary-amd64/
# pve7到这个链接下载
https://mirrors.tuna.tsinghua.edu.cn/proxmox/debian/dists/bullseye/pve-no-subscription/binary-amd64/
# 下载名为pve-qemu-kvm_x.x.x-x_amd64.deb就是了。
pve-qemu-kvm_8.1.5-3_amd64.deb 27.0 MiB 2024-02-22 03:18
pve-qemu-kvm_8.1.5-4_amd64.deb 27.0 MiB 2024-03-12 22:52
pve-qemu-kvm_8.1.5-5_amd64.deb 27.0 MiB 2024-04-12 02:40
pve-qemu-kvm_8.1.5-6_amd64.deb 27.0 MiB 2024-04-29 22:50
pve-qemu-kvm_8.2.2-1_amd64.deb 28.1 MiB 2024-04-29 16:50
去除虚拟化后,大部分游戏都是可以玩的。“高级”点的游戏对虚拟机检测机制不好过,类似腾讯游戏有ACE这类检测机制就玩不了。
原神启动
相关文章参考
佛西博客-在Proxmox VE 7.1 中开启vGPU_unlock,实现显卡虚拟化
佛西博客-在Proxmox VE 7.4 中开启NVIDIA P106显卡虚拟化(vGPU)
pve能代替esxi不