NPU 포트에서 pciehp 분리¶

Rebel100™ATOM™

Important

RHEL 10.2 기반 RHCOS 노드에만 적용합니다. RHEL 9.x 기반 RHCOS 노드에는 필요하지 않습니다.

Important

ATOM™ 노드에는 이 페이지의 설정이 필요하지 않습니다.

같은 PCIe 스위치 아래의 NPU 여러 장이 동시에 리셋되면 링크가 오래 끊겨, 핫플러그 드라이버 pciehp가 사용 중인 카드를 커널에서 제거할 수 있습니다. NPU가 연결된 다운스트림 포트에서 pciehp를 분리하면 이를 막을 수 있습니다.

MachineConfig로 pciehp 분리 훅을 RHCOS 노드에 배포합니다.

사전 요구사항¶

OpenShift Container Platform 4.x 클러스터(4.21, 4.22에서 검증)
cluster-admin 권한
OpenShift CLI(oc) 설치 및 로그인 완료

클러스터가 쓰는 Ignition 스펙 버전을 확인합니다. OCP 4.21과 4.22는 3.5.0을 쓰며, 아래 매니페스트가 선언한 값도 같습니다.

oc get mc 00-worker -o jsonpath='{.spec.config.ignition.version}'

적용 대상 풀¶

MachineConfig는 MachineConfigPool 단위로 적용되므로, RHEL 10.2 기반 RHCOS에서 동작하는 Rebel100™ 노드가 속한 풀을 대상으로 합니다. 커스텀 풀을 만드는 방법은 커스텀 MachineConfigPool 생성에서 확인할 수 있습니다.

노드의 RHCOS가 어느 RHEL 버전 기반인지 먼저 확인합니다. VERSION_ID가 10.2가 아니면 이 문서의 작업은 필요하지 않습니다.

oc debug node/<NODE_NAME> -- chroot /host grep VERSION_ID /etc/os-release

풀 전체에 적용합니다. 적용하지 않은 노드에서는 사용 중인 카드가 사라질 수 있습니다.
풀의 노드가 전부 VM 게스트라면 생략할 수 있습니다. 아래 명령이 아무것도 출력하지 않으면 VM 게스트입니다.

oc debug node/<NODE_NAME> -- chroot /host ls /sys/bus/pci_express/drivers/pciehp/

베어메탈과 VM이 섞인 Rebel100™ 풀도 풀 전체에 적용합니다. VM 노드에서는 훅이 아무 작업 없이 종료됩니다.

아래 매니페스트와 명령은 풀 이름으로 __MCP_NAME__을 씁니다. Rebel100™ 노드가 속한 풀 이름으로 바꿉니다. 클러스터의 worker가 모두 Rebel100™ 노드이면 worker, 그렇지 않으면 커스텀 풀 이름입니다. 이 문서의 모든 명령에 같은 이름을 씁니다. master가 worker를 겸해 master 노드에 NPU가 장착된 3노드 컴팩트 클러스터에서만 master를 쓰며, 이때는 control plane이 순차 재부팅되므로 API 가용성을 함께 확인합니다.

매니페스트¶

매니페스트는 풀의 각 노드에 다음 파일을 배포합니다.

항목	경로	역할
훅 스크립트	`/usr/local/sbin/rbln-pciehp-hook.sh`	NPU가 연결된 다운스트림 포트에서 `pciehp`를 분리
udev rule	`/etc/udev/rules.d/99-rbln-pciehp.rules`	RBLN 디바이스가 추가되면 해당 포트의 systemd 서비스를 시작
systemd 유닛	`rbln-pciehp-unbind@.service`	포트 하나에 대해 훅을 실행

훅 스크립트와 udev rule은 gzip으로 압축되어 매니페스트에 들어 있습니다. 이 값(contents.source)은 수정하지 않고 그대로 적용해야 합니다. __MCP_NAME__만 바꿉니다.

99-rbln-pciehp.yaml
# Generated by Butane; do not edit
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: __MCP_NAME__
  name: 99-rbln-pciehp
spec:
  config:
    ignition:
      version: 3.5.0
    storage:
      files:
        - contents:
            compression: gzip
            source: data:;base64,H4sIAAAAAAAC/7U8a1fcRpbf+1fUtJ2YJqgbvJndpElnBwOOOWMeh0eycxwfEK1qWoNa6qgkoNfhv+99Van0ADs7GT7YIJVuVd2670e9+MvoOk5H16GZ9170Xqj8OkmD5TTW82Uwz7LboZmrIFC3Wi8VP1bZbKaKuVYnuwdaRdl9aopchwu1zPLCwJuwUPPMFADsVF/rJImz1KijkwszpBl+ma9gUGyUfohNYeBJUP+BJ8epCpugK8hGhSkC3LArKsocpghVkYepiXVaqCROb5W+g18B2tre+/dnu6OTvd2BitMiQ9j6Lp5qletFdhcmY3hCX9AU91mZRGoe3uF7ABmnOlJFeKtxcxrgTcM8UllZACYIEbc6T3WiFnEU3Gf5bZKFEcxjCh1GG7DUiAbBXznuiOeYZUmS3ePmozy8MTAc93RwfHh4Ab/DkuIoLABxyhRZviAgoZqVSQJLCnRaLnRO74dKXaRwgFGc3gCwLE1WNNs8K5ZJecOIMzqn3SYa9mTUzv7paO8E8HG4Dx/weky40DwYpr7TGwDrfh5P5wTsJsmuw0RdIbYv6ZQn02yxDIsrNV1EgDitzH1cwGhGXZoVfNR0jG9ywAeMVyf7//Pd9w8PD7hZs0oBchFP1SKLtCzZFMEsnMJOWkcfI6b0okzCQns4lVlp17m29Kf5LKfzML0B0HjcCFvoJ0RQFj2w0ngWTxnVOg2vEx35KLUENgXU5Qg7M1rGmQ1lMiEIWQec1dKoZa4N0B1+TedONEkThEmR3Wh4mAt68kjnOC68CZFeaHyUA/7zNlc0OOQchpZAxSovEw2QMziEaZjChtS1Dn1QY3X13WbAv5shDjdXsHTE1bWeZTlu4Or774OrDfoIzgMhMhEbPKjpPM/SrEQiNbEcFk2NxK5zexwwGIEjSesZTKVimAGH8BGYFTDEIlJlGheA4rPMW+Iro5Azlnl2rYMiXiDjGY2HtVzq1DAxwqLkNHBZRQy8cJ2VKZ4XIi+EvxG1tCVACBxOEq4MkRJMD9si8LhZGX6liHSvV7jgsEyKDaF5gD/HHefA9/AWxEi+YoQAzZM45GOPYSt5uJgZtSa7Cxh7JqDRWc54gofAVaEZbPDfcUp7B/IGds4ymDcEaIswLeGBXWhF5XmZznD3smxC15UdBlCsjL0a4uaCIM0Ct7IrlS3hGFBYITiRACzmgMSze3ivUyZHIimifN5npJP4GuWMBvYqDXyHWkJdl3EC1G2Y7dbSDB8DTyz0KIpz+h/1CAg/ZMMRMNcojmDr13oaAhSUcxXiQJTmMUBOkX5JnCKH5TAz8NjKKgxYbVzAScLAo+NzNc2e4xB4sZul0zLPUREQIRkEbfAcAcGoiWAx89gKERR9wsHl0pc6oB2ALwEeUXwKzJqsiK3z+OYGmFgByohWsls1B85g4WNWi2UB4m6Wwz9WsG3bDQA05NEFqBNfhIFUD42Jb1AKRcsMtBQquZWahUBXONMZSAogIoPzy56WhITOlce8FJTSiV4AHkKg4Fn8QEQA9JKlrHMqFmRUvyPtek+HMM2A7G5K1jMIEQg0vAGWAAbDDxfPy6jOg9nLaPdA0ZaaRyAPhHtRQnqSOtewgikQvlEjXUxp6aPS5KMkm4YJALNSE7X5NAmRW+EMYvwfABYaGfX03e7xGZ8dbfYwnM6R6Xdpc+p4ifSdIR4tykIkURQ9JFl2AackymHPwA+afg8rVCBfAjksWdoBKmcFCcBCTwGsY+EUlRyhECQmLvMmDyNSNWfzeFmRYTHPtSaohoUZEV+elTdzRspCo1qLzcIaGGMmMZ0sURIXda0gs6PIBdkWF3OyyX4rAUfmEGRnYd5m+USZJUgQzSbNmj0NGJIlKCyAcEM+MpTMGyjKAbvxFA0hxiLrnYFIEQ3rivAwT9/tv1dbm8PXw14PpXlQZmoZL0HWxkmv9/P+0d7x6aS/+bClZ7N+7/B47+L9/qRfybN+73znJ3jQsEj7vd7B0dn5zvv3lyc75+8m/YooRgZt2Q4Ttt+72Nv/+fKUZkBqGiGWRqQMh9EItJ/3EetI+OTo4Pzy7YH7RGS8/F+bhsXm34ZiadFuTk6P3+xf7h4fvZXvrciGCf3ZEIn93t7pzu7FuT8cDMNpWdDrzi8OYHmnO4dvzy7fHR//XT5ykjUosiwxI9y/8T8G7AENvHl/dAnm+/67k8uzf5ydHoNQBZkLKCzgwFnfwS5BPpMiyNhmnpHMQgJlw0fdh8ktmcMpyBQw53U+jQ2cPVIa0lKRlchtN6zp5mD/3gOXAtGf5KAmpyRXRCMBkZUpEMmwJ+uZ9F9+6ljmOHhEXP18uXdwikPk8SMeyui6NCPY5aV+QBvMjMTmGdmdn/28+2XfkXMAFABzP/dBNfD98e7fa6OAXZAkb9vEiE/hEJLsZm2gPvUUGBaoTFRQKAAAFP/YR8HVf7neV69/xClGKdr9v/8OuC81fAAiILNjxwrH/fj1695jrxfFuoJJEHD0A+B2C1/nzPmXaHLIuA8IZ//iYG8cbOG8+je1qT7iXABL9RelYc4PDRkqfQTzQu2w3JsBsuZgAkxRRupIpGyDG52GBw8rBM5mMXaNWCCb/kEDkSMhkLok2QV/nO2/j9PyATwdUDbkEZDfRxpXt4WbSrWOUIYXVjmxOIWFmSwdO/twEa5oOzRXiOKr0A+F6DsxAhECgCKVExtSVwgP7RNaHc5rbVkgGJDzntmoH8JpASi5x02iUXeFxjOAGOH6r+zEkb4G5wOnR+cyVzcx8sAqK4dwSIQmOR9Q4AucMLhDeQwaBXWi8qji9Y9fb+FxgaMKPrDahG+8gcFbON+/9bs+IFqi0wz+vB+AdoLmRxQbstFWSMk1e1GcJkAVWehwGM6t8O3CP3lZAG4fmYbMnRh8h3RatCMXeESs3r24hTXHgPR+AbEG9ICWR35TolUlotKOYW8aXCgxMmCXcAq4ObBJtkksqjd7b8FzFD8dyA7DE/gNCVoiMIo0WJpaglUodqWbhTjon8ia6B6h3QOqd5qBCp8W7J47KQBciOI5wmiCNhN4rpfqOpqpO4AGnEKCeRmSpcyOfR9kEzhUMxINL5xI2MYlpPBG1eCB9BAZ+dgfrQ9Q3CRG0zBkRJwJRADS4Dbgm577AL6pQRi9/AQfPPYHNC4CjoNfZjGuB4HBygnWp+r7D3/7+FhB/qCCHN/r5eOI99dnWYaMBuKE14WOCQ4UDPzQ/KI5nkQkvwQJOcG/2HR5bEKn4cjyAP/kqjKg2ClAbQquAYUm7pmSQFyAcZ+iFyjeEqFZDkTASYxpOV+ZeEqnVcytsdoMk1gK5vCapVy77WTyco3WJluu65evv1bLe3w76EBBQIhHGI9dOOUFT2TIV6P1x37jOf/24sX66LFvEbVDOiVgJ+UBfNYCzWLUJHbpr4wlTkSQQTXUB126CT/jzc2+gtFgJAs4YAsKN+29Zdd5HqL0tv4R4Yc0VKoJgwWFJabItEOha1ACbqmwzzgVgv2wGXwfBrOPT/0//uyDof1loLa3Ber6wCHRPtQmnFrsHIIuioNZmbKdhDFHAyo0jayU4ZAdLXZbNqJ0aLcKrpP296VgY8jgj8rb2Do95v2q/np9cSIP3HceZvzlCqd6dgmNf6Ee+/8O/SIBihHGPeCXP1tRAB4vs7RDioZ0DICOLRA4Zo5+nn3F2sOPtJKkNXdThdEQUAdAgEDV+XSy6cPT0WRTgUmmC/gfXEZwi/C7yAngv6ggQoSKvYus58lhsvGqiBgaKhJ6ZAqBZxyY3sZ3ZIcDxUdZX+SBMxhYxPJxg4TwFQmJ7oFdT/C/7ni71gLM1qE4lYkLF28JO5MGT6wIlLaLhbBtiUYYR2bCqa7xAcvX+zwuKsswAfWaw5Fow8COjsEHBD0+Q+MTUQaGZxIkaGiy3L0pMbCPqQ4R2lPw7EFqGIz3gkIGNY5yI2Z44TXi2hl/6TQpMQRksoX2AlzxIrwBH5NR6JlzvIiGYeZhdHEbxeAULBHj6F2QUH3CIVBkRqvvf5SxIn15iu9rSpSjQ6mTBk55isb/xHTOmo55rX7SyqNxjzRHDPBxjATZX2cV7mwBRNe5CzVxeB7wf4PnGc5AB6Lxx7ORgXiTYdxVJgokV4O8NKyBS7J7GFdoqyMB/XRisM84sjoSlxXYDMg1BkLgBBwY9C4vT45Pzy/P9k9/Ptjdv3x3gl9+q9ZODvcnWxuYLZm83lB7J7uT7wa0Ops84AkcKCAGtgWR0wDFm9+SG4AmoluIKlZL+K0sXPRtHt/MaRNjB2gTvqRYGHkO+NmGem2fNVhn2HUo4uc2D2XzWz4WpgZl5RIJJaYOFFlsY1l4dQuLTAFSkjCy0xKooG7J33hsE/5C1D8/7yI4ITe0RexEHoEhpMcmNSqRoC/X1ugX9Y3aGgzcy8biYOtudquuBOyPjdncujyea8zMMhxmpl8aM7NAZwtXYIGzLrPZYR6H+F9ghAxzjply37Y+VahPtur7EptZjFY5CfHrt/i0nsCW1eFolaVZkC3JT76h0HlKCTW9QSqGRLBmQQKylDyWGcYTWeOg2UGBaeYIUQEo81zCAsNFuQ4ooDh3WQ3OxbBXg+H6QqfEbCFB+o/XASVdKTrMScYMTb/7GGwbkfroT4GoTiPZAAK0wazrLCuGvRrROcRsWoJD83X62OXyiHpzSrR2ooTIR0LImhmAX4u26spF1/2IdL/pIEUsi/nkgde+6Xt/PPYfPc7jkM6nSBio5h6J6qT1/zuMLk+diZEOm/+zTS8Oml+6uS4xgik2WF13cmT0GeVJrM2jCOP1z8slOo9VjuxzgBoxVf8M6X3qjgJQT8R4aYn9EkO1NpYDXjsJmVpsGOjthx8o0Oke7x+/BdweSooBFCGH+rbhmIVZKCEBhE2TCedhSI4j8KfCWWIqc5LXz+3GFHPROeXSyXwhJUWeJSYuMRzLoTaABoL4ATiEbSP4EgcE1zrExPXYhXoxSYA0nnDeSEJqnJYgTwx1PSY5Y0MpLOZhlwWThQWBNd4kjcvce3Ryga/KFOdGW6zKV1I4Vr4GVBTOekPlnJG0AkEzZjkFdndiKkHkZFUlqTbqooqjk9pwbMqTU7wuylRjkQnxcii7oJRNSDk4KwvKJcuy5+XYjHLlLpsDgxEUzEGpPpZhys9tTikbbCisyQlxsottwigutn06oBqKONKLZVZoEcpkIqJ1RLlFHUl4jlc/D/NFoo2U7RyXxbKk57W8MOZjMdwHu6J81kFaT/AWbDWTK86RbpwD85YFWPS5tdIoIIaprZRCxFRLkCLh8IkYIO6U4sBAaC6JTVFZEk93mdRw0NpBOJeAmhXaIgUiiDP2RA0eN1hrlCgehkiCDBZaEB5Fgv8zKylGg4CTZNizhECMfPF+/9Fz1jxyHFXvrUHzAN/4qatHq2Raz/nItoGptxUntRxNBkF8k8IWgvY6fn25e7j3/uBo//L45Pys15YqIITmAEht/ue333ZIIlYdlBrNFxUXOHQu4ylmL1taG8lb8BqmK9aSRG3XIY7nkiufKNiVClEaYLQTiWAW5pQhQmM5Q3cvG/bgdOLZqhKmOFnNKUfJk0/rcXK37I6g96dKm7thaNTMsI4E3PrbeLmkqh9A7ywByxx4bK4xX9NwUB/JKgkCM8/uwT9Zgvww9YCfRZBdGWwZWQHzyDal4KQxu5MiK5AciZOughT2cwWLIQ/Ymh9MhjWJ4zgWDoJguXwE557zBeCKU+LsV5RpanMqLPDCiDzb0mhkeIXHg+YS4BdMRY/0avvtO8rrE4bRjgSb9OV/e4YWG1RAI80YMsfaYILHWjzKnUZ//Xf+S4RdsnL1L5H1L6soiDuvsccO+LX7Rq00FmH4ZIvaEyShui4xMkjEFgPwyr6uzpt/6mGv1tzq7c7B+/29sXoKYdXSMCkHa2L8cKgUT3mmXn1lfk1fVajBzJ63lq2e2PqdCFzvN+RIhajmUrNbb5nVulglWdOcCltAFpjCIqWBEkKIoOOzyIjiiINUIEWAUltiBOd4FgsOBySmzud+mIX0B2YBja9B45kv6OMCyHuGbGHl0jayJQimgus/cHVrF9dlWpQY4luS0oIVBNksoLz33t8PzyqeAyYZtGrm0LSSiiFX3+UiUg4XNTrk4j3L/xnKQLHAeC2vTM0Ep8SZzapel2Z1nT3AciUYzmFwLNQiXY7bWLqUAIdHeHkggF21JhsxcOxDQWwScyKBSlY4ekk5i5AJDwtCa/Nd/a5ucr1UwW9XY47VYe6Tij6qmo8rGvMKfLIwh9OixPTZwU8nByf7tmwTi52qwjf8EDHEC05diScCK3MxvUii0SJRNVvfQXKr9Rgu0keiKYar4sWN3aWExvAJBj9GaGh5hRQv10oK4wb5YAhj+rURET7yh7TTUTCgM1QiLLzW6fcMPKnIjsxAuR9ZN4jmxPAqqnm8aIUXzm+4MIM2CCKtz0BZHzgebMlCK5AEcF2qv/zEqEeB5GRIC4Qn48S79TmeeQzQVGHKOoetU4cjcjJteJuJYXMolCxxY1u1aOMaNYGEvtBY7JSqooFserRdJY9HQWOdcg2ZXzFV0bBIIqJfzN6YqhgNADGUML/RRY2fqoK1pSSr7kPjKbMCrCOG/KUIqWsGicQ5f7X64iYptXNYv4hAhTxt+M+LW3s1TRy+rgoVtlwuuLCRNze2conl4ZMOcaOsCo9nWRKCsJ6+LLIFFqqh7bpRUwuGS9QFn5eAgYX5ZtJvmeH9XnMZsuqaFV1buq9v73PwtFT3ACH+JmN2YLFe5/VZRNaHV7jEvxiT3PGB/R5tnPbAavsV5eMEjeBc/2bYe/lEoY5tFtqb22AAv1BnBfidHEYURVvhF9198Xk5KkC2hBSoo/wmhwBzI1QfEg173f4R7pTn7A2ZEQ14Cbqzzs0lSw2VJi5Xl+i/g74xWAwTVVUbNnuuUVSx37iOfLbOoiAEUyWn+s3VWPI9trNCKp1CjoqK8eBUNze02HqWvf2z872DUyx3CznowTzvgJM8EH9KUk0h2uC18qkQK6cotqPU8QIMVnGCKc4hhoYBD3XKosKrRaIapKetjgQrA1HxYe2I4KqJ/16dcmrE/19//WsXuXXR/xNjhAXqNipm7Msl6nniYhdGgFNmx0CtSfhv1ArkDRpm6lbLTK0sRyI9QAgWcOGJXmd32hKG9Y84ffrKdXpgzhmJlQJa4KWi9rDRHspC8gdwUm+ARtVVK9AYlFgWIhpDHMVU33PciudCU8+ujRuI+EAbxdFc3IB1EWiLWCcbM63YIuPgMe0iQc9mmmJGXLGMOsZGRLgZAOBR8J38XfA/e7LXlnJhRXh7RzF+/G/iWT//X61BRy8zcvpMvhrbYG8wA28KpwN6wv8cFbXe9xsDnpe0z8zccXwquG3O/8SoJ1bhiP05EqWutCeUMqOfKmsldYe2q5RZtULLLcXawbH1FDBn3WatPFu+QBTLK1AIgjdcKPqxM7dRu7StnktM1a25l2trYD8GdqAaDP49KQuWdSPlGgz+7ITFdIHcQaBrh4OeUnV4gAh4w/1LtqbupuB4iCBeWGbLs5mDAKF4oq0FtTLL6x0/6Cc0xm7WTHiq7C3T2xQbo1zvxRLV4FjhIpp2ua12sTlCv46453ZNvrXJp5jrfbNz9u7y7PjidHf/w+bHcfByE4gGB4Du6yn7G2WF86ktF/vAeWd6ZSvtTM5pZiA499GwLyD8crbqu+6athHBIrO3V7lm+JX1zQgtYqBwJbzvM6O4F1xR/ZoXcORi4GmRdEUcCWo1wgs2ulYNsow4dCF1xTRD3QSslDIbgPDQdTRUT2zDAj/5lU7PLqLKKXO9tnU7rDkSYx27F7kT3AR6plphpXZy1JpAVnVSoXPjs20yWth+qWVCLREGC2dWyOztmdvbqeLg9aHV4iq/qWOMzaBydXprQk6Kn/uF5n6ICazLdfH7onXfbswoD5IZihtROzTCabiSzpEk7T26CyX40dGjQ81vFA8F1a8JmHzM5smysjHBviuo0418R8m0xYaXDXMMFwhWcVRSp9PV0DExhqyYR+U3n8faVPh0/aj9vOuznp8MdTRbeSr06F9Jgn64gKP62NuTciCs27uQvkYul3OteV3VaK6K9qsDAEE9AXsWVcAkkzTr7WDN0sT2fCIXR7b1B0sUKdeYJRFVCaF6hmm41F2yJITsYGaGzINo2jfJRZwF8agSahCQBgsiEzhy12YzF7uVKJTPm1ysIuPCMIlFYEIGC24pbSlJQR5N6bWCIqh3lK9dUlNomwy5wBcFbQ/TREk8LczEzMsCcSi76b2hTbYe9z6cMYo+9s5XSz0BbWLmoEH2YYngSubFpDsLhudQp4umA+7RUK+D5f3XNdKzArQiPfzrX86/N+JMVW5Z4kzUKVinOZegJvVDBnxIfYSIL5AxUzAfohxLYlxsVNo87V0Aupbo9ypvVvfhqsqXUz825XbZ9Wx2ZteiUJSCBbdh7KeRNTih6FR4DVN8jQKMoAsXjC2/TqOEJRKWAhDL2U4Pv32cthlG1BBUa4QnYOq3Upfo+5794+x8/3Dv8pedo/MzynUZDvMsE6oFnQGcX68sS8LKf72i+LfsDdHNHo/tuQ8CPA/sMhYnuUwLT7BTcA6ZLkutC8bNSS5dj2325AbScZQpBqB3ds8Pjo8mk34YRf0NdXbxhtcNTwBf8GTn/PzUdhNMJl4vwYYCGvtm0rcmwIbaP/r5U23Xj9/UmyGl6fDXl+J2uubDOim3uKUi+05u8V6joeVslyiEU04D7BMMoy5bh0A9NZ71db+ha33p32/NDabIXwjrYbSg2HmegZEgEAPq0nymdoeWY7+urWJbvHz0hOlaDw3e8bSo1EKKXWHX1PBmDQRrFtVN66q2zpu4owSow2zhqrV2Dr7TaukaRNjpSpF3+21dIBQvKs9omutwetuaa9sLwiR6hplScdz6Pa/ykKUL+qkUepLMKjBFlkZ0+QUHYSljY7tfbFrOT9GOBZqrnHXhEmoWDbFWmMpn/QR6KkxYrZTbRWx17guOzki3MgonzEHWMoXl0pZ7SFuFYbPKE/go5QVcu4iE7UIMXnGskZom0Uq4x74NUvAiaOLFQkcxbERg2ZgSN4tZZKF4cspcbjKxFQDsS9brueCdwJNtUBCJq4NfeCfUSDyy0U69eqZ1z0X7mgqpx0RgXMEkJ1khvsqGUKEQt7nXU5N5dqvTsbuqxi3bUZlbACx1hVbPEuOIGKYbWsLvylrUyB7GdCSMGvW7rRCXY1Ofkvh4rPzqqNuVykxXDWSk8qBOk155iRdK7NehrBUeFznlhiYkxppdKblECA0LqiqiGLkIqLClt06C/8vO6dHB0U/j9lUr3t0uXrWnd99GtXmC5JKRa02JIaVoQlSWggJKhLuztbFaWqdbJUGuhzPG9Wti2utuLdE2j+wsl0hhWS2ki0IdyRiIlkOnjEUd1bSCzxAE69pm8eu3L0nzglQ+E7v5DURiMlXeFRXHSKcU6uVFdIllOaXpqOHFly50JfGl7sjLPIz8YA+N6wggoodWH7rlFBtH/XyD2FdoTof4A9oxwLphof6g5vb6WTyNKwvzzITuhXkDOhZW8w/+kE1TWxFdaOaV1EZ5xgztqt6QksdMDzYDg3E2Y7t5iSru8V4P6lgE40Oa0W0fFwsNqrlh5m/RVDdFech6JlTj8NUY00ZZV5jE+567jDrbkXosze2VLnQzF4k6w/myZh1yJdC49UeuW0mVFYVSHUcJNy5PcvKuSiJyBZzMKoKSzRS5MyCcTvEqG0E8Olh0QFoUkzMEvNpRromLUyuCMM8waheDv2CBPc2WMV9kQaGfVc1y2bBb5vI/vxWhxpJerwZyazvD7x9ml/YSy7Fbe6EHS1fvFHLZFGxTSJmFIWC+bwvmn1tc08pu5db/gPZj1voSifhnpwhkon9DVoAh15ICXpOoXiqt7tiuohvMqO+KMpwFeKTJZBMVB74AYe76atHgGrPKdQm3lLQo837O1RCZQWmmMaPgHxPsFg4ooewe1mtjFf3kbHdr8/utqpGhn5mxVesv14YtgECStmnq5HT//Pwfl0c7hyBUB8qmG/1CqIYgf6aTlYGyCBxzMST1sfZbrRb+oJ03Z/tH52oNjY6Dny7fHZ+fvL/4Ce+LoTtj6JpKYDdhvFSsD7GReGVPiUxVbZVDiDRhaxwVJzRH1apO+94kNe1KOHaTuCubxnU1qiSA3xr19CS+pmxMArxoCag2rj5JNerJSZpZxo/VJNYRHau211pN4o1qTNJTxAiTp/LHXukr/v3YzCdXRXJVA6Gf//xYQ4lnZdYH2iKDQWXnejjyPuPlu4sLbG1CeAcCF3ls0H8q/+wRX6Ng5+MTC2wMxKu66iD7X7TExldda1338Kgq5qsB9co0EE7fS2E/1U3+BY3kwuX4ZixE2NlU7vrJv6yV3Afb/1wrNAvhl2tr9IvX1vlZQaZYnk/6Ir+dN1brrX2iZ9rr9Wv32VIJlgB/c3xxtNffBk9ah7fb1ItQa/6UVCqNdo3ctBwGZJUL7FB+9fboeYwyXToKa6EevcRLYuy2tGzKuwam3i7MV7I8dyOLdyeLXMfy9G0s9j6Wz1zF4sDW7zz58itP5OPWjSYyPz7kGkpeBp2j5dzlhO9XgX9ojOrXm3OlyB1Ey1fB1rcG/v3OIFrHCsve+x487xQJ5FeKlmET8V+oWE25WISYk0SZLCeOxEfkXTWuWlrw/JEOBewBE5+F70GQeLrtkLXN/NyO6zpX8tjc9l2vZGnAdPe6I3/44RUnes52fqLY9aveH7iO+sn7nBq3UBPmxFP4UA82fKQu7GWSrYAjQoofgPdGd5xKtIBDBcgq1b2VrZ+5n6QxHFcFnFh8URiBEiYUYCDkYKSdjEbV9YNdJOxNUlQNF4g1a9RWVPmVWHNGR0irI0f1w5u9t8Ph8KMPDIgH21IoHyXAFnzdEF+p8oc+ct1w/MxseNfpYiz3C+5mxFFfdCMj0qDr5u6IxlhAX35PIwbrbZUkBX4M5iGNBfTMhY0OMX5AdYOzS1UqDl4CDAvu+QsdJZqGl2YOuOhmWe9laXhUFupnb3xEqNxIMqDg9oa6i8OqPrbHiBQXvR7/cw67DdUbv70dW1SaMeUehrG5hJCjw1dPNENdcZBjWyK10iJkixyoRyguethZyBfZptS442cqLJPVwpX1FMWw1zuoAvjtqL0t2MXsbcoXx3BMYob3KVGTgMYOUj7Xce/palqq7/dCnkSnHMZBr2PYaFeiuIaXKOnKi4gT5BWfepkQindU6ZBeZzqEIiNfkupg4S25ix6GCOhGlar6yQVJAaN1kSmXnttAqHf9LO4ZC2P9aHCdf3t1/t2gTjyM5n72ZnDJl3PDFSYRZO/eEeMS4Ou10LvBfGDlInViwF5cPDyqLiaWaK4tt+VO9o4biEneukuKo17HdcTcc7n2hZcQD4Zqj5UP7ujzV/p+5mZgf3tc7s73W7rOVMoqoZq8b/C4bUE9h7PieBYaHFXGBy81zBI9enO4i3E1rAAgUmd1T7qZBYy9mMc2rDuqwdoSuSFSWgfkAO+RWKyvPsaA4+ckMQrdXqUR4ACvnu6Vrl2WXrc1/vwIU896qlt4ay07qrKQgS2r3FZeFSlf1kl+lLMuBv64yuaoRrLZMFB1iGJM+ACR8P1hfjJhuytv4X3NlsEXft34Npj/HgRIzb/TP/3+QJHhx2/B1bSBB65IlcpKLkX98evX2/yefuPv+CE2hbxGGORy/h9DOpsix2QAAA==
          mode: 493
          overwrite: true
          path: /usr/local/sbin/rbln-pciehp-hook.sh
        - contents:
            compression: gzip
            source: data:;base64,H4sIAAAAAAAC/1SSwWrbTBSF93qKg/zDv4hs2m2KoG5i2i6cQKy2dJWMZ449Q6Q76sxIjgl59zKSKWQ76J77ffdoga0SdaSBilDYKm2d8MbLwR3hBQ/fbu53nxBJPGzWt9vNqjNwgmRdhHGBOvlwXhWLIiclbWnyWLIExfTeSaogPuVHlyKMP0lMgapD70O6xmA4QivJ3yTGBFUsYDg6zf8jtHWtCZQKSsyUmqfgIihDx6ASDfY8+MB3O6HkfFIXrsYSJx+e85hVYmiQPBTiOSZ2BoO4hKCSZUCyShAGgWXg9ZQ5yN6JATuXYrFAr90jX/rAGBHY+ZEYOFJSnCHzhtbJEcn6SByC7+AkOkOo2TazMEyayrReP0eoo3IS0+yQw/Bn4MAVsPu9azbb28df67tmlw2GyKmBvlWa8IdigaeLyTIM8oTo/5nlY0+ZPhiGDLVc5jbUvj1P750fZN5q2fYTVOtGRnipcLJO2xnZ5+OcXCT2VCnm+01lDKLtqljfNN/v7+q6VMaUFXY/vszUdV322pUV1k3z8DpSjA9vdV1+ePnIw6Gs0Ky/XtXlhbassLn7+fpO+O2qLsO+lWWvHW2/nLv4/N8zg7BdRYb8o5TF3wAAAP//FQW6LMcCAAA=
          mode: 420
          overwrite: true
          path: /etc/udev/rules.d/99-rbln-pciehp.rules
    systemd:
      units:
        - contents: |
            # Managed as a MachineConfig on RHCOS; see README.md in this directory.
            #
            # RequiresMountsFor= is spelled out with the resolved directory because it does
            # not chase symlinks: on an ostree host /usr/local is a symlink to /var/usrlocal
            # and /var is a separate mount.  install resolves this at run time; a static
            # config has to state it.
            [Unit]
            Description=Unbind pciehp from the PCIe downstream port hosting %I
            DefaultDependencies=no
            After=systemd-udevd.service
            # udev coldplug for PCI runs before local-fs.target.  On an ostree host the
            # helper lives under /var, so without this the unit would fail to exec and the
            # boot-time unbind would silently never happen.
            RequiresMountsFor=/var/usrlocal/sbin
            Conflicts=shutdown.target
            Before=shutdown.target

            [Service]
            Type=oneshot
            ExecStart=/usr/local/sbin/rbln-pciehp-hook.sh unbind %I
          name: rbln-pciehp-unbind@.service

적용¶

Warning

적용하면 롤링 재부팅이 발생합니다. MCO가 풀의 노드를 하나씩 drain하고 파일을 쓴 뒤 재부팅합니다. udev rule이 적용되려면 재부팅이 필요합니다.

oc apply -f 99-rbln-pciehp.yaml

# UPDATED=True, UPDATING=False, DEGRADED=False가 될 때까지 대기
oc get mcp __MCP_NAME__ -w

풀이 UPDATED=True, UPDATING=False, DEGRADED=False를 보고하면 롤아웃이 끝난 것입니다.

검증¶

# NPU가 물린 포트가 모두 unbound로 보고되는지
oc debug node/<NODE_NAME> -- chroot /host /usr/local/sbin/rbln-pciehp-hook.sh status

# 헬퍼의 SELinux 라벨
oc debug node/<NODE_NAME> -- chroot /host ls -Z /usr/local/sbin/rbln-pciehp-hook.sh

# rule이 매칭되어 유닛을 호출하는지
oc debug node/<NODE_NAME> -- chroot /host udevadm test /sys/bus/pci/devices/<BDF>

# 포트별 유닛 실행 기록
oc debug node/<NODE_NAME> -- chroot /host journalctl -u 'rbln-pciehp-unbind@*'

# SELinux denial 없음
oc debug node/<NODE_NAME> -- chroot /host ausearch -m AVC -ts recent

훅 스크립트의 SELinux 타입은 bin_t여야 하고, ausearch에는 거부 기록이 없어야 합니다.

VM 게스트에서는 status가 포트를 찾고 rc=0으로 끝나지만 BOUND → unbound 상태 변화는 나타나지 않습니다. 이 상태 변화는 베어메탈 노드에서만 확인할 수 있습니다.

롤백¶

배포된 파일은 MCO가 관리하므로 rbln-pciehp-hook.sh uninstall을 실행해서는 안 됩니다. 롤백하려면 MachineConfig를 삭제합니다. 삭제 후 대상 풀의 노드가 순차적으로 재부팅됩니다.

oc delete -f 99-rbln-pciehp.yaml

# UPDATED=True, UPDATING=False, DEGRADED=False가 될 때까지 대기
oc get mcp __MCP_NAME__ -w

참고¶

Warning

systemd 유닛은 systemd.units로 선언합니다. storage.files로 /etc/systemd/system 아래에 파일을 배포하면 MCO가 선언되지 않은 유닛으로 보고 삭제하므로, 노드가 선언한 설정 상태에 도달하지 못합니다.

유닛의 RequiresMountsFor=에는 /usr/local/sbin 대신 실제 경로인 /var/usrlocal/sbin을 적습니다. RequiresMountsFor=는 심링크를 따라가지 않는데, ostree 호스트에서 /usr/local은 /var/usrlocal의 심링크이고 /var는 별도 마운트입니다. PCI coldplug를 처리하는 udev는 local-fs.target보다 먼저 실행되므로, 심링크를 해석한 실제 경로를 적어두지 않으면 유닛이 실행되지 않아 부팅 시 pciehp 분리가 수행되지 않습니다. 노드에서 심링크를 확인하려면 다음을 실행합니다.

oc debug node/<NODE_NAME> -- chroot /host readlink -f /usr/local/sbin