Troubleshoot¶

Runtime creation fails after compilation¶

1
2
3
4
5

Failed to create RBLN runtime: ...

If you only need to compile the model without loading it to NPU, you can use:
  from_pretrained(..., rbln_create_runtimes=False) or
  from_pretrained(..., rbln_config={..., 'create_runtimes': False})

from_pretrained creates runtimes on the NPU immediately after compilation. Device memory exhaustion is not the only possible cause, so inspect the original exception at the top of the message first. To skip runtime creation:

1
2

model = RBLNModel.from_pretrained(model_id, rbln_create_runtimes=False)
model.save_pretrained(save_dir)

Load the saved artifacts separately:

1

model = RBLNModel.from_pretrained(save_dir)

Device / num_devices configuration errors¶

Validation of device and num_devices may raise any of these errors:

1

Device {device_id} is not a valid NPU device. Please check your NPU status with 'rbln-smi' command.

A device ID that does not exist on the host. When device is not set, it defaults to 0 … num_devices - 1, so asking for more NPUs than the host has also ends here, at the first missing ID.

1

The number of devices ({len_device}) does not match `num_devices` ({num_devices}).

The device list and num_devices disagree in length.

1

`num_devices` ({N}) is greater than the number of available devices {M}.

Every listed ID exists, but there are fewer NPUs than num_devices, which happens only when the device list repeats an ID (for example device=[0, 0] on a host with one NPU).

Run rbln-smi to list available devices, then verify the following:

• The specified device IDs exist on the system.
• num_devices ≤ the number of available devices.
• The device list length equals num_devices.

Flash attention configuration errors¶

Flash attention is enabled by setting attn_impl="flash_attn" or kvcache_partition_len. Compilation fails when any of the following constraints is violated:

• max_seq_len must be a multiple of kvcache_partition_len and at least 2 × kvcache_partition_len.
• kvcache_partition_len must be within the range of the target NPU, so max_seq_len is at least 2,048 on every NPU.

1

`max_seq_len` ({X}) must be a multiple of `kvcache_partition_len` ({Y}) when using 'flash_attn'. Please adjust either value to meet this requirement.

The limits follow the target NPU: npu in rbln_config when it is set, otherwise the NPU attached to the host. On a host with no NPU and no npu set, the ATOM™ limits apply, so set npu (for example rbln_npu="RBLN-CR13") when compiling for Rebel100™ there.

1

`kvcache_partition_len` ({X}) is out of the REBEL supported range for 'flash_attn' (1024 <= `kvcache_partition_len` <= 16384). Please provide a valid value within this range.

1

`kvcache_partition_len` ({X}) is out of the ATOM supported range for 'flash_attn' (1024 <= `kvcache_partition_len` <= 32768). Please provide a valid value within this range.

Logging and debugging¶

Control log verbosity with the OPTIMUM_RBLN_VERBOSE environment variable (default info):

1
2

$ OPTIMUM_RBLN_VERBOSE=debug python inference.py    # detailed logs
$ OPTIMUM_RBLN_VERBOSE=warning python inference.py  # warnings and errors only

Supported levels: debug, info, warning, error, critical.