Symptom

When the 2288H V5 is configured with Tesla A100 40 GB, the MMIOH resources are insufficient after the lspci -vvv -s b9:00.0 command is executed on the Linux OS. That is, the command output contains "Region 1: Memory at <unassigned> (64-bit, prefetchable)", as shown in the following figure.

b9:00.0 is the bus address of Tesla A100 40 GB on the OS. The bus address may vary depending on hardware configurations.

Solution

On the BIOS setup screen, set MMIO High Granularity Size to 256G or larger. The procedure is as follows:

1. Restart the server and go to the BIOS setup screen.
2. Choose Advanced > Socket Configuration > Common RefCode Configuration.

3. Choose MMIO High Granularity Size and press Enter. Select 256G or larger and press Enter.

4. Press F10.

   The Save Changes&Exit dialog box is displayed.

5. Select Yes and press Enter.

   The server automatically restarts for the settings to take effect.

6. Run the lspci -vvv -s b9:00.0 command on the Linux OS again. In the command output, region 1 can properly allocate address space. In the following example, "Region 1: Memory at <eb000000000> (64-bit, prefetchable) [size=64G]" is displayed, where eb000000000 indicates the start address allocated by the MMIO, size=64G indicates the size of the address space allocated by the MMIO.