set 100G memory but only 66G in centos 6.8/6.9 about lis-next HOT 22 CLOSED

It looks the built-in kernels of CentOS 6.8/6.9 are not so resilient to the BIOS defect. Good to know you have a workaround by installing the 4.x kernel.

No easy way to copy text from the VM terminal to the host. You can try SSH or VNC. :-)

from lis-next.

Fair enough. We can look into adding the "disable_mtrr_trim" only for RH6.X installations.

@vyadavmsft; can you look into adding this for the RHEL 6.X RPMs? I will see what we can do in the rhel6.x install script for folks playing with the git sources.

from lis-next.

Did you check "Enable Dynamic Memory" in the Hyper-V "Setting -> Memory" for the VM?

What's the output of "free -m" and "cat /proc/meminfo" in the VM?

from lis-next.

sorry, I download the wrong ubuntu iso, i386 version.

from lis-next.

Ah.. yes, 64GB -- that's the max addressable physical address for 32-PAE

from lis-next.

I try ubuntu 14.04.05 x64, it show 98G, without lis-next.

so it look like a bug at centos 6.9 kernel?? 2.6.32-696

from lis-next.

after I install lis-next from source code, centos 6.9 fail to start after boot menu...

from lis-next.

centos 6.8 x64 livecd show this, 66G memory:

any idea for that ?? thank you a lot

from lis-next.

centos 7.3 1611, show 98G

from lis-next.

I reproduced the issue with CentOS 6.8/6.9. Note the line "WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM.":

BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000f7ff0000 (usable)
BIOS-e820: 00000000f7ff0000 - 00000000f7fff000 (ACPI data)
BIOS-e820: 00000000f7fff000 - 00000000f8000000 (ACPI NVS)
BIOS-e820: 0000000100000000 - 0000000fe0000000 (usable)
BIOS-e820: 0000001000000000 - 0000001928000000 (usable)
SMBIOS version 2.3 @ 0xF57D0
SMBIOS 2.3 present.
DMI: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090006 04/28/2016
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
Hypervisor detected: Microsoft HyperV
HyperV: features 0x2e7f, hints 0xc2c
HyperV: LAPIC Timer Frequency: 0x30d40
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x1928000 max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
e820 update range: 0000001100000000 - 0000001928000000 (usable) ==> (reserved)
WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM.
------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/mtrr/cleanup.c:972 mtrr_trim_uncached_memory+0x2de/0x309() (Not tainted)
Hardware name: Virtual Machine
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-696.el6.x86_64 #1
Call Trace:
[] ? warn_slowpath_common+0x91/0xe0
[] ? warn_slowpath_null+0x1a/0x20
[] ? mtrr_trim_uncached_memory+0x2de/0x309
[] ? setup_arch+0x4e7/0xca6
[] ? vprintk_default+0xe/0x10
[] ? printk+0x4f/0x58
[] ? start_kernel+0xdc/0x436
[] ? x86_64_start_reservations+0x125/0x129
[] ? x86_64_start_kernel+0x115/0x124
---[ end trace a7919e7f17c0a725 ]---
update e820 for mtrr
modified physical RAM map:
modified: 0000000000000000 - 0000000000010000 (reserved)
modified: 0000000000010000 - 000000000009fc00 (usable)
modified: 000000000009fc00 - 00000000000a0000 (reserved)
modified: 00000000000e0000 - 0000000000100000 (reserved)
modified: 0000000000100000 - 00000000f7ff0000 (usable)
modified: 00000000f7ff0000 - 00000000f7fff000 (ACPI data)
modified: 00000000f7fff000 - 00000000f8000000 (ACPI NVS)
modified: 0000000100000000 - 0000000fe0000000 (usable)
modified: 0000001000000000 - 0000001100000000 (usable)
modified: 0000001100000000 - 0000001928000000 (reserved)
last_pfn = 0x1100000 max_arch_pfn = 0x400000000
last_pfn = 0xf7ff0 max_arch_pfn = 0x400000000

from lis-next.

So the question is: why are CentOS 7.3 and Ubuntu 14.04.5 able to show ~100GB rather than 60+ GB...

from lis-next.

after upgrade kernel by http://elrepo.reloumirrors.net/kernel/el6/x86_64/RPMS/kernel-ml-4.10.12-1.el6.elrepo.x86_64.rpm

it works, 98G show in free -m

from lis-next.

after startup, centos 6.9 with kernel 4.10.12

from lis-next.

is there any way to copy text from hyper-v guest?

dmesg show this at centos with kernel 4.10.12 :

from lis-next.

ok, thank you a lot ^_^

from lis-next.

BTW, is it possible a bug in hyper-v, not centos 6.8/6.9?

at the same win10 x64, centos 6.9 VM in virtualbox 5.1.20 can access full memory size, but it is very slow after ~60G, about 20mb/s, but still working.

from lis-next.

Yeah, I think there is a bug in Hyper-V's guest BIOS: "WARNING: BIOS bug: CPU MTRRs don't cover all of memory, losing 33408MB of RAM."

I just don't know how the 4.x kernel can cope with this.

from lis-next.

add a topic at https://social.technet.microsoft.com/Forums/en-US/a92bd5c8-f47e-41b1-86c9-d5b3426f286b/cpu-mtrrs-dont-cover-all-of-memory-losing-33408mb-of-ram?forum=win10itprovirt

from lis-next.

Old RHEL 6.x kernel has a bug, which can be exposed when the VM is running on WS 2016, where the reported max supported physical address has 44 bits. On WS 2012 R2, the max physical address has only 42 bits, and the bug is not triggered.

The old kernel needs to pick the patch:
x86: Fix /proc/mtrr with base/size more than 44bits (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d5c78673b1b28467354c2c30c3d4f003666ff385)

or, we can use the "disable_mtrr_trim" kernel parameter to work around the issue (there is not perf issue with this, because Hyper-V always ignores the MTRR setting for conventional RAM of a VM).

from lis-next.

Great!

from lis-next.

@wendal @dcui Is there any action item here with respect to LIS? Can we close this issue?

from lis-next.

from lis-next.