The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,953 other followers

Archive for the ‘NVMe’ Category

Samsung 980 Pro NVMe SSD serial in ESXi is in different byte order than the sticker

Posted by jpluimers on 2022/02/03

I installed three Samsung 980 Pro NVMe SSD devices in various ESXi rigs.

This is the serial numbers that ESXi came up with:

  1. Local NVMe Disk (t10.NVMe____Samsung_SSD_980_PRO_1TB_________________E824B311B1382500)
  2. Local NVMe Disk (t10.NVMe____Samsung_SSD_980_PRO_1TB_________________782DB311B1382500)
  3. Local NVMe Disk (t10.NVMe____Samsung_SSD_980_PRO_1TB_________________6F2DB311B1382500)

ESXi presents the serial number (actually the EUI64) in reverse byte order of what is on the device labels.

I had to look up what EUI64 was, and it is a kind of UUID (see Universally unique identifier – Wikipedia), but for hardware devices:

  • World Wide Name – WikipediaEach WWN is an 8- or 16-byte number, the length and format of which is determined by the most significant four bits, which are referred to as an NAA (Network Address Authority). The remainder of the value is derived from an IEEE OUI (or from Company Id (CID)) and vendor-supplied information. Each format defines a different way to arrange and/or interpret these components. OUIs are used with the U/L and multicast bits zeroed, or sometimes even omitted (and assumed zero). Though CID has U/L set to 1.The WWN formats include:
    • “Mapped EUI-64” formats manage to fit an EUI-64 address into an 8-byte WWN. Since the NAA is mandatory, and takes up a nibble, this represents a four-bit deficit. These four bits are recouped through the following tricks: First, two bits are stolen from the NAA by allocating NAAs 12, 13, 14, and 15 to all refer to the same format. Second, the remaining two bits are recouped by omitting the U/L and multicast bits from the EUI-64’s OUI. When reconstructing the embedded EUI-64 value, the U/L and multicast bits are assumed to have carried zero values.

  • [Wayback] Base NVM Express – Part One – NVM Express

    A namespace ID (NSID) is an identifier used by a controller to provide access to a namespace (handle to a namespace). An NVMe controller may support multiple namespaces that are referenced using NSID. EUI64 (8 bytes), NGUID (16 bytes) and UUID (128-bit) are globally unique namespace identifiers defined in the Base Specification.

  • [Wayback] VMware Docs: NVMe Devices with NGUID Device Identifiers

    For NVMe devices, ESXi generates device identifiers based on the information it retrieves from the devices. Generally, the NVMe devices support identifiers in EUI64 or NGUID formats, or use both formats. NGUID is a Namespace Globally Unique Identifier that uses the EUI64 16-byte designator format.

  • [Wayback] ESXI6.7 nvme ssd issue – VMware Technology Network VMTN

    Being a software engineer, I am disturbing by the way this software create the disk ID.

    According to the following table from VMware Docs we have some case that the software will not recognize the disk or lost disks.

    ID Formats Supported by Device Device Identifier Generated by Host
    EUI64 ID Format NGUID ID Format ESXi 6.7 and earlier ESXi 6.7 Update 2
    yes yes t10.xxx_EUI64 t10.xxx_EUI64
    yes no t10.xxx_EUI64 t10.xxx_EUI64
    no yes t10.xxx_controller_serial_number eui.xxx (NGUID) as primary ID

    t10.xxx_controller_serial_number as alternative primary ID

    so what if

    1. The NVMe SSD from the same company may use the same EUI64 for every NVMe SSD on the same interface (say using an ASUS Hyper M.2 X16 PCIe 3.0×4 Expansion card with 4 identical NVMe SSD). It is possible because of bad design from the SSD manufacturer, the EUI64 are all the same for the 4 NVMe SSD, under this case, the ESXI will only recognized one of the 4 disks (since the disk will be t10.xxx_EUI64, and EUI64 are the same for all 4 NVMe SSD, The “storage” “adapters” tab did show there are 4 interfaces (adapters), but the “stroage” “devices” will only show 1 disks.
    2. Why there is no “no, no” options?

    I understand that the NVMe standard 1.3 (for ESXi 6.7) is the base of how the NVMe SSD should be designed, but I think the software should be smart enough to cover the mistakes the hardware company may make and so it can recognized most of the NVMe SSD that is available from the market.

    I have the same issue in V7.0! Whatever happened to using something you know is unique like serial numbers or something based off the serial number!

    My two identical NVMe drives show up as one! Very odd how this one got through testing!

    Having same issue with the Asus hyper x16 m. 2 pcie card. I have 4 Intel 660p in it and it’s only showing 1. Even when I changed the bifurcation to x4x4x4x4 for that pcie slot. It actually show nothing. When in auto it shows 1. I’m using esxi 7 though. Trying to test out VMware horizon but esxi not detecting all the nvme in the Asus adapter.

  • [Wayback/Archive.is] linux.kernel: [PATCH 0/7] Implement NVMe Namespace Descriptor Identification

    This patchset implemets NVMe Namespace Descriptor Identification as of
    NVMe 1.3. The Namespace Descriptor Identification allows a NVMe host
    to query several Namespace Identification mechanisms, such as EUI-64,
    NGUID and UUID from the target. If more than one value is set by the
    target, it can transmit all set values to the host.

  • [Wayback] OS-6042: Need to handle NVMe devices with EUI64 values (SmartOS + ZFS)

    NVMe devices with namespaces with an EUI64 value do not attach to the system. It’d be good if these did.

  • [Wayback] ⚙ D19905 bhyve: Add EUI64 to NVMe device (FreeBSD)

    Add the EUI64 field (part of the Identify Namespace data) to NVMe devices to support UEFI drivers.

    The implementation will accept an IEEE Extended Unique Identifier (EUI-64) from the command line. If one isn’t provided, it will create one based on

    • The IEEE OUI reported in the Identify Controller data
    • The PCI bus, device/slot, function values
    • The Namespace ID
  • [Wayback] nvme-scsi: Use correct byte ordering for eui64 in Dev ID VPD – Patchwork
    NVME specifies an EUI64/NGUID in little-endian format, while SCSI
    specifies that the Device Identification VPD use big-endian for EUI
    formats. The current code copies this bytestream directly from the
    Identification Namespace page, meaning we just need to reverse the
    bytestream when passing it on to the VPD.

    This seems to hold true for NGUID devices, but Keith just pointed out to
    me that it may not hold true for EUI64 devices. It seems like that case
    needs byte swiveling within each field. So I'll NAK for now until I can
    figure out if that's the case.

    This will break existing setups that rely on VPD 0x83 for device
    identification (which I think includes older SuSE distros).
    
    And once you change the setup anyway please stop using this buggy
    SCSI emulation.

  • [Wayback] A Quick Tour of NVM Express (NVMe)

    • nguid, Namespace Globally Unique Identifier (NGUID) and, eui64, IEEE Extended Unique Identifier (EUI64) are assigned when the namespace is created and preserved across namespace and controller operations (e.g. reset, format).

These are pictures of the devices in the same order:

Read the rest of this entry »

Posted in ESXi6.5, ESXi6.7, Hardware, NVMe, Power User, SSD, Virtualization, VMware, VMware ESXi | Leave a Comment »

M.2 SSD PCIe 3.0×4 vs 2.0×4 | Tom’s Hardware Forum

Posted by jpluimers on 2021/10/27

I wanted to know the influence of PCIe versions to NVMe support, and found [Archive.is] M.2 SSD PCIE 3.0×4 vs 2.0×4 | Tom’s Hardware Forum answering:

You already know that the Z97 motherboard has one M.2 port. The problem is there are several M.2 variations. Here are the current possibilities:

M.2 3.0 x4 – State of the art M.2 SSD that uses 4 PCIe 3.0 channels for proper operation. The M.2 port on the motherboard is sometimes referred to as an Ultra M.2 port. It is the appropriate port for the Samsung 950 Pro SSD.

There are other M.2 SSDs that either use PCIe 2.0 or use fewer PCIe channels:

  • M.2 2.0 x4
  • M.2 2.0 x2
  • M.2 SATA 3

You will need to check your motherboard manual or the company web site to determine if the M.2 port can support an M.2 3.0 x4 SSD like the 950 Pro. A lot of the Z87 and Z97 motherboards had M.2 ports for M.2 2.0 x2 SSDs that would not fully support an M.2 3.0 x4 SSD. With the exception of ASRock, the other motherboard manufacturers did not do a very good job of fully identifying the M.2 ports. You will have to find a little more information about the M.2 port on your motherboard. Hopefully the information is in the owner’s manual or the manufacturer’s web site.

The addition of M.2 ports on the the motherboard required the use of additional PCIe channels. Unfortunately Intel resisted adding chipset support for additional PCIe channels until recently. The lack of a sufficient number of PCIe channels resulted in some awkward solutions:

  • Some motherboards reduced the number of channels available to graphic cards. The cards might be reduced from PCIe 3.0 x16 down to PCIe 3.0 x8. Graphic card performance is reduced by about 5%. That is not a terrible hit in performance.
  • Some motherboards reduce the number of SATA 3 ports that are available. For example,the MSI Z107 Titanium motherboard has two M.2 3.0 x4 ports. If I install a 950 Pro in each port, then all of the standard SATA ports are disabled except for two ports. The M.2 ports do not reduce the performance of graphic cards
  • Some high end motherboards add an expensive PLX chip to handle M.2 SSDs. An example would be the ASRock Z97 Extreme9 motherboard. The PLX provides direct data transmission between the M.2 SSDs and the cpu. It is actually possible to run two graphic cards in SLI at PCIe 3.0 x16 each and still be able to run two M.2 SSDs.

The Samsung 950 Pro uses the new NVMe protocol instead of AHCI. A motherboard would have to have a UEFI BIOS, an Intel chipset, and a Microsoft Windows operating system that support the NVMe protocol. You would have to do the following:

  • Check for any BIOS updates. Sometimes the updates include new and improved support for the NVMe protocol.
  • Check for any Intel chipset updates. Sometimes the updates include new and improved support for the NVMe protocol.
  • Windows 7, 8, and 10 can support the NVMe protocol. Again you will have to check for updates or fixes that can improve NVMe support. NVMe is new so things could get a little complicated.
  • Finally Samsung has issued their own NVMe driver for the 950 Pro. Reviews and articles indicate the Samsung NVMe driver is better than the Windows NVme drivers.

Some modern motherboards still provide a PCIe 2.0 slot. That does not mean you should purchase a PCIE 2.0 SSD or an M.2 SSD with an adapter card. M.2 3.0 x4 SSDs are backwards compatible.

It might be confusing but for all practical purposes it is just a matter of checking requirements and specifications.

Note I have bulleted some entries for readability and bolded some text relevant to some motherboards I still have running and fixed some typos.

On connectors and B/M keying:

Read the rest of this entry »

Posted in Hardware, NVMe, Power User, SSD | Leave a Comment »

Windows and the current state of S.M.A.R.T. tooling that understands NVMe

Posted by jpluimers on 2021/09/16

I had trouble with two Intel 600p NVMe SSD devices: read-errors.

It appeared only few tools understand how to get S.M.A.R.T. health information from them, and even then they did not explain the read errors.

I’m going to RMA them, but in case anyone else needs to get health information from NVMe SSD devices, here is which tools do what:

So basically, CrystalDiskInfo and HD Tune are my first line of checking for drive issues, followed by smartmontools to get text output, then by vendor specific tools to assist with the RMA.

In the past, I used another smartmontools wrapper, but it was discontinued and had an even older version than GSmartControl: Source: Closed: HDD Guardian – Home.

On Intel 600p becoming locked in read-only mode after failure:

Start of Intel RMA procedure via [Wayback] Warranty Information.

My case looks remarkably similar to [Wayback] Full Diagnostic Scan always fails during Read Scan on my SSD 600p Series 256GB – Intel Community.

A few screenshots of the tools I used for health information:

Read the rest of this entry »

Posted in Hardware, NVMe, Power User, SSD, WayBack machine | Leave a Comment »

Intel SSD 600p Series NVMe SSD Temperatures: ensure you cool it properly even for moderate use

Posted by jpluimers on 2018/04/30

A short while ago, I installed a pair of 1TB Intel 600p NVMe SSD devices.

Performing a straight rsync backup of some 100 gigabyte of data managed to shutdown both of them: the throttling was so much that VMware ESXi lost the volumes on it and showed “Permanently inaccessible device”.

I was quite surprised as the initial reviews did not mention any temperature problems, but apparently there are.

Since then I’ve put a 120mm fan near to it which made the problems go away, but it still every now and then jumped to over 60 degrees Celsius.

Later I “fixed” it in a better way for cooling getting the idle temperature around 30 degrees Celsius and heavy duty temperatures around 40 degrees Celsius:

I did not have to mount any M.2 heatsinks on them yet; they probably won’t fit under the fan anyway

Related links and products:

Related:

–jeroen

Posted in Hardware, NVMe, Power User, SSD | 1 Comment »

 
%d bloggers like this: