The Wiert Corner – irregular stream of stuff

Jeroen W. Pluimers on .NET, C#, Delphi, databases, and personal interests

  • My badges

  • Twitter Updates

  • My Flickr Stream

  • Pages

  • All categories

  • Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,364 other followers

Archive for the ‘IPMI’ Category

MacOS IPMIView and Supermicro X9SRi-F web interface cannot mount ISOs; on Windows, the web started Java iKVM console can

Posted by jpluimers on 2021/05/07

Screenshots so I do not forget.

Works from Windows Java iKVM Viewer v1.69.21 started via the web interface:

C:\Users\jeroenp>java -version
java version "1.8.0_211"
Java(TM) SE Runtime Environment (build 1.8.0_211-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode)

Fails from web interface:

It does not matter if the share is iso$ or iso: both fail in the same way.

Note this was after working around this very confusion error message from the web interface:

The image path is invalid or contain /:*?"<>|

The error message is just telling that the path should start with a back-slash:

[WayBack] Supermicro IPMI and Windows Share – Server Fault

With HTML5 iKVM this seems to work fine, but that requires more recent motherboards:

[WayBack] How to map a network share to boot from ISO, moving Supermicro Java iKVM to browser-only HTML5 iKVM | TinkerTry IT @ Home

Fails on MacOs with Java iKVM Viewer v1.69 r14.

# java -version
java version "1.8.0_74"
Java(TM) SE Runtime Environment (build 1.8.0_74-b02)
Java HotSpot(TM) 64-Bit Server VM (build 25.74-b02, mixed mode)

Gets you the rainbow circle of death and an “application not responding”

The success and failure are the same no matter the firmware; the same results were obtained both before and after upgrading the IPMI firmware:

–jeroen


PS: from Twitter

 

Posted in Hardware, IPMI, Mainboards, Power User, SuperMicro, X9SRi-F | Leave a Comment »

Need to research alternative IPMI tooling

Posted by jpluimers on 2021/05/07

Since IPMIView cannot mount ISOs and some tasks are not that convenient in GUI tools, I want to look more deeply in console based IPMI tooling.

Though my machine to remotely manage stuff is a MacOS based machine, most of my sites have Raspberry Pi devices, so some compatibility there would be nice. imputil seems to work there given IPMI firmware is recent:

Here are some names for future research:

  • impmiutil
    • [WayBack] ipmiutil – IPMI Management Utilities

      IPMI Management Utilities Project provides a series of common utilities for IPMI server management locally or via LAN. The utilities provide source and scriptable command binaries for automating server management functions.

      IPMIUTIL performs a series of common IPMI server management functions to allow administrators to perform management functions without a learning curve. It can gather FRU inventory data, SEL firmware log, sensors, watchdog, power control, health, monitoring, and has an SOL console. It can write sensor thresholds, FRU asset tags, and supports a full IPMI configuration save/restore. Ease-of-use examples with IPMIUTIL:

      • Native builds on many OSs: Linux, Windows, BSD, Solaris, MacOSX, HPUX
      • The only IPMI project natively supporting Windows drivers
      • IPMIUTIL supports any IPMI-compliant vendor firmware
      • IPMIUTIL detects and handles OEM-specific IPMI firmware variants
      • IPMIUTIL interprets various vendor OEM-specific sensor values automatically.
      • Any IPMI values not yet recognized at least return the values, rather than just ‘na’ or Unknown.
      • Shared library for custom applications, sample source included
      • IPMIUTIL BSD license is compatible with open-source or commercial use
      • Linux driverless support is ideal for boot media or embedded
      • Configuring the IPMI LAN requires just a single command
      • The decoding of IPMI SEL events includes a severity.
      • A soft-reboot can be easily performed instead of a hard reboot by using ipmiutil_asy, even if the platform does not support ACPI.
      • IPMI SEL management is automated via checksel.
      • Watchdog management is automated via ipmiutil_wdt.
    • [WayBack] homebrew-core/ipmiutil.rb at master · Homebrew/homebrew-core · GitHub
    • [WayBack] ipmiutil — Homebrew Formulae
    • [WayBack] Install ipmiutil on Mac with Brew | BrewInstall
  • ipmitool
  • freeipmi
    • [WayBack] FreeIPMI – Home

      FreeIPMI provides in-band and out-of-band IPMI software based on the IPMI v1.5/2.0 specification. The IPMI specification defines a set of interfaces for platform management and is implemented by a number vendors for system management. The features of IPMI that most users will be interested in are sensor monitoring, system event monitoring, power control, and serial-over-LAN (SOL). The FreeIPMI tools and libraries listed below should provide users with the ability to access and utilize these and many other features. A number of useful features for large HPC or cluster environments have also been implemented into FreeIPMI. See the README or FAQ for more info.

       

    • [WayBack] homebrew-core/freeipmi.rb at master · Homebrew/homebrew-core · GitHub
    • [WayBack] freeipmi — Homebrew Formulae

Things I am interested in:

  • discovering IPMI capable devices in a network
  • reading sensors
  • setting power state
  • mounting/unmounting ISO images

–jeroen

Posted in Hardware, IPMI, Mainboards, Power User, SuperMicro | Leave a Comment »

Supermicro: 5 BIOS beeps might just mean internal VGA is disabled

Posted by jpluimers on 2021/05/03

A while ago, I got a second hand Supermicro workstation and wondered why the IPMI KVM would not show anything.

The video card in it worked fine, so I used it as a desktop machine setup for a while, happily running VMS.

I finally decided to move into a closet, so I removed the video card, and rebooted.

Then it beeped 5 times, which was odd: I expected it to switch from the video card to the internal video. It didn’t.

Older BIOS codes: [WayBack] AMI BIOS Beep Codes – Thomas-Krenn-Wiki

From PDF [WayBack] BIOS POST Codes for C7/X9/X10/X11/B9/B10/B1/A1 Motherboards

PEI Beep Codes

# of Beeps Description
1 Memory not Installed
1 Memory was installed twice (InstallPeiMemory routine in PEI Core called twice)
2 Recovery started
3 DXEIPL was not found
3 DXE Core Firmware Volume was not found
4 Recovery failed
4 S3 Resume failed
7 Reset PPI is not available

DXE Beep Codes

# of Beeps Description
1 Invalid password
4 Some of the Architectural Protocols are not available
5 No Console Output Devices are found
5 No Console Input Devices are found
6 Flash update is failed
7 Reset protocol is not available
8 Platform PCI resource requirements cannot be met

So I did a bit of reading in the manual, then found about a jumper which had happily been living out of view, under the video card:

VGA Enable (JPG1)

JPG1 allows you to enable or disable the onboard VGA connector. The default position is on pins 1 and 2 to enable VGA.

Setting the jumper to pins 1&2 made internal VGA available again, it happily booted and showed in the IPMI KVM.

Later I understood why the jumper was set to VGA disabled: when having two video cards, by default Windows will extend your desktop to an invisible monitor.

The easiest workaround for that is just to disable VGA. However, you can also change Windows

Older versus newer boards

It appears that most older SuperMicro systems have a hardware switch, but for newer chipsets supporting Intel Quick Sync Video a there is a BIOS setting: [WayBack] Enable internal graphics in SUPERMICRO servers | Any IT here? Help Me!

Windows 10 fixing video cards and ACPI_BIOS_ERROR

The reboot did not work fine: Windows 10 would not initialise properly, but hung when detecting video cards.

Read the rest of this entry »

Posted in Hardware, IPMI, Mainboards, Power User, SuperMicro, X10SRH-CF, X9SRi-3F, X9SRi-F | Leave a Comment »

The tale of [SSH into ESXi 6.7 box resulting in “debug1: expecting SSH2_MSG_KEXDH_REPLY”, delay and after entering password “Permission denied, please try again.”]

Posted by jpluimers on 2021/04/02

A similar ESXi 6.5 box worked well to ssh into, but on ESXi 6.7 it failed:

SSH into ESXi 6.7 box resulting in “debug1: expecting SSH2_MSG_KEXDH_REPLY“, delay and after entering password “Permission denied, please try again.

I had a hard time figuring out why: Login with the same user+password on the web user interface, DCUI and console shell work fine (see [WayBack] Enable SSH on VMware ESXi 6.x – VirtuBytes).

Searches that led me to EBCAK:

It almost felt like the /etc/passwd file thought the user had an empty password, but in fact it did not.

Adding an AllowUsers clause to ESXi in /etc/ssh/ssd_config, then performing /etc/init.d/SSH restart failed as well, and should not be needed anyway (default is all users having a valid shell can login, including root as on ESXi,  by default has PermitRootLogin yes) (via [WayBack] server – Permission denied please try again ssh error – Ask Ubuntu).

Setting LogLevel debug from LogLevel info in /etc/ssh/ssd_config did not change anything (not even after restarting sshd, or rebooting): it did not even add any more logging in /var/log/syslog.log or any of the log files under /var/log or /scratch/log.

Ruling out lock-down mode:

# vim-cmd vimsvc/auth/lockdown_is_possible
false
# vim-cmd vimsvc/auth/lockdown_is_enabled
false

See [WayBack] New vSphere 4.1 CLI Utilities Marketing Did Not Tell You About Part 3 and [WayBack] HOW TO: Enable or Disable Lockdown Mode on VMware vSphere ESXi host | vStrong.info

Q: What is Lockdown Mode?
A: Lockdown Mode prevents users from logging directly to the host. The host will only be accessible through local console or vCenter Server. None of remote management options e.g. vCLI, PowerCLI script, SSH will work. When it is enabled, only vpxuser () has authentication permissions and can connect to the host remotely.

No password login also means no passwordless login

The above rules out easy uploading my public keys for doing passwordless login in [WayBack] ssh root@host – Permission denied, please try again. – Tarran Jones.

Delay annoyance

There is also an annoyance: it takes about 10 seconds before you can enter the password (adding -v -v -v reveals the wait is on debug1: expecting SSH2_MSG_KEXDH_REPLY).

Disabling/enabling SSH from the DCUI: not fully disabled

After disabling SSH from the DCUI, I could still connect over SSH.

So then I disabled the TSM-SSH service from the web interface (despite DCUI telling SSH was disabled, TSM-SSH was still active, strange!) as it hosts the SSH service. I could still perform my ssh command!

Then it occurred to me: the IP address in the web browser was one off from the IP address in my ssh command.

By sheer coincidence, the IPMI IP address was one lower than the LAN1 IP address. I had been ssh-ing into the IPMI interface all the time, never realising IPMI had support for the first place!

Restring the TSM-SSH service now suddenly did get me LogLevel debug output in /var/log/auth.log (backed by /scratch/log/auth.log and duplicated in /vmfs/volumes/<<ssd-volume>>/.locker/log/auth.log).

Learned three things

So learned three things the hard way:

  1. Be more careful with IP-addresses
  2. IPMI does ssh (but it is very undocumented)
  3. DCUI enable/disable of SSH is not complete; TSM-SSH is

Some references:

–jeroen

Posted in ESXi6.5, ESXi6.7, Hardware, IPMI, Mainboards, Power User, SuperMicro, Virtualization, VMware, VMware ESXi | Leave a Comment »

Running SuperMicro IPMIView on MacOS

Posted by jpluimers on 2021/03/16

I wrote about SuperMicro mainboards and IPMIView recently, but that ran only on Windows and Linux. Since I focus my desktop mainly on MacOS, and never on Linux, I did not want to use the Windows IPMView (though it did work most of the time).

Not having a MacOS version sounded odd, as there was an iOS version:

[WayBack] ‎Supermicro IPMIView on the App Store “This app is only available on the App Store for iOS devices.”

A quick search made me find a few links:

The last one looked most promising, so I forked it.

Following the steps already made me write down a few notes for changes in the README.md file.

But then I bumped into a strange error when wanting to use the KVM Console from IPMIView, as it threw the same error all the time:

".jre/Contents/Home/bin/java": error=2, No such file or directory

I made a quick note in [WayBack] KVM Console cannot start due to java not found · Issue #1 · jpluimers/IPMIView.app · GitHub:

When starting a KVM Console, you get this error: ".jre/Contents/Home/bin/java": error=2, No such file or directory

Try to fix this.

Later I dug a bit deeper, and managed to fix it in the script steps of the README.md:

git clone https://github.com/TheCase/IPMIView.app
pushd IPMIView.app/
mkdir -p Resources/IPMIView
pushd Resources/IPMIView/
tar -zxvf ~/Downloads/IPMIView*.tar.gz --strip=1
pushd jre/
mkdir -p Contents/Home/bin
pushd Contents/Home/bin
ln -s `which java` java
popd
popd
popd
popd
rsync -avlo IPMIView.app/ ~/Applications/IPMIView.app/

Of course I ran into another problem on one of my SuperMicro machines: the KVM Console would consistently crash. Luckily that was solved by a IPMI Firmware Upgrade:

[WayBack] java – Supermicro IPMIView KVM Console does not work at all – Server Fault

The problem was the firmware for the IPMI on these boards was too old (not the same as the BIOS – updating the BIOS will not help in this case). Digging around SuperMicro’s site (never did get a reply from them), I found the Firmware Revision of 3.20 & was able to install it. On the IPMI device tab, under “Device Information”, you should see: Firmware Revision 3.20 IPMI Revision: 2.0 I can now see the KVM Console in both the IPMIView software

–jeroen

Posted in Hardware, IPMI, Mainboards, Power User, SuperMicro, X10SRH-CF, X9SRi-3F | Leave a Comment »

 
%d bloggers like this: