microsoft / scxcore Goto Github PK
View Code? Open in Web Editor NEWSystem Center Cross Platform Provider for Operations Manager
License: Microsoft Public License
System Center Cross Platform Provider for Operations Manager
License: Microsoft Public License
Hi,
it seems you have released a new version https://packages.microsoft.com/rhel/8/prod/scx-1.6.9-2.universal.x64.rpm but it's not reflected here.
Greetings
Klaas
Would it be possible to integrate Rocky Linux 8 detection? Would hope this would be fairly straightforward as it's a RHEL derivative?
Thanks!
Hi,
cross linked from microsoft/omi/issues/680 -- hope I found the right github repository
it seems the scxagent-logrotate selinux module is uninstalled after upgrading.
You need to change a lot of the postun scriptlet and put it into the if clause that is not being executed during update
Postuninstall from scx-1.6.6-0.x86_64
postuninstall scriptlet (using /bin/sh):
#!/bin/sh
# If we're called for upgrade, don't do anything
if [ "$1" -ne 1 ]; then
# Check if port 1270 is open
/opt/omi/bin/omiconfigeditor httpsport -q 1270 < /etc/opt/omi/conf/omiserver.conf > /dev/null 2>&1
if [ $? -eq 0 ]; then
touch /etc/opt/microsoft/scx/conf/scxagent-enable-port
# Remove port 1270 from the list of ports that OMI will listen on
/opt/omi/bin/omiconfigeditor httpsport -r 1270 < /etc/opt/omi/conf/omiserver.conf > /etc/opt/omi/conf/omiserver.conf_temp
mv /etc/opt/omi/conf/omiserver.conf_temp /etc/opt/omi/conf/omiserver.conf
fi
fi
# Clean up logrotate
rm -f /etc/logrotate.d/scxagent
rm -f /etc/cron.d/scxagent
DeleteSoftLinkToSudo() {
if [ -L /etc/opt/microsoft/scx/conf/sudodir ]; then
LINKED_DIR=`(cd /etc/opt/microsoft/scx/conf/sudodir ; pwd -P)`
[ x${LINKED_DIR} = x/usr/bin ] && rm /etc/opt/microsoft/scx/conf/sudodir
fi
}
DeleteSoftLinkToTmpDir() {
if [ -L /etc/opt/microsoft/scx/conf/tmpdir ]; then
LINKED_DIR=`(cd /etc/opt/microsoft/scx/conf/tmpdir ; pwd -P)`
[ x${LINKED_DIR} = x/tmp ] && rm /etc/opt/microsoft/scx/conf/tmpdir
fi
}
/opt/omi/bin/service_control reload
# If we're called for upgrade, don't do anything
if [ "$1" -ne 1 ]; then
DeleteSoftLinkToSudo
DeleteSoftLinkToTmpDir
fi
if [ -e /usr/sbin/semodule ]; then
if [ ! -z "$(/usr/sbin/semodule -l | grep scxagent-logrotate)" ]; then
echo "Removing selinux policy module for scxagent-logrotate ..."
/usr/sbin/semodule -r scxagent-logrotate
fi
fi
exit 0
I think this needs to go into the if clause that is not being executed during upgrade:
# Clean up logrotate
rm -f /etc/logrotate.d/scxagent
rm -f /etc/cron.d/scxagent
if [ -e /usr/sbin/semodule ]; then
if [ ! -z "$(/usr/sbin/semodule -l | grep scxagent-logrotate)" ]; then
echo "Removing selinux policy module for scxagent-logrotate ..."
/usr/sbin/semodule -r scxagent-logrotate
fi
fi
Reproducer (copied from other issue):
have a system without omi/scx
$ dnf install omi scx
Last metadata expiration check: 1:22:15 ago on Thu 11 Feb 2021 02:11:10 PM UTC.
Dependencies resolved.
=====================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
=====================================================================================================================================================================================================================================================================================
Installing:
omi x86_64 1.6.6-0 packages-microsoft-com-prod 1.8 M
scx x86_64 1.6.6-0 packages-microsoft-com-prod 2.0 M
Transaction Summary
=====================================================================================================================================================================================================================================================================================
Install 2 Packages
Total download size: 3.8 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): scx-1.6.6-0.universal.x64.rpm 9.6 MB/s | 2.0 MB 00:00
(2/2): omi-1.6.6-0.ssl_110.ulinux.x64.rpm 8.5 MB/s | 1.8 MB 00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 17 MB/s | 3.8 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: omi-1.6.6-0.x86_64 1/2
Creating omi group ...
Creating omi service account ...
Installing : omi-1.6.6-0.x86_64 1/2
Running scriptlet: omi-1.6.6-0.x86_64 1/2
************************************************************
* Warning: The certificate and keyfile were not generated *
* since they already exist. *
************************************************************
2021-02-11 15:33:30 : Crontab not configured to update omi keytab automatically. Skip unconfigure
ktutil not found
Checking if cron is installed...
Checking if cron/crond service is started...
Set up a cron job to OMI logrotate every 15 minutes
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
Trying /usr/share/selinux/packages/omi-selinux/omi-logrotate.pp ...
Trying /usr/share/selinux/packages/omi-selinux/omi-selinux.pp ...
Labeling omi log files ...
Configuring OMI service ...
Created symlink /etc/systemd/system/multi-user.target.wants/omid.service → /usr/lib/systemd/system/omid.service.
Trying to start omi with systemctl
omi is started.
Running scriptlet: scx-1.6.6-0.x86_64 2/2
Installing : scx-1.6.6-0.x86_64 2/2
Running scriptlet: scx-1.6.6-0.x86_64 2/2
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
Trying /usr/share/selinux/packages/scxagent-logrotate/scxagent-logrotate.pp ...
Verifying : omi-1.6.6-0.x86_64 1/2
Verifying : scx-1.6.6-0.x86_64 2/2
Installed products updated.
Installed:
omi-1.6.6-0.x86_64 scx-1.6.6-0.x86_64
Complete!
$ semodule -l|grep -E 'scx|omi'
omi-logrotate
omi-selinux
scxagent-logrotate
$ dnf reinstall scx omi
Last metadata expiration check: 1:24:46 ago on Thu 11 Feb 2021 02:11:10 PM UTC.
Dependencies resolved.
=====================================================================================================================================================================================================================================================================================
Package Architecture Version Repository Size
=====================================================================================================================================================================================================================================================================================
Reinstalling:
omi x86_64 1.6.6-0 packages-microsoft-com-prod 1.8 M
scx x86_64 1.6.6-0 packages-microsoft-com-prod 2.0 M
Transaction Summary
=====================================================================================================================================================================================================================================================================================
Total download size: 3.8 M
Installed size: 11 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): scx-1.6.6-0.universal.x64.rpm 12 MB/s | 2.0 MB 00:00
(2/2): omi-1.6.6-0.ssl_110.ulinux.x64.rpm 10 MB/s | 1.8 MB 00:00
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 21 MB/s | 3.8 MB 00:00
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: omi-1.6.6-0.x86_64 1/4
Unconfiguring omid (systemd) service ...
Removed /etc/systemd/system/multi-user.target.wants/omid.service.
Reinstalling : omi-1.6.6-0.x86_64 1/4
Running scriptlet: omi-1.6.6-0.x86_64 1/4
************************************************************
* Warning: The certificate and keyfile were not generated *
* since they already exist. *
************************************************************
omi already configured
2021-02-11 15:36:02 : Crontab not configured to update omi keytab automatically. Skip unconfigure
ktutil not found
Checking if cron is installed...
Checking if cron/crond service is started...
Set up a cron job to OMI logrotate every 15 minutes
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
Trying /usr/share/selinux/packages/omi-selinux/omi-logrotate.pp ...
Trying /usr/share/selinux/packages/omi-selinux/omi-selinux.pp ...
Labeling omi log files ...
Configuring OMI service ...
Created symlink /etc/systemd/system/multi-user.target.wants/omid.service → /usr/lib/systemd/system/omid.service.
Trying to start omi with systemctl
omi is started.
Running scriptlet: scx-1.6.6-0.x86_64 2/4
Reinstalling : scx-1.6.6-0.x86_64 2/4
Running scriptlet: scx-1.6.6-0.x86_64 2/4
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
System appears to have SELinux installed, attempting to install selinux policy module for logrotate
Trying /usr/share/selinux/packages/scxagent-logrotate/scxagent-logrotate.pp ...
Running scriptlet: scx-1.6.6-0.x86_64 3/4
Cleanup : scx-1.6.6-0.x86_64 3/4
Running scriptlet: scx-1.6.6-0.x86_64 3/4
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
Removing selinux policy module for scxagent-logrotate ...
libsemanage.semanage_direct_remove_key: Removing last scxagent-logrotate module (no other scxagent-logrotate module exists at another priority).
Running scriptlet: omi-1.6.6-0.x86_64 4/4
Cleanup : omi-1.6.6-0.x86_64 4/4
Running scriptlet: omi-1.6.6-0.x86_64 4/4
Removing selinux policy module for omi-logrotate ...
libsemanage.semanage_direct_remove_key: Removing last omi-logrotate module (no other omi-logrotate module exists at another priority).
Verifying : omi-1.6.6-0.x86_64 1/4
Verifying : omi-1.6.6-0.x86_64 2/4
Verifying : scx-1.6.6-0.x86_64 3/4
Verifying : scx-1.6.6-0.x86_64 4/4
Installed products updated.
Reinstalled:
omi-1.6.6-0.x86_64 scx-1.6.6-0.x86_64
Complete!
$ semodule -l|grep -E 'scx|omi'
omi-selinux
Greetings
Klaas
Hi,
for version v1.8.1-0 there are no releases. Only source code.
Can you please provide them under Assets like they are mentioned in the Package Checksum, see scx-1.8.1-0.ssl_100.universal.x64.deb
Thank you
see this issue for details. Was referred here by that parent issue.
2017/09/14 01:47:36 [50706,50706] INFO: null(0): EventId=40012 Priority=INFO Socket: 0x1adc070, closing connection (mask 2)
2017/09/14 01:47:36 [50706,50706] INFO: null(0): EventId=40033 Priority=INFO Selector_RemoveHandler: selector=0x5d29e8, handler=0x1adc070, name=BINARY_SERVER_CONNECTION
Failed to create directory /var/opt/microsoft/scx/lib/state/scxuser, errno=13 (Permission denied)
2017/09/14 01:47:36 [50707,50707] ERROR: null(0): EventId=20115 Priority=ERROR Failed to execute PREEXEC program: {/opt/microsoft/scx/bin/omi_preexec}
2017/09/14 01:47:36 [50707,50707] ERROR: null(0): EventId=20000 Priority=ERROR AgentMgr_HandleRequest failed for namespace: root/scx
2017/09/14 01:47:36 [50707,50707] INFO: null(0): EventId=40028 Priority=INFO Socket: 0x20559d0, Connection Closed while reading header
2017/09/14 01:47:36 [50707,50707] INFO: null(0): EventId=40012 Priority=INFO Socket: 0x20559d0, closing connection (mask 2)
Command:
/opt/omi/bin/omicli iv -u scxuser -p scxuserpassword --auth Basic --hostname yourlinuxhost root/scx { SCX_LogFile } GetMatchedRows { filename /var/log.txt regexps ".*" qid myQID } --port 1270 --encryption https
Stop omiserver and run ` /opt/omi/bin/omiserver --loglevel 5 --httptrace -l ` to get the above log, thanks.
Hello, our Redhat 7 (agent 1.6.2-343) and Redhat 8 (agent 1.6.6-0) correctly provide the OS release version when the agent is installed by the management server (SCOM2016) .
When we update our system, the release version changes in /etc/redhat-release, but this is not updated in SCOM.
The version I see when running 'winrm enumerate http://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_OperatingSystem?__cimnamespace=root/scx' ... is also still the old one.
It seems the info provided by the agent comes from the file '/etc/opt/microsoft/scx/conf/scx-release', which doens't get updated even after a couple of weeks.
To solve the issue manually, I have to:
execute '/opt/microsoft/scx/bin/tools/GetLinuxOS.sh' to update the 'scx-release' file (not sure it's the way to go)
restart the 'omid.service' (At that point, up to date info is available using winrm)
wait the next scheduled discovery on the management server.
Would it be possible that the agent updates this info automatically?
Best regards,
If you're monitoring Solaris System with Zones installed with SCX agent 1.6.2-343, querying SCX_UnixProcess class isn't returning results for services running in a zone other than 'global'.
However, 'ps -ef' or 'ps -efZ' produce expected results.
The only workaround to monitor Unix processes is to use ExecuteCommand methods and parse the output.
OS Version: CentOS 7.3 (3.10.0-514.26.2.el7.x86_64)
OMI: OMI-1.0.8-6
scx: scx-1.6.2-337
we enabled diagnositic extension at 08/02 and noticed that omiagent in our Azure VM would eat up almost 100% CPU of one core since 10/23. This happened suddenly without any changing from our end. The issue lasted for a long time and still bothered us. Some times, it could be solved after we restarted the waagent service which would restarted the omi service as well.
we consulted omiagent engineer. they suggested to open an issue to the providers team since most high cpu would be caused by providers themselves. The diagnostic extension only called SCX providers so we ask for help here.
our troubleshooting:
anyone has clue what's happending or how to do troubleshooting?
I would like to know why the SCX module of the Log Analytics agent does not collect logs for certain file systems, is it a functional limitation?
I have confirmed that the filesystems listed in the following documents are not collected by the SCX module.
https://github.com/microsoft/pal/blob/06c611367b1c31b3b6a6b65a9f58117fe54a9e95/source/code/scxsystemlib/disk/diskdepend.cpp#L719
My customer speculates that this may be due to the following.
Is his understanding incorrect? If that is wrong, I would like to know the correct reason.
OMI tables SCX_DiskDrive and SCX_DiskDriveStatisticalInformation are showling all disk drives except the ones behind a RAID 0.
Disk Stats seems to be correctly populated, but the omi is not aligned with it.
This issue prevents other Azure Extensions like Linux Azure Diagnostic 3.0 to collect disk metrics and potentially detect Throttling at Disk Level.
In sample below, disk drives 'sdd' and 'sdf' are configured in a RAID 0 array with mdadm, and mounted on ‘/p10_RAID’.
OS : Ubuntu Linux 16.04 LTS
Attached 4 data disks to VM:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
fd0 2:0 1 4K 0 disk
sda 8:0 0 30G 0 disk
└─sda1 8:1 0 30G 0 part /
sdb 8:16 0 14G 0 disk
└─sdb1 8:17 0 14G 0 part /mnt
sdc 8:32 0 1023G 0 disk
└─sdc1 8:33 0 1023G 0 part /p30
sdd 8:48 0 127G 0 disk
└─sdd1 8:49 0 127G 0 part
└─md127 9:127 0 253.9G 0 raid0 /p10_RAID
sde 8:64 0 511G 0 disk
└─sde1 8:65 0 511G 0 part /p20
sdf 8:80 0 127G 0 disk
└─sdf1 8:81 0 127G 0 part
└─md127 9:127 0 253.9G 0 raid0 /p10_RAID
Below system disk stats information:
# cat /proc/diskstats
7 0 loop0 0 0 0 0 0 0 0 0 0 0 0
7 1 loop1 0 0 0 0 0 0 0 0 0 0 0
7 2 loop2 0 0 0 0 0 0 0 0 0 0 0
7 3 loop3 0 0 0 0 0 0 0 0 0 0 0
7 4 loop4 0 0 0 0 0 0 0 0 0 0 0
7 5 loop5 0 0 0 0 0 0 0 0 0 0 0
7 6 loop6 0 0 0 0 0 0 0 0 0 0 0
7 7 loop7 0 0 0 0 0 0 0 0 0 0 0
2 0 fd0 3 0 24 144 0 0 0 0 0 144 144
8 0 sda 9290 1 678663 96092 13517 16015 699864 53708 0 62752 149784
8 1 sda1 9258 1 676551 96056 13514 16015 699864 53708 0 62720 149748
8 16 sdb 413 0 29354 168 763 33446 30086720 210760 0 2804 210928
8 17 sdb1 118 0 9506 68 763 33446 30086720 210760 0 2748 210828
8 80 sdf 309 0 12080 1780 0 0 0 0 0 1148 1776
8 81 sdf1 283 0 10016 1592 0 0 0 0 0 1008 1588
8 32 sdc 329 0 19746 1968 3 0 24 12 0 1616 1980
8 33 sdc1 296 0 17626 1716 3 0 24 12 0 1432 1728
8 48 sdd 249 0 12570 312 3 0 24 28 0 332 340
8 49 sdd1 223 0 10506 264 3 0 24 28 0 284 292
8 64 sde 296 0 19482 1824 3 0 24 12 0 1468 1836
8 65 sde1 263 0 17362 1628 3 0 24 12 0 1304 1640
9 126 md126 293 0 15730 0 5 0 24 0 0 0 0
# ls -l /sys/block
total 0
lrwxrwxrwx 1 root root 0 Sep 22 15:57 fd0 -> ../devices/platform/floppy.0/block/fd0
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop0 -> ../devices/virtual/block/loop0
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop1 -> ../devices/virtual/block/loop1
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop2 -> ../devices/virtual/block/loop2
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop3 -> ../devices/virtual/block/loop3
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop4 -> ../devices/virtual/block/loop4
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop5 -> ../devices/virtual/block/loop5
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop6 -> ../devices/virtual/block/loop6
lrwxrwxrwx 1 root root 0 Sep 22 15:57 loop7 -> ../devices/virtual/block/loop7
lrwxrwxrwx 1 root root 0 Sep 22 12:28 md126 -> ../devices/virtual/block/md126
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sda -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/00000000-0000-8899-0000-000000000000/host2/ta rget2:0:0/2:0:0:0/block/sda
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sdb -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/00000000-0001-8899-0000-000000000000/host3/ta rget3:0:1/3:0:1:0/block/sdb
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sdc -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/f8b3781b-1e82-4818-a1c3-63d806ec15bb/host5/ta rget5:0:0/5:0:0:0/block/sdc
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sdd -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/f8b3781b-1e82-4818-a1c3-63d806ec15bb/host5/ta rget5:0:0/5:0:0:3/block/sdd
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sde -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/f8b3781b-1e82-4818-a1c3-63d806ec15bb/host5/ta rget5:0:0/5:0:0:1/block/sde
lrwxrwxrwx 1 root root 0 Sep 22 12:28 sdf -> ../devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/device:07/VMBUS:01/f8b3781b-1e82-4818-a1c3-63d806ec15bb/host5/ta rget5:0:0/5:0:0:2/block/sdf
Below the OMI query:
# /opt/omi/bin/omicli cql root/scx "select Name from scx_diskdrivestatisticalinformation"
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=sda
}
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=md
}
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=sdc
}
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=sde
}
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=sdb
}
instance of SCX_DiskDriveStatisticalInformation
{
[Key] Name=_Total
}
# /opt/omi/bin/omicli cql root/scx "select Name from SCX_DiskDrive"
instance of SCX_DiskDrive
{
Name=sda
}
instance of SCX_DiskDrive
{
Name=md
}
instance of SCX_DiskDrive
{
Name=sdc
}
instance of SCX_DiskDrive
{
Name=sde
}
instance of SCX_DiskDrive
{
Name=sdb
}
Re-raising from microsoft/OMS-docker#76
Comments below were from @kevi5702 .
Really hoping someone can point me in the right direction here, every time we redeploy our docker instances our SCX logs at (/var/opt/microsoft/scx/log/scx.log) beging to fill very rapidly with the following messages:
2017-09-29T15:02:39,523Z Error [scx.core.common.pal.system.disk.statisticallogicaldiskinstance:############] statvfs() failed for /var/lib/docker/overlay/######################/merged; errno = 2
Systemctl restart omsagent##### seems to take care of this, but we were expecting the agent to be aware when a container went away and to stop trying to stat the directory it used to be mounted to.
So far we have tried removing and reinstalling the OMS bundle. But curious if there is something else we are doing wrong here?
Digging around this error seems to match the code here in the PAL software:
I'm wondering if overlay needs to be added to excludes somewhere, however manually making a basic overlay mount doesn't produce the statvfs errors when its unmounted, only the warning about overlay not being recognized.
I was able to reproduce this on a new centos image with fresh OMS deploy and just running a basic hello world container.
Restarting the omid.service looks to make this go away, so not sure if something needs to be aware to update this when a container is removed?
This only seems to trigger when logical disk performance counters are enabled and only when the file system was a docker overlay FS mount.
Since commit hash 6fc4a5d
, unit tests are now failing:
---- All tests run ----
!!!FAILURES!!!
Test Results:
Run: 381 Failures: 1 Errors: 0
1) test: ScxSSLCertTest::testNonRFCcompliantDomainFallback (F) line: 494 /home/serviceb/jenkins/workspace/SCXCore-Runner/label/SUSE_10.0_X64/opsmgr/test/code/shared/tools/scx_ssl_config/scxsslcert_test.cpp
assertion failed
- Expression: ret == 0
- Error running: ./openssl_1.0.0/scxsslconfig, error message is: Error generating SSL certificate. Use scxsslconfig to generate a new certificate, specifying host and domain names if necessary. The error was: 'Unable to add the domain name to the subject.'
, error code is: 255
make: *** [testrun] Error 1
Build step 'Execute shell' marked build as failure
Finished: FAILURE
This is causing nightly build failures.
Normally, pbuild should eliminate these sorts of failures. I'd like to understand how this occurred so that we can prevent failures like this in the future. Please collaborate with me on this so we can improve build reliability, thanks.
omiagent is crashing with messages like these:
Sep 6 11:01:34 hostname kernel: omiagent[44104]: segfault at 7f7c6bc3fe00 ip 00007f7c6bc3fe00 sp 00007f7c69ce4d50 error 14 in libnss_files-2.17.so[7f7c6bdc7000+c000]
Sep 6 11:31:34 hostname kernel: omiagent[60927]: segfault at 7f5309121e00 ip 00007f5309121e00 sp 00007f5303107d50 error 14 in libnss_files-2.17.so[7f53092a9000+c000]
Environment information:
got a core file, could confirm the crash occurs in /lib64/libnss_dns.so.2:
#0 0x00007f737fa78e00 in ?? ()
#1 0x00007f737fabcf47 in ?? ()
#2 0x00007f737fabcb60 in ?? ()
#3 0xffffffff00000073 in ?? ()
#4 0x0000000000000000 in ?? ()
From To Syms Read Shared Object Library
0x00007f738a0e08b0 0x00007f738a0ebe01 Yes (*) /lib64/libpthread.so.0
0x00007f7389ed7d90 0x00007f7389ed888e Yes (*) /lib64/libdl.so.2
0x00007f7389cca620 0x00007f7389cd1cba Yes (*) /lib64/libpam.so.0
0x00007f7389a71b00 0x00007f7389aad4ed Yes (*) /opt/omi/lib/libssl.so.1.0.0
0x00007f7389662f00 0x00007f738979ac37 Yes (*) /opt/omi/lib/libcrypto.so.1.0.0
0x00007f73892488d0 0x00007f738939822f Yes (*) /lib64/libc.so.6
0x00007f738a2f7ad0 0x00007f738a313080 Yes (*) /lib64/ld-linux-x86-64.so.2
0x00007f7389003240 0x00007f73890097f2 Yes (*) /lib64/libaudit.so.1
0x00007f7388dbf060 0x00007f7388df150a Yes (*) /lib64/libgssapi_krb5.so.2
0x00007f7388af0910 0x00007f7388b57cea Yes (*) /lib64/libkrb5.so.3
0x00007f73888c8500 0x00007f73888c90d3 Yes (*) /lib64/libcom_err.so.2
0x00007f7388698890 0x00007f73886b6b4f Yes (*) /lib64/libk5crypto.so.3
0x00007f7388480170 0x00007f738848c6f8 Yes (*) /lib64/libz.so.1
0x00007f7388279580 0x00007f738827b3bc Yes (*) /lib64/libcap-ng.so.0
0x00007f738806d720 0x00007f73880742bb Yes (*) /lib64/libkrb5support.so.0
0x00007f7387e675b0 0x00007f7387e681cc Yes (*) /lib64/libkeyutils.so.1
0x00007f7387c508f0 0x00007f7387c5ee92 Yes (*) /lib64/libresolv.so.2
0x00007f7387a2c9d0 0x00007f7387a427d6 Yes (*) /lib64/libselinux.so.1
0x00007f73877c55f0 0x00007f738780b5b0 Yes (*) /lib64/libpcre.so.1
0x00007f738737ec00 0x00007f73874dc898 Yes (*) /opt/omi/lib/libSCXCoreProviderModule.so
0x00007f738a3afcc0 0x00007f738a3cb568 Yes (*) /opt/omi/lib/libmicxx.so
0x00007f7387026db0 0x00007f738702bc6c Yes (*) /lib64/libcrypt.so.1
0x00007f7386e201e0 0x00007f7386e2316c Yes (*) /lib64/librt.so.1
0x00007f7386b72220 0x00007f7386bd937a Yes (*) /lib64/libstdc++.so.6
0x00007f738681a310 0x00007f73868852d6 Yes (*) /lib64/libm.so.6
0x00007f7386601a90 0x00007f7386611245 Yes (*) /lib64/libgcc_s.so.1
0x00007f73863fcb40 0x00007f73863fd1bb Yes (*) /lib64/libfreebl3.so
0x00007f737fcc2130 0x00007f737fcc9481 Yes (*) /lib64/libnss_files.so.2
0x00007f737ef762e0 0x00007f737efa531f Yes (*) /lib64/libssl3.so
0x00007f737ed4c2c0 0x00007f737ed60d97 Yes (*) /lib64/libsmime3.so
0x00007f737ea2fdf0 0x00007f737eb04e24 Yes (*) /lib64/libnss3.so
0x00007f737e7f38d0 0x00007f737e8035d8 Yes (*) /lib64/libnssutil3.so
0x00007f737e5e3ec0 0x00007f737e5e4c28 Yes (*) /lib64/libplds4.so
0x00007f737e3df4b0 0x00007f737e3e0b16 Yes (*) /lib64/libplc4.so
0x00007f737e1acc30 0x00007f737e1cceaf Yes (*) /lib64/libnspr4.so
0x00007f737c116020 0x00007f737c11987f Yes (*) /lib64/libnss_dns.so.2
0x00007f735fdc6280 0x00007f735fdeffac Yes (*) /lib64/libsoftokn3.so
0x00007f735fb17a00 0x00007f735fb9f7e0 Yes (*) /lib64/libsqlite3.so.0
0x00007f735f887600 0x00007f735f8e5a6d Yes (*) /lib64/libfreeblpriv3.so
0x00007f735f681c70 0x00007f735f68236b Yes (*) /lib64/libnsssysinit.so
0x00007f735f45a7e0 0x00007f735f4747c2 Yes (*) /lib64/libnsspem.so
SCXcore’s scxsslconfig program should set the permissions to OMI due to the non-root changes. It was always weird that SCX was generating an OMI certificate, and now we were bitten by this. This can’t be done in postinstall, because scxsslconfig can be run after installation. So the program itself needs to set the permissions.
Repro steps:
sh scx-1.6.3-372.universal.x64.sh –install
/opt/omi/bin/service_control restart
Another way to repro the issue is to install omi*.rpm, then install scx*.rpm, it shows that scx package installing reverts /etc/opt/omi/ssl/omi.pem file permission from omi:omi to root:root.
On our hardened instances of RHEL the postinstallation script embedded in scx fails, which causes the SSL certs to not be properly generated, or even existing certificates being removed during upgrades.
Running scriptlet: scx-1.7.0-0.x86_64 2/4
/etc/profile.d/bash_timeout.sh: line 1: TMOUT: readonly variable
warning: %post(scx-1.7.0-0.x86_64) scriptlet failed, exit status 1
Error in POSTIN scriptlet in rpm package scx
We work around it for upgrades by removing the file /etc/opt/omi/ssl/.omi_cert_marker
that the scx script is looking for.
Removing the readonly TMOUT is not an option as it is required by our security policy.
I've observed this behavior on all supported versions of RHEL. I've verified the script exits at sourcing /etc/profile by extracting it and running the script manually using sh -x
There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.
Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.
Example needs to be updated for:
Use 0 to disable line wrapping (-w 0)
multi line output won't work in SCOM MPs
Build: \\redmond\wsscfs\OSTCData\Builds\opsmgr\develop\1.6.3-595
and \\redmond\wsscfs\OSTCData\Builds\opsmgr\develop\1.6.3-606
root@omi86-sles10-01 # rpm -ivh /root/scxbundle.15252/098/scx-1.6.3-606.universal.x86.rpm
Preparing... ########################################### [100%]
1:scx ########################################### [100%]
/var/tmp/rpm-tmp.28002: line 63: 15298 Floating point exception(core dumped) /opt/microsoft/scx/bin/tools/scxadmin -config-reset RunAs AllowRoot >/dev/null 2>&1
/var/tmp/rpm-tmp.28002: line 83: 15311 Floating point exception(core dumped) /opt/microsoft/scx/bin/tools/scxsslconfig
error: %post(scx-1.6.3-606.i386) scriptlet failed, exit status 1
Hi
The installation of SCX fails on Ubuntu 18.04.4
azure# dpkg -i ./scx-1.6.4-7.ssl_110.universal.x64.deb
(Reading database ... 297782 files and directories currently installed.)
Preparing to unpack .../scx-1.6.4-7.ssl_110.universal.x64.deb ...
Unpacking scx (1.6.4.7) over (1.6.4.7) ...
Trying to stop omi with systemctl
omi is stopped.
Trying to start omi with systemctl
omi is started.
Setting up scx (1.6.4.7) ...
/var/lib/dpkg/info/scx.postinst: 26: /usr/share/bash-completion/bash_completion: Syntax error: "(" unexpected
dpkg: error processing package scx (--install):
installed scx package post-installation script subprocess returned error exit status 2
Errors were encountered while processing:
scx
The API annotations (IN, OUT, etc.) in the SCX Provider Description.md doc are all hyperlinks that point to the wiki back at scx.codeplex.com.
Customer used the LAD to collect File system statistics and have some file systems that are used NFS/CIFS from Azure File Share. After configuring to collect the mount point metrics of NFS/CIFS, Cx is unable to get the data.
We narrowed down that those metrics are collected by SCXcore, also confirmed that cmd /opt/omi/bin/omicli ei root/scx SCX_FileSystemStatisticalInformation cannot print NFS mount point as well. But there is no where to state that LAD didn't support NFS/CIFS mount file system. So can you help to confirm?
Recently I try to install Log Analytics agent on Ubuntu 18.04 however I'm encountered below errors. Anyone could advise me how to resolve this. Thank you.
The installation screen showing below info and stop.
/var/lib/dpkg/info/.scx.postinst: 58 : [ : EXISTS : unexpected operator ]
/var/lib/dpkg/info/.scx.postinst: 64 : [ : EXISTS : unexpected operator ]
/var/lib/dpkg/info/.scx.postinst: 68 : [ : EXISTS : unexpected operator ]
INFO: Please select correct value !!
After installing running command scxadmin failes with error:
/opt/microsoft/scx/bin/tools/.scxadmin: libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by /opt/microsoft/scx/bin/tools/.scxadmin)
OS: CentOS 6.x
Previous version 1.7.0 works.
Copied over from microsoft/omi#491 (please see this for full communication on this issue).
On our RM provisioned VMs in Azure we noticed that the root partition is filling up with large numbers of "core.###" files in the /var/opt/omi/run directory.
Further investigation shows segmentation faults (in /var/log/messages) as follows:
Jan 30 10:17:17 ML001 kernel: omiagent[2054]: segfault at 7f6b7e181e00 ip 00007f6b7e181e00 sp 00007f6b78846d50 error 14
Jan 30 10:32:12 ML001 kernel: omiagent[3298]: segfault at 7fa8356f4e00 ip 00007fa8356f4e00 sp 00007fa82fdb9d50 error 14
Jan 30 11:02:19 ML001 kernel: omiagent[5873]: segfault at 7fbd84d3de00 ip 00007fbd84d3de00 sp 00007fbd7ede0d50 error 14 in libnss_dns-2.17.so[7fbd84ec5000+5000]
Jan 30 11:17:19 ML001 kernel: omiagent[13175]: segfault at 7fbac740ae00 ip 00007fbac740ae00 sp 00007fbac54bcd50 error 14 in libnss_dns-2.17.so[7fbac7592000+5000]
Jan 30 11:32:21 ML001 kernel: omiagent[20049]: segfault at 7f79230dfe00 ip 00007f79230dfe00 sp 00007f791d7a4d50 error 14
Jan 30 12:02:14 ML001 kernel: omiagent[46782]: segfault at 7f9fa8939e00 ip 00007f9fa8939e00 sp 00007f9fa296dd50 error 14 in libnss_dns-2.17.so[7f9fa8ac1000+5000]
Environment information:
Operating System: CentOS Release 7.4.1708 (fully patched, that is, "yum update" shows no updates pending).
So far the workaround has been to write a cron job (!) to periodically wipe the core files but obviously this is not an ideal situation.
Further information from "JumpingYang001":
Following debug info shows omiagent loaded scx provider:
(gdb) info sharedlibrary
From To Syms Read Shared Object Library
0x00007fa598b2f900 0x00007fa598b3ace1 Yes () /lib64/libpthread.so.0
0x00007fa598926e60 0x00007fa59892795e Yes () /lib64/libdl.so.2
0x00007fa598719670 0x00007fa598720d0c Yes () /lib64/libpam.so.0
0x00007fa5984bfbb0 0x00007fa5984fb58d Yes () /opt/omi/lib/libssl.so.1.0.0
0x00007fa5980b0f00 0x00007fa5981e8bd7 Yes () /opt/omi/lib/libcrypto.so.1.0.0
0x00007fa597ca0480 0x00007fa597de6bcf Yes () /lib64/libc.so.6
0x00007fa598d46b10 0x00007fa598d61440 Yes () /lib64/ld-linux-x86-64.so.2
0x00007fa597a5c100 0x00007fa597a62402 Yes () /lib64/libaudit.so.1
0x00007fa597818650 0x00007fa59784aa1a Yes () /lib64/libgssapi_krb5.so.2
0x00007fa597549a10 0x00007fa5975b0e8a Yes () /lib64/libkrb5.so.3
0x00007fa597321570 0x00007fa597322143 Yes () /lib64/libcom_err.so.2
0x00007fa5970f18c0 0x00007fa59710fc0f Yes () /lib64/libk5crypto.so.3
0x00007fa596ed9170 0x00007fa596ee56f8 Yes () /lib64/libz.so.1
0x00007fa596cd2580 0x00007fa596cd43bc Yes () /lib64/libcap-ng.so.0
0x00007fa596ac6890 0x00007fa596acd42b Yes () /lib64/libkrb5support.so.0
0x00007fa5968c05b0 0x00007fa5968c11cc Yes () /lib64/libkeyutils.so.1
0x00007fa5966a89d0 0x00007fa5966b77e1 Yes () /lib64/libresolv.so.2
0x00007fa596484ac0 0x00007fa59649a8c6 Yes () /lib64/libselinux.so.1
0x00007fa59621d5f0 0x00007fa5962635b0 Yes () /lib64/libpcre.so.1
0x00007fa595ed6430 0x00007fa596034438 Yes /opt/omi/lib/libSCXCoreProviderModule.so
0x00007fa598e0fcc0 0x00007fa598e2b568 Yes /opt/omi/lib/libmicxx.so
0x00007fa595b78e50 0x00007fa595b7daac Yes () /lib64/libcrypt.so.1
0x00007fa595972250 0x00007fa59597504c Yes () /lib64/librt.so.1
0x00007fa5956c3510 0x00007fa59572a5ba Yes () /lib64/libstdc++.so.6
0x00007fa59536b370 0x00007fa5953d6276 Yes () /lib64/libm.so.6
0x00007fa595152af0 0x00007fa5951622a5 Yes () /lib64/libgcc_s.so.1
0x00007fa594f4dba0 0x00007fa594f4e309 Yes () /lib64/libfreebl3.so
0x00007fa58e8131d0 0x00007fa58e81a3e1 Yes () /lib64/libnss_files.so.2
0x00007fa58e60c090 0x00007fa58e60f4f0 Yes () /lib64/libnss_dns.so.2
0x00007fa58d908ec0 0x00007fa58d933b0f Yes () /lib64/libssl3.so
0x00007fa58d6df380 0x00007fa58d6f3e57 Yes () /lib64/libsmime3.so
0x00007fa58d3c5740 0x00007fa58d498654 Yes () /lib64/libnss3.so
0x00007fa58d18b390 0x00007fa58d199d45 Yes () /lib64/libnssutil3.so
0x00007fa58cf7bf10 0x00007fa58cf7cc78 Yes () /lib64/libplds4.so
0x00007fa58cd77510 0x00007fa58cd78b78 Yes () /lib64/libplc4.so
0x00007fa58cb44ca0 0x00007fa58cb64cc0 Yes () /lib64/libnspr4.so
0x00007fa58c27e2d0 0x00007fa58c2a7f5c Yes () /lib64/libsoftokn3.so
---Type to continue, or q to quit---
0x00007fa57e552a00 0x00007fa57e5da860 Yes () /lib64/libsqlite3.so.0
0x00007fa57e2c8bc0 0x00007fa57e32196d Yes () /lib64/libfreeblpriv3.so
0x00007fa58c077cd0 0x00007fa58c0783cb Yes () /lib64/libnsssysinit.so
0x00007fa57e09e7e0 0x00007fa57e0b8496 Yes (*) /lib64/libnsspem.so
(*): Shared library is missing debugging information.
(gdb) The crash is on 0x00007fa58e405e00 which is in /lib64/libnss_dns.so.2, that is same as your segmentation faults in /var/log/messages.
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /opt/omi/bin/omiagent...done.
[New LWP 123588]
[New LWP 108293]
[New LWP 108369]
[New LWP 108394]
[New LWP 108295]
[New LWP 108294]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `/opt/omi/bin/omiagent 9 10 --destdir / --providerdir /opt/omi/lib --loglevel WA'.
Program terminated with signal 11, Segmentation fault.
#0 0x00007fa58e405e00 in ?? ()
Missing separate debuginfos, use: debuginfo-install omi-1.4.2-1.x86_64
(gdb) bt
#0 0x00007fa58e405e00 in ?? ()
#1 0x00007fa58e449f47 in ?? ()
#2 0x00007fa58e449b60 in ?? ()
#3 0xffffffff00000073 in ?? ()
#4 0x0000000000000000 in ?? ()
Here are the threads:
(gdb) info threads
Id Target Id Frame
6 Thread 0x7fa598e01f00 (LWP 108294) 0x00007fa598b35cf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
5 Thread 0x7fa598dc2f00 (LWP 108295) 0x00007fa598b35cf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
4 Thread 0x7fa58e5caf00 (LWP 108394) 0x00007fa598b35cf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
3 Thread 0x7fa58e609f00 (LWP 108369) 0x00007fa598b35cf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
2 Thread 0x7fa598f53880 (LWP 108293) 0x00007fa597d707a3 in select () from /lib64/libc.so.6
(*) 1 Thread 0x7fa57ffff700 (LWP 123588) 0x00007fa58e405e00 in ?? ()
Please let me know if any further debug information is required.
Hi,
when trying to mirror the repository at https://packages.microsoft.com/centos/7/prod/ there is a checksum error for scx-1.6.4-7.universal.x64.rpm.
The repository metadata file says the checksum should be
7f4693c10b130ccfb40db6bcee88e98d5d67116b7984b1f5a81b721b86af4940
Downloading the file shows the actual checksum (SHA256) is
42b8943a8bbb45a810d6656d9887cf14c480e1d7a4e3ba20fc3d42518787665a.
Cheers,
kkbis
We are having a problem with a VM where SCX is reporting wrong values for the memory.
Please see the free and omnicli command output below and let me know if you need more information.
OS is Centos 7.2
kernel: 3.10.0-514.26.2.el7.x86_64
$ rpm -q scx omi
scx-1.6.3-274.x86_64
omi-1.2.0-35.x86_64
$ free -m
total used free shared buff/cache available
Mem: 3440 309 180 39 2949 2782
Swap: 6499 0 6499
$ /opt/omi/bin/omicli ei root/scx SCx_MemoryStatisticalInformation
instance of SCX_MemoryStatisticalInformation
{
Caption=Memory information
Description=Memory usage and performance statistics
[Key] Name=Memory
IsAggregate=true
AvailableMemory=498
PercentAvailableMemory=14
UsedMemory=2942
PercentUsedMemory=86
PercentUsedByCache=0
PagesPerSec=8
PagesReadPerSec=0
PagesWrittenPerSec=8
AvailableSwap=6499
PercentAvailableSwap=100
UsedSwap=0
PercentUsedSwap=0
}
Original issue was posted in OMS agent repo. SCX team please make time to respond. @vimish @rubroboletus
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.