Fixes #36973 - Use dnf needs-restarting to collect tracer information #149

sbernhard · 2023-12-04T12:35:06Z

Discussion, see: https://community.theforeman.org/t/tracer-for-dnf/35633?u=bernhard_suttner

m-bucher

Looks good to me

ehelms · 2023-12-08T13:21:30Z

@parthaa @ianballou could one of y'all take a look?

ianballou · 2023-12-08T21:43:20Z

Can we get a redmine for this please?

ianballou · 2023-12-11T18:37:42Z

@ehelms I think https://yum.theforeman.org/client/nightly/el7/x86_64/foreman-client-release.rpm might be leading to the wrong repository:

failure: repodata/repomd.xml from foreman-client: [Errno 256] No more mirrors to try.
https://yum.theforeman.org/client/nightly/el7_9/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

src/katello/tracer/SystemdDbus.py

ehelms · 2023-12-12T14:19:07Z

@ehelms I think https://yum.theforeman.org/client/nightly/el7/x86_64/foreman-client-release.rpm might be leading to the wrong repository:
failure: repodata/repomd.xml from foreman-client: [Errno 256] No more mirrors to try.
https://yum.theforeman.org/client/nightly/el7_9/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found

Thanks for the heads up!

src/katello/tracer/dnf.py

ehelms · 2023-12-14T19:51:10Z

@ehelms I think https://yum.theforeman.org/client/nightly/el7/x86_64/foreman-client-release.rpm might be leading to the wrong repository:
failure: repodata/repomd.xml from foreman-client: [Errno 256] No more mirrors to try.
https://yum.theforeman.org/client/nightly/el7_9/x86_64/repodata/repomd.xml: [Errno 14] HTTPS Error 404 - Not Found
Thanks for the heads up!

The fix is now deployed and the tests are passing for EL7.

src/katello/tracer/SystemdDbus.py

src/katello/tracer/dnf.py

src/katello/tracer/__init__.py

src/katello/tracer/dnf.py

sbernhard · 2023-12-27T12:47:18Z

src/katello/tracer/dnf.py

+from katello.tracer.SystemdDbus import SystemdDbus
+
+STATIC_SERVICES = [
+  "systemd",


Inspired by https://github.com/FrostyX/tracer/blob/master/data/applications.xml#L84

src/katello/tracer/dnf.py

ianballou · 2024-03-15T19:23:01Z

As an aside, I had to hack my system by copying the /usr/local/bin/katello-tracer-upload to /usr/sbin/katello-tracer-upload and /etc/cron.d/katello-tracer-upload to ensure that the dnf plugin would call the new code.

ianballou

Based on my comments above, I'd like to know if this issue with session traces is a surprise or not. Perhaps it was tested with a different client package upgrade?

sbernhard · 2024-03-15T22:14:46Z

Based on my comments above, I'd like to know if this issue with session traces is a surprise or not. Perhaps it was tested with a different client package upgrade?

I have tested this some time ago and session traces were recognized by https://github.com/Katello/katello-host-tools/pull/149/files#diff-21052bd00993aa53f643b56c70d49991ad54e23c97da7378b12a0b5b31197e28R36

The is_session is similar to https://github.com/FrostyX/tracer/blob/master/tracer/resources/processes.py#L268

See also this comment: #149 (comment)

Ah, maybe I just missed this one to grab: https://github.com/FrostyX/tracer/blob/ff8fc924fcbe2f638dd88b50549813dab2b8595b/tracer/resources/applications.py#L208

ianballou · 2024-03-18T20:11:23Z

Ah, maybe I just missed this one to grab: https://github.com/FrostyX/tracer/blob/ff8fc924fcbe2f638dd88b50549813dab2b8595b/tracer/resources/applications.py#L208

Wow, I am surprised there is such specialized logic in there for ssh sessions. Nice catch!

ianballou · 2024-03-18T20:29:16Z

@sbernhard after your update, I'm still seeing the same issue with ssh. The packages I'm playing with are:

[root@rhel9 katello-host-tools]# rpm -qa | grep ssh
libssh-config-0.10.4-8.el9.noarch
libssh-0.10.4-8.el9.x86_64
openssh-8.7p1-34.el9.x86_64
openssh-clients-8.7p1-34.el9.x86_64
openssh-server-8.7p1-34.el9.x86_64

Tracer is still showing:

[root@rhel9 katello-host-tools]# tracer -a
You should restart:
  * Some applications using:
      sudo systemctl restart NetworkManager

  * These applications restarting your session:
      ssh-vagrant-session

but I'm not seeing the session info yet. Is there any debugging / log statements you'd like me to try to help figure out what's going wrong?

sbernhard · 2024-03-20T12:55:55Z

@sbernhard after your update, I'm still seeing the same issue with ssh. The packages I'm playing with are:
[root@rhel9 katello-host-tools]# rpm -qa | grep ssh
libssh-config-0.10.4-8.el9.noarch
libssh-0.10.4-8.el9.x86_64
openssh-8.7p1-34.el9.x86_64
openssh-clients-8.7p1-34.el9.x86_64
openssh-server-8.7p1-34.el9.x86_64
Tracer is still showing:
[root@rhel9 katello-host-tools]# tracer -a
You should restart:
  * Some applications using:
      sudo systemctl restart NetworkManager

  * These applications restarting your session:
      ssh-vagrant-session
but I'm not seeing the session info yet. Is there any debugging / log statements you'd like me to try to help figure out what's going wrong?

Added some more handling for ssh sessions https://github.com/Katello/katello-host-tools/pull/149/files#diff-21052bd00993aa53f643b56c70d49991ad54e23c97da7378b12a0b5b31197e28R29

Similar to https://github.com/FrostyX/tracer/blob/master/tracer/resources/processes.py#L79

ianballou · 2024-03-20T12:57:19Z

Awesome, I'll give it a re-test

ianballou · 2024-03-20T13:13:36Z

Session is working now!

ianballou · 2024-03-20T13:20:28Z

Just need to check the static type and also test on EL7 to make sure the legacy stuff isn't broken.

ianballou · 2024-03-20T13:26:27Z

Another photo of the traces working on EL9:

ianballou · 2024-03-20T13:33:42Z

I'm a little confused why sudo is being marked as a daemon that needs restarting.

Here is the tracer -a output:

You should restart:
  * Some applications using:
      sudo systemctl restart NetworkManager
      sudo systemctl restart auditd
      sudo systemctl restart chronyd
      sudo systemctl restart crond
      sudo systemctl restart dbus-broker
      sudo systemctl restart gssproxy
      sudo systemctl restart irqbalance
      sudo systemctl restart rhsmcertd
      sudo systemctl restart rpcbind
      sudo systemctl restart sshd
      sudo systemctl restart systemd-journald
      sudo systemctl restart systemd-logind
      sudo systemctl restart systemd-udevd

  * These applications manually:
      (sd-pam)

  * These applications restarting your session:
      bash
      getty@tty1
      ssh-vagrant-session

  * These applications rebooting your computer:
      systemd

Here is the dnf needs-restarting output:

[root@rhel9 katello-host-tools]# dnf needs-restarting
Updating Subscription Management repositories.
1 : /usr/lib/systemd/systemd --system --deserialize 42 
534 : /usr/bin/rpcbind -w -f 
538 : /sbin/auditd 
559 : /usr/sbin/NetworkManager --no-daemon 
564 : /usr/lib/systemd/systemd-logind 
570 : /usr/sbin/chronyd -F 2 
574 : /usr/bin/dbus-broker-launch --scope system --audit 
578 : dbus-broker --log 4 --controller 9 --machine-id 1d74ad9f991d4d3eb78e0b2f33e7d45c --max-bytes 536870912 --max-fds 4096 --max-matches 131072 --audit 
587 : /usr/bin/rhsmcertd 
603 : /sbin/agetty -o -p -- \u --noclear - linux 
1950 : sshd: vagrant [priv] 
1954 : /usr/lib/systemd/systemd --user 
1957 : (sd-pam) 
1964 : sshd: vagrant@pts/0  
1965 : -bash 
1985 : sudo -i -u root 
1987 : -bash 
2068 : /usr/sbin/anacron -s

Also (sd-pam) seems to be an outlier since it's not a service. The old tracer reports it as needing "manual restart".

Here's it in systemctl status

           └─user.slice
             └─user-1000.slice
               ├─session-5.scope
               │ ├─ 1950 "sshd: vagrant [priv]"
               │ ├─ 1964 "sshd: vagrant@pts/0"
               │ ├─ 1965 -bash
               │ ├─ 1985 sudo -i -u root
               │ ├─ 1987 -bash
               │ ├─23256 systemctl status
               │ └─23257 less
               └─[email protected]
                 └─init.scope
                   ├─1954 /usr/lib/systemd/systemd --user
                   └─1957 "(sd-pam)"

ianballou · 2024-03-20T13:38:11Z

I'm thinking sudo might count under the session type?

ianballou · 2024-03-20T19:02:33Z

It looks like with old tracer, sudo is ignored:

vs

[root@rhel7 katello-host-tools]# tracer -a
You should restart:
  * Some applications using:
      sudo systemctl restart NetworkManager
      sudo systemctl restart NetworkManager
      sudo systemctl restart chronyd
      sudo systemctl restart crond
      sudo systemctl restart gssproxy
      sudo systemctl restart irqbalance
      sudo systemctl restart polkit
      sudo systemctl restart rhsmcertd
      sudo systemctl restart rpcbind
      sudo systemctl restart rsyslog
      sudo systemctl restart sshd
      sudo systemctl restart systemd-journald
      sudo systemctl restart systemd-logind
      sudo systemctl restart systemd-udevd
      sudo systemctl restart tuned

  * These applications manually:
      sudo

  * These applications restarting your session:
      bash
      getty@tty1
      ssh-vagrant-session

  * These applications rebooting your computer:
      dbus
      kernel
      systemd

This is on EL7.

ianballou · 2024-03-20T19:03:47Z

I think anything that the old tracer puts under the "manual restart" category can be ignored, like (sd-pam) and sudo.

Co-Authored-by: Ewoud Kohl van Wijngaarden <[email protected]>

sbernhard · 2024-03-22T16:27:28Z

I have added a workaround @ianballou

ianballou

Looking pretty much perfect! Just one small addition:

src/katello/tracer/dnf.py

Co-authored-by: Ian Ballou <[email protected]>

ianballou · 2024-04-02T14:23:21Z

Looking good, doing some final checks

ianballou

Okay, looking good to me!

jeremylenz · 2024-04-02T14:42:01Z

@sbernhard can you please fix the Redmine issue and/or squash? :)

ianballou · 2024-04-02T14:42:44Z

@sbernhard can you please fix the Redmine issue and/or squash? :)

I took care of it during the squash merge 👍

jeremylenz · 2024-04-02T14:43:52Z

oh right, of course squash fixes that 🙈

pr-processor bot added Not yet reviewed Waiting on contributor labels Dec 4, 2023

sbernhard force-pushed the use_dnf branch 2 times, most recently from e928ff2 to 1f722a5 Compare December 4, 2023 12:41

m-bucher approved these changes Dec 7, 2023

View reviewed changes

pr-processor bot removed the Not yet reviewed label Dec 7, 2023

sbernhard force-pushed the use_dnf branch from 1f722a5 to b179459 Compare December 11, 2023 08:15

pr-processor bot added Needs re-review and removed Waiting on contributor labels Dec 11, 2023

sbernhard changed the title ~~Use dnf needs-restarting to collect tracer information~~ Fixes #36973 - Use dnf needs-restarting to collect tracer information Dec 11, 2023

ianballou reviewed Dec 11, 2023

View reviewed changes

src/katello/tracer/SystemdDbus.py Show resolved Hide resolved

ianballou reviewed Dec 11, 2023

View reviewed changes

src/katello/tracer/SystemdDbus.py Show resolved Hide resolved

ianballou reviewed Dec 13, 2023

View reviewed changes

src/katello/tracer/dnf.py Outdated Show resolved Hide resolved

ianballou reviewed Dec 13, 2023

View reviewed changes

src/katello/tracer/dnf.py Outdated Show resolved Hide resolved

ekohl requested changes Dec 18, 2023

View reviewed changes

pr-processor bot added Waiting on contributor and removed Needs re-review labels Dec 18, 2023

sbernhard force-pushed the use_dnf branch from b179459 to a5ded0c Compare December 27, 2023 12:40

pr-processor bot added the Needs re-review label Dec 27, 2023

sbernhard force-pushed the use_dnf branch from a5ded0c to ad72873 Compare December 27, 2023 12:45

sbernhard commented Dec 27, 2023

View reviewed changes

sbernhard force-pushed the use_dnf branch 3 times, most recently from a060d5b to 086dc5a Compare December 27, 2023 13:07

ekohl requested changes Dec 27, 2023

View reviewed changes

src/katello/tracer/dnf.py Outdated Show resolved Hide resolved

ianballou reviewed Mar 15, 2024

View reviewed changes

sbernhard force-pushed the use_dnf branch from 70febc1 to 8d82c21 Compare March 18, 2024 15:44

sbernhard force-pushed the use_dnf branch from 8d82c21 to 977c262 Compare March 20, 2024 12:54

sbernhard force-pushed the use_dnf branch from 977c262 to bea8b81 Compare March 20, 2024 12:56

Fixes #36973 - Use dnf needs-restarting to collect tracer information

ca250ba

Co-Authored-by: Ewoud Kohl van Wijngaarden <[email protected]>

sbernhard force-pushed the use_dnf branch from bea8b81 to ca250ba Compare March 21, 2024 16:28

ianballou reviewed Mar 29, 2024

View reviewed changes

src/katello/tracer/dnf.py Outdated Show resolved Hide resolved

Update src/katello/tracer/dnf.py

d1fbe2c

Co-authored-by: Ian Ballou <[email protected]>

pr-processor bot added the Waiting on contributor label Mar 29, 2024

ianballou approved these changes Apr 2, 2024

View reviewed changes

ianballou merged commit 1408cbd into Katello:master Apr 2, 2024

maximiliankolb deleted the use_dnf branch September 10, 2024 15:07

Fixes #36973 - Use dnf needs-restarting to collect tracer information #149

Fixes #36973 - Use dnf needs-restarting to collect tracer information #149

Uh oh!

Conversation

sbernhard commented Dec 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m-bucher left a comment

Choose a reason for hiding this comment

Uh oh!

ehelms commented Dec 8, 2023

Uh oh!

ianballou commented Dec 8, 2023

Uh oh!

ianballou commented Dec 11, 2023

Uh oh!

Uh oh!

Uh oh!

ehelms commented Dec 12, 2023

Uh oh!

Uh oh!

Uh oh!

ehelms commented Dec 14, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbernhard Dec 27, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ianballou commented Mar 15, 2024

Uh oh!

ianballou left a comment

Choose a reason for hiding this comment

Uh oh!

sbernhard commented Mar 15, 2024

Uh oh!

ianballou commented Mar 18, 2024

Uh oh!

ianballou commented Mar 18, 2024

Uh oh!

sbernhard commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

ianballou commented Mar 20, 2024

Uh oh!

sbernhard commented Mar 22, 2024

Uh oh!

ianballou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ianballou commented Apr 2, 2024

Uh oh!

ianballou left a comment

Choose a reason for hiding this comment

Uh oh!

jeremylenz commented Apr 2, 2024

Uh oh!

sbernhard commented Dec 4, 2023 •

edited

Loading