[Centos9] Add support for prometheus node-exporter container #54

andypitcher · 2025-01-02T16:43:44Z

Targeted Env: Centos9
Context: The Monitoring chart in Rancher can be used with SELinux enabled, however with the container-selinux policy installed (comes with k3s-selinux and rancher-selinux), the node-exporter container inherits container_t, which is not allowed to run several tasks. This makes the Monitoring app to stop running.

Changes: This PR introduces the addition of a new type prom_node_exporter_t along with the required rules to allow node-exporter to run with the least permissions.

Example of AVC denials that come along the assignment of prom_node_exporter_t to the node-exporter container, and that are addressed in this PR.

AVCs

1. avc:  denied  { read } for  pid=243339 comm="k3s-server" scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:prom_node_exporter_t:s0:c339,c717 tclass=file permissive=1
2. avc:  denied  { open } for  pid=243339 comm="k3s-server" path="/proc/1249142/limits" dev="proc" ino=29581995 scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:prom_node_exporter_t:s0:c339,c717 tclass=file permissive=1
3. avc:  denied  { getattr } for  pid=243339 comm="k3s-server" path="/proc/1249142/limits" dev="proc" ino=29581995 scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:system_r:prom_node_exporter_t:s0:c339,c717 tclass=file permissive=1
4. avc:  denied  { read } for  pid=1249174 comm="node_exporter" path="socket:[29581173]" dev="sockfs" ino=29581173 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tclass=tcp_socket permissive=1
5. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="stat" dev="proc" ino=29581175 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tclass=file permissive=1
6. avc:  denied  { open } for  pid=1249174 comm="node_exporter" path="/proc/1249174/stat" dev="proc" ino=29581175 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tclass=file permissive=1
7. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="net" dev="proc" ino=4026531845 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:proc_net_t:s0 tclass=lnk_file permissive=1
8. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="self" dev="proc" ino=4026531842 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:proc_t:s0 tclass=lnk_file permissive=1
9. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="clocksource" dev="sysfs" ino=7504 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:sysfs_t:s0 tclass=dir permissive=1
10. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="netstat" dev="proc" ino=4026532056 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:proc_net_t:s0 tclass=file permissive=1
11. avc:  denied  { open } for  pid=1249174 comm="node_exporter" path="/host/proc/1249174/net/netstat" dev="proc" ino=4026532056 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:proc_net_t:s0 tclass=file permissive=1
12. avc:  denied  { getattr } for  pid=1249174 comm="node_exporter" path="/host/sys/fs/xfs/stats/stats" dev="sysfs" ino=19937 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:sysfs_t:s0 tclass=file permissive=1
13. avc:  denied  { read } for  pid=1249174 comm="node_exporter" name="stats" dev="sysfs" ino=20003 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:sysfs_t:s0 tclass=file permissive=1
14. avc:  denied  { open } for  pid=1249174 comm="node_exporter" path="/host/sys/fs/xfs/vda1/stats/stats" dev="sysfs" ino=20003 scontext=system_u:system_r:prom_node_exporter_t:s0:c272,c355 tcontext=system_u:object_r:sysfs_t:s0 tclass=file permissive=1

The Monitoring chart in Rancher can be used with SELinux enabled, however with the container-selinux policy installed the node-exporter container inherits container_t, which is not allowed to run several tasks. This commit adds a new type prom_node_exporter_t along with the required rules to allow node-exporter to run with least permissions.

andypitcher · 2025-01-03T13:55:32Z

cc @ca-hu for review

ca-hu · 2025-01-09T19:17:33Z

done with the review :) ftr i did not have a test system so i did not check if permissions are missing, but i believe you can do that better than me anyways :)

ca-hu · 2025-01-09T17:11:15Z

policy/centos9/rancher.te

+############################################################################
+require {
+	type container_runtime_t;
+	type prom_node_exporter_t;


we might not need this type prom_node_exporter_t; in the require block since it is defined below

ca-hu · 2025-01-09T17:32:57Z

policy/centos9/rancher.te

+# type prom_node_exporter_t   				 	           #
+# target: prometheus-node-exporter container for Rancher monitoring chart  #
+############################################################################
+require {


nitpick: above gen_require is used, here require. should not make a difference, but maybe for consistency

not only for this PR, but generally in this file: i am a bit confused why there are many duplications stated in the gen_requires sections. they should not be needed i think. not sure if you want to split the file up in the future or what the plan is/was here? usually its just one block at the top

ca-hu · 2025-01-09T18:32:19Z

policy/centos9/rancher.te

+dev_list_sysfs(prom_node_exporter_t)
+dev_read_sysfs(prom_node_exporter_t)
+files_read_etc_symlinks(prom_node_exporter_t)
+init_read_state(prom_node_exporter_t)
+kernel_read_network_state(prom_node_exporter_t)
+kernel_read_network_state_symlinks(prom_node_exporter_t)
+kernel_read_proc_files(prom_node_exporter_t)
+kernel_read_proc_symlinks(prom_node_exporter_t)
+kernel_read_software_raid_state(prom_node_exporter_t)
+libs_read_lib_files(prom_node_exporter_t)
+selinux_read_security_files(prom_node_exporter_t)


make sense as the node_exporter reads system state, ok

ca-hu · 2025-01-09T18:34:44Z

policy/centos9/rancher.te

+corenet_tcp_bind_generic_node(prom_node_exporter_t)
+corenet_tcp_bind_generic_port(prom_node_exporter_t)


makes sense, ok

ca-hu · 2025-01-09T18:56:01Z

policy/centos9/rancher.te

+	class fifo_file write;
+}
+type prom_node_exporter_t;
+container_domain_template(prom_node_exporter_t, container)


this needs to be

container_domain_template(prom_node_exporter, container)

otherwise you will get this (note the _t_t suffix):

seinfo -x -a container_domain | grep prom_node prom_node_exporter_t_t

ca-hu · 2025-01-09T18:57:45Z

policy/centos9/rancher.te

+	class fd use;
+	class fifo_file write;
+}
+type prom_node_exporter_t;


when you fix the container_domain_template you dont need this line anymore

ca-hu · 2025-01-09T19:03:32Z

policy/centos9/rancher.te

+allow container_runtime_t prom_node_exporter_t:dir { open read search };
+allow container_runtime_t prom_node_exporter_t:file { getattr open read };
+allow container_runtime_t prom_node_exporter_t:key { create search setattr view };
+allow container_runtime_t prom_node_exporter_t:lnk_file { getattr read };
+allow container_runtime_t prom_node_exporter_t:process { noatsecure rlimitinh siginh sigkill signal transition };
+allow prom_node_exporter_t container_runtime_t:fd use;
+allow prom_node_exporter_t container_runtime_t:fifo_file write;


i think these are not needed if you fix the container_domain_template line because they should be already included

you can check with sesearch -A -s container_runtime_t -t prom_node_exporter_t

ca-hu · 2025-01-09T19:07:34Z

policy/centos9/rancher.te

+allow prom_node_exporter_t self:dir { getattr search };
+allow prom_node_exporter_t self:file { open read };
+allow prom_node_exporter_t self:lnk_file read;
+allow prom_node_exporter_t self:netlink_route_socket { bind create getattr getopt nlmsg_read read write };
+allow prom_node_exporter_t self:process fork;
+allow prom_node_exporter_t self:tcp_socket { accept bind create getattr listen read setopt write };


probably these as well

ca-hu · 2025-01-09T19:12:23Z

policy/centos9/rancher.te

+allow prom_node_exporter_t self:netlink_route_socket { bind create getattr getopt nlmsg_read read write };
+allow prom_node_exporter_t self:process fork;
+allow prom_node_exporter_t self:tcp_socket { accept bind create getattr listen read setopt write };
+container_runtime_typebounds(prom_node_exporter_t)


interesting that this is needed, did you see a nnp_transition? if so i did not find the denial, could you paste it for my curiousity? thanks :)

ca-hu · 2025-01-09T19:13:44Z

policy/centos9/rancher.te

+type prom_node_exporter_t;
+container_domain_template(prom_node_exporter_t, container)
+virt_sandbox_domain(prom_node_exporter_t)
+allow container_runtime_t prom_node_exporter_t:dir { open read search };


final nitpick, might make sense to do interfaces first and allow rules afterwards just for consistency

ca-hu · 2025-01-10T12:01:52Z

sorry, forgot to click "submit review", so you probably only saw my last comment....

* ref: rancher#54

andypitcher marked this pull request as ready for review January 3, 2025 11:08

andypitcher requested a review from a team as a code owner January 3, 2025 11:08

andypitcher self-assigned this Jan 3, 2025

pjbgf approved these changes Jan 10, 2025

View reviewed changes

andypitcher merged commit 5cc1352 into rancher:main Jan 10, 2025
1 check passed

andypitcher changed the title ~~Add support for prometheus node-exporter container~~ [Centos9] Add support for prometheus node-exporter container Jan 10, 2025

ca-hu reviewed Jan 10, 2025

View reviewed changes

andypitcher mentioned this pull request Jan 15, 2025

Refine the policy and add other distro support for the Prometheus Node Exporter #55

Merged

andypitcher added a commit to andypitcher/rancher-selinux that referenced this pull request Mar 7, 2025

policy: add prometheus_node_export_t policy for centos8

3f5dd11

* ref: rancher#54

andypitcher added a commit to andypitcher/rancher-selinux that referenced this pull request Mar 7, 2025

policy: add prometheus_node_export_t policy for centos8

cc8b400

* ref: rancher#54

andypitcher mentioned this pull request Mar 14, 2025

[Monitoring] Add new SELinux type prom_node_exporter_t to node-exporter when SELinux is enabled rancher/ob-team-charts#44

Closed

		corenet_tcp_bind_generic_node(prom_node_exporter_t)
		corenet_tcp_bind_generic_port(prom_node_exporter_t)

[Centos9] Add support for prometheus node-exporter container #54

[Centos9] Add support for prometheus node-exporter container #54

Uh oh!

Conversation

andypitcher commented Jan 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andypitcher commented Jan 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ca-hu commented Jan 9, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ca-hu commented Jan 10, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

andypitcher commented Jan 2, 2025 •

edited

Loading

andypitcher commented Jan 3, 2025 •

edited

Loading