Objective:
I’m attempting to run nmstate within a container in an Ubuntu machine in this blog (Ubuntu Box info below). I partially succeeded, in that I was able to perform several commands but was unable to patch network configuration changes. Why? Continue reading to find out.
$ uname -a
Linux goglides 5.4.0-72-generic #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
NMState
Here is my understanding about NMState after reading a few articles (article references given below in reference section.)
NMState is a library that maintains a host’s network in a declarative manner. Simply said, it’s a networking configuration tool that uses the NetworkManager API to apply the appropriate networking configuration state on a host.
NMState has a number of advantages.
- Since NMState configure hosts following a declarative approach, it satisfies business demands for host networking management via a northbound declarative API and the tool applies the configuration through a southbound provider.
- Multiprovider support in the future; presently, NetworkManager is the only provider supported.
- NMState aids in the enablement of Infrastructure as Code and facilitates networking job automation.
- It also aids in the reduction of potential human error-related misconfigurations.
- Transaction and network configuration verification are also supported by NMState.
If you wish to administer NetworkManager with the Python programming language, you may use the libnmstate Python library, or you can use the nmstatectl command-line tool directly.
Installation
If you want to install NMState on a Centos or Fedora machine, run the following command.
$ sudo dnf install nmstate
...
Installed:
nmstate-1.0.2-14.el8_4.noarch python3-jsonschema-2.6.0-4.el8.noarch python3-libnmstate-1.0.2-14.el8_4.noarch python3-varlink-29.0.0-1.el8.noarch
Complete!
And that’s it! Let’s check the version as follows
$ nmstatectl version
1.0.2
If you’re using CentOS/Fedora on the host system, it’s simple. However, I’m attempting to run NMState in a Docker container on an Ubuntu computer. As a result, I built a custom Dockerfile for this purpose.
FROM docker.io/library/centos:8
RUN dnf -y install nmstate
ENTRYPOINT ["/usr/bin/nmstatectl"]
Save the above text to a file called Dockerfile and create the Docker image as follows:
docker build . -t nmstatectl
Then, for regular activities, establish a Linux alias.
alias nmstatectl="docker run -it --privileged=true --net host --volume /var/run/dbus:/var/run/dbus --volume /var/run/docker/netns:/var/run/netns nmstatectl:latest"
Now you should be able to run nmstatectl command as normal.
$ nmstatecli version
1.0.2
Check a networking interface’s current configuration, such as the eno3 configuration:
$ nmstatecli show eno3
---
dns-resolver:
config: {}
running: {}
route-rules:
config: []
routes:
config: []
running:
- destination: fe80::/64
metric: 256
next-hop-address: '::'
next-hop-interface: eno3
table-id: 254
interfaces:
- name: eno3
type: ethernet
state: up
ethernet:
sr-iov:
total-vfs: 0
vfs: []
ipv4:
enabled: false
address: []
ipv6:
enabled: true
address:
- ip: fe80::266e:96ff:fe30:dffc
prefix-length: 64
mac-address: 24:6E:96:30:DF:FC
mtu: 1500
The networking setup, as seen above, is divided into four sections:
- dns-resolver: We’re simply defining nameserver settings for this interface in this section. You might use something like this as an example.
dns-resolver:
config:
search:
- goglides.com
- goglides.io
server:
- 8.8.8.8
- 2001:4860:4860::8888
- route-rules: it specifies the networking interface’s routing rules. Some of the route-rules are as follows:
route-rules:
config:
ip_to: 192.0.2.0/24
priority: 1000
route_table: 50
- routes: There are both dynamic and static routes in this category.
routes:
config:
- destination: 198.51.100.0/24
metric: 150
next-hop-address: 192.0.2.1
next-hop-interface: eth1
table-id: 254
- Interfaces: An interface is a network interface that is either hardware or software. This section covers both IPv4 and IPv6 configurations. The examples below illustrate IPv4 setups with static IP addresses and DHCP disabled.
interfaces:
- name: eth1
type: ethernet
state: up
ipv4:
address:
- ip: 192.0.2.251
prefix-length: 24
dhcp: false
enabled: true
Testing Phase
I was running into some issue when trying to access host network from docker container.
$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-kolla state UP group default qlen 1000
link/ether 24:6e:96:30:df:fc brd ff:ff:ff:ff:ff:ff
inet6 fe80::266e:96ff:fe30:dffc/64 scope link
valid_lft forever preferred_lft forever
$ nmstatectl show eno3
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
Failed to query ethtool info: Received a netlink error message No such file or directory (os error 2)
Traceback (most recent call last):
File "/usr/bin/nmstatectl", line 11, in <module>
load_entry_point('nmstate==1.0.2', 'console_scripts', 'nmstatectl')()
File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 73, in main
return args.func(args)
File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 319, in show
full_state = libnmstate.show()
File "/usr/lib/python3.6/site-packages/libnmstate/netinfo.py", line 36, in show
return show_with_plugins(plugins, include_status_data)
File "/usr/lib/python3.6/site-packages/libnmstate/nmstate.py", line 79, in show_with_plugins
plugins, info_type
File "/usr/lib/python3.6/site-packages/libnmstate/nmstate.py", line 182, in _get_interface_info_from_plugins
ifaces = plugin.get_interfaces()
File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 145, in get_interfaces
for dev in list_devices(self.client)
File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 113, in client
return self.context.client if self.context else None
File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 118, in context
self._ctx = NmContext()
File "/usr/lib/python3.6/site-packages/libnmstate/nm/context.py", line 45, in __init__
self._client = NM.Client.new(cancellable=None)
GLib.Error: g-io-error-quark: Could not connect: No such file or directory (1)
Apparently, my container was not able to access some files in the host system. The thing about NetworkManager is that it runs on dbus. So I tried to mount the host dbus inside the container as follows:
sudo docker run -it --privileged=true --net host --volume /var/run/dbus:/var/run/dbus nmstatecli:latest bash
Now I made some progress and seeing different output this time. At least the program is not crashing and I can see network configuration in yaml format,
$ nmstatectl show br-internal
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
2021-09-08 18:06:23,076 root WARNING libnm version 1.30.0 mismatches NetworkManager version 1.10.6
2021-09-08 18:06:23,076 root DEBUG NetworkManager version 1.10.6
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
---
dns-resolver:
config: {}
running: {}
route-rules:
config: []
routes:
config: []
running:
- destination: fe80::/64
metric: 256
next-hop-address: '::'
next-hop-interface: br-internal
table-id: 254
- destination: 10.40.0.0/22
metric: 0
next-hop-address: 0.0.0.0
next-hop-interface: br-internal
table-id: 254
interfaces:
- name: br-internal
type: linux-bridge
state: up
bridge:
options:
gc-timer: 4591
group-addr: 01:80:C2:00:00:00
group-forward-mask: 0
hash-max: 4096
hello-timer: 0
mac-ageing-time: 300
multicast-last-member-count: 2
multicast-last-member-interval: 100
multicast-querier: false
multicast-querier-interval: 25500
multicast-query-interval: 12500
multicast-query-response-interval: 1000
multicast-query-use-ifaddr: false
multicast-router: 1
multicast-snooping: true
multicast-startup-query-count: 2
multicast-startup-query-interval: 3124
stp:
enabled: false
forward-delay: 0
hello-time: 2
max-age: 20
priority: 32768
port:
- name: br-kolla.103
stp-hairpin-mode: false
stp-path-cost: 100
stp-priority: 32
ipv4:
enabled: true
address:
- ip: 10.40.0.40
prefix-length: 22
ipv6:
enabled: true
address:
- ip: fe80::266e:96ff:fe30:dffc
prefix-length: 64
mac-address: 24:6E:96:30:DF:FC
mtu: 1500
Some reason I keep seeing the following issue during the edit g-dbus-error-quark: No such method ‘AddConnection2’ (19). It is possibly related to the version being miss-matched.
$ nmstatectl edit br-kolla
--- output omitted ---
2021-09-08 18:23:13,741 root DEBUG Applying desire state: {'dns-resolver': {'config': {}, 'running': {}}, 'route-rules': {'config': []}, 'routes': {'config': [{'destination': '0.0.0.0/0', 'metric': 0, 'next-hop-address': '192.168.4.1', 'next-hop-interface': 'br-kolla', 'table-id': 254}], 'running': [{'destination': '0.0.0.0/0', 'metric': 0, 'next-hop-address': '192.168.4.1', 'next-hop-interface': 'br-kolla', 'table-id': 254}, {'destination': '192.168.4.0/22', 'metric': 0, 'next-hop-address': '0.0.0.0', 'next-hop-interface': 'br-kolla', 'table-id': 254}]}, 'interfaces': [{'name': 'br-kolla', 'type': 'linux-bridge', 'state': 'up', 'bridge': {'options': {'gc-timer': 12316, 'group-addr': '01:80:C2:00:00:00', 'group-forward-mask': 0, 'hash-max': 4096, 'hello-timer': 0, 'mac-ageing-time': 300, 'multicast-last-member-count': 3, 'multicast-last-member-interval': 100, 'multicast-querier': False, 'multicast-querier-interval': 25500, 'multicast-query-interval': 12500, 'multicast-query-response-interval': 1000, 'multicast-query-use-ifaddr': False, 'multicast-router': 1, 'multicast-snooping': True, 'multicast-startup-query-count': 2, 'multicast-startup-query-interval': 3124, 'stp': {'enabled': False, 'forward-delay': 0, 'hello-time': 2, 'max-age': 20, 'priority': 32768}}, 'port': [{'name': 'br-kolla-veth', 'stp-hairpin-mode': False, 'stp-path-cost': 2, 'stp-priority': 32}, {'name': 'eno3', 'stp-hairpin-mode': False, 'stp-path-cost': 4, 'stp-priority': 32}]}, 'ipv4': {'enabled': True, 'address': [{'ip': '192.168.4.8', 'prefix-length': 22}]}, 'ipv6': {'enabled': False, 'address': []}, 'mac-address': '24:6E:96:30:DF:FC', 'mtu': 1500}]}
--- output omitted ---
freedesktop/NetworkManager/Checkpoint/2 finished
Traceback (most recent call last):
File "/usr/bin/nmstatectl", line 11, in <module>
load_entry_point('nmstate==1.0.2', 'console_scripts', 'nmstatectl')()
File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 73, in main
return args.func(args)
File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 303, in edit
new_state, verify_change=args.verify, save_to_disk=args.save_to_disk
File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 81, in apply
_apply_ifaces_state(plugins, net_state, verify_change, save_to_disk)
File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 114, in _apply_ifaces_state
plugin.apply_changes(net_state, save_to_disk)
File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 233, in apply_changes
NmProfiles(self.context).apply_config(net_state, save_to_disk)
File "/usr/lib/python3.6/site-packages/libnmstate/nm/profiles.py", line 89, in apply_config
self._ctx.wait_all_finish()
File "/usr/lib/python3.6/site-packages/libnmstate/nm/context.py", line 213, in wait_all_finish
raise tmp_error
libnmstate.error.NmstateLibnmError: Add profile: 33fe0aeb-6fc5-42d7-a7cb-286307488707, iface:br-kolla, type:linux-bridge failed with error: g-dbus-error-quark: No such method 'AddConnection2' (19)
Even if I upgrade Ubuntu, my host NetworkManager version is “1.10.6", and Ubuntu states this is the latest stable version. So I haven't attempted to patch it manually.
In host:
$ sudo NetworkManager --version
1.10.6
Inside the container:
$ NetworkManager --version
1.30.7-27896.copr.183f2a1a59.el8
Anyway, I rebuilt the container in CoreOS using podman, which isn’t the same thing because CoreOS isn’t based on Debian, but I simply wanted to see whether I could modify network configuration from the container.
$ podman build .
STEP 1: FROM docker.io/library/centos:8
STEP 2: RUN dnf -y install nmstate
--> Using cache 725688f4443408115573ea6754a9bcd743b3b4bae45ce51ed3c93fa4963dad1a
--> 725688f4443
STEP 3: ENTRYPOINT ["/usr/bin/nmstatectl"]
--> Using cache 39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
--> 39a7157ba12
39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
And attached to the container,
podman run -it --privileged=true --entrypoint=bash --net host --volume /var/run/dbus:/var/run/dbus 39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
Generate config for the existing interface,
$ nmstatectl show eno4 > en04.yaml
# content of eno04.yaml
---
dns-resolver:
config:
search: []
server:
- 192.168.1.70
running:
search: []
server:
- 192.168.1.70
route-rules:
config: []
routes:
config: []
running: []
interfaces:
- name: eno4
type: ethernet
state: up
ethernet:
auto-negotiation: true
duplex: full
speed: 1000
sr-iov:
total-vfs: 0
vfs: []
ipv4:
enabled: false
address: []
dhcp: false
ipv6:
enabled: false
address: []
autoconf: false
dhcp: false
lldp:
enabled: false
mac-address: 24:6E:96:19:F3:65
mtu: 1500
ovs-db:
external_ids: {}
Change mtu: 1500 to mtu: 1800 and apply the configuration.
2021-09-08 22:20:53,405 root DEBUG Applying desire state: {'dns-resolver': {'config': {'search': [], 'server': ['192.168.1.70']}, 'running': {'search': [], 'server': ['192.168.1.70']}}, 'route-rules': {'config': []}, 'routes': {'config': [], 'running': []}, 'interfaces': [{'name': 'eno4', 'type': 'ethernet', 'state': 'up', 'ethernet': {'auto-negotiation': True, 'duplex': 'full', 'speed': 1000, 'sr-iov': {'total-vfs': 0, 'vfs': []}}, 'ipv4': {'enabled': False, 'address': [], 'dhcp': False}, 'ipv6': {'enabled': False, 'address': [], 'autoconf': False, 'dhcp': False}, 'lldp': {'enabled': False}, 'mac-address': '24:6E:96:19:F3:65', 'mtu': 1500, 'ovs-db': {'external_ids': {}}}]}
--- output omitted ---
2021-09-08 22:20:58,716 root DEBUG Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 started
2021-09-08 22:20:58,719 root DEBUG Checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 destroyed
2021-09-08 22:20:58,720 root DEBUG Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 finished
Desired state applied:
---
--- output omitted ---
...
mtu: 1800
ovs-db:
external_ids: {}
This demonstrates I was able to change it from the container. Check the following:
$ ip link show eno4
3: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1800 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000
link/ether 24:6e:96:19:f3:65 brd ff:ff:ff:ff:ff:ff
References:
https://nmstate.io/
https://fedoramagazine.org/nmstate-a-declarative-networking-config-tool/
https://resources.ovirt.org/site-files/2020/Using\_NetworkManager\_for\_host\_networking\_in\_oVirt\_44.pdf
https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/8/html/configuring\_and\_managing\_networking/assembly\_introduction-to-nmstate\_configuring-and-managing-networking
https://stackoverflow.com/questions/52654962/nmcli-in-a-docker-container
https://github.com/intelsdi-x/snap-plugin-collector-ethtool/issues/17
Top comments (0)