Goglides Dev 🌱

Balkrishna Pandey
Balkrishna Pandey

Posted on • Originally published at pandeybk.Medium on

Run NMState within a Docker container on an Ubuntu machine

#networkingtools #ubuntu #docker #nmstate

Objective:

I’m attempting to run nmstate within a container in an Ubuntu machine in this blog (Ubuntu Box info below). I partially succeeded, in that I was able to perform several commands but was unable to patch network configuration changes. Why? Continue reading to find out.

$ uname -a
Linux goglides 5.4.0-72-generic #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Enter fullscreen mode Exit fullscreen mode

NMState

Here is my understanding about NMState after reading a few articles (article references given below in reference section.)

NMState is a library that maintains a host’s network in a declarative manner. Simply said, it’s a networking configuration tool that uses the NetworkManager API to apply the appropriate networking configuration state on a host.

NMState has a number of advantages.

  1. Since NMState configure hosts following a declarative approach, it satisfies business demands for host networking management via a northbound declarative API and the tool applies the configuration through a southbound provider.
  2. Multiprovider support in the future; presently, NetworkManager is the only provider supported.
  3. NMState aids in the enablement of Infrastructure as Code and facilitates networking job automation.
  4. It also aids in the reduction of potential human error-related misconfigurations.
  5. Transaction and network configuration verification are also supported by NMState.

If you wish to administer NetworkManager with the Python programming language, you may use the libnmstate Python library, or you can use the nmstatectl command-line tool directly.

Installation

If you want to install NMState on a Centos or Fedora machine, run the following command.

$ sudo dnf install nmstate
 ...
Installed:
  nmstate-1.0.2-14.el8_4.noarch python3-jsonschema-2.6.0-4.el8.noarch python3-libnmstate-1.0.2-14.el8_4.noarch python3-varlink-29.0.0-1.el8.noarch                           

Complete!
Enter fullscreen mode Exit fullscreen mode

And that’s it! Let’s check the version as follows

$ nmstatectl version
1.0.2
Enter fullscreen mode Exit fullscreen mode

If you’re using CentOS/Fedora on the host system, it’s simple. However, I’m attempting to run NMState in a Docker container on an Ubuntu computer. As a result, I built a custom Dockerfile for this purpose.

FROM docker.io/library/centos:8
RUN dnf -y install nmstate
ENTRYPOINT ["/usr/bin/nmstatectl"]
Enter fullscreen mode Exit fullscreen mode

Save the above text to a file called Dockerfile and create the Docker image as follows:

docker build . -t nmstatectl
Enter fullscreen mode Exit fullscreen mode

Then, for regular activities, establish a Linux alias.

alias nmstatectl="docker run -it --privileged=true --net host --volume /var/run/dbus:/var/run/dbus --volume /var/run/docker/netns:/var/run/netns nmstatectl:latest"
Enter fullscreen mode Exit fullscreen mode

Now you should be able to run nmstatectl command as normal.

$ nmstatecli version
1.0.2
Enter fullscreen mode Exit fullscreen mode

Check a networking interface’s current configuration, such as the eno3 configuration:

$ nmstatecli show eno3
---
dns-resolver:
  config: {}
  running: {}
route-rules:
  config: []
routes:
  config: []
  running:
  - destination: fe80::/64
    metric: 256
    next-hop-address: '::'
    next-hop-interface: eno3
    table-id: 254
interfaces:
- name: eno3
  type: ethernet
  state: up
  ethernet:
    sr-iov:
      total-vfs: 0
      vfs: []
  ipv4:
    enabled: false
    address: []
  ipv6:
    enabled: true
    address:
    - ip: fe80::266e:96ff:fe30:dffc
      prefix-length: 64
  mac-address: 24:6E:96:30:DF:FC
  mtu: 1500
Enter fullscreen mode Exit fullscreen mode

The networking setup, as seen above, is divided into four sections:

  • dns-resolver: We’re simply defining nameserver settings for this interface in this section. You might use something like this as an example. 
dns-resolver:
  config:
    search:
    - goglides.com
    - goglides.io
    server:
    - 8.8.8.8
    - 2001:4860:4860::8888
Enter fullscreen mode Exit fullscreen mode
  • route-rules: it specifies the networking interface’s routing rules. Some of the route-rules are as follows: 
route-rules:
  config:
    ip_to: 192.0.2.0/24
    priority: 1000
    route_table: 50
Enter fullscreen mode Exit fullscreen mode
  • routes: There are both dynamic and static routes in this category. 
routes:
  config:
  - destination: 198.51.100.0/24
    metric: 150
    next-hop-address: 192.0.2.1
    next-hop-interface: eth1
    table-id: 254
Enter fullscreen mode Exit fullscreen mode
  • Interfaces: An interface is a network interface that is either hardware or software. This section covers both IPv4 and IPv6 configurations. The examples below illustrate IPv4 setups with static IP addresses and DHCP disabled. 
interfaces:
  - name: eth1
    type: ethernet
    state: up
    ipv4:
      address:
      - ip: 192.0.2.251
        prefix-length: 24
      dhcp: false
      enabled: true
Enter fullscreen mode Exit fullscreen mode

Testing Phase

I was running into some issue when trying to access host network from docker container.

$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-kolla state UP group default qlen 1000
    link/ether 24:6e:96:30:df:fc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::266e:96ff:fe30:dffc/64 scope link 
       valid_lft forever preferred_lft forever


$ nmstatectl show eno3 
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
Failed to query ethtool info: Received a netlink error message No such file or directory (os error 2)
Traceback (most recent call last):
  File "/usr/bin/nmstatectl", line 11, in <module>
    load_entry_point('nmstate==1.0.2', 'console_scripts', 'nmstatectl')()
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 73, in main
    return args.func(args)
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 319, in show
    full_state = libnmstate.show()
  File "/usr/lib/python3.6/site-packages/libnmstate/netinfo.py", line 36, in show
    return show_with_plugins(plugins, include_status_data)
  File "/usr/lib/python3.6/site-packages/libnmstate/nmstate.py", line 79, in show_with_plugins
    plugins, info_type
  File "/usr/lib/python3.6/site-packages/libnmstate/nmstate.py", line 182, in _get_interface_info_from_plugins
    ifaces = plugin.get_interfaces()
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 145, in get_interfaces
    for dev in list_devices(self.client)
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 113, in client
    return self.context.client if self.context else None
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 118, in context
    self._ctx = NmContext()
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/context.py", line 45, in __init__
    self._client = NM.Client.new(cancellable=None)
GLib.Error: g-io-error-quark: Could not connect: No such file or directory (1)
Enter fullscreen mode Exit fullscreen mode

Apparently, my container was not able to access some files in the host system. The thing about NetworkManager is that it runs on dbus. So I tried to mount the host dbus inside the container as follows:

sudo docker run -it --privileged=true --net host --volume /var/run/dbus:/var/run/dbus nmstatecli:latest bash
Enter fullscreen mode Exit fullscreen mode

Now I made some progress and seeing different output this time. At least the program is not crashing and I can see network configuration in yaml format,

$ nmstatectl show br-internal

Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
2021-09-08 18:06:23,076 root WARNING libnm version 1.30.0 mismatches NetworkManager version 1.10.6
2021-09-08 18:06:23,076 root DEBUG NetworkManager version 1.10.6
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
Unhandled IFLA_INFO_DATA for iface type Other("IpTun")
---
dns-resolver:
  config: {}
  running: {}
route-rules:
  config: []
routes:
  config: []
  running:
  - destination: fe80::/64
    metric: 256
    next-hop-address: '::'
    next-hop-interface: br-internal
    table-id: 254
  - destination: 10.40.0.0/22
    metric: 0
    next-hop-address: 0.0.0.0
    next-hop-interface: br-internal
    table-id: 254
interfaces:
- name: br-internal
  type: linux-bridge
  state: up
  bridge:
    options:
      gc-timer: 4591
      group-addr: 01:80:C2:00:00:00
      group-forward-mask: 0
      hash-max: 4096
      hello-timer: 0
      mac-ageing-time: 300
      multicast-last-member-count: 2
      multicast-last-member-interval: 100
      multicast-querier: false
      multicast-querier-interval: 25500
      multicast-query-interval: 12500
      multicast-query-response-interval: 1000
      multicast-query-use-ifaddr: false
      multicast-router: 1
      multicast-snooping: true
      multicast-startup-query-count: 2
      multicast-startup-query-interval: 3124
      stp:
        enabled: false
        forward-delay: 0
        hello-time: 2
        max-age: 20
        priority: 32768
    port:
    - name: br-kolla.103
      stp-hairpin-mode: false
      stp-path-cost: 100
      stp-priority: 32
  ipv4:
    enabled: true
    address:
    - ip: 10.40.0.40
      prefix-length: 22
  ipv6:
    enabled: true
    address:
    - ip: fe80::266e:96ff:fe30:dffc
      prefix-length: 64
  mac-address: 24:6E:96:30:DF:FC
  mtu: 1500
Enter fullscreen mode Exit fullscreen mode

Some reason I keep seeing the following issue during the edit g-dbus-error-quark: No such method ‘AddConnection2’ (19). It is possibly related to the version being miss-matched.

$ nmstatectl edit br-kolla
--- output omitted ---

2021-09-08 18:23:13,741 root DEBUG Applying desire state: {'dns-resolver': {'config': {}, 'running': {}}, 'route-rules': {'config': []}, 'routes': {'config': [{'destination': '0.0.0.0/0', 'metric': 0, 'next-hop-address': '192.168.4.1', 'next-hop-interface': 'br-kolla', 'table-id': 254}], 'running': [{'destination': '0.0.0.0/0', 'metric': 0, 'next-hop-address': '192.168.4.1', 'next-hop-interface': 'br-kolla', 'table-id': 254}, {'destination': '192.168.4.0/22', 'metric': 0, 'next-hop-address': '0.0.0.0', 'next-hop-interface': 'br-kolla', 'table-id': 254}]}, 'interfaces': [{'name': 'br-kolla', 'type': 'linux-bridge', 'state': 'up', 'bridge': {'options': {'gc-timer': 12316, 'group-addr': '01:80:C2:00:00:00', 'group-forward-mask': 0, 'hash-max': 4096, 'hello-timer': 0, 'mac-ageing-time': 300, 'multicast-last-member-count': 3, 'multicast-last-member-interval': 100, 'multicast-querier': False, 'multicast-querier-interval': 25500, 'multicast-query-interval': 12500, 'multicast-query-response-interval': 1000, 'multicast-query-use-ifaddr': False, 'multicast-router': 1, 'multicast-snooping': True, 'multicast-startup-query-count': 2, 'multicast-startup-query-interval': 3124, 'stp': {'enabled': False, 'forward-delay': 0, 'hello-time': 2, 'max-age': 20, 'priority': 32768}}, 'port': [{'name': 'br-kolla-veth', 'stp-hairpin-mode': False, 'stp-path-cost': 2, 'stp-priority': 32}, {'name': 'eno3', 'stp-hairpin-mode': False, 'stp-path-cost': 4, 'stp-priority': 32}]}, 'ipv4': {'enabled': True, 'address': [{'ip': '192.168.4.8', 'prefix-length': 22}]}, 'ipv6': {'enabled': False, 'address': []}, 'mac-address': '24:6E:96:30:DF:FC', 'mtu': 1500}]}


--- output omitted ---

freedesktop/NetworkManager/Checkpoint/2 finished
Traceback (most recent call last):
  File "/usr/bin/nmstatectl", line 11, in <module>
    load_entry_point('nmstate==1.0.2', 'console_scripts', 'nmstatectl')()
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 73, in main
    return args.func(args)
  File "/usr/lib/python3.6/site-packages/nmstatectl/nmstatectl.py", line 303, in edit
    new_state, verify_change=args.verify, save_to_disk=args.save_to_disk
  File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 81, in apply
    _apply_ifaces_state(plugins, net_state, verify_change, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/netapplier.py", line 114, in _apply_ifaces_state
    plugin.apply_changes(net_state, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/plugin.py", line 233, in apply_changes
    NmProfiles(self.context).apply_config(net_state, save_to_disk)
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/profiles.py", line 89, in apply_config
    self._ctx.wait_all_finish()
  File "/usr/lib/python3.6/site-packages/libnmstate/nm/context.py", line 213, in wait_all_finish
    raise tmp_error
libnmstate.error.NmstateLibnmError: Add profile: 33fe0aeb-6fc5-42d7-a7cb-286307488707, iface:br-kolla, type:linux-bridge failed with error: g-dbus-error-quark: No such method 'AddConnection2' (19)
Enter fullscreen mode Exit fullscreen mode

Even if I upgrade Ubuntu, my host NetworkManager version is “1.10.6", and Ubuntu states this is the latest stable version. So I haven't attempted to patch it manually.

In host:

$ sudo NetworkManager --version
1.10.6
Enter fullscreen mode Exit fullscreen mode

Inside the container:

$ NetworkManager --version
1.30.7-27896.copr.183f2a1a59.el8
Enter fullscreen mode Exit fullscreen mode

Anyway, I rebuilt the container in CoreOS using podman, which isn’t the same thing because CoreOS isn’t based on Debian, but I simply wanted to see whether I could modify network configuration from the container.

$ podman build . 
STEP 1: FROM docker.io/library/centos:8
STEP 2: RUN dnf -y install nmstate
--> Using cache 725688f4443408115573ea6754a9bcd743b3b4bae45ce51ed3c93fa4963dad1a
--> 725688f4443
STEP 3: ENTRYPOINT ["/usr/bin/nmstatectl"]
--> Using cache 39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
--> 39a7157ba12
39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
Enter fullscreen mode Exit fullscreen mode

And attached to the container,

podman run -it --privileged=true --entrypoint=bash --net host --volume /var/run/dbus:/var/run/dbus 39a7157ba124db6acb070766bab765d16ca7999acaf150e53d89631a374ea729
Enter fullscreen mode Exit fullscreen mode

Generate config for the existing interface,

$ nmstatectl show eno4 > en04.yaml


# content of eno04.yaml
---
dns-resolver:
  config:
    search: []
    server:
    - 192.168.1.70
  running:
    search: []
    server:
    - 192.168.1.70
route-rules:
  config: []
routes:
  config: []
  running: []
interfaces:
- name: eno4
  type: ethernet
  state: up
  ethernet:
    auto-negotiation: true
    duplex: full
    speed: 1000
    sr-iov:
      total-vfs: 0
      vfs: []
  ipv4:
    enabled: false
    address: []
    dhcp: false
  ipv6:
    enabled: false
    address: []
    autoconf: false
    dhcp: false
  lldp:
    enabled: false
  mac-address: 24:6E:96:19:F3:65
  mtu: 1500
  ovs-db:
    external_ids: {}
Enter fullscreen mode Exit fullscreen mode

Change mtu: 1500 to mtu: 1800 and apply the configuration.

2021-09-08 22:20:53,405 root DEBUG Applying desire state: {'dns-resolver': {'config': {'search': [], 'server': ['192.168.1.70']}, 'running': {'search': [], 'server': ['192.168.1.70']}}, 'route-rules': {'config': []}, 'routes': {'config': [], 'running': []}, 'interfaces': [{'name': 'eno4', 'type': 'ethernet', 'state': 'up', 'ethernet': {'auto-negotiation': True, 'duplex': 'full', 'speed': 1000, 'sr-iov': {'total-vfs': 0, 'vfs': []}}, 'ipv4': {'enabled': False, 'address': [], 'dhcp': False}, 'ipv6': {'enabled': False, 'address': [], 'autoconf': False, 'dhcp': False}, 'lldp': {'enabled': False}, 'mac-address': '24:6E:96:19:F3:65', 'mtu': 1500, 'ovs-db': {'external_ids': {}}}]}

--- output omitted ---

2021-09-08 22:20:58,716 root DEBUG Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 started
2021-09-08 22:20:58,719 root DEBUG Checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 destroyed
2021-09-08 22:20:58,720 root DEBUG Async action: Destroy checkpoint /org/freedesktop/NetworkManager/Checkpoint/8 finished
Desired state applied: 
---
--- output omitted ---
...
  mtu: 1800
  ovs-db:
    external_ids: {}
Enter fullscreen mode Exit fullscreen mode

This demonstrates I was able to change it from the container. Check the following:

$ ip link show eno4
3: eno4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1800 qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 24:6e:96:19:f3:65 brd ff:ff:ff:ff:ff:ff
Enter fullscreen mode Exit fullscreen mode

References:

https://nmstate.io/

https://fedoramagazine.org/nmstate-a-declarative-networking-config-tool/

https://resources.ovirt.org/site-files/2020/Using\_NetworkManager\_for\_host\_networking\_in\_oVirt\_44.pdf

https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/8/html/configuring\_and\_managing\_networking/assembly\_introduction-to-nmstate\_configuring-and-managing-networking

https://stackoverflow.com/questions/52654962/nmcli-in-a-docker-container

https://github.com/intelsdi-x/snap-plugin-collector-ethtool/issues/17

Top comments (0)

Read next

devleader profile image

When Should Code Be Refactored? What You Need To Know

Dev Leader -

dappdiva profile image

Knowing About P2P Trading & Crypto Exchange

Debasmita Ghosh -

thattechguy profile image

Transforming Agriculture: The Impact of Blockchain Technology on the Supply Chain

Aryan K Khanna -

etherempress profile image

Navigating the Crystal Ball of AI: Can AI Algorithms Really Predict Your Future?

Anushree Mitra -