ENIB S7-NSA 06_infrastructure - Infrastructure services

{} Network setup

First, you need to recreate the network architecture from the previous tutorial.
Before doing so, wait until the instructor has started the device that will interconnect all participants in the room.

Instructor workstation configuration

The instructor will then tell you the name of the physical host running this device.
This name is represented by teacher in the following diagram, which you should follow:

First, start your three switches with s7nsa_hub_switch.py s:9991 s:9992 s:9993 and keep this terminal open.
Once the switches are started, from another terminal, start your four machines:

s7nsa_machine.py A 9991
s7nsa_machine.py B 9992
s7nsa_machine.py C 9993
s7nsa_machine.py D teacher:9999 9991 9992 9993 # substitute teacher

Note that for your machine D, the symbolic name teacher must be replaced with the name of the physical host indicated by the instructor.

Record the number corresponding to your host system name✍: 7 for laurier7, 12 for citron12, 4 for pomme4, 23 for thym23...
This number is represented by XX in the diagram, as part of the address assigned to the eth0 interface of machine D✍.
On the host system, download and install the file tuto_06_setup.sh into the SHARED directory.
Log in to root account (with the s7nsa password) of your machines A, B, C and D, and run the command sh SHARED/tuto_06_setup.sh XX, replacing XX with the number you have just recorded from your host system.
This script configures these machines in a similar way to the configuration obtained at the end of the previous tutorial.
In addition to assigning the addresses shown in the diagram, routing is configured accordingly, and the following firewall rules are applied on router D:

Address translation:
- The “Internal servers” subnet, when communicating with destinations outside your network, is seen as having the public address 100.64.XX.1✍
  XX is the number you recorded from your host system.
  .
- Machine C, in the “DMZ” subnet, is exposed as an HTTP server (TCP-80) with the public address 100.64.XX.2✍
  XX is the number you recorded from your host system.
  .
- Machine C, in the “DMZ” subnet, is exposed as a DNS server (UDP-53) with the public address 100.64.XX.3✍
  XX is the number you recorded from your host system.
  .
Filtering:
- By default, unless a rule states otherwise, all packets are dropped.
- Router D responds to ping requests.
- Machine C, in the “DMZ” subnet, is reachable over HTTP (TCP-80).
- Machine C, in the “DMZ” subnet, is reachable over DNS (UDP-53).
- Router D relays DHCP traffic (UDP-67,68).
- The “Internal servers” subnet can communicate without restriction with any destination.
- The “Workstations” subnet can access machine A over DNS (UDP-53).
- The “Workstations” subnet can access machine A over HTTP-proxy (TCP-8888).

At this stage, the initial network architecture is in place:

Services in the “DMZ” subnet will be accessible from anywhere.
The “Workstations” subnet provides workstations with open access for users.
The “Internal servers” subnet orchestrates most of the infrastructure.

Use ping to verify that your machine A can reach:

your machines B and C,
the external interface of your router 100.64.0.XX✍
XX is the number you recorded from your host system.
,
the instructor's device 100.64.0.254,
the external interface of another participant's router 100.64.0.YY✍
YY is the number the other participant recorded from their host system.
.

{} Internal DNS

In the previous tutorial, all communication relied on the explicit use of assigned IP addresses.
In practice, this is rather inconvenient, and it is very common to assign names to machines so that they can be identified by names that humans find easier to remember.
If, from machine A, you try the commands ping A, ping B, ping C or ping D you will see that these names are currently unknown.

Additional activity: local resolution

The usual way to assign names to machines is to deploy a DNS server.
On machine A, write the configuration for such a server as follows✍:

XX=1234 # substitute with your host number
cat <<EOF >/etc/named.conf
options {
  dnssec-validation no;
  listen-on-v6 { none; };
  allow-transfer { 127.0.0.1; };
  directory "/var/named";
};
zone "s7nsa-${XX}.lab" IN { type master; file "s7nsa-${XX}.lab.zone"; };
EOF

Be sure to set the XX variable to the number you recorded from your host system.
This configuration specifies that the server is authoritative for the domain s7nsa-XX.lab✍ and that the file /var/named/s7nsa-XX.lab.zone✍ should be used to describe its contents.
This file should be prepared as follows✍:

XX=1234 # substitute with your host number
SERIAL=`date '+%Y%m%d'`01
cat <<EOF >/var/named/s7nsa-${XX}.lab.zone
\$TTL 1H
@        IN  SOA    ns admin ( ${SERIAL} 1H 15M 1W 1H )
@        IN  NS     ns
ns       IN  A      192.168.10.2
A        IN  A      192.168.10.2
proxy    IN  CNAME  A
B        IN  A      192.168.20.2
C        IN  A      192.168.30.2
www      IN  CNAME  C
D        IN  A      192.168.10.1
router   IN  CNAME  D
EOF
chown root:named /var/named/s7nsa-${XX}.lab.zone

Be sure to set the XX variable to the number you recorded from your host system.
The serial number is important when multiple interacting DNS servers are updated; it is usually made up of the date and a counter that is incremented each time a modification is made.
Without going into too much detail, you should identify:

an NS record specifying the name of the DNS server itself✍
An A record must exist to provide its address.
,
some A records specifying the IP addresses associated with the names ns, A, B, C and D✍
It is perfectly valid to associate the same address with multiple names; for example ns and A here.
,
some CNAME records defining aliases for other names.

This service can now be enabled with the command systemctl enable --now named.service✍.
Verify with ps aux and ss -panu that the named process is running and bound to UDP port 53.

Machine A can now use its own DNS service to resolve domain names✍:

XX=1234 # substitute with your host number
cat <<EOF >/etc/resolv.conf
nameserver 127.0.0.1
search s7nsa-${XX}.lab
EOF

Be sure to set the XX variable to the number you recorded from your host system.
The nameserver entry specifies the IP address of the DNS server to query; here it is the local machine.
The search entry is a convenience feature that automatically completes a relative domain name (B, www...) into a fully qualified domain name (FQDN) (B.s7nsa-XX.lab, www.s7nsa-XX.lab...).
For example, try the command host www; a query to determine the address of www.s7nsa-XX.lab is sent to the DNS server on the local machine. Since this server is authoritative for the domain s7nsa-XX.lab, it responds according to its configuration file: it is an alias for C.s7nsa-XX.lab, whose address is then returned.
The configuration of this server allows zone transfers from the local machine; use dig @localhost s7nsa-XX.lab AXFR✍ to list the entire contents of this zone.

The firewall allows this DNS server to be queried from the “Workstations” subnet.
Now configure machine B to query this DNS server✍:

XX=1234 # substitute with your host number
cat <<EOF >/etc/resolv.conf
nameserver 192.168.10.2
search s7nsa-${XX}.lab
EOF

Be sure to set the XX variable to the number you recorded from your host system.
Machine B can now query this server using host A, host ns, host proxy, host C, host www...

Additional activity: reverse DNS

Although this is less commonly useful, it is also possible to request reverse DNS lookups from the DNS server.
This consists of finding the domain name corresponding to an IP address.

If you wish to experiment with this feature, append the following new zone to the /etc/named.conf file✍:

zone "168.192.in-addr.arpa" IN { type master; file "168.192.zone"; };

The file /var/named/168.192.zone for this zone should be prepared as follows✍:

XX=1234 # substitute with your host number
SERIAL=`date '+%Y%m%d'`01
cat <<EOF >/var/named/168.192.zone
\$TTL 1H
@  IN  SOA  ns.s7nsa-${XX}.lab. admin.s7nsa-${XX}.lab. ( ${SERIAL} 1H 15M 1W 1H )
@  IN  NS   ns.s7nsa-${XX}.lab.
2.10  IN  PTR  A.s7nsa-${XX}.lab.
2.20  IN  PTR  B.s7nsa-${XX}.lab.
2.30  IN  PTR  C.s7nsa-${XX}.lab.
1.10  IN  PTR  D.s7nsa-${XX}.lab.
1.20  IN  PTR  D.s7nsa-${XX}.lab.
1.30  IN  PTR  D.s7nsa-${XX}.lab.
EOF
chown root:named /var/named/168.192.zone

Be sure to set the XX variable to the number you recorded from your host system.
The serial number is important when multiple interacting DNS servers are updated; it is usually made up of the date and a counter that is incremented each time a modification is made.
Without going into too much detail, you should identify:

an NS record specifying the FQDN of the DNS server itself,
some PTR records specifying the FQDN for some IP addresses✍
Note that only the least significant part of the address is specified to complete the zone name, again in reverse order.
Multiple addresses may be associated with the same FQDN.
.

This service can now be restarted with the command systemctl restart named.service.

From machines A or B, try reverse DNS lookups such as host 192.168.10.2, host 192.168.20.2, host 192.168.30.2...

{} Automatic configuration

This series of tutorials uses only the minimum number of machines required to introduce a few concepts.
However, it is very common to have hundreds or even thousands of machines to manage.
Manually specifying IP addresses, routes, and DNS information on every machine is simply not a practical approach.
In practice, machines use a DHCP client which broadcasts a configuration request on the local subnet.
A DHCP server receiving such a message exchanges information with the client to provide its configuration (address, route, DNS...).

Machine A will act as the DHCP server; configure it as follows✍:

XX=1234 # substitute with your host number
MAC_C="aa:bb:cc:dd:ee:ff" # substitute with MAC address of C/eth0
cat <<EOF >/etc/kea/kea-dhcp4.conf
{
  "Dhcp4": {
    "interfaces-config": {
      "interfaces": [ "eth0" ],
      "service-sockets-max-retries": 200000,
      "service-sockets-retry-wait-time": 1000
    },
    "ddns-generated-prefix": "client",
    "ddns-replace-client-name": "always",
    "subnet4": [
      {
        "id": 1,
        "subnet": "192.168.20.0/24",
        "option-data": [
          { "name": "routers", "data": "192.168.20.1" },
          { "name": "domain-name-servers", "data": "192.168.10.2" },
          { "name": "domain-name", "data": "s7nsa-${XX}.lab" }
        ],
        "pools": [ { "pool": "192.168.20.100 - 192.168.20.254" } ]
      },
      {
        "id": 2,
        "subnet": "192.168.30.0/24",
        "option-data": [ { "name": "routers",  "data": "192.168.30.1" } ],
        "reservations": [
          {
            "hw-address": "${MAC_C}",
            "ip-address": "192.168.30.2",
            "hostname": "C"
          }
        ]
      }
    ]
  }
}
EOF

Be sure to set the XX variable to the number you recorded from your host system and MAC_C with the mac address of interface eth0 on machine C.
The sections relating to the “Workstations” and “DMZ” subnets contain the appropriate default route information for each subnet.
The section relating to the “Workstations” subnet contains the information intended for the clients' /etc/resolv.conf file.
It also contains a range of IP addresses; each machine on this subnet that requests its configuration will be assigned an arbitrary IP address from this range.
The section relating to the “DMZ” subnet contains a reservation for the MAC address of machine C's eth0 interface; if this MAC address requests a configuration, it will be assigned the reserved IP address and domain name (all other requests on this subnet will be rejected).
This service can now be enabled with the command systemctl enable --now kea-dhcp4.service✍.
Verify with ps aux and ss -panu that the kea-dhcp4 process is running and bound to UDP port 67.

However, the DHCP clients managed by this server are located in the “Workstations” and “DMZ” subnets, whereas the DHCP server is in the “Internal servers” subnet.
Under these circumstances, the broadcast requests sent by the clients cannot reach the server✍.
Router D can facilitate the process by relaying the broadcast requests to the DHCP server.
Configure it as follows✍:

cat <<'EOF' >>/etc/systemd/system/dhcp-helper.service
[Unit]
Description=daemon for DHCP relay agent

[Service]
Type=forking
PIDFile=/run/dhcp-helper.pid
ExecStart=/usr/bin/dhcp-helper -r /run/dhcp-helper.pid -s 192.168.10.2

[Install]
WantedBy=multi-user.target
EOF

Note that the dhcp-helper service takes as a parameter the address of the DHCP server to which it must relay requests. This service can now be enabled with the command systemctl enable --now dhcp-helper.service✍.
Verify with ps aux and ss -panu that the dhcp-helper process is running and bound to UDP port 67.

The client machines can now be configured to use automatic configuration.
On machine B, remove the two DNS settings that you previously added to the /etc/resolv.conf file.
On machines B and C, edit the script /etc/custom/10_network_setup.sh and remove the commands and comments about ip addr add... and ip route add....
Add the dhcpcd -4 -n eth0 instead; this is the DHCP client program.
Rerun the /etc/custom/10_network_setup.sh script on machines B and C, then observe the exchanges with the DHCP server.

Using ip addr, verify that a new address has been assigned from the range defined on the server for machine B, while machine C has correctly retained its specifically assigned address.
In both cases, ip route should show a default route appropriate for the subnet.
The command hostname should show a new generated name on machine B and still the chosen name C on machine C.
On machine B, cat /etc/resolv.conf should point to your DNS server and its domain.
Observe that most host commands that worked previously still produce the same results, even though the only configuration on machines B and C now consists of using DHCP.
On machine A, the ping C command should still work but not ping B since its network configuration now uses an arbitrary address; however, ping to this new address should work.

{} Public DNS

The “DMZ” subnet is used to expose services outside your network; it is therefore necessary for the names of these services to be resolvable from outside so that their addresses can be determined.
Your DNS server on A is dedicated to internal use and is not accessible from outside.
Furthermore, for security reasons, it is out of the question to disclose the details of the internal infrastructure to the rest of the world.

A second DNS server, this time for public use, must therefore be installed in the “DMZ” subnet.
On machine C, prepare it as follows✍:

XX=1234 # substitute with your host number
cat <<EOF >/etc/named.conf
options {
  dnssec-validation no;
  listen-on-v6 { none; };
  allow-transfer { 127.0.0.1; };
  directory "/var/named";
};
zone "s7nsa-${XX}.lab" IN { type master; file "s7nsa-${XX}.lab.zone"; };
EOF

SERIAL=`date '+%Y%m%d'`01
cat <<EOF >/var/named/s7nsa-${XX}.lab.zone
\$TTL 1H
@        IN  SOA    ns admin ( ${SERIAL} 1H 15M 1W 1H )
@        IN  NS     ns
ns       IN  A      100.64.9.3
www      IN  A      100.64.9.2
EOF
chown root:named /var/named/s7nsa-${XX}.lab.zone

Be sure to set the XX variable to the number you recorded from your host system.
This service can now be enabled with the command systemctl enable --now named.service✍.
Verify with ps aux and ss -panu that the named process is running and bound to UDP port 53.
The configuration of this server allows zone transfers from the local machine; use dig @localhost s7nsa-XX.lab AXFR✍ to list the entire contents of this zone.
Note that this zone exposes ns (for DNS) and www (for HTTP) as public addresses, even though both services will be provided by machine C using a private address; this is consistent with the network address translation rules configured on the router/firewall at the very beginning of this tutorial.

For such a server in the DMZ to be useful, it must receive queries originating from outside your network.
The various participants'A machines are precisely authorised by their network firewall to communicate freely with the outside world.
On your machine A, edit the appropriate file using nano /etc/named.conf (or vi if you are already familiar with it) and add the two following lines into the option block:

  allow-recursion { 127.0.0.1; 192.168.10.0/24; 192.168.20.0/24; };
  forwarders { 100.64.0.254; }; forward only;

Recursions are queries that do not concern the s7nsa-XX.lab domain, for which this server is authoritative; they are permitted from the local machine and the “Internal servers” and “Workstations” subnets.
Recursions are performed here by forwarding queries to the DNS server on the instructor's device, which can be regarded as the Internet service provider for the purposes of this tutorial✍.
Restart your DNS server on machine A with systemctl restart named.service.

Under these new conditions, observe that commands such as host poivre0.enib.fr or host web.enib.fr work from your A or B machines.
Indeed, your DNS server on A is not authoritative for the .enib.fr domain, so it performs a recursion to the instructor's DNS server, which is itself not authoritative for this domain and therefore performs a recursion to the school's physical DNS server, which is authoritative and provides the answer.
Similarly, you should observe that commands such as host www.nginx.org or host www.rust-lang.org also work; since the school's DNS server is not authoritative for the requested domains, it forwards the queries to the root servers, which will direct them to the authoritative servers for those domains.

The instructor's DNS server is configured to know about the participants' public DNS servers: the various s7nsa-XX.lab domains have 100.64.XX.3 as their authoritative server, and you have just configured yours on machine C.
By coordinating with other participants, your A or B machines should be able to successfully perform queries such as host ns.s7nsa-YY.lab or host www.s7nsa-YY.lab, where YY represents the number these participants recorded from their host system.

{} Application services

Now that the essential infrastructure is in place, it is possible to build upon it to provide application services.
On the host system, download and place the index.html, login.cgi and nginx_http.conf files in the SHARED directory.
Machine C must prepare the content and configuration of an Nginx HTTP server as follows✍:

rm -rf /srv/http
mkdir -p /srv/http/cgi-bin /var/log/nginx
cp SHARED/index.html /srv/http
cp SHARED/login.cgi /srv/http/cgi-bin
chmod +x /srv/http/cgi-bin/login.cgi
chown -R http:http /srv/http /var/log/nginx
cp SHARED/nginx_http.conf /etc/nginx/nginx.conf
systemctl enable --now nginx.service

Verify with ps aux and ss -pant that the nginx process is running and listening on TCP port 80.

From your A machine, you can visit the website hosted by this HTTP server with lynx http://www/.
Thanks to the search setting in /etc/named.conf, www is understood as www.s7nsa-XX.lab✍, and this text-mode browser visits the HTTP server on your C machine.
Similarly, by coordinating with other participants, you should be able to visit the website they host using lynx http://www.s7nsa-YY.lab/ where YY represents the number these participants recorded from their host system.
Of course, since you have already observed through DNS resolution that the instructor's device is connected to the external network, your browser can also visit external websites using lynx http://www.nginx.org/ or lynx http://www.rust-lang.org/ for example.

The firewall on your D router allows unrestricted access to the HTTP server on your C machine; workstations in the “Workstations” subnet therefore have access to it.
Verify that your B machine can successfully use lynx http://www/.
However, access to other websites from your B machine is filtered by your firewall; only DNS resolution is permitted.

The firewall does, however, allow the use of an HTTP proxy on machine A from hosts in the “Workstations” subnet.
Such an HTTP proxy can be used directly with its minimal configuration on machine A; start it with systemctl enable --now tinyproxy.service.
Verify with ps aux and ss -pant that the tinyproxy process is running and listening on TCP port 8888.

On machine B, using the commands export http_proxy="http://proxy:8888/" and export https_proxy="${http_proxy}" sets two environment variables that the browser will consult when communicating over HTTP and HTTPS.
Their effect will be to contact the HTTP proxy that you have just started✍ and ask it to retrieve the requested websites.
Then, on machine B, run commands such as lynx http://www.s7nsa-YY.lab/✍, lynx http://www.nginx.org/ or lynx http://www.rust-lang.org/, and observe that the websites are successfully accessed.

On machine A, monitor the HTTP proxy activity log with journalctl -feu tinyproxy.service while repeating the visits from machine B.
Observe that the HTTP proxy records the websites that are visited.
This is one of the reasons for using an HTTP proxy; if the authorities detect malicious activity originating from your network, the network administrator must be able to provide such logs in an attempt to identify the person responsible for the offending actions.

Another very common use of an HTTP proxy is to filter the websites that may be visited✍.
To test this capability, for example use the command echo 'nginx[.]org' >>/etc/tinyproxy/filter on machine A to add a filter.
Then restart the HTTP proxy with systemctl restart tinyproxy.service and once again monitor its activity with journalctl -feu tinyproxy.service.
On machine B, observe that you can still visit various websites, but that http://www.nginx.org is blocked; the HTTP proxy activity log should report this refusal.

Additional activity: remote administration

When the fleet of machines (physical or virtual) becomes large, the only practical way to administer it is to operate remotely over the network.
For systematic automated operations, tools such as Chef or Ansible are commonly used.
However, for troubleshooting or occasional diagnostics, it is necessary to take control of the remote machine in order to execute commands as though interacting with it physically.

The usual entry point for this is the SSH suite of utilities.
On machines C and D, start an SSH server with systemctl enable --now sshd.service.
Verify with ps aux and ss -pant that the sshd process is running and listening on TCP port 22.

Since the “Internal servers” subnet is considered trustworthy and tightly controlled, the firewall grants it unrestricted access to the other machines.
From machine A, you can then request access to a remote machine using ssh www or ssh router.
The first time you connect to a remote machine, you will be asked for confirmation; simply answer yes, and a key identifying that remote machine will be stored in order to verify future connections.
To connect to a remote machine, you must authenticate as though you were connecting to it directly; enter the root password of the target machine to obtain an interactive shell on that remote machine.
Use hostname and ip addr to verify that the commands you enter are indeed executed on the target machine and not on machine A; leave this interactive shell with exit to return to machine A.

If you frequently work on remote machines, having to enter passwords every time is inconvenient (and even discouraged).
This can be avoided by providing a way to prove that the remote operation originates from a trusted account (the root account on machine A here).
To do this, generate a key pair in the root account on machine A using the ssh-keygen command; simply press Enter in response to each prompt.
Verify that the key pair is present in the account with ll .ssh.
To copy the public key to the remote machines, use ssh-copy-id www and ssh-copy-id router; at this stage you will still need to enter the password to authenticate.
Try ssh www and ssh router again to observe that the password is no longer requested when the connection originates from the root account on machine A.

The SSH suite of utilities is not limited to obtaining an interactive shell.
It can also be used to execute a single remote command, such as ssh www ip addr or ssh router hostname.
It is also possible to transfer files using the scp or sftp commands (consult the documentation if you wish to use them).
Another very useful feature is secure tunnelling (consult the documentation if you wish to use it).