To run Awanio, several supporting infrastructure components are needed, such as good quality servers, network devices to connect between nodes in the cluster, and storage. For storage devices, Awanio can use local storage (DAS), network storage (SAN) and distributed storage (Ceph).

The operating system required to run Awanio is Ubuntu Server version 20.04.

• Intel EMT64 or AMD64 with Intel VT/AMD-V CPU flag. 8 cores minimum, 16 cores or above preferred.
• For PCI(e) passthrough CPU configuration with VT-d/AMD-d CPU flag must be enabled.
• Memory, minimum 16 GB for OS and services used by Awanio. The remaining memory will be allocated for guests. For Ceph or ZFS additional memory is required, at least 1 GB of memory for every 1 TB of used storage.
• Fast and redundant storage, it is recommended to use SSD disks.
• Disk Capacity: at least 2 disks are required in each node. The first is for the OS boot disk and the other is for data. For OS disk at least 50GB minimum, 100GB preferred. And for minimum 120GB data disk, recommended for more than 500GB. Make sure the disk has 5,000+ random IOPS performance.
• For OS storage: use Hardware RAID with batteries protected write cache (“BBU”) or if it is non-RAID use ZFS and SSD cache.
• For VM storage: If local storage, use hardware RAID with battery backed write cache (BBU) or for non-RAID use ZFS.
• Redundant Gbit NICs.
• Three NICs

• CPU: 64bit (Intel EMT64 or AMD64), 8 cores minimum.
• Intel VT/AMD-V capable CPU/Mainboard (for KVM Full Virtualization support)
• Minimum 16GB of RAM
• Two SSD Hard drives. One for the OS and one for workload data.
• Two NICs

Each node that is joined to the cluster must have at least 2 NICs. The first NIC will be used for internal cluster management. The second NIC is used for external networking between workloads inside the cluster and workloads outside the cluster. The NIC name used in each node must be the same.

In a public cluster environment, the external network is a public IP that will be used by each vm.

In a private cluster environment, the external network is the IP range that is shared with servers or VMs that are outside the cluster. In this topology model, if it is necessary to ingress from a public IP, then one or several nodes must have a third NIC connected directly to the WAN. This IP is then defined in the ingressExternalIPs directive in the cluster-config file.

In this model, external requests can only be made via Ingress Service and Load Balancer service.

Awanio uses Ceph as SDS in managing storage. There are two options for storage configurations that can be used, i.e. internal and external.

This configuration is where some or all of the nodes joined in the custer act as storage nodes. The storage used can be a local disk with the SSD or NVME type or an external distributed storage type.

In order to configure the storage node, at least one of these local storage types is required:

• Raw devices (no partitions or formatted filesystems)
• Raw partitions (no formatted filesystem)
• LVM Logical Volumes (no formatted filesystem)

To check, you can use the lsblk -f command

Information in FSTYPE must be empty.

$ lsblk -f
NAME                  FSTYPE      LABEL UUID                                   MOUNTPOINT
vda
└─vda1                LVM2_member       >eSO50t-GkUV-YKTH-WsGq-hNJY-eKNf-3i07IB
  ├─ubuntu--vg-root   ext4              c2366f76-6e21-4f10-a8f3-6776212e2fe4   /
  └─ubuntu--vg-swap_1 swap              9492a3dc-ad75-47cd-9596-678e8cf17ff9   [SWAP]
vdb

In the example above, vdb disk can be used.

To make a disk or partition raw disk or raw formatted, use the following command:

DISK="/dev/sdX"

# Zap the disk to a fresh, usable state (zap-all is important, b/c MBR has to be clean)
sgdisk --zap-all $DISK

# Wipe a large portion of the beginning of the disk to remove more LVM metadata that may be present
dd if=/dev/zero of="$DISK" bs=1M count=100 oflag=direct,dsync

# SSDs may be better cleaned with blkdiscard instead of dd
blkdiscard $DISK

# Inform the OS of partition table changes
partprobe $DISK

External storage must use the LUN protocol. Disks that have been attached to the storage node must be ensured in raw device format. To be sure, run the following commands:

sudo multipath -ll

The following command is to make a raw disk:

sudo multipath -f mpathb

To activate the disk, run the command:

sudo multipath

Todo…

• The user used must be sudo user
• The sudo user should be able to run commands without needing to enter a password again. Check out the guide here..
• Each node must have other’s SSH private keys.

Each Node must be in a swap-off state.

1. Identify configured swap devices and files with cat /proc/swaps.
2. Turn off all swap devices and files with swapoff -a.
3. Remove any matching reference found in /etc/fstab.

License key is required for the installation process. To get a license please login to https://partner.awan.io/.

The following is an example of a configuration for a cluster installation.

apiVersion: awanio.io/v1alpha1
kind: ClusterConfig
metadata:
  license: "xxx-xxx-xxx-xxx"
  features: "computes,object-storage,kubernetes,support"
spec:
  rootUser:
    name: Awanio
    username: awanio
    password: xxx-xxx-xxx
    email: ask@awan.io
  server:
    ingressExternalIPs:
      - 172.31.31.19
      - 108.137.91.215
    balancer:
      name: eth2
      ip: 108.137.91.215/24
    nics:
      internal:
        name: ens5
      external:
        name: eth1
        ip: 103.179.254.253/24
        mac: 16:52:f3:13:6a:25
        gateway: 103.179.254.254/24
        gatewayNodes: awid1
    subnets:
      workload: 10.0.0.0/16
      workloadGateway: 10.0.0.1
      service: 10.1.0.0/16
      external: 103.179.255.0/24
    nodes:
      - name: awid1
        role: control-plane-worker
        internalIP: 172.31.31.19/20
        disk:
          - /dev/nvme1n1
      - name: awid3
      # the option is: control-plane, worker and control-plane-worker
        role: worker
        # this user should sudoers and have the public ssh key in remote servers
        user: ubuntu
        sshPort: "22"
        sshPrikeyFile: /home/ubuntu/.ssh/id_rsa
        internalIP: 172.31.17.44/20
        disk:
          - /dev/nvme1n1
    ExternalCephCluster:
      fsid: null
      mondata: null
      adminSecret: null

Awanio provides a downloadable CLI format installer here.

wget https://storage.awan.io/assets/awctl-linux-amd64.tar.gz
tar -xvf awctl-linux-amd64.tar.gz
sudo chmod +x ./awctl
sudo mv ./awctl /usr/local/bin/
sudo awctl --help

After all the configuration files are ready, run the following command to start the installation process:

sudo awctl -f cluster-config.yaml

Once the process completed, the output would be:

+----------------------------------------------------+
|                Awanio cluster access               |
+-----------------------+----------------------------+
| Dashboard             | http://172.31.31.19:32337/ |
+-----------------------+----------------------------+
| Platform              | http://172.31.31.19:31080/ |
+-----------------------+----------------------------+
| API                   | http://172.31.31.19/v2     |
+-----------------------+----------------------------+