refact(apis): refact NDM apis to adhere to k8s standard (openebs-arch…

…ive#301)

* refact(apis): refact BD and BDC apis to adhere to k8s api definition
* chore(code-gen): autogenerated code
* fix typo in block device apis

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

refact(apis): refact NDM apis to adhere to k8s standard (openebs-arch…

…ive#301)

* refact(apis): refact BD and BDC apis to adhere to k8s api definition
* chore(code-gen): autogenerated code
* fix typo in block device apis

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

feat(upgrade): add pre-upgrade tasks for 0.4.1 to 0.4.2 (openebs-arch…

…ive#303)

* fix node hostname label
* refactor the upgrade package. 
* add pre-upgrade steps for 0.4.1 to 0.4.2

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

fix(daemon, controller): change nodeName to hostname in disk and BD (o…

…penebs-archive#290)

get the hostname of the node from the node object and use it while
creating CRs

if hostname label is not set, or has empty string, use node name as the
hostname

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

fix(daemon, controller): change nodeName to hostname in disk and BD (o…

…penebs-archive#290)

get the hostname of the node from the node object and use it while
creating CRs

if hostname label is not set, or has empty string, use node name as the
hostname

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

fix(controller): add extra checks before removing finalizer (openebs-…

…archive#283)

To prevent accidental deletion of BDC, the BDC controller will now perform an additional check to make sure that the finalizer added by NDM will be the last one to be removed from the BDC object.

The new workflow will be:
- NDM Operator
1. NDM Operator will be watching all the BDCs
2. If DeletionTimeStamp is set on the BDC, then, Controller will check for the no.of finalizers on the BDC, if it has only one finalizer, then check whether NDM finalizer is present. If yes, the BD will be marked as released and the NDM finalizer will be removed.
3. NDM operator will make sure that, its finalizer is the last one to be removed from the BDC

- Maya (Owner of BDC)
1. Maya will create the BDC and add a finalizer to it.
2. When pool/localpv is deleted and cleanedup, this finalizer should be removed by maya, and will delete the BDC
3. Once deletion timestamp is set, NDM operator takes up the work.

This approach will give partial control of how the BDC should be deleted to the owner of BDC. If the user who creates the claim, want to prevent accidental deletion, user can add a finalizer but then it will be the responsibility of the user to remove it.

This approach also makes sure that BD will be marked as Released only when no one else is using the corresponding BDC.


Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

fix(cleaner): change dd to wipefs (openebs-archive#273)

Fixed high cpu/io utilization of cleanup job. Using dd caused heavy load on the CPU, and prevented other pods from getting scheduled on the node. It also caused kube-api server crash at times. By using wipefs instead of dd only the file-system signatures will be erased. This process is much faster than writing zeros, since we are only working with certain block(superblocks).

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

fix(cleaner): change dd to wipefs (openebs-archive#273)

Fixed high cpu/io utilization of cleanup job. Using dd caused heavy load on the CPU, and prevented other pods from getting scheduled on the node. It also caused kube-api server crash at times. By using wipefs instead of dd only the file-system signatures will be erased. This process is much faster than writing zeros, since we are only working with certain block(superblocks).

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

feat(scrub-job): scrub job to clean data on released block devices (o…

…penebs-archive#267)

When a BDC was deleted, the BD was unclaimed and could be claimed by other BDCs. In this case the data on the BD remains. This PR introduces a scrub job which will delete the data on the BD.

The new workflow will be like: When a BDC is deleted, the BD will be marked as Released. The reconcile loop will pick up all BDs in released state and creates a scrub job for those BDs. If a job is already in progress, then new job will not be created.

The job can perform 2 opertions depending on the BD:
- If the BD is used as a raw block device, a dd command will be issued on the BD to wipe out all the data. This takes significantly long duration.
- If the BD is used as a volume i.e it was preformatted and mounted before claiming, then an rm -rf will be issued by the job. Only the data will be removed. The filesystem on the BD will remain as it is. This job is much quicker than wiping the disk

Once the job is completed, the BD will be marked as Unclaimed and can be claimed by other BDCs.

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>

feat(bdd, deviceclaim): implemented BDD for device claim. (openebs-ar…

…chive#257)

1) Added YAMLs for integration testing
2) Udev functions for simulating devices. The udev functions could be
used to trigger fake udev events so that ndm detects the events and
updates the CRs. functions will be required to create a fake disk,
trigger add/remove/change events
3) sanity tests will be done in a single node minikube cluster and
functions for interfacing with the cluster is included.The cluster
start and stop is not included as this will be done by the VM.
Status of cluster, and generation of kube-config files will be checked
4) package to query kube-API server and retrieve results in a predefined
format is included. The various resources like configmap, CRD etc can
also be created using this package. These resources will be created
using the YAMLs
5) The basic tests include:
        a) running NDM and checking for pod
        b) checking for sparse disks
        c) check for external disk already attached
        d) check for external disk attached at runtime

Signed-off-by: Akhil Mohan <akhil.mohan@mayadata.io>