Data Retention

📘

Why do we need Data Retention?

Learn more about Data Retention Policy and its importance in the organisation.

Organisations need to mitigate risks and threats to the security of their data while complying with industry regulations and standards. Because of this, they need to have data retention policies in place.

Data retention policies define, among others, what types of data they need to keep or archive, where and how long will these be kept, and what happens after the archival period passes. Data backup is one of the processes that help implement these policies.

maaiiconnect's Data Retention feature lets Staff administrators regularly create a backup of data stored within the service account and securely move them outside of the service (e.g., to their locally-hosted servers or other preferred backup locations).

 

System Requirements

maaiiconnect has very minimal or lightweight system requirements for its data retention feature. You can immediately configure data retention as long as you have an SFTP server that is readily available and has enough storage to back up your system data.

Recommended Setup:
Hardware: H/W or Virtual Machine (CPU Core 2; RAM 4GB; Storage 1TB +; OS Ubuntu)
Software: OpenSSH and SFTP Server
MongoDB version: v4.2

High Availability (HA) Setup:
You will need three (3) separated Hardware (H/W)
Two (master/slave): H/W or Virtual Machine (CPU Core 2; RAM 4GB; Storage 1TB +; OS Ubuntu)
One (arbiter): H/W or Virtual Machine (CPU Core 1; RAM 500MB; Storage 200G +; OS Ubuntu)

Virtual Machine Requirements

  • You will need a VM Platform:

    VMware ESXi version 6.0 or later
    KVM (CentOS 6.6, 6.7, or 6.8 as the KVM Hypervisor)

  • Virtual machine resource (H/W requirements mentioned above)

Network Requirements

  • Setup IPSec Tunnel
  • Setup a SFTP Server
  • Setup the DNAT to SFTP Sever for the upload service

 

Enabling Data Retention

To enable Data Retention:

  • From the navigation menu, go to Administration > Security > Retention.

  • On the Retention page, under the Settings tab, turn the toggle button under Retention Settings ON. A new set of configuration options will then appear.

Retention Settings

  • Under General:
    • Type in the maximum number of days the data will be retained in your maaiiconnect service account
    • Beside Retention schedule, on the dropdown box, select the hour of the day (in UTC) you would like to back up the data.
    • Create an Archive Key that will be used to open/unzip the downloaded backup file. The key should have at least eight characters containing uppercase and lowercase letters, numbers, and symbols.

General Settings

  • Configure Online mode or Offline mode or both settings, depending on what you have requested to provisioned to your maaiiconnect service. Refer to the succeeding steps below for more information.

 

Data Retention Modes

The data retention feature currently supports the following backup modes:

Online Mode

The customer's backup data are transferred to their SFTP server and synced to their own MongoDB. The data can be viewed through the maaiiconnect Retention Portal.

Online mode retention portal login

To configure Online mode settings:

  • Under Retention Portal Permissions, set the administrator password for your maaiiconnect minisite.
  • Under MongoDB Permissions, type the following details of your MongoDB database:
    • Host
    • User
    • Password
    • AuthDB
    • TargetDB

Online mode settings for Data Retention Portal

  • You may click on Test this configuration now to start a test connection.

 

Offline Mode

The customer's backup data are transferred to their SFTP server and viewed in a raw JSON file format.

Offline mode sample retained data

To configure offline mode settings via SFTP Information:

  • On the SFTP User text box, type in the user name that will be used to log in to the SFTP server where you will archive your data.
  • On the SFTP Address text box, type in your SFTP server's IP address and port (port 22).
  • Click on the SFTP Public Key text box to copy the key. This serves as the SFTP server's authentication to automatically back up your data there.
    • You may also click on Generate New Key to get a new SFTP key.

Offline mode (SFTP) settings

  • You may click on Test this configuration now to start a test connection.

 

Data Types and Backup File Structure

Below are the categories of data that will be backed up:

  • Call and message histories: includes chats and call logs in all chatrooms, shared multimedia files, voicemail, call recording, conference calls, and others.
  • Contact lists: includes Internal/Staff and External Contacts. Contact lists will be backed up but will not be deleted.
  • User accounts: includes Staff members' user profiles, personal Weblink and QR codes, display names, routing settings, and others. User accounts will be backed up but will not be deleted.

 

For Online Mode, when you access the Retention Portal site, you will be able to view:

  • Call History
  • Conference Call History
  • File
  • Voicemail
  • Call Recording
  • Text Message
  • Staff
  • External Contacts
  • and, Account management

Online mode - Retention Portal

For more details about online mode retained data, please refer to the Retention Portal section of this Guide.

 

For Offline Mode, the data archived in a ZIP file has the following naming format:

backup_{service ID}_{retenion job ID}_{backup date and time}.zip

Where:

  • service ID: refers to the unique maaiiconnect ID of your Service Account
  • retention job ID: refers to a 36-character system-generated ID that maaiiconnect triggers when the data backup and retention is initiated
  • backup date and time: refers to the date and time the backup was initiated, in yyyymmddhhmmss format

Sample maaiiconnect backup file downloaded into a local machine

When you open the ZIP file, it will contain the following folders and their respective contents:

Folder Name

Contents

lc-data-server

Database records of Staff List and closed enquiries

history

Database records of chat, direct call, and enquiry call histories

file-management

Database records of files shared in rooms and other service files such as staff avatars and tag icons

dynamic-phonebook

Database records of the External Contact List

data-retention

All files shared in rooms, such as images, audio and video files, and documents

 

Backup Log

Once the Data Retention Settings is enabled, maaiiconnect proceeds to back up the data to the SFTP server according to the set schedule, it also creates a Backup Log after every upload.

Backup Log

To view the Backup Log:

  • From the navigation menu, go to Administration > Security > Retention.
  • On the Retention page, click on the Backup Log tab.

The Backup Log contains the following details:

  • Date and time the backup was created
  • File Name
  • Total Data Size
  • Total number of backup files
  • Status

 

Retention Status

The Retention Status in the Backup Log indicates the backup data's actual condition in your local server.

Retention Status

The retention status message may be any of the following:

Status Message

Description

Uploading backup...

Data in maaiiconnect has been backed up and is currently being uploaded to the SFTP server(s).

Backup uploaded

Data backup has been completed. A Delete from the maaiiconnect button appears, prompting Staff administrators to proceed with the retention process's next step.

Deleting data...

maaiiconnect is removing all backed up data from its system.

Failed to delete data

An error occurred while trying to remove the data from maaiiconnect. A Try again button appears, prompting Staff administrators to repeat the process.

Completed

maaiiconnect has finished removing the backed up data from its system.

 

Removing Data from maaiiconnect

To remove the backed up data from maaiiconnect, on the Backup Log, click on the Delete from maaiiconnect button next to the Backup uploaded status of the backup file. maaiiconnect will then process this request and display a Deleting data... status.

Deleting data from maaiiconnect

Once the data have been successfully deleted, maaiiconnect will change its status to Completed. Staff members will no longer view or access the deleted data, such as messages in a chatroom.

Sample notification indicating that the data have been deleted from maaiiconnect

 

Disabling Data Retention

To disable Data Retention:

  • From the navigation menu, go to Administration > Security > Retention.

  • On the Retention page, under the Settings tab, turn the toggle button under Retention Settings OFF.

Disable Data Retention

Updated about a month ago



Data Retention


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.