Fabric Minimum System Requirements for Development and QA Installations

Introduction

  • This document provides information regarding the hardware, software and the operating system requirements for Fabric installations.
  • TDM installation is based on Fabric with the additions of TDM library and PostgreSQL installation.

Fabric Execution Servers - Node Specifications

The Fabric server installation is a Linux-based solution. A dedicated servers for Fabric instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined .

Minimum System Configuration for Each Fabric Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher.

Hardware Requirements:

Processor

For simple scenarios: Intel Xeon Quad-core or equivalent.

For high-load or complex scenarios: Octa-cores are  recommended.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 150 GB free disk space is recommended.

Network

1 Gbps network speed between the Fabric nodes, Cassandra Nodes, Kafka nodes and source and destination Dbs.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.4xlarge

Google - GCP

D16d v4

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration must be as follows:

  • Volume of 150G /opt/apps/fabric/ - will be used also as the home directory for a Fabric user.

Kafka Server Node Specifications

The Kafka server installation is a Linux-based solution. A dedicated servers for Kafka instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined.

Minimum System Configuration for Each Fabric Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.

Hardware Requirements:

Processor

For simple scenarios: Intel Xeon Quad-core or equivalent.

For high-load or complex scenarios: Octa-cores are  recommended.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 50 GB free disk space is recommended.

Network

1 Gbps network speed between the Kafka nodes and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration must be as follows:

Volume of 50G /opt/apps/kafka/ - will be used also as the home directory for a Kafka user.

Cassandra Server Node Specifications

The Cassandra server installation is a Linux-based solution. A dedicated servers for Cassandra instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined.

Minimum System Configuration for Each Cassandra Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.
  • Python version 2.7 (this is a Cassandra limit; until Cassandra version 4.0.x - Python version 2.7 is the latest supported; newer versions of Cassandra are not yet certified by K2view for Fabric)

Hardware Requirements:

Processor

For most scenarios: Intel Xeon Octa-core CPU or equivalent.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 200 GB free disk space is recommended.

Network

1 Gbps network speed between the Kafka nodes and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration should be as follows:

  • Volume of 200G /opt/apps/cassandra/

PostgreSQL Server Node Specifications

  • PostgreSQL is required for TDM projects only.
  • K2view supports PostgreSQL version 9.6 & 13.x .
  • PostgreSQL can be deployed in one of the following two ways:
    • On-premise as a virtual or physical machine.
    • As a cloud service (SaaS)

Suggested System Configuration for PostgreSQL Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.

Hardware Requirements:

Processor

Intel Xeon quad-core or equivalent.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 100 GB free disk space is required.

Disk space can be increased according to the project scope and retention requirements.

Network

1 Gbps network speed between the PostgreSQL and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.xlarge, a1.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-4

Example of Cloud Supported SaaS

  • AWS RDS PostgreSQL - read more here
  • GCP Cloud SQL PostgreSQL - read more here
  • Azure Database for PostgreSQL - read more here

Fabric Studio Requirements:

Fabric Studio is used to configure, manage and control the Fabric server operation and its installation is Windows-based. The application can be installed locally on either a workstation or a server with Terminal Services for RDP connection.

Minimum System Configuration

Operation System

Workstation: Windows 10 Professional 64 Bit or higher.

Server: Windows server standard 2012 64 bit or higher.

Processors

Intel I3 quad-core 5th Gen or equivalent.

Physical Memory (RAM)

8 GB RAM

Disk Space

100 GB available disk space.

 

Windows Ports

The following ports should be open on the Windows server:

  • 3389 - used for RDP

Windows Permissions

Local administrator privileges are needed for the Fabric Studio installation.

Text/Code editor

Visual Studio Code / notepad++

https://code.visualstudio.com/

https://notepad-plus-plus.org/downloads/

file archiver

7zip

https://7-zip.org/

FTP/SCP client

winSCP

https://winscp.net/eng/download.php

SSH client

Putty

https://www.putty.org/

Git Shell

TortoiseGit

https://tortoisegit.org/

Java IDE

Intellij IDE (community edition):

https://www.jetbrains.com/idea/download/#section=windows

Linux Servers Ports

Open the following ports on the Linux server and make sure they are accessible from Management stations.

Depending on your setup, not all ports are required:

Ports Number

Role

22

SSH, SCP

3213, 9443, 5124, 6379

Fabric

9042, 9142

Cassandra

9093, 7270

Kafka

3000, 9090

Grafana stack

5432

PostgreSQL

Connectivity

Fabric server and Fabric Studio should have access (permissions) and connectivity (network) to all sources & destinations. Cassandra and Kafka servers should be accessible from Fabric servers’ nodes.

Fabric Minimum System Requirements for Development and QA Installations

Introduction

  • This document provides information regarding the hardware, software and the operating system requirements for Fabric installations.
  • TDM installation is based on Fabric with the additions of TDM library and PostgreSQL installation.

Fabric Execution Servers - Node Specifications

The Fabric server installation is a Linux-based solution. A dedicated servers for Fabric instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined .

Minimum System Configuration for Each Fabric Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher.

Hardware Requirements:

Processor

For simple scenarios: Intel Xeon Quad-core or equivalent.

For high-load or complex scenarios: Octa-cores are  recommended.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 150 GB free disk space is recommended.

Network

1 Gbps network speed between the Fabric nodes, Cassandra Nodes, Kafka nodes and source and destination Dbs.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.4xlarge

Google - GCP

D16d v4

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration must be as follows:

  • Volume of 150G /opt/apps/fabric/ - will be used also as the home directory for a Fabric user.

Kafka Server Node Specifications

The Kafka server installation is a Linux-based solution. A dedicated servers for Kafka instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined.

Minimum System Configuration for Each Fabric Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.

Hardware Requirements:

Processor

For simple scenarios: Intel Xeon Quad-core or equivalent.

For high-load or complex scenarios: Octa-cores are  recommended.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 50 GB free disk space is recommended.

Network

1 Gbps network speed between the Kafka nodes and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration must be as follows:

Volume of 50G /opt/apps/kafka/ - will be used also as the home directory for a Kafka user.

Cassandra Server Node Specifications

The Cassandra server installation is a Linux-based solution. A dedicated servers for Cassandra instance is prefered. Following a sizing process, the exact number of nodes for each environment needs to be determined.

Minimum System Configuration for Each Cassandra Node Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.
  • Python version 2.7 (this is a Cassandra limit; until Cassandra version 4.0.x - Python version 2.7 is the latest supported; newer versions of Cassandra are not yet certified by K2view for Fabric)

Hardware Requirements:

Processor

For most scenarios: Intel Xeon Octa-core CPU or equivalent.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 200 GB free disk space is recommended.

Network

1 Gbps network speed between the Kafka nodes and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-8

For all installations, the FS configuration should be as follows:

  • Volume of 200G /opt/apps/cassandra/

PostgreSQL Server Node Specifications

  • PostgreSQL is required for TDM projects only.
  • K2view supports PostgreSQL version 9.6 & 13.x .
  • PostgreSQL can be deployed in one of the following two ways:
    • On-premise as a virtual or physical machine.
    • As a cloud service (SaaS)

Suggested System Configuration for PostgreSQL Server

Software Requirements:

Operating system:

  • RedHat (version 7 or higher) based distribution with latest patches (for example: Centos, Oracle Linux).
  • Ubuntu Server, version 18.04 or higher, with latest patches.

Hardware Requirements:

Processor

Intel Xeon quad-core or equivalent.

Physical Memory (RAM)

16 GB RAM

Storage

Direct attached SSD/NVME based storage is recommended for best performance.

SAN supported with flash based disks. RAID-0 is preferred.

NAS is not certified.

Disk Space

For application data, at least 100 GB free disk space is required.

Disk space can be increased according to the project scope and retention requirements.

Network

1 Gbps network speed between the PostgreSQL and Fabric nodes.

Example of Cloud Supported Instance Types:

Cloud provider

Instance Type

Amazon - AWS

m5.xlarge, a1.2xlarge

Google - GCP

D16d v2

Microsoft - Azure

e2-standard-4

Example of Cloud Supported SaaS

  • AWS RDS PostgreSQL - read more here
  • GCP Cloud SQL PostgreSQL - read more here
  • Azure Database for PostgreSQL - read more here

Fabric Studio Requirements:

Fabric Studio is used to configure, manage and control the Fabric server operation and its installation is Windows-based. The application can be installed locally on either a workstation or a server with Terminal Services for RDP connection.

Minimum System Configuration

Operation System

Workstation: Windows 10 Professional 64 Bit or higher.

Server: Windows server standard 2012 64 bit or higher.

Processors

Intel I3 quad-core 5th Gen or equivalent.

Physical Memory (RAM)

8 GB RAM

Disk Space

100 GB available disk space.

 

Windows Ports

The following ports should be open on the Windows server:

  • 3389 - used for RDP

Windows Permissions

Local administrator privileges are needed for the Fabric Studio installation.

Text/Code editor

Visual Studio Code / notepad++

https://code.visualstudio.com/

https://notepad-plus-plus.org/downloads/

file archiver

7zip

https://7-zip.org/

FTP/SCP client

winSCP

https://winscp.net/eng/download.php

SSH client

Putty

https://www.putty.org/

Git Shell

TortoiseGit

https://tortoisegit.org/

Java IDE

Intellij IDE (community edition):

https://www.jetbrains.com/idea/download/#section=windows

Linux Servers Ports

Open the following ports on the Linux server and make sure they are accessible from Management stations.

Depending on your setup, not all ports are required:

Ports Number

Role

22

SSH, SCP

3213, 9443, 5124, 6379

Fabric

9042, 9142

Cassandra

9093, 7270

Kafka

3000, 9090

Grafana stack

5432

PostgreSQL

Connectivity

Fabric server and Fabric Studio should have access (permissions) and connectivity (network) to all sources & destinations. Cassandra and Kafka servers should be accessible from Fabric servers’ nodes.