Clusters Hub

Overview

Once the user is authenticated to Indexima, he gets access to a hub of indexima Clusters.

A VisualDoop Administrator would be able to attach a cluster into this hub.

A simple user (i.e., not an admin) would only see the list and use the already created connections to actually connect to the cluster.

Part 1: Administrate Clusters attachments

User must be VisualDoop Admin user to perform the following actions.

Attach a cluster

Click on the "Attach Cluster" button,

then fill up all the information mandatory to connect to an existing Indexima cluster.

How to fill the connection attachment string?

Parameter Definition Valid value Mandatory

Cluster Name Name of the cluster (e.g, to distinguish several contexts or environments). Any string that only contains letters & digits. (Nb: the Space character is not allowed) Yes

Cluster Hostname

The hostname of the load balancer or of the master node for the console to connect to the cluster.

A valid internet hostname.

Yes

Cluster Port

The port of the load balancer or of the master node for the console to connect to the cluster.

An integer port number.

Default value: 9999 corresponds to the default value of webui.port defined in (2022.2) galactica.conf#Connectivity toward Indexima

Note: the console will interact with the core engine on the following resulting URL: http(s)://<cluster_hostname>:<cluster_port>

SSL
- Note: see parameter webui.ssl.enable to enable ssl for interactions between core engine and console
- Note: SSL parameter would have to be set on both galactica & Visualdoop

Yes

Cluster API secret key

The secret key for the console to interact with the cluster.

Any string that only contains letters & digits

Default Value: EvdbpGMCWPzpSzgkjTqq9SjM

Is equal to the value defined in monitor.api.key defined in (2022.2) galactica.conf#Connectivity toward Indexima

Yes

Hive Hostname

The hostname of the load balancer or of the master node for any sql client to connect to indexima.

A valid internet hostname

Zookeeper
- If Zookeeper is used on top of indexima, the hostname must be the concatenation of available nodes. The port number (Hive port) will be suffixed automatically if missing. Eg: master0.co.internal:2181,master1.co.internal:2181,master2.co.internal:2181

Examples

#	Hive Hostname	Hive Port	Resulting Jdbc Connection
1	master1,master2,master3	2180	jdbc:hive2://master1:2180,master2:2180,master3:2180/<hive_parameters>
2	master1:2181,master2:2182,master3:2183	0000	jdbc:hive2://master1:2181,master2:2182,master3:2183/<hive_parameters>
3	master1:2181,master2,master3:2183	0000	jdbc:hive2://master1:2181,master2:0000,master3:2183/<hive_parameters>

Yes

Hive Port

The port of the load balancer or of the master node for any sql client to connect to indexima.

An integer port number.

Default value: 10000

Yes

Hive Parameters

The optional hive parameters to add to the JDBC URL to connect to indexima (depending on hive customization).

Any valid hive parameters.

If there are any extra parameters, then the string must start by default. Otherwise, the string can be left empty
kerberized cluster
- hive parameters are required to provide the Kerberos parameters.
  - Principal: ;principal=indexima/ip-192.168.0.1.internal@INDEXIMA.COM
Zookeeper
- the zookeeper parameters must be added.
- serviceDiscoveryMode: ;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=<indexima_zookeeper_namespace>
Note: The resulting url used by console to access core engine will be jdbc:hive2://<hive_hostname>:<hive_port>/<hive_parameters>
For a complete description of hive parameters, refer to https://cwiki.apache.org/confluence/display/hive/HiveServer2+Clients#HiveServer2Clients-ConnectionURLs

Yes

Example:

Edit cluster link connection string

Click on the Edit button so you can modify the connection string

Detach a cluster

Click Edit then Delete to detach a cluster

Part2: Administrate connections within Clusters attachments

Register a connection to a cluster

User/Pwd Authentication or LDAP Authentication

User/Password

Fill in a valid User/password of a user who can connect to the cluster.

If you did not customize this in the Indexima installation, the user/password admin/admin is the default user.

Kerberos Authentication

Keytab user

Fill in with the value of the Principal used in the hive parameter.

Keytab Path

Keytab path

Example

Common options

Impersonate

When the box is:

ticked, User/PWD used to authenticate to the Indexima Interface (Visualdoop) will be used to connect & authenticate to Indexima cluster
unticked, User/PWD defined in the connection will always be used to connect & authenticate to Indexima cluster

Note: In the case where the Developer Console UI has been configured to use a local LDAP server as the authentication method, and the Indexima cluster has been configured to have only one local service user named "indexima", with a password "secret", I login into the Developer console with my own user and password.

Is Shared

When the box is:

ticked, the connection is shared with all other users.
unticked, the connection is visible only to its creator.

Edit a connection

Click on the pen to edit connection details

Delete a connection

Edit the connexion and click on Delete

Recommendations

In most cases, we recommend creating 1 cluster attachment & 1 connection using impersonate option (with the credentials of an admin user filled in the connection details)