Architecture Diagram
Elements created by the Indexima AWS Marketplace CloudFormation template :
VPC
The default template does not create the VPC resources. It is assumed you already have a VPC ID and a Subnet ID to deploy the application to.
Unless your Dataviz servers are on the same VPC, it will be required that the target Subnet is a public subnet.
Security groups
Two security groups are created by the template.
UI Security group
The UI EC2 will need to open the ports on the allowed CIDR for
- SSH (22)
- Indexima Developer Console Web application (8082)
- Zookeeper client connection (2181)
All traffic is open on the UI and Indexima security groups source.
Indexima Security group
The Indexima EC2 machines need to open the ports on the allowed CIDR for
- SSH (22)
- Indexima Monitor API (9999)
- Indexima Hive server (10000)
All traffic is open on the UI and Indexima security groups source.
IAM
For the IAM resources, the template creates a role that can be assumed by an EC2, a custom policy, and an instance profile.
Indexima Role
The role needs to allow EC2 resources to assume it.
Policy
The policy attached to the role allows to list all the bucket in the account, and allows all operations on the Indexima bucket and prefix (these values are inputted as parameters of the template).
Instance Profile
In order to attach a role to an EC2 instance, an instance profile needs to be created. The role above is attached to it.
Compute
A number of on-demand EC2 instances will be created: an Indexima master machine, the UI machine, and the Zookeeper server.
An autoscaling group and a launch configuration will also be created for the Indexima workers.
Resource Type | Resource Name | Description |
---|---|---|
On-demand EC2 | Indexima Master | The primary machine Indexima needs to start. |
On-demand EC2 | Indexima UI | The machine hosting the web application allowing the administration of the Indexima cluster. |
On-demand EC2 | Zookeeper | A server with Zookeeper is installed on this machine. All Indexima instances join the Zookeeper namespace, allowing them to be reachable thru Zookeeper. |
Autoscaling Group | Indexima Cluster ASG | The worker instances are created with an autoscaling group. Its initial size is the size inputted in the Cluster Size parameter. It can be changed by updating the CloudFormation stack and changing this value. |
Launch Config | Indexima Cluster LC | The configuration and start script for the Indexima workers. Similar to the Indexima Master configuration except for the fact that it needs the master to be up and running so the workers can attach themselves to the Master. |