This guide provides technical details and general considerations related to running Gemini Enterprise: Manage on Amazon Web Services (AWS).
Running Gemini Enterprise: Manage on Amazon Web Services
Due the nature of the Amazon Web Services, network interfaces and their configuration are automatically applied during the Instance creation process. For this reason, the Network Configuration section in Gemini Enterprise: Manage (Manage) provides read-only access. Additional virtual network interfaces, including any bonding options, are not available.
A key feature of Amazon Web Services is the identity and access management (IAM) and password-less authentication with Key Pairs when connected with SSH. This is recommended and also enforced by Amazon Web Services for security reason. Gemini appliance AMI supports automatic injection of assigned Key Pairs during Instance creation process.
Note on Storage
It is highly recommended to use EBS Volumes from “General Purpose SSD (GP2)” type to achieve best results regarding performance. Refer to section 2.2 for further instructions how to attach Storage to Gemini appliance.
Manager supports disk volume management feature that allows to expand disk spaces with mounting and merging new disks. To achieve this, create a new virtual disk, mount and merge it into the existing volume from GUI. Read the Node -> Storage section in administration guide for more details.
Note on Security Groups
To administrate and run the Gemini Appliance and services, certain communication channels between clients and nodes are required. As a minimum, ports tcp/443 (HTTPS) and tcp/22 (SSH) are required for basic system configuration and operations. As the Web Interface and SSH console offer low-level system access, make sure to not expose these ports towards public access (Anywhere, 0.0.0.0/0). Depending on the Deployment, add inbound/outbound rules as needed.
For a complete list of used network ports, see the respective article on our Support Portal.
In general, best practice is to follow the Splunk sizing principles when deploying Gemini appliance with Amazon Web Services.
Please refer to the “DEPLOYING SPLUNK® ENTERPRISE ON AMAZON WEB SERVICES” document, to be found here: https://www.splunk.com/pdfs/technical-briefs/deploying-splunk-enterprise-on-amazon-web-services-technical-brief.pdf.
For specific Data Ingestion Rates and Retention Periods, make your selection based on the recommendations below:
The table below lists requirements for resources used with Cloudera.
- CPU: 4
- RAM: 64 GB
- Disk: 500GB
- Recommended AWS instance: Cloudera Manager: m4.xlarge or m4.4xlarge
Getting Started with AWS
- You will need to have an AWS account issued and managed by your organization.
- You must be familiar with the process of finding Amazon Machine Images (AMIs).
- Prepare a Security Group according to the Port Matrix in chapter 1.1.4.
- (optional) A Key Pair for SSH connection.
- If the AMI is not available at all or in a specific region, please contact Gemini Support.
Discover & Launch Instance
This section provides information on finding and using the publicly available AMI for the Gemini Appliance.
- Log in to the AWS Console and access the EC2 Services section.
- Select AMIs on the left navigation bar to access the available images.
- Change the selector next to the search bar to Public Images, enter gemini-appliance and choose Search.
- Ensure that the discovered image corresponds to the Gemini appliance. Right-click on the found image, and select Launch.
- Select an Instance Type according to the Sizing Guide in Chapter 1.2.
- Configure Instance Details according to your company standards.
- Leave 10 GiB General Purpose SSD (GP2) storage for the Root Volume.
- Add a new EBS Volume, set Size to your needs, choose the General Purpose SSD (GP2) option. Do not enable Encryption as it will significantly impact the performance.
- Add tags if required.
- Select your pre-created Security Group.
- Review the Instance configuration summary and choose Launch.
- Instance creation takes a couple of minutes. Note the instance ID as this will be used for the first time authentication.
After the Instance Status indicates Running, a IP address and a FQDN are assigned to this instance. Note the IP address and FQDN, and proceed with next chapter.
- Gemini Enterprise Manage Final configuration of the appliance is completed using the Manage web interface. Using a supported web browser, navigate to:
A security warning or message may be displayed. This is expected and does not indicate a problem. Depending on your browser you may have to choose Proceed Anyway or Continue for the page to load.
For security purposes, please specify the Instance ID of the AWS instance used for this installation. This step ensures that this configuration process is accessed by the authorized owner of the instance. It is only required upon initial configuration. Ongoing access to the appliance will be configured in a later step.
The Instance ID is the 17-character identifier corresponding to the EC2 instance, and shown in the AWS console.
When you first log in to Manage, the End User Software License Agreement is shown. After reading the terms, choose Accept to advance to the next screen.
Manage supports multiple languages and setting a preferred language adjusts the entire user experience accordingly. If the hostname and timezone settings were skipped in previous basic configurations, this step provides an opportunity to configure them.
Select License Option
There are three options:
- Activate Enterprise Edition using Purchased License: Select this if a purchased license has been provided or there is a license server configured.
- Select Use a License File if you have a purchased license.
- Select Connect to a License Server if you have a license server to manage all the licenses and can be activated from this license server. Input the License server IP address and token string for validation.
- Activate Free Edition: With this option several features are restricted. Ignore this option if you already have apurchased license or you want to have a trial.
- Activate Enterprise Edition Free Trial: Select this if you want to start a 30-day free trial.
- Follow the steps to generate a license request file, send it to Gemini support and apply the received license.
Choose Next to proceed.
If you have multiple appliances to be configured, select “Bulk Provisioning (provision several appliances with this node as the master)” would help you to apply configurations and complete the initial setup on the appliances at one time.
To operate as a standalone appliance, select “Stand-Alone (single) provisioning”.
This is a step-by-step wizard to guide you complete the initial setup configurations.
If you have a known appliances IP list, e.g. acquired from DHCP server or reported by AWS, you can create a text file and make an IP list, one IP address per line, and then upload this text file to locate the appliances.
Another way is to perform a IP subnet scan to discover the appliances. Use CIDR notation to specify the subnet, e.g. 192.168.156.0/24.
Please note if you specify a large subnet, it might take long time on scanning.
If the IP address of all the appliances are assigned by DHCP server and you want to keep them as static IP configurations, select “Network settings - Static assigned”. This is useful when building a DHCP server for deployment temporarily. User may assign IP addresses by using DHCP server and use this option to turn the configurations permanently. In the end DHCP server can be removed after deployment.
Keep the default settings to assign IP address by DHCP, select “Network settings - DHCP assigned”.
When you have assigned DNS records for each appliance, select “Use Reverse DNS Loopkup” to acquire it’s own hostname.
If you want to name the appliances with a custom pattern, select “Specify Custom Pattern” and specify the custom pattern with valid tokens.
Change Admin Password
Update the password for the account ‘admin’ in Manager here. It is recommended that you use a strong password or if applicable, follow appropriate password security policy as required for your enterprise.
Please note all the appliance will be updated with the same admin password.
Connect to LDAP
You may configure LDAP resources here to support LDAP authentication. When LDAP resources are configured successfully and correctly, a user will be able to login to Manager with their LDAP account.
Read LDAP Authentication in Settings chapter for more details. Please note this is optional and you can skip it.
For SSH login, you may specify the password for SSH authentication. You may also upload SSH key to complete the key exchange and it allows you to login Manager via SSH without password required.
Please note this is optional and you can skip it.
Here you can see the summary of provision appliances. Click “Start” to start provisioning. You can see the live status during provisioning and download the CSV result for further use.
Click FINISHED to complete this wizard.
Stand-Alone Provisioning - Join Cluster
If this node is going to join into an existing Gemini Cluster, select “Join an existing appliance cluster” and provide the IP Address and the Token String that were assigned on the Master Node.
If this is an standalone appliance, or you wish to configure a cluster at a later time, select “Operate as a standalone appliance”.
Stand-Alone Provisioning - Change Admin Password
Update the password for the account ‘admin’ in Manager here. It is recommended that you use a strong password or if applicable, follow your password security policy in your enterprise.
Stand-Alone Provisioning - Success
Congratulations! The Completed screen lets you know that this appliance has been configured. Click “Get Started” to launch Manager.
Welcome to Gemini Enterprise: Manager
Log in to Manager with username ‘admin’ and password configured in the setup process.
Install Featured Platforms
Follow the on screen directions to install featured platforms like Splunk Enterprise and Cloudera CDH. Note that this process involves accessing the application website and downloading binaries. Ensure that you have access to the public internet (from the browser used, not the appliance) before proceeding further.
To access the Gemini Support documentation and knowledge base, or to open a support ticket, please visit http://support.geminidata.com. Also refer to the support site frequently for updates, patches and other information related to your appliance.