[Announcement] Automated Indexer Deployment from PrimeLab

Automated Deployment of Indexer

PrimeLab is excited to launch our Automated Indexer Deployment Utility for NEAR Protocol. Quickly deploy your own PRODUCTION READY Indexer 82% FASTER!

Whether you’re a new dApp or a seasoned publisher, you have a need for reliable & verifiable blockchain data. Running an Indexer node enables you to maintain a reliable & verifiable “Source of Truth” for blockchain data.

Current Public Solutions

The near-indexer-for-explorer repository available in the NEAR GitHub org provides you with a basic Indexer project that can be deployed to start your own indexer. It assumes that you have experience and knowledge of Postgres Databases, Cloud Architecture, Docker, and a high-level understanding of using Cargo.

Our Improved Solution

We’re providing a Terraform based solution that enables you to deploy the near-indexer-for-explorer project AND all of the infrastructure, databases, security group settings, backups, etc. needed to ensure your project is up and running quickly.

Automatic Resource Deployment

Inputs

When deploying an Indexer with our proposed solution, you’re able to build and deploy using the default parameters, or adjust the input parameters to fit your infrastructure or use-case.

Name Description Type Default Required
account The account number associated with the environment. number null no
ecs_asg_desired_capacity EC2 Capacity Providers ASG - Min Number of Instances number 1 no
ecs_asg_max_size EC2 Capacity Providers ASG - Max Number of Instances number 1 no
ecs_asg_min_size EC2 Capacity Providers ASG - Min Number of Instances number 1 no
ecs_instance_type Instance Type for ECS Cluster string “c5.2xlarge” no
efs_filesystems A map of EFS filesystems to create using the EFS module. map {} no
environment Name of Environment for deployment. string null no
indexer_archive_health_check_block_id Block ID To Check from for the Healthcheck number 0 no
indexer_archive_node_cpu ECS Container - Indexer Node CPU number 4096 no
indexer_archive_node_ram ECS Container - Indexer Node RAM number 8192 no
indexer_aurora_autoscaling_enabled n/a bool false no
indexer_aurora_autoscaling_max_capacity n/a number 1 no
indexer_aurora_autoscaling_min_capacity n/a number 0 no
indexer_aurora_engine n/a string “aurora-postgresql” no
indexer_aurora_engine_version n/a string “13.4” no
indexer_aurora_instance_class n/a string “db.r6g.large” no
indexer_aurora_instances n/a any { “one”: {}} no
indexer_aurora_monitoring_interval n/a number 60 no
indexer_chain_id n/a string “localnet” no
indexer_command_concurrency The number of outbound connections to the blockchain number 1 no
indexer_data_archive_url n/a string “” no
indexer_docker_tag n/a string null no
indexer_generate_config n/a string “” no
indexer_is_archive Weither to set the indexer node in archive mode bool true no
indexer_near_dirpath n/a string “/indexer/near/” no
indexer_run_migration n/a string “” no
indexer_set_archive n/a string “” no
indexer_sync_delta n/a string “” no
indexer_sync_from_command n/a string “” no
redshift_automated_snapshot_retention_period How long will we retain backups number 0 no
redshift_cluster_database_name The name of the database to create string “” no
redshift_cluster_master_username Master username string “etl” no
redshift_cluster_node_type Node Type of Redshift cluster string “” no
redshift_cluster_number_of_nodes Number of nodes in the cluster (values greater than 1 will trigger ‘cluster_type’ of ‘multi-node’) number 1 no
region Name of region for deployment. string “us-east-1” no

Build/Deployment Process

Terraform automatically provisions and deploys all required infrastructure, databases, and packages needed to run your Indexer.

Outputs

After deploying your core infrastructure, Indexer, and the required databases, you’re provided with all of the resources created during the deployment.

Name Description
ecs_cluster_name The name of the ECS cluster
multi_region_primary_aurora_kms_key_arn Multi-Region primary key for indexer-aurora-primary Cluster
primary_aurora_global_cluster_engine Name of the database engine to be used for this DB cluster.
primary_aurora_global_cluster_engine_version Engine version of the Aurora global database.
primary_aurora_global_cluster_id RDS Global Cluster identifier
primary_aurora_reader_endpoint A read-only endpoint for the cluster, automatically load-balanced across replicas
primary_aurora_security_group_id The security group ID of the cluster
primary_aurora_writer_endpoint Writer endpoint for the cluster
redshift_cluster_arn The Redshift cluster ARN
redshift_cluster_automated_snapshot_retention_period The backup retention period
redshift_cluster_availability_zone The availability zone of the Cluster
redshift_cluster_database_name The name of the default database in the Cluster
redshift_cluster_encrypted Whether the data in the cluster is encrypted
redshift_cluster_endpoint The connection endpoint
redshift_cluster_hostname The hostname of the Redshift cluster
redshift_cluster_id The Redshift cluster ID
redshift_cluster_identifier The Redshift cluster identifier
redshift_cluster_node_type The type of nodes in the cluster
redshift_cluster_nodes Cluster nodes in the Redshift cluster
redshift_cluster_parameter_group_name The name of the parameter group to be associated with this cluster
redshift_cluster_port The port the cluster responds on
redshift_cluster_preferred_maintenance_window The backup window
redshift_cluster_public_key The public key for the cluster
redshift_cluster_revision_number The specific revision number of the database in the cluster
redshift_cluster_security_groups The security groups associated with the cluster
redshift_cluster_subnet_group_name The name of a cluster subnet group to be associated with this cluster
redshift_cluster_type The Redshift cluster type
redshift_cluster_version The version of Redshift engine software
redshift_cluster_vpc_security_group_ids The VPC security group ids associated with the cluster
redshift_parameter_group_id The ID of Redshift parameter group created by this module
redshift_subnet_group_id The ID of Redshift subnet group created by this module
secondary_aurora_reader_endpoint A read-only endpoint for the cluster, automatically load-balanced across replicas
secondary_aurora_security_group_id The security group ID of the cluster

That’s it! :partying_face:

You’re now running a Production-Ready Indexer deployed and syncing with the NEAR MainNet!

Release Roadmap:

  1. RDS Restore Points & Compute Restore Points
    1. Requester-Pays RDS backups enabling you to skip the long initial sync process.
  2. Indexer APIs
    1. Basic REST/GraphQL APIs that enable any development team to quickly query the indexer data without creating complex indices or queries.
2 Likes

Thanks for open sourcing this! I hope it makes it easier for people to build indexers of their own! It may be best categorized under the development tag to get visibility from the devs who would use it.

Awesome, thanks for the feedback! I’ve included the development tag :slight_smile: