CODEWITHSUNDEEP
amazon-web-servicesgoelastic-map-reduce
NSR
NSR

Reputation: 877

How to create an EMR cluster using AWS SDK for Go

I want to create EMR clusters using AWS SDK for Go, but I can't find a way in the official documentation.

Package: emr — AWS SDK for Go

Cound you please help me with a detailed code?

Upvotes: 2

Views: 1370

Answers (1)

V. Samma
V. Samma

Reputation: 2608

Actually, coming up to the same problem, there is a way which is described in the documentation. For me, it was not straightforward as well because the wording is different. It appears that a "running a job flow" is basically what equals creating a cluster and adding steps to it.

So what you want is the function RunJobFlow found here:

https://docs.aws.amazon.com/sdk-for-go/api/service/emr/EMR.html#RunJobFlow-instance_method

So, a simple code example which creates a cluster without steps is the following (make sure you have correct credentials configured):

package main

import (
    "fmt"

    "github.com/aws/aws-sdk-go/aws"
    "github.com/aws/aws-sdk-go/aws/session"
    "github.com/aws/aws-sdk-go/service/emr"
)

func main() {
    sess := session.New(&aws.Config{Region: aws.String("eu-west-1")})
    svc := emr.New(sess)

    params := &emr.RunJobFlowInput{
        Instances: &emr.JobFlowInstancesConfig{ // Required
            Ec2KeyName:                    aws.String("keyname"),
            HadoopVersion:                 aws.String("2.7.2"),
            InstanceCount:                 aws.Int64(1),
            KeepJobFlowAliveWhenNoSteps: aws.Bool(true),
            MasterInstanceType:          aws.String("m1.medium"),
            Placement: &emr.PlacementType{
                AvailabilityZone: aws.String("eu-west-1a"), // Required
            },
            TerminationProtected:       aws.Bool(true),
        },
        Name:           aws.String("Go Test Cluster"), // Required
        Applications: []*emr.Application{
            { // Required
                Name:    aws.String("Ganglia"),
            },
            { 
                Name: aws.String("Spark"),
            },
            // More values...
        },
        JobFlowRole: aws.String("EMR_EC2_DefaultRole"),
        LogUri:      aws.String("s3://aws-logs-0000000000-eu-west-1/elasticmapreduce/"),
        ReleaseLabel: aws.String("emr-4.6.0"),
        ServiceRole:  aws.String("EMR_DefaultRole"),
        VisibleToAllUsers: aws.Bool(true),
    }
    resp, err := svc.RunJobFlow(params)

    if err != nil {
        // Print the error, cast err to awserr.Error to get the Code and
        // Message from an error.
        fmt.Println(err.Error())
        return
    }

    // Pretty-print the response data.
    fmt.Println(resp)
}

Upvotes: 5

Related Questions