amazon web services - Is there a way to add multiple jobs using HadoopJarStepConfig jarConfig = new HadoopJarStepConfig(HADOOP_JAR); -
i have written aws swf workflow , first action boot cluster , run mapreduce program. action has 2 other mapreduce jars executed depending upon first jar's output. using add jars
hadoopjarstepconfig jarconfig = new hadoopjarstepconfig(s3n_hadoop_jar); jarconfig.setargs(args_as_list); hadoopjarstepconfig jarconfig1 = new hadoopjarstepconfig(s3n_hadoop_jar); jarconfig1.setargs(args_as_list1); try { stepconfig enabledebugging = new stepconfig() .withname("enable debugging") .withactiononfailure("terminate_job_flow") .withhadoopjarstep(new stepfactory().newenabledebuggingstep()); stepconfig runjar = new stepconfig(hadoop_jar, jarconfig); stepconfig runjar1 = new stepconfig(hadoop_jar, jarconfig1); request.setsteps(arrays.aslist(new stepconfig[]{enabledebugging, runjar, runjar1})); runjobflowresult result = emr.runjobflow(request); is correct way add multiple jars. thanks.
use:
request.withsteps(enabledebugging, runjar, runjar1);
don't use:
new stepconfig[]{enabledebugging, runjar, runjar1})); wrong, dont need spetconfig again here..
Comments
Post a Comment