Can I submit an oozie job with multiple configuration files?

Question

From the Oozie CLI I want to do something like this:

oozie job -oozie http://host:port/oozie -config jobConfig.properties, baseConfig.properties -submit

I have a lot of different jobs I'm running where a portion of the .properties file is identical. I'd like to be able to move that duplicated portion into a single baseConfig.properties file, and combine this with the job-specific part when I submit the job.

Chris White · Accepted Answer · 2012-12-19T23:16:32.003

11

If the number of overridden properties is small, you can just reference the base config file, and then use the -Dprop1=value notation for each property you want to customize.

Otherwise i think you have two (possibly three) alternatives (as you can't pass multiple config options to the current OOZIE client):

Create a script that concatenates the base and job specific property files together, and the run oozie with the name of the merged file
Using the source from OozieCLI as a starting point, write an extension class that can handle a CSV value for the -config option, or can handle multiple -config argument instances
Are you able to move the common attributes into static jobXml files for each action, then you would only need a single config file for the actual differences between the job runs.

edited Dec 19 '12 at 23:16

answered Dec 19 '12 at 21:22

Chris White

29,241
4
69
92

Thanks, Chris. I was kind of suspecting I would have to make my build script concatenate the two files, but you've given me a bunch of other good options. – Tim Goodman Dec 20 '12 at 15:02
@TimGoodman what did you wind up doing? I have a similar use case. – Chris Marotta Jul 12 '16 at 19:36

score 3 · Answer 2 · answered Aug 05 '13 at 21:29

3

I could put default values into config-default.xml file as desribed here: http://oozie.apache.org/docs/3.3.2/WorkflowFunctionalSpec.html#a4.1_Workflow_Job_Properties_or_Parameters

It works in most of cases.

answered Aug 05 '13 at 21:29

Oleksii

1,071
7
12

Can I submit an oozie job with multiple configuration files?

2 Answers2