316 questions
0
votes
1
answer
42
views
Java action in Apache Oozie workflow
I am trying to configure an Apache Oozie workflow to execute different actions depending on the day of the week. After reading https://stackoverflow.com/questions/71422257/oozie-coordinator-get-day-of-...
0
votes
0
answers
16
views
Oozie workflow arguments interprets double quotes weirdly for spark-submit command parameters
I have a Spark job which takes a bunch of configurable parameters. I am facing an issue specifically in this portion:
--conf spark.executor.extraJavaOptions="-Duser.timezone=PST -XX:+UseG1GC -...
- 826
0
votes
0
answers
34
views
How often does the oozie coordinator check for the presence of files in Synchronous Datasets?
<coordinator-app name=""
frequency="1 17 * * *"
start="${start_date}" end="${end_date}" timezone="Asia/Seoul"
xmlns="uri:oozie:...
- 1
0
votes
0
answers
41
views
OOZIE coordinator jobs always showing in RUNNING state - max concurrency reached
I have submitted a coordinator job where my workflow.xml is -
<workflow-app xmlns="uri:oozie:workflow:0.5" name="my_workflow">
<start to="abc"/>
<action ...
1
vote
1
answer
132
views
On renaming a column in hive table, it removed all values of that column for its previous data prior to deployment
We just went ahead with a deployment for one of our Hive based table. We renamed our column risk_old to risk_new (renamed). The table is period partitioned. However post deployment, we saw a strange ...
- 11
0
votes
1
answer
41
views
Oozie coordinator app, how to configure action triggered by external data source?
I would like to run a job every time when a external data source is updated, for example, some government file is update, http://www.ic.gc.ca/folder/filename.zip. Is there way of doing it?
Please ...
- 1
0
votes
0
answers
449
views
How to schedule job on Oozie every exactly n day regardless of the month
I would like to schedule a job to run every n day where n is not 1 or 7. I know the syntax 0 0 */n * *, but when I check for the next execution dates, it always runs on 1st of the next month, ...
- 534
0
votes
1
answer
110
views
oozie intial instance and start time giving error on missing dataset
I am new to oozie and trying to understand dataset.xml. I have following dataset and trying to understand what exactly oozie is trying to validate here. what is the meaning of initial instance and ...
- 181
1
vote
1
answer
322
views
Oozie coordinator get day of the week
I am trying to create a condition in my Oozie workflow, where an action should be executed only on mondays (at the end of the workflow).
So far I added a decision node in the workflow, and the current ...
- 21
0
votes
1
answer
404
views
Running Python Script in OOzie with special libraries without install them to server
I want to run Python script in OOzie workflow with special libraries. But, I want to run this script without installing these special libraries to Hadoop nodes. I tried to run with virtualenv but ...
- 250
0
votes
0
answers
734
views
Getting "Delegation Token can be issued only with kerberos or web authentication" executing Hive query in shell script
I am trying to connect Hive and execute query in shell script triggered by oozie, the server is kerberos enabled. I am passing hive credentials in workflow but I am still getting the error output ...
- 355
0
votes
1
answer
423
views
Using Beeline as an example (vs hive cli)?
I have a sqoop job ran via oozie coordinator. After a major upgrade we can no longer use hive cli and were told to use beeline. I'm not sure how to do this? Here is the current process:
I have a ...
- 1,226
0
votes
1
answer
645
views
How to kill/cancel a coordinator action before the workflow starts?
I need a way to kill a coordinator action that is in status READY and doesn't yet have a workflow ID.
My coordinator looks as follows:
$ oozie job -info 0000123-000000123456789-oozie-oozi-C
Job ID : ...
- 2,407
0
votes
1
answer
735
views
How to mark an Oozie workflow action's status as OK
I am using Apache oozie. I want to mark the status of one of the shell action as OK, in my oozie workflow. It is in Running state.
Can we please share the command to use in Apache Oozie to do this.
- 91
0
votes
1
answer
421
views
How to get the first day of the month in input-events in oozie coordinator?
I need to configure data waiting for the whole previous month. How to get the first day of last month in start-instance if the start is on 2nd of month?
<input-events>
<data-in ...
- 41
0
votes
1
answer
245
views
Running Oozie Action on a future date
I have a requirement for which a workflow is on demand run.But there is a task ( curl command) to get triggered at a future time .
- 119
0
votes
2
answers
59
views
Oozie property file value not reading from spark
I have a property file in oozie and getting value from shell script like below:
filter_cond = record = 'n' and name = 'abc' and age = '14'
in Shell script
val cond = ${getproperty filter_cond}
...
- 607
5
votes
1
answer
704
views
How to trigger now scheduled oozie job?
I've submitted a job to Oozie using the following command:
oozie job -config ${config_file} -submit
My job is scheduled to run at 5 UTC every day (frequency = 1440). My question is - how to trigger ...
- 1,223
0
votes
0
answers
134
views
Setting Kerberos Authentification to Oozie Coordinator
I'm doing a Spark Copy from a remote HDFS to my HDFS.
I have a Oozie Coordinator which check every day, if the data is available in the specified directory in the remote HDFS and then run the ...
- 165
1
vote
1
answer
958
views
Oozie - run a workflow every day or every hour
I have a oozie workflow(hive_insertion.xml) that executes a .hive file, which inserts data into a table.
The Oozie workflow is:
<workflow-app xmlns = "uri:oozie:workflow:0.4" name = "simple-...
- 3,093
0
votes
1
answer
33
views
Setup an oozie that stops after a few runs
I need to setup an Oozie job that runs every 1 hour from 9am till 5pm and then stop for the day. And then do the same for all week days. I know how to set up something that starts at a given time and ...
- 3,443
0
votes
1
answer
199
views
how oozie timezone work, it does not pick the right timezone?
I have oozie job running on CDH cluster. I have the following coordinator
<coordinator-app name="name" frequency="0 */5 * * *" start="2020-03-05T16:00Z" end="2020-03-07T16:00Z" timezone="America/...
- 6,425
-1
votes
1
answer
240
views
Get oozie job information in oozie workflow by REST
How can I find jobs with the parent id is null? I tried 3 methods but none of them worked for me.
/oozie/v1/jobs?jobtype=wf&filter=parent_id=%00 NOT WORKING
/oozie/v1/jobs?jobtype=wf&filter=...
- 9
0
votes
0
answers
277
views
How to design the Oozie coordinator on arrival of input multiple times in a day
I have a requirement to schedule my coordinator on arrival of input from other application. I may receive one or multiple times in a day. So, whenever I receive an input I need to trigger my treatment....
- 689
0
votes
0
answers
1k
views
oozie - java.lang.NoClassDefFoundError: Could not initialize class java.net.NetworkInterface
While running the oozie sample examples, Oozie jobs are getting scheduled and status showing as Running. After some times jobs get KILLED. While digging the hadoop logs found these exceptions.
I have ...
0
votes
1
answer
187
views
oozie java.io.IOException: No FileSystem for scheme: hdfs
I have setup the oozie 4.3.1 with Hadoop 2.7.3.
oozie has been setup and running successfully and able to see web console http://localhost:11000/oozie/
and also confirm using oozie status command.
...
0
votes
1
answer
275
views
How to use date in oozie workflow
I have an oozie coordinator and runs an oozie workflow. One of the action in my workflow is to run a bash script file and I need to use year, month and day in the bash script. From what I understand, $...
- 6,425
0
votes
1
answer
257
views
How to configure this email action globally in Oozie?
I have around 50 workflows like this, below code just an example,
<workflow-app name="exit_1_email_test" xmlns="uri:oozie:workflow:0.5">
<start to="ssh-8e73"/>
<action name="...
- 4,970
0
votes
0
answers
152
views
Limited amount of future actions for Oozie Coordinator job
I have created a bundle job which launches 4 coordinator job's with each coordinator job in turn starting 1 work flow ( so 4 concurrent workflows in total).
When I look at the action details for each ...
- 73
0
votes
1
answer
909
views
How to get Oozie's dependency check to list the dataset full path name instead of coord:latest(0)?
With an Oozie coordinator and workflow, I see the following in the Coord Job Log for a specific action:
JOB[0134742-190911204352052-oozie-oozi-C] ACTION[0134742-190911204352052-oozie-oozi-C@1] [...
- 1,118
0
votes
1
answer
590
views
What does the input-events tag in coordinator.xml do?
I am trying to understand a coordinator.xml, but not able to understand what does the input-events tag do.can you please help me in understanding, when will this workflow get triggered.
<...
1
vote
2
answers
2k
views
Rerun sucessful oozie jobs
Is there a way to fully rerun successful oozie jobs. Let assume that we schedule creation a table and we want to rebuild it on demand - is there easy way to do it in oozie?
I try oozie -rerun command ...
- 13
1
vote
0
answers
105
views
Trigg Oozie workflow from input-event even if missing
I have classic coordinator with input-event on HDFS path.
<datasets>
<dataset name="rawData" frequency="${coord:days(1)}" initial-instance="${startDate}" timezone="UTC">
<...
- 1,286
0
votes
1
answer
161
views
Can some one let me know why following Oozie coordinator is running in loop
I was new to oozie process . I was testing the following coordinator.xml,when i submit the job it running in loop but I want to run everyday at 1:00 am .Can someone let me know what mistake i was ...
- 654
0
votes
1
answer
214
views
Oozie coordinator creates 12 workflow jobs for one action
guys.
I am using Hortonwork's HDP Sandbox. I've configured the simple spark job (which takes text file and outputs into another file the word counts).
Anyway, the problems I have is with the ...
- 633
2
votes
2
answers
731
views
OOZIE workflow.xml No function is mapped to the name coord:nominalTime
I'm using Oozie's SLA feature. I'm trying to use ${coord:nominalTime()} for nominal time, but it throws an error when I schedule the workflow:
E0803 : E0803: IO error, E1004: Expression language ...
- 526
1
vote
3
answers
1k
views
How to detect errors in an oozie workflow before actually running it?
I am new to big data and was wondering if there is a way to detect errors in an oozie workflow before running it? Something like a compilation error sort of. Please help.
Thanks,
Annie
- 13
0
votes
2
answers
677
views
How to handle day light saving in oozie coordinator?
Due to day light saving all my scheduled jobs(using oozie) are running later one hour.
Before DST : My jobs runs every day at 8:00 pm
after DST: Now job is running at 9:00 pm
could any tell is there ...
- 1
1
vote
1
answer
531
views
Check programatically the status of an action in oozie workflow from another oozie workflow
I am running some code in oozie workflow named WF1's action named AC1.. This workflow is not scheduled but runs continuously.. usually action AC1 will get its turn 4 times a day. Time at which this ...
- 94
0
votes
2
answers
2k
views
How to execute one particular workflow action in Oozie. If I killed Oozie workflow manually?
I have below Oozie workflow,Suppose manually I killed the job when action "Do_task1" was executing, but still I want to execute action "Do_task2" in spite of killing oozie job manually(when action "...
- 21
1
vote
1
answer
1k
views
How to read config properties in sub-workflow (separate xml file)?
I am getting below mentioned error message while reading config properties in separate sub-workflow file. I am posting the sample code. Appreciate your help in resolving this issue. Thank you!
2019-...
- 1,311
0
votes
2
answers
227
views
Find stack trace for workflow job launched from Oozie coordinator
I have a bare-bones Oozie coordinator with this specification:
<coordinator-app name="my-coord" frequency="${coord:days(1)}"
start="${startDate}" end="${endDate}" timezone="UTC"
...
- 1,467
2
votes
0
answers
133
views
Getting END_MISS notification when my Job Status - WAITING state
I have set sla-nominal time as ${coord:nominalTime()}, so my sla starts calculating taking coordinator triggering time as the start time for sla calculations.
But the problem is I have many workflows ...
0
votes
2
answers
220
views
How to find hadoop applications ran by oozie (hadoop) job
We know that first oozie runs a hadoop job and using that job it runs other hadoop applications. So I want to find the list of those hadoop applications (eg. application_231232133) ran by oozie (...
- 1,823
0
votes
0
answers
214
views
duplicate java actions executed by oozie launcher
I am facing below issue with oozie-5.0.0.
My oozie workflow has a java action which is getting executed twice by the same oozie launcher.
I found that this is happening as oozie launcher is ...
- 41
0
votes
1
answer
1k
views
Oozie commandline filters
oozie job -info $coordinator
the command gives you the details of workflows belong to the coordinator, print their ID, status, created time and nominal time.
I'm trying to print the workflows of ...
0
votes
0
answers
405
views
How to schedule Oozie coordinator cron to launch at same time in local timezone accounting for daylight savings
I have an Oozie coordinator setup with the following information:
frequency="00 09 * * *" timezone="America/New_York"
Currently we are UTC-4 (EDT), hence if I want the job to run at 5am I have to ...
1
vote
1
answer
1k
views
Killing oozie coordinator is not killing the subsequent spark job
When i tried to kill a spark job which is invoked from an oozie job it's not getting killed.
It's just showing the workflow job is killed.
Then i checked the workflow log from this command
oozie ...
- 33
1
vote
0
answers
416
views
Is it possible cancel scheduled action from Oozie?
We have a coordinator running a workflow every hour.
Recently, the cluster was KO for a week. The problem is solved and now I see Oozie executing all those jobs it missed during the outage.
¿Can I ...
- 753
0
votes
1
answer
828
views
Error while executing shell-script using oozie
I'm trying to run kafka-connect-hdfs using Oozie version: 4.2.0.2.6.5.0-292 via script file sample.sh.
Yes I do know we can run the kafka-hdfs connector directly, but it should happen via oozie.
Kafka ...
- 6,149