java - Yarn mini-cluster container log directories don't contain syslog files -


i have setup yarn mapreduce mini-cluster 1 node manager, 4 local , 4 log directories , on based on hadoop 2.3.0 cdh 5.1.0. looks more or less working. failed achieve syslog logging containers. see container log directories, stdout , stderr files not syslog mapreduce container logging. appropriate stderr warns have no log4j configuration , contains no other string:

log4j:warn no appenders found logger (org.apache.hadoop.metrics2.impl.metricssystemimpl). log4j:warn please initialize log4j system properly. log4j:warn see http://logging.apache.org/log4j/1.2/faq.html#noconfig more info.

how can add normal logging containers? yet time, yarn mini-cluaster.

any piece of advice or useful point?

just lower amount of tried ways answers:

  • yes, i'm sure logging directories correct , see correlation between container log directories , applications.
  • yes, mapreduce jobs work. @ least expected work.
  • mini-cluster logging @ normal way , in accordance have setup. related containers.
  • lower layers dfs clsuter works normally. have hbase , zk mini-clusters here , work ok. need logging mapreduce jobs debugging.

ok, happened classpath, client configuration , packaging.

  1. client configuration shall include proper classpath yarn applications. in case have added following lines yarn-site.xml (please note $hadoop_common_home substitution):
<property>     <name>yarn.application.classpath</name>     <value>$hadoop_common_home/*,$hadoop_common_home/lib/*</value> </property>
  1. i have added following variable definition mini-cluster start-up script (it worth note have mini-cluster server-side jars ./lib relatively mini-cluster startup script:

base_path="pwd" export hadoop_common_home=${base_path}

the root cause of not working logging client map-reduce job starting inside new vm on yarn without knowledge locate hadoop-yarn-server-nodemanager.jar contains container-log4j.properties file in turn responsible container default logging configuration. working fine.


Comments

Post a Comment

Popular posts from this blog

javascript - RequestAnimationFrame not working when exiting fullscreen switching space on Safari -

request animation frame stops working when exiting fullscreen clicking on original space's safari window. fine if fullscreen mode canceled escape key or calling cancelfullscreen(). steps reproduce: open https://dl.dropboxusercontent.com/u/769042/prezi/safari-fullscreen.html click "draw raf", kittie appears click "fullscreen", go fullscreen click "draw raf", kittie appears go space original safari was, showing "click exit fullscreen mode", click anywhere, out fullscreen click "draw raf", nothing happens what handling click calling: window.requestanimationframe(draw); which draws on canvas context: function draw() { ctx.drawimage(img, math.random()*500|0, math.random()*400|0, 100, 100); } i checked .hidden , .visibilitystate, updated correctly. tested on osx 10.9.3, safari 7.0.4 (9537.76.4). has workaround/solution other switching old settimeout? this sounds webkit bug 88940 : using reque...
Read more

jsf - How to ajax update an item in the footer of a PrimeFaces dataTable? -

this visual of table have: +---------+------------+-------------------+ | header1 | header2 | header3 | +---------+------------+-------------------+ | row | input | calc'ed output | | row b | input b | calc'ed output b | | etc.. | etc.. | etc.. | +---------+------------+-------------------+ | total: total calc'ed output | +------------------------------------------+ and stripped-down code: <p:datatable id="mytable" value="#{mybean.somelist}" var="currentitem"> <p:column headertext="header1"> <h:outputtext value="#{currentitem.name}" /> </p:column> <p:column headertext="header2"> <pe:inputnumber value="#{currentitem.inputval}"> <p:ajax event="change" listener="#{mybean.changelistener}" update="outputval outputtotal...
Read more

django - CSRF verification failed. Request aborted. CSRF cookie not set -

error: reason given failure: csrf cookie not set. in general, can occur when there genuine cross site request forgery, or when django's csrf mechanism has not been used correctly. post forms, need ensure: browser accepting cookies. view function uses requestcontext template, instead of context. in template, there {% csrf_token %} template tag inside each post form targets internal url. if not using csrfviewmiddleware, must use csrf_protect on views use csrf_token template tag, accept post data. you're seeing section of page because have debug = true in django settings file. change false, , initial error message displayed. can customize page using csrf_failure_view setting. did find, nothing helped, must foolish(( my template <form method="post">{% csrf_token %} {% field in form %} <div class="control-group"> {{ field }} </div> {% endfor %} <input type=...
Read more