You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
One of the common pain points in debugging CDAP MapReduce (which includes Hydrator pipelines) has been missing logs - which means we need to ask the user to go to MR container logs using UI or cmdline and this is not ideal. Sometimes this is fixed in Release X and in Release X+1 there is retrogression and we don't catch it. In order to avoid this completely, we need an integration test case to make sure we collect MR logs - this might be accomplished by hitting the logs endpoint or we might need to add a new LogClient. The test should also make sure that the MR program throws exceptions and those logs are also captured (in addition to the successful completion path). Also add System.out/err.println in Mapper/Reducer logic to make sure we are capturing those statements too.
The text was updated successfully, but these errors were encountered:
One of the common pain points in debugging CDAP MapReduce (which includes Hydrator pipelines) has been missing logs - which means we need to ask the user to go to MR container logs using UI or cmdline and this is not ideal. Sometimes this is fixed in Release X and in Release X+1 there is retrogression and we don't catch it. In order to avoid this completely, we need an integration test case to make sure we collect MR logs - this might be accomplished by hitting the logs endpoint or we might need to add a new LogClient. The test should also make sure that the MR program throws exceptions and those logs are also captured (in addition to the successful completion path). Also add System.out/err.println in Mapper/Reducer logic to make sure we are capturing those statements too.
The text was updated successfully, but these errors were encountered: