DistCp does not honour config-overrides for mapreduce.[map,reduce].memory.mb

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major
    • 3.0.0-alpha1
    • Affects Version/s: 0.23.9, 2.2.0
    • Component/s: distcp
    • None
    • Incompatible change
    • Prior to this change, distcp had hard-coded values for memory usage. Now distcp will honor memory settings in a way compatible with the rest of MapReduce.

      When a DistCp job is run through Oozie (through a Java action that launches DistCp), one sees that mapred.child.java.opts as set from the caller is honoured by DistCp. But, DistCp doesn't seem to honour any overrides for configs mapreduce.[map,reduce].memory.mb.

      Problem has been identified. I'll post a patch shortly.

        1. MAPREDUCE-5653.trunk.patch
          1 kB
          Mithun Radhakrishnan
        2. MAPREDUCE-5653.branch-2.patch
          1 kB
          Mithun Radhakrishnan
        3. MAPREDUCE-5653.branch-0.23.patch
          1 kB
          Mithun Radhakrishnan
        4. MAPREDUCE-5653.trunk.2.patch
          0.7 kB
          Ratandeep Ratti

            Assignee:
            Ratandeep Ratti
            Reporter:
            Mithun Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated:
              Resolved: