反向排序减速器键

Reverse Sorting Reducer Keys(反向排序减速器键)

问题描述

以相反的顺序将 Map 输出键获取到减速器的最佳方法是什么?默认情况下，reducer 以键的升序接收所有键.任何帮助或评论都受到广泛赞赏.

What is the best approach to get the Map Output keys to a reducer in reverse order? By default the reducer receives all keys in ascending order of keys. Any help or comments widely appreciated.

简单来说，在正常情况下，如果 map 发出键 1,4,3,5,2，reducer 收到的键与 1,2,3,4 相同，5.我希望减速器改为接收 5,4,3,2,1.

In simple words, in the normal scenario, if a map emits keys 1,4,3,5,2 the reducer receives the same as 1,2,3,4,5. I would like the reducer to receive 5,4,3,2,1 instead.

推荐答案

在 Hadoop 1.X 中，您可以使用 JobConf.setOutputKeyComparatorClass.

In Hadoop 1.X, you can specify a custom comparator class for your outputs using JobConf.setOutputKeyComparatorClass.

您的比较器必须实现 RawComparator接口.

Your comparator must implement the RawComparator interface.

在 Hadoop 2.X 中，这是通过使用 Job.setSortComparatorClass，仍然带有 RawComparator 的实现.

With Hadoop 2.X, this is done by using Job.setSortComparatorClass, still with an implementation of RawComparator.

这篇关于反向排序减速器键的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！