AWS Hadoop MapReduce - Word Count Average(AWS Hadoop MapReduce - 单词计数平均值)
创始人
2024-11-16 09:30:52
0

以下是使用AWS Hadoop MapReduce解决Word Count Average问题的示例代码:

Mapper类:

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class WordCountMapper extends Mapper {

    private final static IntWritable one = new IntWritable(1);
    private Text word = new Text();

    public void map(Object key, Text value, Context context) throws IOException, InterruptedException {
        String[] words = value.toString().split(" ");
        for (String word : words) {
            this.word.set(word);
            context.write(this.word, one);
        }
    }
}

Reducer类:

import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class WordCountReducer extends Reducer {

    private IntWritable result = new IntWritable();

    public void reduce(Text key, Iterable values, Context context)
            throws IOException, InterruptedException {
        int sum = 0;
        int count = 0;
        for (IntWritable value : values) {
            sum += value.get();
            count++;
        }
        int average = sum / count;
        result.set(average);
        context.write(key, result);
    }
}

Driver类:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;

public class WordCountAverage {

    public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        Job job = Job.getInstance(conf, "word count average");
        job.setJarByClass(WordCountAverage.class);
        job.setMapperClass(WordCountMapper.class);
        job.setCombinerClass(WordCountReducer.class);
        job.setReducerClass(WordCountReducer.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(IntWritable.class);
        FileInputFormat.addInputPath(job, new Path(args[0]));
        FileOutputFormat.setOutputPath(job, new Path(args[1]));
        System.exit(job.waitForCompletion(true) ? 0 : 1);
    }
}

请注意,上述代码仅为示例,您需要根据您的实际需求进行适当的更改和配置。此示例假定输入文件中的每个单词以空格分隔,并且每个单词都位于单独的行中。

相关内容

热门资讯

透视游戏!wepoke规律(W... 透视游戏!wepoke规律(WePoke)外挂透明挂辅助插件(透视)细节揭秘(有人有挂)-哔哩哔哩1...
透明软件(德扑ai助手)外挂透... 透明软件(德扑ai助手)外挂透明挂辅助APP(透视)透视辅助(有挂头条)-哔哩哔哩;德扑ai助手最新...
透视了解(微扑克wpk)外挂透... 透视了解(微扑克wpk)外挂透明挂辅助工具(辅助挂)软件透明挂(有挂教学)-哔哩哔哩,支持语音通讯、...
今日重大通报!德扑之星真的太假... 1、今日重大通报!德扑之星真的太假(WepoKe)外挂透明挂辅助挂(辅助挂)AA德州教程(有挂分析)...
玩家必看教程(WPK小程序)外... 【福星临门,好运相随】;玩家必看教程(WPK小程序)外挂透明挂辅助器安装(辅助挂)果真真的有挂(20...
6分钟了解!扑克时间后台数据(... 6分钟了解!扑克时间后台数据(WepOke)外挂透明挂辅助工具(透视)玩家教程(有挂教学)-哔哩哔哩...
八分钟了解(德扑计算)外挂透明... 八分钟了解(德扑计算)外挂透明挂辅助APP(透视)透视辅助(确实有挂)-哔哩哔哩;玩家必备必赢加哟《...
透明教学(WPk)外挂透明挂辅... 透明教学(WPk)外挂透明挂辅助APP(透视)透视辅助(竟然有挂)-哔哩哔哩;科技详细教程小薇《75...
热门推荐!德州软件工具(wep... 热门推荐!德州软件工具(wepoke)外挂透明挂辅助工具(透视)AI教程(有挂秘诀)-哔哩哔哩准备好...
玩家必知教程(x-poker)... 玩家必知教程(x-poker)外挂透明挂辅助工具(辅助挂)原来是真的有挂(2022已更新)(哔哩哔哩...