hadoop教程.ppt

Hadoop使用聂志******@?概念?狭义云计算是指IT基础设施的交付和使用模式,通过网络以按需、易扩展的方式获得所需的资源(硬件、平台、软件)。?广义云计算是指服务的交付和使用模式,通过网络以按需、易扩展的方式获得所需的服务。这种服务可以是IT和软件、互联网相关的,也可以是任意其他的服务。?三层模型?Saas:more?Paas:hadoop?Iaas:openstack2016-11-5google vs hadoopGoogle calls it:Hadoop equivalent:MapReduceHadoopGFSHDFSBigtableHBaseChubbyZookeeper2Hadoop使用?编写map和reduce函数?运行测试–命令行运行–eclipse运行?查看结果2016-11-5map public static class TokenizerMapper extends Mapper<Object, Text, Text, IntWritable>{ private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(Object key, Text value, Context context ) throws IOException, InterruptedException { StringTokenizer itr = new StringTokenizer(()); while (()) { (()); (word, one); //设置key value } } }说明:map的输出key 、value和reduce的输入key、value要一致,见上面红色部分reducepublic static class IntSumReducer extends Reducer<Text,IntWritable,Text,IntWritable> { private IntWritable result = new IntWritable(); public void reduce(Text key, Iterable<IntWritable> values, Context context ) throws IOException, InterruptedException { int sum = 0; for (IntWritable val : values) { sum += (); //聚集操作 } (sum);(key, result); } }说明:map的输出key 、value和reduce的输入key、value要一致,见上面红色部分2016-11-5Job 配置public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs(); if ( != 2) { ("Usage: wordcount <in> <out>"); (2); } Job job = new Job(conf, "word count"); //job name (); (