在 Map reduce java 中跳过 .csv 的第一行

Skipping the first line of the .csv in Map reduce java(在 Map reduce java 中跳过 .csv 的第一行)

问题描述

由于 mapper 函数对每一行都运行，我可以知道如何跳过第一行吗?对于某些文件，它包含我想忽略的列标题

As mapper function runs for every line , can i know the way how to skip the first line. For some file it consists of column header which i want to ignore

推荐答案

在mapper中读取文件时，数据以键值对的形式读入.关键是下一行开始的字节偏移量.对于第 1 行，它始终为零.所以在映射器函数中执行以下操作

In mapper while reading the file, the data is read in as key-value pair. The key is the byte offset where the next line starts. For line 1 it is always zero. So in mapper function do the following

    @Override
    public void map(LongWritable key, Text value, Context context) throws IOException {
        try {
            if (key.get() == 0 && value.toString().contains("header") /*Some condition satisfying it is header*/)
                return;
            else {
                // For rest of data it goes here
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }     

这篇关于在 Map reduce java 中跳过 .csv 的第一行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持编程学习网！