Thursday, March 9, 2017

Hive External Table

Script to create an external table in Hive to read records from a HDFS folder:

CREATE EXTERNAL TABLE Mytable (
    Id int,
    PlanID string,
    ServiceID string,
    SessionDuration int)
STORED AS TEXTFILE
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ","
LOCATION "/user/hive/staging/"
TBLPROPERTIES(
    "skip.header.line.count"="1"
);

Using the script above, it'll use the files in /user/hive/staging and skip the first line of each file.