Quantcast
Channel: Read parquet file having mixed data type in a column - Stack Overflow
Viewing all articles
Browse latest Browse all 2

Read parquet file having mixed data type in a column

$
0
0

I want to read a parquet file using spark sql in which one column has mixed datatype (string and integer).

val sqlContext = new SQLContext(sparkContext)val df = sqlContext.read.parquet("/tmp/data")

This throws me exception : Failed to merge incompatible data types IntegerType and StringType

Is there a way to explicitly type cast the column during read ?


Viewing all articles
Browse latest Browse all 2

Latest Images

Trending Articles





Latest Images