Pivotal Greenplum®-Spark® Connector v1.1

Greenplum Database -> Spark Data Type Mapping

When you load Greenplum data into Spark, the Greenplum-Spark Connector maps Greenplum Database data types to Spark data types for you.

For additional information on Greenplum Database or Spark data types, refer to:

Supported Data Types

The mapping between supported Greenplum Database column types and Spark data types is identified in the table below:

Greenplum Column Type Spark Type
bigint LongType
bigSerial LongType
boolean BooleanType
char StringType
date DateType
decimal DecimalType
float DoubleType
int IntegerType
real FloatType
serial IntegerType
smallInt ShortType
text StringType
time TimeStampType
timestamp TimeStampType
timestamptz TimeStampType
timetz TimeStampType
varchar StringType

The Greenplum-Spark Connector does not support complex types, nor does it support any data type not listed in the table above.