TiSpark 2.3.0

New Features

• Support working with TiDB-3.1 and TiDB-4.0
• Support working with TiFlash
• Support directly writing data to TiKV

Fixes

• Support new row format encoding #1492
• Fix test: add TiFlash replica available check #1495
• Fix string type pushdown #1500

TiSpark 2.3.0-rc.1

• Fix column name case bug #1487
• Fix resolve current lock npe bug #1485
• Fix resolve lock bug with tiflash #1483

TiSpark 2.3.0-rc

New Features

• Support working with TiDB-3.1 and TiDB-4.0
• Support working with TiFlash
• Support directly writing data to TiKV

TiSpark 2.1.9

Fixes

• Fix desc temp view #1328
• Fix prefix index on blob #1334
• Shade io.opencensus to resolve grpc conflict #1352
• Fix parition table isn't shown in show command #1374
• Fix partition pruning when partition definition contains big integer #1385
• Support TiDB-4.0 #1398

TiSpark 2.1.8

Fixes

• Fix UnsupportedOperationException: using stream rather than removeIf #1303

TiSpark 2.1.7

Fixes

• Add task retry if tikv is down #1207
• Fix output offsets: add field type to Constant and ColumnRef when encoding proto #1231
• Register udf ti_version for every sparksession #1258
• Add timezone check #1275
• Disable set, enum and bit pushed down #1242

TiSpark 2.1.6

Fixes

• Fix TopN push down bug #1185
• Consider nulls order in TopN pushdown #1187
• Fix Stack Overflow Error when reading from partition table #1179
• Fix parsing view table's json bug #1174
• Fix No Matching column bug #1162
• Fix behavior of estimateTableSize #845
• Fix Bit Type default value bug #1148
• Fix fastxml security alert #1127
• Fix bug: TiSpark Catalog has 10-20s delay #1108
• Fix reading data from TiDB in Spark Structured Streaming #1104

TiSpark 2.1.5

Fixes

• Remove useless scala and jackson dependencies #1079
• Fix range partition throw UnsupportedSyntaxException error #1088
• Make TiSpark reading data from a hash partition table #1089

TiSpark 2.2.0

New Features

• Natively support writing data to TiKV (ACID) using Spark Data Source API

WARNING

DO NOT set spark.tispark.write.without_lock_table to true on production environment (you may lost data).

Improvements

• Release one TiSpark jar (both support Spark-2.3.x and Spark-2.4.x) instead of two #933
• Add spark version to TiSpark's udf ti_version #943
• Bump grpc to 1.17 #982
• Add retry mechanism for batchGet #986

Fixes

• Catch UnsupportedSyntaxException when generating partition expressions #960
• Fix TiSpark cannot read from a hash partition table #966
• Prohibit extra index data type pushdown when doing index scan to avoid decoding extra column #995
• Prohibit agg or groupby pushdown on double read #1004

TiSpark 2.1.4

Fixes

• Fix distinct without alias bug: disable pushdown aggregate with alias #1055
• Fix reflection bug: pass in different arguments for a different version of same function #1037