Build new Spark-Tensorflow-Connector for scala 2.12

[wuxianxingkong]

Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.

[yiwei00000]

Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.

You can build 2.12 by yourself!!!

[yiwei00000]

Spark has released 3.x which built with scala 2.12, but th newest version of Spark-Tensorflow-Connector is built with scala 2.11 which means not compatiable with spark 3.x. Maven repository is still 2.11.

Also, you use spark-tfrecord https://github.com/linkedin/spark-tfrecord, that support scala 2.12.

[dennisobrien]

It would be great to have an example of how to build this for Spark 3.x and Tensorflow 2.x

I tried a number of combinations with no success. I can build the jar, but it writes an empty TFRecord file.

If others have successfully built the jar for Spark 3.x and Tensorflow 2.x I'd love to hear the details.

[wuxianxingkong]

@dennisobrien Hi, try this: https://github.com/wuxianxingkong/spark-tfrecord (Built with spark 3.1.1 and scala 2.12)

[yundai424]

It would be super helpful if TF team could kindly help releasing spark-tensorflow-connector built with scala 2.12 to maven artifact 🙌 Right now the best workaround seems to be spark-tfrecord but there'll be migration effort across products using it ..

[ssabzevari-antuit]

Is this still not available? It seems it is not in Maven but Databricks is working fine, seems they compiled it for scala 2.12.