1

I have a dataflow job that writes results to a Cloud Spanner table. Everything works fine when I run the job locally but running it on Dataflow gives the following error -

java.lang.IllegalArgumentException: Jetty ALPN/NPN has not been properly configured.
    io.grpc.netty.GrpcSslContexts.selectApplicationProtocolConfig(GrpcSslContexts.java:174)
    io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:151)
    io.grpc.netty.GrpcSslContexts.configure(GrpcSslContexts.java:139)
    io.grpc.netty.GrpcSslContexts.forClient(GrpcSslContexts.java:109)
    com.google.cloud.spanner.SpannerOptions$NettyRpcChannelFactory.newSslContext(SpannerOptions.java:289)
    com.google.cloud.spanner.SpannerOptions$NettyRpcChannelFactory.newChannel(SpannerOptions.java:278)
    com.google.cloud.spanner.SpannerOptions.createChannel(SpannerOptions.java:252)
    com.google.cloud.spanner.SpannerOptions.createChannels(SpannerOptions.java:239)
    com.google.cloud.spanner.SpannerOptions.<init>(SpannerOptions.java:88)
    com.google.cloud.spanner.SpannerOptions.<init>(SpannerOptions.java:43)
    com.google.cloud.spanner.SpannerOptions$Builder.build(SpannerOptions.java:179)

I get this error when trying to create SpannerOptions instance like so -

val options: SpannerOptions = SpannerOptions.newBuilder().build()

I saw this question but excluding netty-all like below does not help.

libraryDependencies ++= libraries(
  apache_beam,
  circe,
  scalatest % Test,
  junit % Test,
  hamcrest % Test
).map(_.exclude("io.netty", "netty-all"))

What can I do to fix this?

Kakaji
  • 1,260
  • 1
  • 13
  • 21
  • What version of the Java library of Cloud Spanner are you using? This error is typical if you have multiple conflicting versions of tcnative in your build path, or if you are missing it. The tcnative package has moved from "io.netty.tcnative" to "io.netty.internal.tcnative" in recent versions of the Cloud Spanner libraries. You could try to exclude the latter from your dependencies to see if it helps. – Knut Olav Loite Aug 16 '17 at 07:09
  • @KnutOlavLoite I have observed that this error occurs then `"com.google.cloud" % "google-cloud-pubsub" % "0.21.1-beta"` is a dependency of the project. Do you have any information on this please? Thanks! – Kakaji Nov 08 '17 at 03:02
  • I can see that `"com.google.cloud" % "google-cloud-pubsub" % "0.21.1-beta"` introduces `io.netty:netty-tcnative-boringssl-static:2.0.3.Final` into the project which evicts `io.netty:netty-tcnative-boringssl-static:1.1.33.Fork26` that Beam requires. Directly overriding `2.0.3` by `1.1.33.Fork26` does not help. – Kakaji Nov 08 '17 at 03:11
  • Sorry, I don't know about that. I created a tutorial on how to use Google Cloud Spanner JDBC with Dataflow. The JDBC driver shades its dependencies, which should (in most cases) fix these kind of problems. I don't know if that's an option for you, but you can find the tutorial here: http://www.googlecloudspanner.com/2017/10/google-cloud-spanner-with-apache-beam.html – Knut Olav Loite Nov 08 '17 at 07:53

1 Answers1

1

Fixed by matching google-cloud-spanner version with the one specified in Apache Beam's pom.xml.

Kakaji
  • 1,260
  • 1
  • 13
  • 21