What's the magic behind ScalaFX to make OpenJDK 9+ actually work?

Question

Environment:

• OpenJDK 64-Bit Server VM Zulu12.2+3-CA (build 12.0.1+12, mixed mode, sharing)
• Scala 2.12.7
• Windows 10 Professional, X86_64
• IntelliJ IDEA 2019.1.3 (Ultimate Edition)

I checked out the scalafx-hello-world from GitHub, built and ran it in IntelliJ and it worked all fine. Here quickly the significant application implementation:

package hello

import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.geometry.Insets
import scalafx.scene.Scene
import scalafx.scene.effect.DropShadow
import scalafx.scene.layout.HBox
import scalafx.scene.paint.Color._
import scalafx.scene.paint._
import scalafx.scene.text.Text

object ScalaFXHelloWorld extends JFXApp {

  stage = new PrimaryStage {
    //    initStyle(StageStyle.Unified)
    title = "ScalaFX Hello World"
    scene = new Scene {
      fill = Color.rgb(38, 38, 38)
      content = new HBox {
        padding = Insets(50, 80, 50, 80)
        children = Seq(
          new Text {
            text = "Scala"
            style = "-fx-font: normal bold 100pt sans-serif"
            fill = new LinearGradient(
              endX = 0,
              stops = Stops(Red, DarkRed))
          },
          new Text {
            text = "FX"
            style = "-fx-font: italic bold 100pt sans-serif"
            fill = new LinearGradient(
              endX = 0,
              stops = Stops(White, DarkGray)
            )
            effect = new DropShadow {
              color = DarkGray
              radius = 15
              spread = 0.25
            }
          }
        )
      }
    }

  }
}

EDIT: My build.sbt:

// Name of the project
name := "ScalaFX Hello World"

// Project version
version := "11-R16"

// Version of Scala used by the project
scalaVersion := "2.12.7"

// Add dependency on ScalaFX library
libraryDependencies += "org.scalafx" %% "scalafx" % "11-R16"
resolvers += Resolver.sonatypeRepo("snapshots")

scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8", "-feature")

// Fork a new JVM for 'run' and 'test:run', to avoid JavaFX double initialization problems
fork := true

// Determine OS version of JavaFX binaries
lazy val osName = System.getProperty("os.name") match {
  case n if n.startsWith("Linux") => "linux"
  case n if n.startsWith("Mac") => "mac"
  case n if n.startsWith("Windows") => "win"
  case _ => throw new Exception("Unknown platform!")
}

// Add JavaFX dependencies
lazy val javaFXModules = Seq("base", "controls", "fxml", "graphics", "media", "swing", "web")
libraryDependencies ++= javaFXModules.map( m=>
  "org.openjfx" % s"javafx-$m" % "11" classifier osName
)

After that, I changed the implementation to:

package hello

import javafx.application.Application
import javafx.scene.Scene
import javafx.scene.control.Label
import javafx.stage.Stage

class ScalaFXHelloWorld extends Application {
  override def start(stage: Stage): Unit = {
    stage.setTitle("Does it work?")
    stage.setScene(new Scene(
      new Label("It works!")
    ))
    stage.show()
  }
}

object ScalaFXHelloWorld {
  def main(args: Array[String]): Unit = {
    Application.launch(classOf[ScalaFXHelloWorld], args: _*)
  }
}

Here I get the following error:

Exception in Application start method
java.lang.reflect.InvocationTargetException
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:464)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:363)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:567)
    at java.base/sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:1051)
Caused by: java.lang.RuntimeException: Exception in Application start method
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:900)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication$2(LauncherImpl.java:195)
    at java.base/java.lang.Thread.run(Thread.java:835)
Caused by: java.lang.IllegalAccessError: superclass access check failed: class com.sun.javafx.scene.control.ControlHelper (in unnamed module @0x40ac0fa0) cannot access class com.sun.javafx.scene.layout.RegionHelper (in module javafx.graphics) because module javafx.graphics does not export com.sun.javafx.scene.layout to unnamed module @0x40ac0fa0
    at java.base/java.lang.ClassLoader.defineClass1(Native Method)
    at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1016)
    at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:151)
    at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:802)
    at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:700)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:623)
    at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
    at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    at javafx.scene.control.Control.<clinit>(Control.java:86)
    at hello.ScalaFXHelloWorld.start(ScalaFXHelloWorld.scala:39)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
    at java.base/java.security.AccessController.doPrivileged(AccessController.java:389)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
    ... 1 more
Exception running application hello.ScalaFXHelloWorld

Now my question is: What does ScalaFX that the module problem does not occur?

Answer 1

I've not been able to exactly reproduce your problem, but I have been able to get a project that uses JavaFX-only (that is, it doesn't make use of ScalaFX) to build and run.

Here's what I'm using (everything else is specified in the build file):

• Zulu OpenJDK 11
• SBT 1.2.8

(I did try using Zulu OpenJDK 12 to build and run the project, and that worked too. However, it's probably best if you use the version of OpenJFX that matches the JDK.)

When I tried your original sources and build.sbt, I encountered the following error when executing an sbt run command from the command line:

D:\src\javafx11>sbt run
[info] Loading global plugins from {my home directory}\.sbt\1.0\plugins
[info] Loading project definition from D:\src\javafx11\project
[info] Loading settings for project javafx11 from build.sbt ...
[info] Set current project to JavaFX 11 Hello World (in build file:/D:/src/javafx11/)
[info] Running (fork) hello.ScalaFXHelloWorld
[error] Error: JavaFX runtime components are missing, and are required to run this application
[error] Nonzero exit code returned from runner: 1
[error] (Compile / run) Nonzero exit code returned from runner: 1
[error] Total time: 1 s, completed Aug 11, 2019, 3:17:07 PM

as I mentioned in my original comments to your question.

I thought that was odd because the code compiled, which meant that the compiler was able to find the JavaFX runtime just fine.

I then tried running the program without forking, by commenting out the fork := true in the build file. Guess what? The program ran without error!

I may be missing something, regarding using SBT with JDK versions 9+, but this indicated that SBT was somehow not running the forked process correctly. I could force the forked process to run correctly by adding the following to the end of the build file:

val fs = File.separator
val fxRoot = s"${sys.props("user.home")}${fs}.ivy2${fs}cache${fs}org.openjfx${fs}javafx-"
val fxPaths = javaFXModules.map {m =>
  s"$fxRoot$m${fs}jars${fs}javafx-$m-11-$osName.jar"
}
javaOptions ++= Seq(
  "--module-path", fxPaths.mkString(";"),
  "--add-modules", "ALL-MODULE-PATH"
)

This works by adding the downloaded ivy-managed JavaFX jar files to Java's module path. However, this is not a good solution for running standalone applications. It may be possible for the sbt-native-packager to provide the necessary environment for the completed application to run, but I haven't tried that.

I've posted the complete solution on GitHub

Let me know whether this helps. In the meantime, I'll look into SBT's support for JDK 9+ modules to see whether there is a simpler solution...

UPDATE:

I have raised an issue (#4941) with the SBT team to look into this in more detail.

UPDATE 2

I patched an issue that stopped the solution from working on Linux. Perform a git pull to update the sources.

UPDATE 3

I should also mention that it's best to have IntelliJ run the application using SBT, which keeps things simple and ensures that the application's environment is properly configured.

To do this, got to the IntelliJ Run menu, and select the Edit Configurations... option. Click the + button at the top left corner of the dialog, select sbt Task" from the list under **Add New Configuration, then configure as follows:

This will compile and build the application first, if required.

Note: The _VM parameters are for running SBT, and do not relate to how SBT runs your forked application.

(You can also add SBT run configurations to test your code, as well.)

Answer 2

Adding to Jonathan Crosmer's answer:

The reason that naming the class and the object differently works is because the Java launcher actually has special behaviour in place if the main class extends javafx.application.Application. If you have the Java sources available, the relevant code can be found in JAVA_HOME/lib/src.zip/java.base/sun/launcher/LauncherHelper.java. In particular there are two methods which are of interest:

public static Class<?> checkAndLoadMain(boolean, int ,String)

//In nested class FXHelper
private static void setFXLaunchParameters(String, int)

The first methods has a check in place that looks if the main class extends javafx.application.Application. If it does, this method replaces the main class with the nested class FXHelper, which has its own public static void main(String[] args).

The second method, which is directly called by the first method, attempts to load the JavaFX runtime. However, the way it does this is by first loading the module javafx.graphics via java.lang.ModuleLayer.boot().findModule(JAVAFX_GRAPHICS_MODULE_NAME). If this call fails, Java will complain about not having found the JavaFX runtime and then immediatly exit via System.exit(1).

Going back to SBT and Scala, some other details are in play. First, if both the main object and the class extending javafx.application.Application have the same name, the Scala compiler will generate a class file which both extends Application and has a public static void main(...). That means that the special behaviour described above will be triggered and the Java launcher will try to load the JavaFX runtime as a module. Since SBT currently has no notion about modules, the JavaFX runtime will not be on the module path and the call to findModule(...) will fail.

On the other hand, if the main object has a different name from the main class, the Scala compiler will place public static void main(...) in a class which does not extend Application, which in turn means that the main() method will execute normally.

Before we go on, we should note that while SBT did not put the JavaFX runtime on the module path, it DID in fact put it on the classpath. That means that the JavaFX classes are visible to JVM, they just can not be loaded as a module. After all

A modular JAR file is like an ordinary JAR file in all possible ways, except that it also includes a module-info.class file in its root directory.

(from The State of the Module System)

However, if a method happens to call, let's say Application.launch(...), Java will happily load javafx.application.Application from the classpath. Application.launch(...) will similarly have access to the rest of JavaFX and everything works out.

That is also the reason why running a JavaFX app without forking works. In that case SBT will always invoke public static void main(...) directly, which means that no special behaviours from the java launcher are triggered and the JavaFX runtime will be found on the classpath.

---

Here is a snippet to see the above behaviour in action:

Main.scala:

object Main {
  def main(args: Array[String]): Unit = {
    /*
    Try to load the JavaFX runtime as a module. This is what happens if the main class extends
    javafx.application.Application.
     */
    val foundModule = ModuleLayer.boot().findModule("javafx.graphics").isPresent
    println("ModuleLayer.boot().findModule(\"javafx.graphics\").isPresent = " + foundModule) // false

    /*
    Try to load javafx.application.Application directly, bypassing the module system. This is what happens if you
    call Application.launch(...)
     */
    var foundClass = false
    try{
      Class.forName("javafx.application.Application")
      foundClass = true
    }catch {
      case e: ClassNotFoundException => foundClass = false
    }
    println("Class.forName(\"javafx.application.Application\") = " + foundClass) //true
  }
}

build.sbt:

name := "JavaFXLoadTest"

version := "0.1"

scalaVersion := "2.13.2"

libraryDependencies += "org.openjfx" % "javafx-controls" % "14"

fork := true

Answer 3

I ran across this same exact problem and found a disturbingly odd and easy solution. tldr; make the main class have a different name from the JavaFX Application class. First an example:

import javafx.application.Application
import javafx.event.ActionEvent
import javafx.event.EventHandler
import javafx.scene.Scene
import javafx.scene.control.Button
import javafx.scene.layout.StackPane
import javafx.stage.Stage

object HelloWorld {
  def main(args: Array[String]): Unit = {
    Application.launch(classOf[HelloWorld], args: _*)
  }
}

// Note: Application class name must be different than main class name to avoid JavaFX path initialization problems!  Try renaming HelloWorld -> HelloWorld2
class HelloWorld extends Application {
  override def start(primaryStage: Stage): Unit = {
    primaryStage.setTitle("Hello World!")
    val btn = new Button
    btn.setText("Say 'Hello World'")
    btn.setOnAction(new EventHandler[ActionEvent]() {
      override def handle(event: ActionEvent): Unit = {
        System.out.println("Hello World!")
      }
    })
    val root = new StackPane
    root.getChildren.add(btn)
    primaryStage.setScene(new Scene(root, 300, 250))
    primaryStage.show()
  }
}

The code as written above throws the exception from the original question. If I rename the class HelloWorld to HelloWorld2 (keeping object HelloWorld, and changing the launch call to classOf[HelloWorld2]), it runs fine. I suspect this is the "magic" that makes ScalaFX work as well, because it is wrapping the JavaFX Application in its own JFXApp type, creating a hidden Application class.

Why does it work? I'm not completely sure, but when running each piece of code in IntelliJ using a standard Run config (right-click HelloWorld and "run HelloWorld.main()") , then in the output clicking "/home/jonathan/.jdks/openjdk-14.0.1/bin/java ..." to expand it shows a command that includes "--add-modules javafx.base,javafx.graphics ", among other things. In the second version, with the renamed HelloWorld2 app, the command does not include this. I can't figure out how IntelliJ has decided to make the command different, but I can only speculate it has something to do with inferring it is a JavaFX app and trying to be helpful by automatically adding "--add-modules "...? In any case, the modules list doesn't include all the modules needed, so for example creating a button requires "javafx.controls", and you get the error. But when the main class doesn't match the Application name, whatever magic inference it does gets turned off, and the standard classpath from the build.sbt just works.

Fun follow up: if I run the application from the sbt shell using sbt run, then the pattern is the same (HelloWorld fails, but renaming the application class fixes it), but the error message is the more-straightforward-but-still-unhelpful "Error: JavaFX runtime components are missing, and are required to run this application". So maybe not entirely an IntelliJ problem, but something to do with JavaFX and Jigsaw? Anyway it's a mystery, but at least we have an easy solution.