You have to build 2 different scripts/installers, one for each platform. Even looking at very popular software like Chrome, the platform choice is still there (even though you're usually directed to the correct choice based on the information your browser provides them with)
Depending on how much time you can put in this task, the quickest (and dirtiest) solution would be having an archive for each platform that contains the script .bat
for Windows and .sh
for OS X together with the jre
(also different per platform), ask the user to unarchive and run the script which will run your jar with the packed jre. Otherwise, you'll need to create an MSI/exe
for windows and a dmg
(or other installer type) for MacOs.
I have done the dmg before with a bundled jre and can try to look for the details if you need them (I no longer have access to the code but can probably find the details). It was a free solution but it did require an OS X computer to create the dmg
.
One option that I used before and works very well is install4j but the price is not small.
LE: Self contained packaging - although I haven't used this before, it seems like the best current option for your problem.
And an open source option - packr.