GraalVM and OpenXION 'Native' building

[OpenXTalkPaul]

As discussed elsewhere in this forum, (in one of many threads that go off into different topics in wildly different directions)
I built a macOS X86-64bit 'native' build of OpenXION. OpenXION is a real nice xTalk interpreter written in JAVA that was built as a reference implementation for the XION xtalk dialect. In most basic syntax its xTalk interpreter is very true to HyperCard's interpreter, but also has additional syntax well beyond HyperTalk like OOP related syntax (which may be very handy) and is a bit more cross-platform polyglot-friendly. The biggest problem with it is that it's a 'terminal' style CLI, even answer/ask answer file, answer folder, etc. but those can be overridden and replaced with Graphical Dialogs fairly easily (see the OXT OpenXION repo for Mac/AppleScript based GUI Ask/Answer dialogs).

Since people have expressed some interest in also having Win, Linux, ARM, AppleSilicon, etc. builds of OpenXION, and I may not have time, and I definitely don't have the hardware to build these 'native' builds of it... and because I've basically forgotten the steps I used to build it, I thought I'd post this nice article, it's probably what I read that prompted me to do it when I built the Mac X86-64 build.

https://medium.com/graalvm/cli-applicat ... 29a40aa0be

This one goes into finer detail:

https://medium.com/graalvm/working-with ... 12ccdcd61b

As far as I remember the steps boiled down to:
Install GraalVM and run some commands to make sure you're using GraalVM as your JVM,
Then install natifier, run another command that points it to the OpenXION distribution .java files,
And viola, spits out a self contained 'native' executable, the JVM thereafter is no longer required
in order to run that executable on whatever platform you built the binary for.

Looking at GraalVM's site, and some of the API documentation, it seems that the VM supports Linux, Mac, Windows + IOS/ANDROID, on Intel/ARM/AppleSilicon/RISC processors, so anything that Oracle supports I guess:

• Platform.AARCH64
• Platform.AMD64
• Platform.ANDROID
• Platform.ANDROID_AARCH64
• Platform.DARWIN
• Platform.DARWIN_AARCH64
• Platform.DARWIN_AMD64
• Platform.HOSTED_ONLY
• Platform.IOS
• Platform.IOS_AARCH64
• Platform.IOS_AMD64
• Platform.LINUX
• Platform.LINUX_AARCH64
• Platform.LINUX_AARCH64_BASE
• Platform.LINUX_AMD64
• Platform.LINUX_AMD64_BASE
• Platform.LINUX_RISCV64
• Platform.MACOS
• Platform.MACOS_AARCH64
• Platform.MACOS_AMD64
• Platform.RISCV64
• Platform.WINDOWS
• Platform.WINDOWS_AARCH64
• Platform.WINDOWS_AMD64

[tperry2x]

Well, I was definitely one of those people, just because I find it very interesting.
Even more so, now reading the list of possible architectures.

Sounds like it would be possible to produce an Apple Silicon / MacOS Arm build, but do you know if it's possible to output for a different platform if the system you are building it on is intel?

I'll rtfm !, I'm sure it's mentioned somewhere.

Thanks for the info Paul, I'm hopeful this will prove helpful.

[OpenXTalkPaul]

Well, I was definitely one of those people, just because I find it very interesting.
Even more so, now reading the list of possible architectures.

Sounds like it would be possible to produce an Apple Silicon / MacOS Arm build, but do you know if it's possible to output for a different platform if the system you are building it on is intel?

I'll rtfm !, I'm sure it's mentioned somewhere.

Thanks for the info Paul, I'm hopeful this will prove helpful.

I'm not sure if GraalVM can or how to do cross-compiling for different CPU/Platform architectures then the one your Java install is, but since the original promise of Java was to deliver write-once-run-on-many App development, I could see that being doable I'll try to look into that as well.
EDIT:
some quick googles for GraalVM cross-compilation earned this:
https://blogs.oracle.com/developers/pos ... th-graalvm

[tperry2x]

Doesn't actually provide a truly 'native' solution by the look of it.
My understanding was that this turned a java program into a C+ program. Perhaps I understood incorrectly.
If you don't have java installed, then it's a no-go... so is it still in Java?

Screenshot at 2024-01-04 17-35-08.png (164.93 KiB) Viewed 846 times

[FourthWorld]

Does this interest in Java-based solutions indicate that OXT engine development is no longer a primary target?

[tperry2x]

This is just it. With OXT Lite, the engine seems frozen in time.
I'd actually like to get away from Java (that's what this is all about), into native C, and eventually adding wxWidgets for genuine native controls. Once I've got these hooks in place, and can get into an xTalk interpreter, then I'd be happy to build a new interface up from there. Probably using some of the IDE ideas you mentioned on your site at http://fourthworld.com/products/devolution/
(with your permission, of course)

Thing is, I'm continuously running into hurdles caused at the engine level which are not solvable in its current form. I know foxtrot47 was looking at the engine, but don't know how far that has progressed.

I had taken the code apart, and tried to dissect it, but from what I can gather: It's been added to again and again without any comments anywhere. A lot of it does not seem to make any sense as it's full of outdated API references that will no longer compile.

I think it better to start from scratch if we are to solve the engine issues properly, but that's a huge task and out of scope for just me. (As you have said before, we need a c+ dev team).

[OpenXTalkPaul]

Doesn't actually provide a truly 'native' solution by the look of it.
My understanding was that this turned a java program into a C+ program. Perhaps I understood incorrectly.
If you don't have java installed, then it's a no-go... so is it still in Java?
Screenshot at 2024-01-04 17-35-08.png

I'm not sure what you did, if you have the correct binary or not (should be around 12MB). The nativefier should scab for the parts of a JavaVM that are actually used and combine them (the 'linker' part of a typical compilation process) into a self-contained binary executable, no pre-installed JVM required (because its required parts are combined into the binary). I've tested this binary with a macOS clean install of Ventura with no JVM installed and it worked for me. A JVM is the 'Engine' OpenXION runs on. But it's only a tiny bit less native than C++ with some platform specifics abstracted away in JAVA bitcode.

Alternatively OXT has a 'Java' (JNI C++ bridged) engine as it's Android implimentation. Some of OXT engine(s) use stuff like SDL and old drawing APIs (and even older themes like Motif) underneath so those aren't so 'native' to begin with.

[OpenXTalkPaul]

Does this interest in Java-based solutions indicate that OXT engine development is no longer a primary target?

I can only speak for myself but OXT engine(s) development, for me was never the 'primary target'. The primary target was/is to have a usable, free and open-source xTalk available for the general public. I would like it to be one that is sustainable, as future-proof as possible.

I've become rather agnostic about what the underlying 'engine' is, even the whole xCard metaphor is not really requirement for me (Book/Pages, Video/Frames/Sprites, etc.) as long as there is a live-scriptable GUI elements (its the Rapid App Dev), at least must provide for basic Answer/Ask sort of dialogs (if included all of them, like answer color, answer record, etc. that would be super).

The OXT Engines are still, and without a doubt, the best available xTalk engines on Desktop and Mobile Platforms, so I intend to continue to work with/on them, but I'm Open to any/all xTalks really.

[OpenXTalkPaul]

I've also come to think that what Bill Atkinson viewed as a mistake, making HyperCard more Disk file / local-talk network oriented and not concentrating more on the wider WAN networking aspects (and I think he's right that the 'Web' could've been based on HyperCard, even more than it was), could now be seen as an advantage. Because Stacks (along with Resource Forks) are somewhat of encapsulated format much like PDF format, as in all the resources a stack required (beyond what the 'Engine' provided) could be contained within the stack itself, no external files required. Dan Gelder's HC simulator, with it's ability to convert many of the thousands of HyperCard stacks still available on the internet and convert their embedded resources into HTML elements including JavaScript-transpiling functionality while retaining most or all of the original content is proof that stack encapsulation should be seen as a good thing.

I would prefer that any implementation includes good media EXPORT capabilities though, as I still haven't figured out how to extract embedded audioClips in rev stacks format back out into a wave samples file on disk.

[tperry2x]

I still haven't figured out how to extract embedded audioClips in rev stacks format back out into a wave samples file on disk.

Was going to try some tests, but may have also discovered that playing sounds is just plain broken too.
Works on Windows and Mac, no luck on Linux it would seem

I'd concur that I want to see an xTalk language with GUI hooks in something other than anything based on the LCC engine. This is why I keep banging on about wxWidgets, as they use the native window controls. They do not try and emulate the window controls as an approximation *cough* LC *cough*. Plus, it's free to use.

I just don't know how to link OpenXION up to it. Once I get 'out' of a C or java enviroment, and into an xTalk one, then I'm quite happy. Java is close (in parts) to xTalk, but it's not xTalk. That's why I'm here and not a java dev on another forum.

I don't really care if that's in LCC, OXT, or whatever - I'd want to start afresh ideally, with modern UI and create an IDE that does what it says it can.

I'm increasingly finding with anything based on the LCC engine (or perhaps that should be LCCC - Livecode Community Cast-off) that the more stones I look under, the more nasty surprises there are. And you can only polish a you-know-what so much.

[tperry2x]

I'm not sure what you did, if you have the correct binary or not (should be around 12MB).

It is 12MB. Perhaps nativefier isn't built with MacOS 10.9 in mind? This is the one you provided on your github page, directly downloaded. So, this isn't me messing anything up as far as I know.

I've tested this binary with a macOS clean install of Ventura with no JVM installed and it worked for me. A JVM is the 'Engine' OpenXION runs on. But it's only a tiny bit less native than C++ with some platform specifics abstracted away in JAVA bitcode.

I'd of thought that if any of it was abstracted away in JAVA, then you'll need a bit of Java to run it, no?
I'll save you the BS, cookie warnings, and advertising cr4p at https://appleinsider.com/articles/11/02 ... setta.html, but Java was dropped from MacOS back in Lion, and you had to install it manually (or install something that also installed Java).
This continued in 10.9 Mavericks too, so this is why it's not there.

Screenshot_2024-01-05_17-49-30.png (44.66 KiB) Viewed 804 times

Screenshot_2024-01-05_17-50-08.png (41.33 KiB) Viewed 804 times

Perhaps I was being difficult (not deliberately) by running this on 10.9, but I knew that didn't contain an ounce of Java, so was a good testbed to find out if it actually needs Java.

Alternatively OXT has a 'Java' (JNI C++ bridged) engine as it's Android implimentation. Some of OXT engine(s) use stuff like SDL and old drawing APIs (and even older themes like Motif) underneath so those aren't so 'native' to begin with.

There's hardly anything that's truly native about the drawing of UI elements in LCC. This is why I'm in favour of going to a modern GUI (see comment above re wxWidgets)

[OpenXTalkPaul]

A lot has happened since 10.7 (and 10.9) came out, and I'm aware that Apple no longer includes Java with macOS. But that binary shouldn't need any JVM to run, because the required JVM bits were turned into a 'Standalone' and combined with OpenXION.java. I'll try to do some testing this OpenXION standalone on older macOS builds this weekend, maybe I'm wrong about it not needing a Java JDK installed (although that does seem like the whole point of 'native image' building).
You can install official Java from Oracle or one of the open-source JDKs (GraalVM, OpenJDK) on newer macOS versions. In fact having a Java install is already a requirement if you want to build apps for Android with LCCE or OXT.

[OpenXTalkPaul]

There's hardly anything that's truly native about the drawing of UI elements in LCC. This is why I'm in favour of going to a modern GUI (see comment above re wxWidgets)

True for the 'classic controls' but there are a few widgets that do actually use native UI (I think that was part of the motivation for extensions to begin with). Like there are MacOS Native Fld and macOS Native Button. They just haven't been worked on at all in order to make them more useable. You can see their truly native appearance but ONLY when you're in 'Run' mode, they're basically invisible when the IDE is in Edit mode. I suppose you could group the native button with a proxy non-native button or use some other workaround, but it would be better to improve on the widget and have it render some sort of non-native representation when in edit mode so you can see SOMETHING while designing your stack. Similarly a combined widget set could maybe be produced that uses Cocoa UI on macOS, GTK UI on Linux, Android UI on Android, etc. etc. So that same 'button' would have native appearance on all the platforms that these hypothetical universal button widget support.

[OpenXTalkPaul]

Error: No bin/java and no environment variable JAVA_HOME

Hmm, I wonder if that is because its looking for the JAVA_HOME environment variable, which hasn't been set since there's no JDK installed, you can set an environmental variable from OXT, or if the macOS app is in a app bundle then you can set environmental variables via a key in its info.plist. I wasn't executing the OpenXION binary from terminal, I was using it with 'open process' from the OXT engine, and also set he defaultFolder (working directory) first.

[OpenXTalkPaul]

Another thought...
If you really want to build on a not-Java but C++ compiled xTalk Engine that's stripped down, sans the xCard UI Widget toolkit, then you should probably be looking at the compiling the 'server' edition engine, which should be easier to compile as there should be less linking, less dependencies involved with that version... or maybe look at the source-code to Hammer (the xTalk dialect of Uli Kusterer's StackSmith (it's also written in C++ ).
Either one you'll need to add in wxWidgets library and use some sort of binding mechanism like libFFI (as used by Lua, Python, Extension Builder, etc.) or some sort of 'extension' wrapper library to make wxWidgets useable from xTalk. With OpenXION you may be able to use the wxWidgets Python bindings by calling proxy Python code (OpenXION supports 'do tPythonScript as Python' syntax). StackSmith built for macOS so it uses Cocoa but I believe I read that Uli set it up in such a way as to allow for swapping out with GTK or other UI Widget toolkit. I compiled StackSmith from its source very easily a few years back, so that's nice.