July 29th 2022 (one year since last official community release)

Updates on the progress of this project
User avatar
richmond62
Posts: 2621
Joined: Sun Sep 12, 2021 11:03 am
Location: Bulgaria
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by richmond62 »

I do not know we need a recompiled engine: the current one works.

Does it contain LC branding?

The only pressing need is for a flavour of the engine to produce Macintosh ARM native standalones.
https://richmondmathewson.owlstown.net/
mdm
Posts: 22
Joined: Thu Sep 16, 2021 2:15 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by mdm »

Ok. The Apple Silicon gap is just one of many issues why the project needs to be able to compile, otherwise it will not be worth a lot.

We can find frequent mention of dubious components in LC on any of the LC/xTalk discussion outlets if we look. I never made a list. This is mostly technical stuff, maybe you do not follow that kind of talk too closely. But you definitely should.

Of course the binaries "work". Now they do. Just some of the stuff in there is already mouldy, depending on what platform we are talking about. As a start we might look at the pre-compiled shared libraries that Paul has recently acquired from somewhere and which are needed in any MacOS re-compile. They are pretty old shit I think. You want current versions of stuff that you integrate.

Such a need might be triggered by changes in the OS context (like Apple Silicon or just some stupid API) or by available functional upgrades that you need to have (e.g. new protocol for browser interaction). Worst of all and most prominent are security breaches. You've heard of those, haven't you? If some hacker publishes a leak in the version of the crypting library or browser component that sadly sits in your xTalk engine, what do you do? This happens all the time. Read the news. Your binary might become a threat overnight. Any time. If you cannot repair, you have a broken software.

I know you love vintage computers and recycling/upcycling/tinkering. In such a scenario, you can maybe seal off a personal installation from attackers for a while. If you do not connect to the internet or are very careful when receiving mail or browsing the web, you might have a solution for yourself. If you just need a playground, you can hold out for a long time. Otherwise not.

Pain levels may vary for sure. There might be opinions if a certain software is still on the responsible side. You may be aware of the case of Adobe Flash. This has been dragging on for a long time, but now it is gone.

You could not recommend to other people to use really vulnerable code, meaning an engine with documented flaws. You should warn them to stay away. Anyone with a little knowledge in the field (like IT people in companies, public sector, even schools) will do their little reseach and will not touch such stuff anyway. Your target group may become extremely limited. You cannot tell your students to learn programming in such an environment because they will hardly be allowed later to use it outside of the classroom, in the "real world" (corporate or else).

All this not only applies to foreign components, but also to home-grown bugs (currently meaning the code that LC Ltd. has produced over the years). If a project cannot change any of its core code, compile a new binary, publish a repair, then it is fatally crippled. If upgrades are impossible, the engine (meaning the existing binaries) will simply rot away. The Windows version might hold out longest (because Microsoft are compatibility Nazis) but I would take no bets.

The pace out there is relentless, it might even be accelerating, one has to stay current.
You must have heard some of this before. People have written it here and elsewhere not just once. I think there was a thread by Paul and Richard about the compiling issue.

This also means you can invest in IDE work or other upgrades on the scripting level, but at some point there will be no more free engine to run it on. One platform will go first, others will follow. You will be able to fall back to a licensed commercial LC engine as long as there is one and probably run your stacks there. But this is not OXT. If there is no OXT engine (but just repurposed antique LC binaries), there is no OXT. It is as simple as that. Compile or die.
User avatar
richmond62
Posts: 2621
Joined: Sun Sep 12, 2021 11:03 am
Location: Bulgaria
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by richmond62 »

Paul can play around with any code that he likes to. He is the one to decide about his time.
Indeed he can.

Just as I can shut my school up for week and push off to the mountains.

However, having said I will teach kids during that week, there is at least a moral duty . . .
https://richmondmathewson.owlstown.net/
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

richmond62 wrote: Sat Aug 06, 2022 5:20 pm
Well I wish they would say that more publicly so I know what the consensus is.
Maybe they don't have the brass neck that I do. :)

And 'consensus' may be overdoing things as we're talk about myself + 3: you may have noticed that there are very, very
few people still looking at these forums.

I suspect this is because, to a large extent, you, having produced a very good, 98% IDE for Linux, then went off
on all sorts of abstruse tangents without checking whether anyone apart from yourself was interetsed in them.
Oh come on! I don't bite! I really hope no one would be afraid to say anything they want to me. I can take it.
I might not agree, or do what you want me to, but I always try to be open to suggestions.
3 - 4 people or not, you guys are currently 'the community'!

That said (and with the knowledge of what else is going on in the rest of my life, and therefore what I actually have time to do, and what I have been doing), I don't agree with that assessment at all. Virtually everything I've done has been de-branding AND about differentiating this here from 'over there', but it also necessarily has been about learning as much as possible about how the IDE is constructed and 'themed' (which currently still has those settings as static/constants, but they should be User Prefs IMO), and there is a small list of bug fixes as well (particularly in the Extension Builder stack). I've barely touched my Windows install though ( least favorite OS ).

There's also been some preliminary research into building the thing(s) from source, because that's one of the next big thing we will eventually need to do. As well there has been some researching alternative xTalks that are available to use. That is because this is OPENxTalk, not OpenLCC! Then there's things like WebAssembly/bytecode modules, we could potentially have a path there for hardware/platform agnosticism/portability, we want to be able to run xTalk everywhere, right? How about on a xTalk on a BeOS clone? https://github.com/electron/electron/is ... -754066075

The playSentence music stuff and using my own builder extensions to add things to OXT, which I consider part of 'differentiating', is also partially about keeping it fun for me, that's really reward. Musical dev is a good part of what inspires me to work on this stuff, since late 1980s / early 1990s when I first started to get some MIDI gear, experimenting with HyperMIDI (a great XCMD/XFCN by "Ear Level Training"). There's also been a few inspirational individuals in the xTalk community over the years, like Frederic Rinaldi, Hermann Hoch, Becky Bettencourt, and others who have kept the xTalk-sphere interesting even while companies like Apple have neglected and/or abandoned these things.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

richmond62 wrote: Sun Aug 07, 2022 1:38 pm I do not know we need a recompiled engine: the current one works.

Does it contain LC branding?

The only pressing need is for a flavour of the engine to produce Macintosh ARM native standalones.
We sort-of have options there, I'm convinced that the Emscripten engine would run 'near native' inside a WebApp wrapper.
I'm pretty sure WebAssembly is based on Intel x86 assembly to make porting existing things easier.
There's also OpenXION which would run on ARM build of Oracle's JavaVM, a very capable (more HyperTalk compatible) cross-platform xTalk engine, but that needs GUI stuff.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

mdm wrote: Sun Aug 07, 2022 11:55 pm Ok. The Apple Silicon gap is just one of many issues why the project needs to be able to compile, otherwise it will not be worth a lot.

We can find frequent mention of dubious components in LC on any of the LC/xTalk discussion outlets if we look. I never made a list. This is mostly technical stuff, maybe you do not follow that kind of talk too closely. But you definitely should.

Of course the binaries "work". Now they do. Just some of the stuff in there is already mouldy, depending on what platform we are talking about. As a start we might look at the pre-compiled shared libraries that Paul has recently acquired from somewhere and which are needed in any MacOS re-compile. They are pretty old shit I think. You want current versions of stuff that you integrate.
My goal for compiling will be as much as possible to whittle it down to the core xTalk language interpreting engines, sans-as-many-dependencies as can be cut-out. I think this work should make it easier to compile individual engines and components. I believe most-all of those components could be replaced with Builder Extensions, which makes updating those things potentially as simple as dropping in the latest shared library binaries, and we might not even need to update the Extension Builder bindings for small patches. One exception being Chromium (CEF builds) that are included. Has anybody tried dropping in different (more up to date) builds of that component? Seems like that should be a drop-in replacement too. I think ultimately it would be better if we could use whatever browser-view, webkit or whatever web engine, that the operating system provides (as most do), on all platforms not just macOS (which I believe is what they're doing in the commercial zone), then things like security of data transfer would be inherently as secure as the OS is.
mdm wrote: Sun Aug 07, 2022 11:55 pm Such a need might be triggered by changes in the OS context (like Apple Silicon or just some stupid API) or by available functional upgrades that you need to have (e.g. new protocol for browser interaction). Worst of all and most prominent are security breaches. You've heard of those, haven't you? If some hacker publishes a leak in the version of the crypting library or browser component that sadly sits in your xTalk engine, what do you do? This happens all the time. Read the news. Your binary might become a threat overnight. Any time. If you cannot repair, you have a broken software.
I wonder how SuperCard is doing? The problem with that (macOS only) commercial xTalk was that it was built around Carbon APIs, never ported to Cocoa, and then (inherently because Carbon was deprecated ages ago) never got a 64bit build. I did see some talk of a 64bit port, but not sure if that will ever surface.
mdm wrote: Sun Aug 07, 2022 11:55 pm Pain levels may vary for sure. There might be opinions if a certain software is still on the responsible side. You may be aware of the case of Adobe Flash. This has been dragging on for a long time, but now it is gone.
Is it really gone or is it under Apache umbrella now?
https://en.wikipedia.org/wiki/Apache_Flex
mdm wrote: Sun Aug 07, 2022 11:55 pm This also means you can invest in IDE work or other upgrades on the scripting level, but at some point there will be no more free engine to run it on. One platform will go first, others will follow. You will be able to fall back to a licensed commercial LC engine as long as there is one and probably run your stacks there. But this is not OXT. If there is no OXT engine (but just repurposed antique LC binaries), there is no OXT. It is as simple as that. Compile or die.
I'm not sure about that, at least that particular phrasing. There will always be this source out available for those interested in open source xTalk, as well as a few others that could be worked from, but yes, we need to compile that source, and we need ways to keep running xTalk everywhere we possible can, in preferably the most hardware and platform agnostic ways possible.
mdm
Posts: 22
Joined: Thu Sep 16, 2021 2:15 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by mdm »

Paul --

"Compile or die" was a deliberately pointed remark. Still it is also true. I was just answering to Richmond's rather starry-eyed question why the project would need compilation capabilities.

To maybe explain my motivation: I am interested in an xTalk that is usable and attractive for serious work, that draws attention and that is worth learning and teaching (three aspects that play together obviously) and thus an xTalk that grows and stays. LC Ltd has never been able to procure this even after going open-source, and now their door is closed again anyway.

If the abandoned open code is not taken by someone and developed into something more promising, then so be it. We can use it as a plaything like many of the other platforms that you name and also like the Zombie systems that you mention (SuperCard, Apache Flex). If compilation does not happen, then the OXT (copied) binaries will go down the drain platform by platform. OXT can still be used there as a toy and for vintage fun and behind tightly shut security doors but no serious developer will invest in it. They will use all the other stuff that draws a crowd and has support, and rightly so.

To deliver something more productive along my line of thought would need a structured approach and a concerted effort of more than one. Nothing like this came out of propositions and debates last fall. Your de-branding efforts and IDE explorations thus became a rather lonely affair, it seems to me, but why bother. Sure they go in the right direction and will not do any harm, on the contrary. Still, there was a lot of stuff that you added out of your fancy right from the beginning, like toolbar icon and logo discussions (which are always rewarding because everybody can add an opinion). Now these aspects have completely taken over the picture (with WebAssembly capabilities, Midi libraries, Electron on FreeBSD etc), so Richmond+3's observation is correct and made me chime in. As I said, I think you should follow your fancy and not be fitted into something that other people assume you should do.
User avatar
richmond62
Posts: 2621
Joined: Sun Sep 12, 2021 11:03 am
Location: Bulgaria
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by richmond62 »

Richmond's rather starry-eyed question
Richmond lives in an EFL school and makes most of his money teaching small Bulgarians to speak (or squeak)
English . . . and after a day of teaching that lot (specially the hormonally imbued teenagers), his eyes are
so full of stars he sometimes has to stop for a 5 minute rest in his 15 km commute home.

As Richmond does NOT make his money in the happy world of computer software directly it would be
a big, big surprise if he knew much about the realities of the need to compile.

Richmond sees part of his role in the xTalk world to ask the Big, Naive, Goofy Questions
that everyone else is scared to ask in case they end up looking like a 'prawn'. As Richmond
has managed to make himself look like a 'prawn' countless times in the 60 years of his life
it is meat and drink to him. :P
https://richmondmathewson.owlstown.net/
User avatar
richmond62
Posts: 2621
Joined: Sun Sep 12, 2021 11:03 am
Location: Bulgaria
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by richmond62 »

Oh come on! I don't bite!
I know that, and Thou knowest that, but some people having been bitten several times on a Forum "somewhere else"
might assume you were a 'bit nibbly' round here as well. 8-)
https://richmondmathewson.owlstown.net/
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

Still, there was a lot of stuff that you added out of your fancy right from the beginning, like toolbar icon and logo discussions (which are always rewarding because everybody can add an opinion). Now these aspects have completely taken over the picture (with WebAssembly capabilities, Midi libraries, Electron on FreeBSD etc), so Richmond+3's observation is correct and made me chime in. As I said, I think you should follow your fancy and not be fitted into something that other people assume you should do.
Most of those things were NOT because I fancied doing them (I'd rather play my bass guitar):
Toolbar icon / logo = required.
MIDI stuff was largely already done (in fact I have not had any time at all to continue working on that stuff), but in my opinion it's not a proper xTalk without 'play playSentence' and a 'speak' command. Although, I do fancy MIDI, music, and sounds, Richard may confirm this has been an obsession of mine since our first interaction in 2014.

WebAssembly capabilities? The Emscripten engine needed de-branding, the playground was for testing since I've mostly never played with the JS engine. I'd also point out that particular Engine isn't so much 'compiled', at least not pre-compiled, but I believe that is the direction they're headed in 'over there'.
The web engine is very important. If the web engine can be turned into the Web IDE, well then you have a fairly permanent FOSS cross-platform/cross-CPU solution to build xTalk stacks and scripts, one that you can use anywhere there's a modern browser, right? You can even deploy your web apps on GitHub like I did with the Playground stack.
Now if that .js based xTalk interpreter / IDE is wrapped into a 'native' app, with something like Electron (and Electron might be overkill for that task), the you have highly cross platform 'regular' app as far as the user is concerned, no compiling C++ necessary (although we could compile more module bits of it to WASM for speed). I do think that WASM may be the future of at least light duty apps in general.

I haven't done anything with FreeBSD, accept for like a few OSes built from it.

I don't understand why you are so stuck on that SSL / data transfer security thing. Currently that is handled with an Externals (not a Builder Extension), not integrated into the engines. I'm certain it can be, and indeed has been, replaced with something else (tsNet External for example). In fact here's an External that could be updated to use the latest libSSL: https://github.com/trevordevore/SSH-Ext ... r-LiveCode
Alternatively we also could use web-APIs (I know the included Chromium build is old, maybe we go 'native' / webkit browser view too), or use something like curl via shell()/open process, to do secure-socket transfers, but if you really want all of that made easy and have someone tell you (albeit with limited liability) that they guarantee it is secure, then I would most emphatically suggest that you sign-up for the LiveCode Ltd plan.

The main reasons for needing to compile from source for me are:
remove some last bits of branding (the integrated installer stack window on macOS), to update or preferably REMOVE any libraries (such as OpenSSL), to support new CPUs or platforms, and to make changes to this xTalk language for things that can't be overridden in script.

I agree that it would take more than just me to really build this thing into some big FOSS community with planned out build cycles and all of that, but until that happens I'll chipping away at it on my own.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

OXT (copied) binaries
Actually they aren't straight copies except for Linux64, the rest have been modified with metadata, info.plist key/values added/changed, AppleScript dictionary, new icons, code-signing, and for Emscripten to remove branding from inside that engines 'binary' (it's a big JavaScript.js file).
User avatar
richmond62
Posts: 2621
Joined: Sun Sep 12, 2021 11:03 am
Location: Bulgaria
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by richmond62 »

The Linux version of LC is seriously . . .
-
SShot 2022-08-11 at 12.35.04.png
SShot 2022-08-11 at 12.35.04.png (31.61 KiB) Viewed 5244 times
-
https://downloads.livecode.com/livecode ... 9_rc_1.pdf

https://downloads.livecode.com/livecode ... 0_dp_4.pdf
-
It has been like that, static since 2016, which, in view of the cross-platform claim is shameful.

Fedora is at version 36

Ubuntu and its derivatives at 22.04
https://richmondmathewson.owlstown.net/
mdm
Posts: 22
Joined: Thu Sep 16, 2021 2:15 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by mdm »

OpenXTalkPaul wrote: Wed Aug 10, 2022 11:09 pm Most of those things were NOT because I fancied doing them (I'd rather play my bass guitar)
Look Paul, this thread took its current direction when, on Aug 2, Joshua asked you "please do a buildset of the IDE we can use for Windows, Linux, and macOS" and Richmond stated that all he would need at present is "a feature identical IDE that has all references to LiveCode removed" and micmac said "Ditto". This was so heartbreaking given the direction your efforts had taken in the past months that I was lured to jump in to confirm that their point is valid.

You may not remember, but among my first suggestions to you in September 2021 was just this, a first edition of OXT that is 100% feature-identical and nothing else. You replied to me in a similar manner then as you reply to the three now by saying YES, BUT "the goal is specifically an IDE that is significantly differentiated from its progenitor". This is your door out to the 1000 wild flowers blooming that you love to visit so much. Your latest reply again explains why the flowers on the list are all interesting and a natural and cannot be done without, and how can we be so stuck on the nitty-gritty.

Your mantra is "de-branding AND differentiating" (and experimenting) all in one. All proposals to cleanly separate these two steps (and others as well) seem to lead nowhere. What should we do. So be it.
User avatar
overclockedmind
Posts: 268
Joined: Sat Apr 30, 2022 9:05 pm
Location: Midwest US
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by overclockedmind »

I had mentioned to another member of the community that, in order to try and understand everything that is there within that mountain of JavaScript, et cetera, that perhaps Paul was using it "out-of-context," to try and become familiar with, well, the mountain. I'd probably do that, too.

I couldn't have known what had been mentioned to someone else nearly a year ago in regard to "differentiation."

What I don't understand is having something (the binaries,) and not making it/that available to everyone.

Paul is indeed allowed to do whatever Paul would like to with his free time. I see this as different from that, uploading something that takes no further work, so that other people can use it... is what I see.

I am indeed one of the "+" people.
MBA (Early 2015, 8GB/512G SSD, Monterey 12.7.2)
System76 serv12 (64GB RAM, 2TB SSD, 2TB HD, Win 10 Pro x64)
Power Mac 3,1 Project - Needs TLC will get it soon
User avatar
tperry2x
Posts: 1341
Joined: Tue Dec 21, 2021 9:10 pm
Location: Britain (Previously known as Great Britain)
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by tperry2x »

Yes, I must admit, having the binaries downloadable from the github page (as I mentioned here)
viewtopic.php?f=2&p=2210#p2210
Even if they are labelled as a 'beta' or 'test version', or whatever - it's better to have some binary out there for people to try than to have nothing for the general public to tinker with.

As far as a security aspect, having both source and compiled binary would be highly useful.
If just for watching what ports are opened by OpenXT and any libraries in something like wireshark. Security hardening of OpenXT and included libraries are possible, but having things to test with is a good first step.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

tperry2x wrote: Sun Aug 14, 2022 1:23 pm Yes, I must admit, having the binaries downloadable from the github page (as I mentioned here)
viewtopic.php?f=2&p=2210#p2210
Even if they are labelled as a 'beta' or 'test version', or whatever - it's better to have some binary out there for people to try than to have nothing for the general public to tinker with.

As far as a security aspect, having both source and compiled binary would be highly useful.
If just for watching what ports are opened by OpenXT and any libraries in something like wireshark. Security hardening of OpenXT and included libraries are possible, but having things to test with is a good first step.
The OXT binaries have been available on GitHub, since fairly early on:

macOS binary here:
https://github.com/OpenXTalk-org/OpenXt ... ents/MacOS
Win/Linux binaries here:
https://github.com/OpenXTalk-org/OpenXt ... ents/Tools

In that repo contains the actual binaries I'm using, they're just not packaged up nicely, like in an Linux AppImage or in macOS .dmg. disc image. The rest of the OXT IDE files are in there too, although they're works in progress, and also may not reflect the very latest things I'm working on.

For the past week, I've (slowly) been trying to collect all of my latest versions of all of those IDE scripts and stacks files.
They're a bit scattered, across three platforms and multiple workstations. On top of that, I also need to merge some folders / files, if I make platform specific change on say Linux, and also made different platform specific changes to the same file on Windows or macOS. I also have a few multiple variations on the same file (experiments with along the way, such as styled IDE stacks for a cross-platform 'darkmode', or adding a macOS Dock-Menu for the IDE which was too clunky to do built-in xTalk handlers for that and should maybe be a Builder Extension instead).

I'm trying to get those nice release packages ready to post on the internet for a OXT 1-year anniversary release.

Besides incorporating the latest changes, I would like for this initial release to have some sort of package system in place, even if it's just a quick preview version.

One of the things that I really wanted to fix with the IDE was the Extension Builder and Extension Manager stacks.
I did fix a some annoying bugginess (such as icons not displaying correctly in the Builder Window, Example stacks not opening from the Extension Manager's context menu, etc.). The remaining problem that has really bothered me since soon after it's addition to the IDE many years ago now is the 'Extension Store'. It only ever showed the same four widgets, which are basically all of the LCC demo builder extensions (such as the Clock widget or the Pink Square'), it was never updated to show the actual LC 'Extension Store' you would normally see visiting that web site with a normal web browser. It turns out that their server serves up a different, long neglected, 'store' page based on it's Browser Widget's web "User Agent" sending "LiveCode" in the 'GET' header to their web server (as opposed to "Chrome ..." or "Safari ..." which then sends back demo page based on passing the IDE's agent info test.

So I'm working to replace all of that with nice, open extensible package management system, that pulls its data from a GitHub repo. I wanted to test how this would work out, but ironically libURL can't seem to retrieve URLs of files from GitHub repos the way it does in a regular web browser. Now I ran a very simple, small company website decades ago, so I have a pretty good understanding of HTTP / FTP work on socket communications level (and even started to invent my own protocol at one point), so I know that I could likely get my scripts to emulate a popular browser, but it turns out we don't need all that noise just to pull in files while getting around things like web cross-origin restrictions or GitHub issues.

Why don't we need to use the libURL in the IDE? Because the libURL in the Emscripten engine, running inside a Browser widget, can retrieve data from GitHub repo exactly like a normal browser, without any of the problems I've encountered trying to do that with the IDEs macOS implementation of libURL (one issue is conflicting data in the cache, which I don't see a command to empty the browser widget cache available to our xTalk script currently). Remember, libURL is a high-level API wrapper scripting front-end for the underlying sockets communications, handled by an External (built on OpenSSL) on Desktop platforms but is implemented differently on other platforms. On mobile devices libURL is implemented natively. I have yet to look at how libURL is implemented in the Emscripten Engine, but it works fine for this particular purpose ( example "put URL "https://openxtalk-org.github.io/OpenXTa ... index.html" <-- currently the actual URL I'm testing with). So we intercept this data's URL before the BrowserWidget navigates to it, and instead we use libeURL in the Emscripten Engine running in the Browser Widget, to pull in that data from our OXT repo's web page, base64Encode it or whatever, and then pass that to our IDE, which avoids any issues with using a GitHub Repo. Once we have a local, decoded copy in memory or on local disk, we install it into our IDE customization folder (ie "~/My OpenXTalk/) using the existing Extension utility commands as normal. This would be very similar to the way the Dictionary stack already operates within the IDE, with some of it's functionality coming from (some interesting) javaScript libraries that get loaded into its browser widget.

And that's it! That's how our IDE can have continuously updated packages repo, web apps built with xTalk, running in the IDE. We don't even need to download the (28mb) .js engine, because it's already in the IDE (it's in the Standalones 'Runtime'(s) folder). So we just need to build our web-app and have the Extension Manager load that into it's Browser Widget instead of that old LC page. Later on I want to do the same for "Sample Stacks" and maybe we could do in-IDE "xTalk News" too. Content is KING!

BTW, as a result of experimenting with this idea, I also discovered that you can use Data URLs in the Browser Widget's URL property! I wonder if it can handle raw (bytes) 'Blob" URLs? ICYDK, a data URL is a URL in a format similar to:
"data:mime type/kind/Base64,#A190SFGSETCBASE64DATAGOESHERE.." (not actual base64 data).
You can take that, delete the first item of it and Base64Decode( it ) into memory or write it to file on disk.
But if you navigate to such a URL with the BrowserWidget, if it's a MIME type the web engine can render, it will automagically decode the base64 and then render the resulting data.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

Dare I say we could have 1000 Widgets by Christmas? Lol

In case anyone else is interested in in-web-page JavaScript-based file transfers, here's a pretty good breakdown of the common methods for doing that:
https://itnext.io/how-to-download-files ... a69b749896

The last method on that page is what I'll be using, since that async method can return progress of the download(s). For most widgets that shouldn't be needed as they're usually no more than about 10Kb of data, but for a library that includes multiple shared-library binaries for all the different platforms, for example libFluidSynth which is 10s of Mbs, it would be needed to let the user know that the browser widget is busy downloading the file and not just locked up/failed.
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

Code: Select all

put URL "https://openxtalk-org.github.io/OpenXTalk-Playground/Widgets/org.openxtalk.paulmcclernan.xygrid.0.1.lce.b64"
Only seemed to work intermittently with libURL giving me back response headers indicating redirects / cached files.
I cleared the macOS generated cache files for org.otpenxtalk.* and it started to work, sans any other, more convoluted method to pull in this base64 encoded text of a widget extension (an .lce builder package file are actually zip compressed directory(s) ). LibURL SHOULD have a way to reset any cached files so it retrieves fresh copy from the server (GitHub Pages server in this case), but doesn't appear to have that capability currently (could probably fix that fairly easily).

Here's a very old somewhat related discussion (includes some libURL mod from Trevor DeVore):
https://forums.livecode.com/viewtopic.php?t=2066
User avatar
OpenXTalkPaul
Posts: 1486
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by OpenXTalkPaul »

OpenXTalkPaul wrote: Wed Aug 24, 2022 5:35 pm

Code: Select all

put URL "https://openxtalk-org.github.io/OpenXTalk-Playground/Widgets/org.openxtalk.paulmcclernan.xygrid.0.1.lce.b64"
Only seemed to work intermittently with libURL giving me back response headers indicating redirects / cached files.
I cleared the macOS generated cache files for org.otpenxtalk.* and it started to work, sans any other, more convoluted method to pull in this base64 encoded text of a widget extension (an .lce builder package file are actually zip compressed directory(s) ). LibURL SHOULD have a way to reset any cached files so it retrieves fresh copy from the server (GitHub Pages server in this case), but doesn't appear to have that capability currently (could probably fix that fairly easily).

Here's a very old somewhat related discussion (includes some libURL mod from Trevor DeVore):
https://forums.livecode.com/viewtopic.php?t=2066
Disregard most of that (actually it's a quick read of interest anyway)...
I was having the same problems again but on a different IDE install / different workstation, but if I'm correct, and it turns out that macOS desktop libURL and it's underlying webkit cached files were most likely trying to shared with LCC'S domains files. It also turns out that I had edited libURL to replaced "LiveCode" and the "MetaCard" (wow that's going back a ways) fall back user agent with "OpenXTalk" (in both spots), so I made that change again, restarted the IDE and now I can:

Code: Select all

put URL "put URL "https://openxtalk-org.github.io/OpenXTalk-Playground/Widgets/org.openxtalk.paulmcclernan.xygrid.0.1.lce.b64"
and my base64 text gets dumped into the message box as expected.
I think some of these problems may be related to code-signing/privileges/provisioning/entitlements and due to my work towards fully separating OXT from LCC (as far as macOS is concerned).
The thing is the Emscripten Engine, running inside a BrowserWidget, has been able to retrieve that data without (the same) problems, so that may be the more reliable implementation of libURL's back end.

I did find that there is an IDE script for revlibURLHTML5, so we can look through that so see how that lib is bound to libURL and then possibly use that as a template to hook in a different backend (such as libCURL, which has been ported to web-assembly, so it could theoretically work virtually anywhere)
User avatar
tperry2x
Posts: 1341
Joined: Tue Dec 21, 2021 9:10 pm
Location: Britain (Previously known as Great Britain)
Contact:

Re: July 29th 2022 (one year since last official community release)

Post by tperry2x »

OpenXTalkPaul wrote: Fri Aug 19, 2022 8:35 pm The OXT binaries have been available on GitHub, since fairly early on:

macOS binary here:
https://github.com/OpenXTalk-org/OpenXt ... ents/MacOS
Win/Linux binaries here:
https://github.com/OpenXTalk-org/OpenXt ... ents/Tools
Sorry, to clarify: what I was looking for was a DMG with a .app inside that can be double-clicked on the mac, and just run.
Likewise, for Windows - an exe that is a standalone portable program.
For Linux, the same - a standalone portable appimage.

I can find none of these on that github. Github is (and will always be) an absolute mystery to me. Let's just have an FTP site or a directory where these can be downloaded from?
Post Reply

Who is online

Users browsing this forum: Ahrefs [Bot] and 3 guests