OXT Lite Moving Forward

[tperry2x]

Haha, yes absolutely. That yellow text on black background was just in the script editor, so not actually a part of my 'all guides' stack:

Screenshot at 2023-12-10 17-31-04.png (5.46 KiB) Viewed 1647 times

But I completely know what you mean. Light mode has it's place (handy when printing / print preview), but I seldom print anything myself. Perhaps I'm the aberration in as much as my typical use does not involve printing ever.
I tend to spend 90% of my time in apps that are set to dark mode, and the other 10% is usually because the app that I'm tasked with using is probably too old and doesn't support it.

Didn't want to force any colour scheme choices on anyone though, as that comes down to personal preference.

Paul does raise a good point: One advantage of using the api method (perhaps the only advantage I can think of though at the moment), is that if a plugin-addon is loaded, the api would be aware of that plugin [*if] the plug also contains the right elements to add to the all guides api.

This is a very valid point, but I'd come back with two counter points to this (again, where's that disclaimer from earlier).

1. I've not seen the 'all guides' ever get updated by anything 'free' that I've tried adding. (Don't know if that's different if I were to add a commercial plugin, but I won't be doing that any time during this plain of existence).

2. Should we even worry about providing support for a 3rd-party plugin anyway? After all, you'd hope that anything developed would either come with a sample stack of how to use the thing, or at least commented throughout.

Another point, not raised here, is what if something changes? (If I want to replace the word XTalk with XionTalk throughout the entire guide) - This could be easily done within the stack as a function, to

(sudo code)

Code: Select all

lock screen
repeat the number of cards, 
replace x with y,
 send the tUpdateTheContent to the editor field.
unlock screen

The other reason I'm doing this: because for Windows and Mac, the API :should: load in a window within the IDE (does not always do so), but on linux - it passes this off to a browser. Whoever did the dictionary and API originally was well aware of the browser deficiencies on Linux, so purposefully redirects the api and guide to load there if Linux is detected.

Screenshot at 2023-12-10 17-48-17.png (9.52 KiB) Viewed 1647 times

(detects if Linux is being used and returns 0 to use the CEF engine for the documentation)

Rather than fix the browser widget. Sorry. Rant over before this turns into a LC-bashing post.

[richmond62]

Rant over before this turns into a LC-bashing post.

Yesterday I peeled a bag of potatoes that had been lying around: I chucked out none of them, but had to excise quite a few rotten bits.

LiveCode is a great starting point: but it is, as far as I can tell, the product of a team that became complacent about certain parts of it, or were so hypnotised by a 'bright future' that they overlooked a lot of stuff.

This reminds me of a chap I met on a train in England who had been at school with me in the 1970s, and was, apparently, a brilliant chap with a bright future ahead of him (as a lot of teachers who obviously were a bit shallow kept on banging on about); anywhere, there he was in an old moth-eaten coat looking as if he had slept in a doss-house. So, bought the fellow a cup of coffee and one of those things that pass for grub on the trains, and bent my ear.

He had, indeed, slept the previous night in some homeless shelter in London, but had been sent a train ticket by his sister to go and spend Christmas with her near Exeter. LUckily he raised the topic I wanted to bring up: his bright future.

He had become so full of himself at school he'd left with 3 O levels and nothing else, and then lived in a squat in London, and there, 35 years later, he was in a tatty coat done up with bailer twine. And as he said: a bright future is only possible with a hard-working past: and, while he looked like the average gay guy's dream at 15 [bronzed Adonis, and so on], and could do all sorts of tricks like the high-jump, he had sat at the back of the classroom picking his beautiful nose and making "clever" teenage remarks.

LiveCode seems to be getting itself in a hole of its own making without either Thy or My help.

It has been anticipating its bright future without doing any consolidation of its foundations for far, far too long.

[richmond62]

Hereabouts (Mac OS 12) OXT Lite 0.95 is loading the documentation into my web browser . . . something I'd far rather did not happen.

[tperry2x]

Hereabouts (Mac OS 12) OXT Lite 0.95 is loading the documentation into my web browser . . . something I'd far rather did not happen.

Hmmm... I'd not inflicted any api changes on anyone in 0.95, so I'd have thought perhaps this is a preference.
I know it sounds silly, but can you check Preferences > Dictionary > "open dictionary in external browser". If that's on, then it explains why.

If it's not on, then there's a bug that I'd somehow created (probably set the checkbox to true when I was adjusting the revPreferencesGUI stack). I can see how that would happen.

But I built in an override just in case, to ignore my preference setting. If you hold shift while clicking on the 'dictionary' toolbar icon, does it then load within the IDE?

Either way, I should mention that in 0.96, this has been replaced and the default is to use the in-IDE method for MacOS and Windows. Linux does still farm out to a browser (like what you get on your mac currently). That's life for Linux users.

However, turning on the newly added preference "Use 'All Guides' experimental stack" will keep it all within the IDE instead of opening a browser on ANY platform, because it's stack-based, it will be loaded without issue. At least that's the current plan.

Regarding the 'bright future', I think it's all too easy to "rest on your laurels" / sit on your hands. But as things change constantly, this approach only gets people & software so far. Things need to constantly be improved upon and skills kept updated. I look back at code I put together 5 years ago and think "I can see how it works, but why did I do it like that?"
That's part of the problem with us inheriting something. We might have taken a different approach (is calling everything in from external stacks really the most efficient method?) No is the answer to that, and a better alternative would be to call most things in from a single reference stack - or at least a single reference stack for say the datagrid, rather than:

Screenshot at 2023-12-10 18-26-34.png (10.91 KiB) Viewed 1633 times

It's quite frankly a bit of a mess, and is quite memory inefficient. I'm sure they had their reasons for doing so (perhaps it made it easier for development at their end), but the deeper you dig in the IDE - the more evidence of bailer-twine I'm finding.

[tperry2x]

Under the category of "more things I've just discovered are broken, and I don't know how long it's been that way for..." (or something similar)

Been working my way through the 'All Guides' stack, and found the first non-existant link:

guides-nope.png (78.09 KiB) Viewed 1623 times

That 'datagrid_sampler.zip' no longer exists

nope.png (13.56 KiB) Viewed 1623 times

I don't suppose anyone has a copy of this stashed away somewhere do they?

[mdm]

That 'datagrid_sampler.zip' no longer exists
I don't suppose anyone has a copy of this stashed away somewhere do they?

Google is our friend (as always).
Tyrone had the same question in 2018
and Heather suggested a solution:
https://lessons.livecode.com/m/datagrid ... -with-data
But well, is it the same, is it under GPL, and need+can it be de-LC-fied?

[tperry2x]

Thank you mdm
I think I found a copy here instead:
https://lessons.livecode.com/m/datagrid ... -of-people

It looks just like what the documentation says it should be.
I'm now submitting the entire lessons site to the waybackmachine, just in case.

The next question would be, what to do when their online site never had the link in the first place?

Screenshot_2023-12-11_22-05-18.png (79.87 KiB) Viewed 1606 times

I guess I know the answer to that - create it myself by following the guide perhaps

[OpenXTalkPaul]

how does anyone realistically edit this api version of the 'all guides'?
Yes - you can mess about with it in a something like libreoffice, or (database editor app of your own preference), but it's not as easy as editing a stack. With my version, it also has a built in editor:

Looks nice... no offense and all of that and I'm not trying to dissuade you here, but...

That's probably mainly because you are not editing the docs the way this documentation system was designed to be edited, which is NOT by editing the sqlite db, but rather edit the files that that sqlite file gets generated from, and then use the 'build docs' button in the repo "Builder.rev' stack (which was in the Engine repo, NOT the IDE Repo). This scans though the docs directories and subdirectories looking for .lcdoc and .md files and packs them into sqlite for distribution (this is the file you incorrectly thought needed to be edited). As you know, this is what the display files then get generated from in the end. But other things happen via the scripts of that 'build docs' button as well, such as that bug fix notes and new feature notes (which are usually tiny markdown files, just a few lines), are merged together to form a larger release notes .md which then gets packed in as the 'release guide'.

If you edited the original 'source' files (.md/.lcdoc), then you would see that they are formatted in an extremely simple 'markdown' (.md) format, commonly found in GitHub repos (and there's built-in supported in GitHub's web UI). This is a format which you can edit in any text editor. If you use a coding text editor like Pulsar (the FOSS successor to the once Google led Atom editor, which has a LCS, LCB, .lc server support packages available in its package repo), Pulsar, VSCode, and the like, it can have many advantages over editing them in the OXT IDE. For one thing there's a package that can live preview .md files while you're editing them. Most of these sorts of text editors also have instantaneous spell checking available (which is something we already have the resources to add xTalk-aware spell-checking to the IDE on all 3 desktop platforms).

However, I can see this being useful If we did something like, oh I don't know...build an IDE version that runs on the Emscipten engine in a browser, then this guide stack (and the stack format of 'Dictionary') can be included in a more self-contained, sandboxed friendly way. It could also be set up to load in same way that I'm loading the SVG Glyphs Palette, which is that if you hold the optionKey down when opening the stack, it opens topLevel for editing ( a nice trick for while you're still developing a stack that will ultimately be palette, modal dialogue, etc.).

What might be helpful for community while sticking with our card/stack format might be to have each topic be a separate stack that an main stack collects into an index and can "go stack in window'. That's not an original idea of mine, if you look at the older-old format for the Dictionary, it consisted of a whole lot of numbered stack files that are used by a main stack. The next iteration of the docs system was builtin a similar way, except instead of a bunch of separate stacks, it would build itself from a whole lot of separate XML files.

The problem I have with manually converting docs and guides (even with a nice editor built in) into cards in a stack, is that when I'm developing dictionary entries or a guide for a new library or widget that I'm working on, then I'm going to have no way to look at my docs until I manually convert them (a step which I would probably tend to skip often). If wanted my docs/guides edits included in OXT Lite docs/guides stacks, then I'd have to manually edit those stacks when I'm all finished (I'm never finished), and then send them in to you in order to share them with others. With this way docs/guides can not be included with an extension package and delivered via a package manager (those .lce files are really just .zip files that can include docs, images, and other resources). Since I'm very keen extending OXT and also having a package manager delivery system, I'm probably not going to use that in my OXT DPE builds. Of course if your stack had an "export to .md format..." button then I could still easily add any edits / corrections that you make in the stack / card version to the SQLite/HTML/JS version.

[OpenXTalkPaul]

I'm pretty sure, there is already an extension or something, by Monte IIRC, that converts markdown<->HTMLText. It would probably not be that hard to roll your own function, but why reinvent the wheel?

I'm starting to like the idea of having a stack that's an editor for .md (+ linked images). It would also be useful as an offline editor for anyone that uses GitHub or similar for any sort of project at all. And although I know a lot of you xTalk peoples don't seem to like using those repo sites, such an .md editor app distributed as FOSS could have a wider appeal and bring attention to xTalk along with it.

Also, Brian Millby created an .lcdocs editor stack, that he specifically granted permission for us to use. So that can be helpful for editing the Dictionary entries (.lcdocs are basically just markdown with special formatting for related terms tags, params, keywords, library associations, and such).

[OpenXTalkPaul]

This is a very valid point, but I'd come back with two counter points to this (again, where's that disclaimer from earlier).

1. I've not seen the 'all guides' ever get updated by anything 'free' that I've tried adding. (Don't know if that's different if I were to add a commercial plugin, but I won't be doing that any time during this plain of existence).

2. Should we even worry about providing support for a 3rd-party plugin anyway? After all, you'd hope that anything developed would either come with a sample stack of how to use the thing, or at least commented throughout.

You can do whatever you want off course, I certainly won't be offended I didn't write this system, I just felt I had to learn all about it to manage the dictionary builds...

Some of the externals (such as revDB, and Monte's stuff), extensions, and also scripts that are included with, and even used by, the IDE itself, such as the Clock Widget or the ICONSVG library, come with a guide .md file, and alll of them have inline syntax docs, and even the revIDE library itself (which is a script lib NOT even LCB stuff guys) has inline documentation mixed in with its scripts (which I had started to add to because they're incomplete), that all get packed into that SQLite db during the Dictionary / Docs building process. I use this system in my extensions (as well has some of Trevor and HH's Extensions do too), I at least have inline Syntax docs, that I tend to update, and I've started to also include guides with mine, which is in addition to the sample (demo) stack(s). This is actually a system could really be used by any xTalk scripter (DataGrid API has docs built in and that's not an 'Extension'), perhaps it could've been set up better for people to advantage of for more general use. But my point is that it is not a question of supporting 3rd parties (there's no significant amount of 3rd parties to even consider in this space). You mentioned commenting code, well these inline-syntax docs are actually exactly that, they're special multi-line 'comments' above each handler that they are related to.
The revDocsParser script lib is what parses them and it is used by Extension Builder and by the Dictionary/Guides build process (and by and the repo stack that builds the .sqlite db), that is the library I think you should take into consideration before doing anything significantly different what that.

[tperry2x]

If you want to skip to the short version, just scroll down and ignore most of my ramblings.

If you want the long, boring version (as here's a lot to go through here), so here I go:

Looks nice... no offense and all of that and I'm not trying to dissuade you here, but...

First of all, thank you - no offence taken at anything you've written.

That's probably mainly because you are not editing the docs the way this documentation system was designed to be edited, which is NOT by editing the sqlite db...

Just to be clear, I'm not editing the sqlite db at all. I just meant, how is someone supposed to go about editing these without using a third-party tool? We have a perfectly good editing model within OXT, so why not use that as we have this available which gives the same results across all platforms?

...but rather edit the files that that sqlite file gets generated from, and then use the 'build docs' button in the repo "Builder.rev' stack (which was in the Engine repo, NOT the IDE Repo). This scans though the docs directories and subdirectories looking for .lcdoc and .md files and packs them into sqlite for distribution (this is the file you incorrectly thought needed to be edited). As you know, this is what the display files then get generated from in the end. But other things happen via the scripts of that 'build docs' button as well, such as that bug fix notes and new feature notes (which are usually tiny markdown files, just a few lines), are merged together to form a larger release notes .md which then gets packed in as the 'release guide'.
If you edited the original 'source' files (.md/.lcdoc), then you would see that they are formatted in an extremely simple 'markdown' (.md) format, commonly found in GitHub repos (and there's built-in supported in GitHub's web UI). This is a format which you can edit in any text editor. If you use a coding text editor like Pulsar (the FOSS successor to the once Google led Atom editor, which has a LCS, LCB, .lc server support packages available in its package repo), Pulsar, VSCode, and the like, it can have many advantages over editing them in the OXT IDE. For one thing there's a package that can live preview .md files while you're editing them. Most of these sorts of text editors also have instantaneous spell checking available (which is something we already have the resources to add xTalk-aware spell-checking to the IDE on all 3 desktop platforms).

This is all good, and handy, but it does not solve the issue of things being incorrect and broken in the original documentation. This is another reason I'm going through it. Nice to have a spell check, but that does not detect broken links, and neither does it go trawling through the waybackmachine to find them. Not being funny when I say that, but this is where human input is required.

However, I can see this being useful If we did something like, oh I don't know...build an IDE version that runs on the Emscipten engine in a browser, then this guide stack (and the stack format of 'Dictionary') can be included in a more self-contained, sandboxed friendly way. It could also be set up to load in same way that I'm loading the SVG Glyphs Palette, which is that if you hold the optionKey down when opening the stack, it opens topLevel for editing ( a nice trick for while you're still developing a stack that will ultimately be palette, modal dialogue, etc.).

Again, these are all interesting experiments and something that can be considered in the future, or in your RC builds. With OXT Lite, I'm mainly concentrating on getting something stable that has the incompatibilities and bug fixes sorted wherever I spot them.

What might be helpful for community while sticking with our card/stack format might be to have each topic be a separate stack that an main stack collects into an index and can "go stack in window'. That's not an original idea of mine, if you look at the older-old format for the Dictionary, it consisted of a whole lot of numbered stack files that are used by a main stack. The next iteration of the docs system was builtin a similar way, except instead of a bunch of separate stacks, it would build itself from a whole lot of separate XML files.

I did think about this approach, but it's again getting away from a single file to edit. Plus loading in 10's (or hundreds / even possibly thousands) of assets as you scroll through the guides will be very memory inefficient. It will also make for an additional headache when editing, plus the more loose files floating around, the more chance for error and something not to be included where it should be *in my opinion*.

The problem I have with manually converting docs and guides (even with a nice editor built in) into cards in a stack, is that when I'm developing dictionary entries or a guide for a new library or widget that I'm working on, then I'm going to have no way to look at my docs until I manually convert them (a step which I would probably tend to skip often). If wanted my docs/guides edits included in OXT Lite docs/guides stacks, then I'd have to manually edit those stacks when I'm all finished (I'm never finished), and then send them in to you in order to share them with others.

Not really, I mean that's the point of my thinking when including the editor. Anyone can edit them and post an updated copy. It doesn't just rest on my shoulders - anyone could update it. If I got run over by a bus tomorrow, then it does not mean that the project is dead, anyone can continue to contribute.
As a side note, it would also be really easy to have the 'all guides' stack scan an 'extras' folder say, for anything that you have added. If someone is adding a plugin, it could just open a sub-stack in there and amalgamate it into the 'all guides' stack automatically.

I guess, as you say, it would be perfectly possible to have a button within the 'All Guides' stack I'm putting together, that allows you to export back to these formats (so it becomes interchangeable between my standard stack & card format, to whatever files you need to generate). I'd need you to code that button though as I have no idea how to begin writing this format back as to how you'd like it. Again, not being funny when I say that - this is also why I went for a stack & card method, because it's there already and works (and I understand it). And if I can understand it, I'm sure everyone else can too
I don't expect the majority of users to know how to use livecode builder, I count myself within that group too. Perhaps I'll learn as I go through these guides. (I've already learnt about data grids which I had never used previously!)

With this way docs/guides can not be included with an extension package and delivered via a package manager (those .lce files are really just .zip files that can include docs, images, and other resources). Since I'm very keen extending OXT and also having a package manager delivery system, I'm probably not going to use that in my OXT DPE builds. Of course if your stack had an "export to .md format..." button then I could still easily add any edits / corrections that you make in the stack / card version to the SQLite/HTML/JS version.

If you make any edits to the 'all guides' section, please feel free to let me know and I'll include them in my stack version. None of this prevents you from using your api method at all, and of course, feel free to keep what you want to do in the RC version as you see fit. As with anything we've inherited here, the underpinnings are O.K., but there are lots of typos, broken links, and examples that are incorrectly formatted within the guides. This is why I can't simply convert the existing guides verbatim, as I'll just bring these errors across.

Short version:
I do believe we should be using the stack / card format - you can't beat this for editability and for speed of loading. Plus (the biggest one for me here), it works the same across all 3 platforms, and does not rely at all on any out of date / vulnerable web/network libraries to do so.

Github is, and likely to always be, a mystery to me. I'd much rather edit in the card and stack format - and as that's the format we are ultimately championing with OXT, rather than some sqlite, md text editing malarkey, or any other method. I think that's what we should be using. Why not use the tools we have at hand and are encouraging other people to use? - Again, that's just my opinion.

Yes, it's all very clever being able to use all these alternate methods, but to a new user (which is who the 'all guides' stack is primarily aimed at I guess), if they have to go off to github - download md files, open them in a text editor, resubmit back to github (?) / put in a special place within the documentation folder "/[path to application]/Documentation/html_viewer/resources/data/api/"... all seems a bit overly complex when they could just shift-click a button in a stack.

Also worth mentioning, this is just a preference. If someone turns on the 'use experimental all guides stack' in the preferences, they'll get my version. The default will still be to use the 'all guides' api method, however broken or inaccurate that version may be.

Again, that's not a dig at anyone as a lot of work has gone into it, but it plainly has not been kept current or updated (or has not been entirely proof-read before being published).... okay, perhaps that does sound like a dig

[OpenXTalkPaul]

OXT Lite Moving Forward

I got the message in that Dropbox text file.
Wow, great work! I hope you know that I do very much appreciate all of the work you've done with OXT Lite.
And it's great that you were able to recompile the Linux binaries with updated dependencies and block the reg-screen from poping up, this may get the .appImage running on helloSystem (freeBSD w/ a 'buntu layer). If you needed to update any other files (like 'Make' files) to get it to compile with updated dependencies, then please try to push those to GitHub repo (or just upload them somewhere we can grab them). Thanks.

If you're really out then good luck to you, live long and prosper, and thanks again.

[tperry2x]

It was a pain. There were lots of fixes to the gyp build tool. It took me ages because I'm still learning C++, and most of that build seems to be in a mixture of C++ and bash.

(I still can't upload back to github though )
I'll put a file dump of the build files somewhere.

I've been taking a back seat from development on OXT-Lite, as I've got to v1.0 and I hoped that would be a better release than 0.99.
It turns out that removing / pruning a lot of the resources has broken a lot of things and created some fairly odd behavior. So, I think it's probably better if I go back to 0.99 and put just those small patches back in to make a revised 1.0

In other news, I got the browser working in Linux by editing the libcef.so file. I spotted that there were two typos which would stop it loading completely, and after correcting these:

fixed-browser-in-linux.png (336.13 KiB) Viewed 1095 times

[OpenXTalkPaul]

In other news, I got the browser working in Linux by editing the libcef.so file. I spotted that there were two typos which would stop it loading completely, and after correcting these:
fixed-browser-in-linux.png

Oh yeah! That's awesome! Great work! It's invaluable to be able the use server and client side web content & JS libraries.

Did you use a newer build of Chromium Embedded Framework?
I believe there's a Spotifiy site where you can get built binaries (nightly builds too), I'll see if I can find the URL..
FOUND IT: https://cef-builds.spotifycdn.com/index.html

Linux 32bit CEF binaries discontinued (there are still builds from 2022 that are newer that what was in 9.6.3 ) :
https://cef-builds.spotifycdn.com/index.html#linux32
I didn't want to bother with 32bit personally but I understand the desire to continue using on older computers)

I wonder if we couldn't have all of the dependencies like that be dynamic linked libraries, instead of being merged into the binary by the linker, so that we could drop in updated binaries without recompiling engine.

Anyway, thanks again!
it's genuinely inspiring! Now I really want take some time to rebuild that AppImage!

[richmond62]

It was a pain.

Yes, it was, and I:

1. Sympathise, and
2. realise that something went wonky with version 10. (as I pointed out).
3. Cannot say how much I admire all the work you have done.

Being, possibly, rather stupid, I did NOT download 0.99 Linux before it was replaced by 1.0 Linux, and as today is my Linux-test day it would be super to have access to both.

I think it's probably better if I go back to 0.99 and put just those small patches back in to make a revised 1.0

A brave and sensible decision.

Oh, and by-ther-way: I've got lots more fish waiting for you.

[tperry2x]

Oh, and by-ther-way: I've got lots more fish waiting for you.

Can you start with the minnows first, moving up to the salmon-sized issues last!

If you can post a list in the 'bugs found' topic:
https://www.openxtalk.org/forum/viewforum.php?f=9

My hope is everyone can work on these - all contributions gratefully received and considered for review. I think this is the best way forward for future changes. We can all then have a look before committing it to the OXT lite / OXT (full fat with extra cream) builds.

@Paul - when I say that, I mean that your version has a lot more features (extra cream) than lite does, and that's a good thing - I do not mean that in a negative way in case it's misconstrued at all.

[tperry2x]

I'm in the middle of compiling the MacOS engine.
In the "LCCE_PrebuiltBinaries", I'm missing the file:

Thirdparty-e5e050573c226f60acfbb9107c2b4aea853b0cbe-mac-Universal-PIC.tar.bz2

There may be more, but that's what it's told me I'm missing so far.

Can anyone help me locate this file as I can't move forward without it.

[overclockedmind]

I've looked everywhere I know where to look; which means, can you define the scope of where I should be looking?

[tperry2x]

I've looked everywhere I know where to look; which means, can you define the scope of where I should be looking?

This is just it. I can't. (I wish I had more info to give you)
It's not something that google will turn up. It's not something that anyone seems to have a copy of and is publicly hosting.
I'm hopeful that someone has this sitting on their personal backup drive somewhere and can send me a link in a PM to. Even if they can upload it to a private google drive share, I'll let them know when I've got what I need by way of a post here. It seems to be something that's behind the LC login / password barrier now.

https://downloads.livecode.com/prebuilts

Or, if anyone can 'let me know' of a working login and password to this, even better.

Currently, not having access to this - it will also prevent me moving forward with a recompiled Windows engine when I come to that too.

In fact, it'll stop anyone compiling a new engine based on the LCC source.
Anyway, this is where my posts have crossed over a bit. Please see here for more:

https://www.openxtalk.org/forum/viewtop ... 6293#p6293