OpenXTalk IDE DPE RC3 Test Release

[tperry2x]

I've been meaning to look at your updater to see how I could incorporate that into OXT 'heavy'.

I wrote it while drinking energy drinks, if I recall, so it was a bit frantic - and the script looks like it
But I have commented most of it as I went along.

[OpenXTalkPaul]

I'd still really like to get to a point where 'Heavy' could be largely based on OXT lite just with certain extras included, particularly the larger ones like fluidSynth (which has multi-OS, multi-architecture, multiple dependence libraries, and a small soundbank all included in the same package) that might not be as easily delivered via a package management system as most smaller libraries or widgets.

Or the other way and just stick to one OXT that has a package manager built into it so user can installed whatever extras they might want to try out. That could help concentrate and focus efforts.

[tperry2x]

I'd still really like to get to a point where 'Heavy' could be largely based on OXT lite just with certain extras included
...
Or the other way and just stick to one OXT that has a package manager built into it so user can installed whatever extras they might want to try out. That could help concentrate and focus efforts.

That's an interesting idea, but for me, GitHub is out of the question. However, I do have an alternative which I'll post here in a while. I need to generate a mockup of what I mean first...

Edit:
Here we go, this is the kind of thing I was thinking of:

collab.png (738.62 KiB) Viewed 1815 times

How does this work? We both install the mega desktop app. I drop OXT Lite into that shared folder which we both have access to. OXT Lite goes into this 'collaboration mode' which tracks what changes are being made when we edit files. (I know how I could get it to do this). Just then means we can both edit the IDE live, without bumping into eachother, as the "collab-mode" (as I'm calling it) will stop you editing a stack if it's being edited by someone else.

(right click and open in new window for large version)

[tperry2x]

Just to add to this, I haven't deleted the datagrid library or anything, so you can turn it back on - and then our two versions should be compatible. (aside from any extra libraries and stuff that Lite doesn't have of course).

There's a topic with pictures of the various preferences in Lite here.

You should be able to turn all the disabled stuff back on, then restart the IDE, and it should all be back again:

...Tom does move fast (and breaks things, but that comes with the territory I think), much faster than I currently can (and maybe I'm overly cautious)...

I can't remember where exactly it is now, but I describe your RCx/'Heavy' version as a more considered, stable release. OXT Lite you could (generously ) call the "rolling release" / frantic-patching version. As we mentioned on the main page, the two approaches are good - and they both yield their own benefits by having a dual approach I think.



I am trying to make more of an effort to test on a variety of platforms before I make updates available though. As said, sometimes it can be hard to find the time.

However, that's also my worry with combining our efforts as I would need to ensure we aren't editing the same file (as per my mockup idea above).

[OpenXTalkPaul]

The mock-up collaboration thing with chat looks fantastic @tPerry2x !
I'm not religious about GitHub, I've mostly use it as a free hosting (that has web-UI based text editing capabilities and webpage serving)
I don't have a problem with using alternative system for collaborating, and I have used jointly-owned-MEGA accounts to share (a buddy of mine lives hundreds of miles away now but we exchange music recordings using it). I can always mirror that merged IDE on MEGA to a GitHub Repo so that random people can have a copy of the work too.

If we're going to work on a single version of the IDE, maybe we should work out some more detailed parameters for that first, because I'm concerned that a bunch of things in the IDE have multiple file edits involved in order to make certain changes work and some things may work on one platform but cause errors on another. I just don't want to be stepping on each-other's toes, you know what I mean? I think detailed (but not strict) work parameters might help avoid conflicts.

Currently if I just copy my modified files, overwriting those files in OXT Lite, then Lite becomes the same as my version (just with different Icon / splash screen). The files need to be DIFF'd and then merged together. I started doing that before on several occasions, but it's a very time consuming task. I'll try to use this weekend (4-day holiday weekend here in the USA) to make a big push on that..

Probably after that I should make having a proper package manager for add-on things be a priority (rather than Tools-redo, which has been making me crazy anyway), With that out of the way, I'd be more likely to concentrate on core IDE things but still be able to deliver any "clever-clevers" when I think they're worthy of sharing.

I'm not sure why you'd want to disable the DataGrid but NOT remove it? Is it a drag on anything if it's not in use? If it was removed I don't think that would be a very big disk space savings. Anyway,

But that's a good way to handle it, having prefs for every modification, the only problem is editing revPrefGUI stack is a real pain using the Project Browser (or Application Browser).

[tperry2x]

The mock-up collaboration thing with chat looks fantastic @tPerry2x !
I'm not religious about GitHub..I don't have a problem with using alternative system for collaborating... I can always mirror that merged IDE on MEGA to a GitHub Repo so that random people can have a copy of the work too.

That sounds like an ideal set of compromises that would suit everyone.

If we're going to work on a single version of the IDE, ...I'm concerned that a bunch of things in the IDE have multiple file edits involved in order to make certain changes work and some things may work on one platform but cause errors on another. I just don't want to be stepping on each-other's toes, you know what I mean?

Yes, that makes perfect sense. My approach does seem to be "what can I fix next?" -- and I'll go off and try something. We've also approached different solutions to the same problem at times, I'm sure. It is a lot of work to merge them, all in the name of possible speed of releases. However, speed isn't everything - and I like the fact that there's two approaches). That said, I'm not adverse to merging them.

Currently if I just copy my modified files, overwriting those files in OXT Lite, then Lite becomes the same as my version (just with different Icon / splash screen). The files need to be DIFF'd and then merged together. I started doing that before on several occasions, but it's a very time consuming task. I'll try to use this weekend (4-day holiday weekend here in the USA) to make a big push on that..

I'd like to understand how the diff tool works better in the IDE too, as I hoped we can use this, rather than relying on an external one. Something I'd build into that collab stack. There's some bits I'd definitely take from RCx/'Heavy' like your revMenubar - and using that as an example, that won't have the replicate option in, so a diff merge rather than a replace is definitely the way forward. I'd say initially, I'd expect a lot to break.
I'd like to take the preferences from OXT Lite though, as there's pref validation - and the entire stack has been re-laid out and made bigger (if you compare it to the LCC one). This is not simply scaled, I have resized the stack, moved controls around, added extra cards... you name it. It's not just script changes. There's the differences in the actual stacks themselves to bear in mind.

Probably after that I should make having a proper package manager for add-on things be a priority (rather than Tools-redo, which has been making me crazy anyway)

I didn't want to say anything, but yes - that way madness lies. It doesn't help the fact that LC used the same keywords for identifying sections, and the stack gets confused (which is why mine does a merry-dance with variables) to make things like the number of columns, the sections shown (which are labelled in OXT lite), and the conditional loading of light or dark tool icons (based on currently detected appearance).

...With that [package manager] out of the way, I'd be more likely to concentrate on core IDE things but still be able to deliver any "clever-clevers" when I think they're worthy of sharing.

For me, this is always part of the fun - seeing what other people have come up with as solutions. Mainly because I learn something by it.

I'm not sure why you'd want to disable the DataGrid but NOT remove it? Is it a drag on anything if it's not in use? If it was removed I don't think that would be a very big disk space savings. Anyway,

No - you are correct, it saves next to nothing on disk space, but it saves quite a great deal of CPU use in the background as there's datagrid (dg) messages happening all the time when it's active. Also (as shown on this post) - and the following picture:

...you'll see all the stacks it loads into ram when it's in use. Closing them off saves extra processing and saves available ram in the IDE. As a result, it does make it noticeably more responsive. But I'm aware some people use it, so that's why it's a pref.

But that's a good way to handle it, having prefs for every modification, the only problem is editing revPrefGUI stack is a real pain using the Project Browser (or Application Browser).

I don't find it too hard to edit. I do it all with the inspector and Application browser, and can edit the stack through there by adjusting sizes, locations etc. It's like when I used to spend ages correcting artwork as a repro operator in illustrator - moving things around all day by just using the transform palette (and a calculator... because maths )

The thing that comes to mind though is structuring of this shared folder.
We know it shares a common file structure when you get into the "Toolsets" folder, but MacOS has the "Tools" directory before that. I suspect this is so each modification does not break codesigning, because the codesigning is not interested in this "Tools" folder, so the mac engine is set to go there.
The Linux and Windows engine never looks for a "Tools" / "Toolsets" folder - they are straight into the "Toolsets" folders as default... then onto the similar folder structure.
What I'm getting at is, in theory I could be running the IDE in "collab-mode" and on my Linux computer. You could be doing the same, but sat at the Mac, also in "collab-mode". I wouldn't want to see your stuff before that "Toolsets" level, as remotely editing that on linux would definitely break your codesigning. So the IDE would need to know what was going on in "collab-mode" and lock the editing of stacks to prevent duplicate edits. This is also what I was thinking of the group chat in that mode, because that way - things can be tested 'live'.
I'd use UTC for universal co-ordinated timestamps.
I'm wondering if the IDE on Linux and MacOS would see the "Toolsets" folder if they were symlinked into the engine folder? In theory this would then mean they could be platform agnostic edits. (Although Windows still doesn't handle symlinks properly) More on this later, I need to do some testing later to show you what I mean...

Edit. This is what I mean:

proof-of-concept.png (414.4 KiB) Viewed 1747 times

Like this, the "Toolsets" folder isn't in the expected location for the IDE (it could be sat in our Mega sync'd folder through the mega desktop app) - the IDE still loads it, because as far as it's concerned, it is there - it's symbolically linked.

This way, you can be editing using the mac engine, and I can be editing the SAME ide using the Linux engine simultaneously.
Of course, you can use the command line to make symlinks, or you can use my tool here.

[OpenXTalkPaul]

We've also approached different solutions to the same problem at times, I'm sure. It is a lot of work to merge them, all in the name of possible speed of releases. However, speed isn't everything - and I like the fact that there's two approaches). That said, I'm not adverse to merging them.

At least for some things that we might want to approach differently, we could go the Linux way and make it a user preference of which method to use. It could be one method works better than another for certain systems. I'm thinking of the myriad of options for Linux here, but it could also allow for a variety of 'Tools' palette re-designs and multiple choices for syntax dictionary viewing methods (in Browser Widget, in Default Browser, in OXT stack dictionary viewer, etc.). But to still allow for trimmed down Lite version and a full-on Heavy version, the startup would need check that any additional stacks are present before adding them to the IDE UI. I bring this up because as an example, your revMenubar shows a menu item for the 'SVG Glyphs Browser', but the stack (and library of additional icons can optionally go with it) are not installed in Lite so that menu item is non-functional in Lite.

I'd like to understand how the diff tool works better in the IDE too, as I hoped we can use this, rather than relying on an external one. Something I'd build into that collab stack. There's some bits I'd definitely take from RCx/'Heavy' like your revMenubar - and using that as an example, that won't have the replicate option in, so a diff merge rather than a replace is definitely the way forward. I'd say initially, I'd expect a lot to break.

I think it works a lot like command-line diff in that it compares text for similarities and flags differences usually with an option to ignore white spaces. The problem as I see it is that a lot of the text we're interested in comparing is contained in different objects in a binary-format UI stack. There are ways to can compare binary data though that is usually significantly slower. Most stack files it seems are mostly script text and image data. This is why LC Team put some significant effort into converting IDE GUI stacks to 'UI stack + behavior script-only stacks' the 'script-only' stacks can be easily compared with DIFF tools (+GitHub).

I'd like to take the preferences from OXT Lite though, as there's pref validation - and the entire stack has been re-laid out and made bigger (if you compare it to the LCC one). This is not simply scaled, I have resized the stack, moved controls around, added extra cards... you name it. It's not just script changes. There's the differences in the actual stacks themselves to bear in mind.

For sure, I haven't made very many changes to the revPrefGUI stack, and some of the changes I did make were just to mirror what you were doing anyway. I REALLY Like the idea of having s Linux-check-up to ensure any dependencies the IDE Engine needs are present.

Probably after that I should make having a proper package manager for add-on things be a priority (rather than Tools-redo, which has been making me crazy anyway)

I didn't want to say anything, but yes - that way madness lies.

That's why I really wanted to build a Tools palette from scratch, but I don't think I went far enough in that regard. It may also be that I was a little over-ambitious with the live-resizing/relayout thing. I'd get it close to the way I'd like it to be, then I'd go to tweak one more little tiny thing and something else would go completely haywire as a result. Very frustrating.

be able to deliver any "clever-clevers" when I think they're worthy of sharing.

For me, this is always part of the fun - seeing what other people have come up with as solutions. Mainly because I learn something by it.

For me as well, that's why I'd like a package manager that would be semi-decentralized in a way, so that user could add other peoples repos. There would be one or two main repos (I'm thinking 'stable' repo and 'testing' repo), and then allow for pasting in additional URLs for repos other users could set up on their own. It could all work in a cohesive way provided there is a set structure to use (for listings .tsv , icons-screenshots-logos image formats, description .txt, etc.) I'd like to keep that simple with plain text descriptions and tab separated plain text data for listings)

No - you are correct, it saves next to nothing on disk space, but it saves quite a great deal of CPU use in the background as there's datagrid (dg) messages happening all the time when it's active. Also (as shown on

...you'll see all the stacks it loads into ram when it's in use. Closing them off saves extra processing and saves available ram in the IDE. As a result, it does make it noticeably more responsive. But I'm aware some people use it, so that's why it's a pref.

Ah OK I have noticed those sort of excessive IDE messages, but maybe we could fix the DG library so that it doesn't do that, or at least checks whatever it's checking on a conditional basis or at least check a bit less frequently.

t's like when I used to spend ages correcting artwork as a repro operator in illustrator - moving things around all day by just using the transform palette (and a calculator... because maths )

That's one of the reasons why I always preferred QuarkXPress back when (when PageMaker still existed), you could do the maths (or have the computer do it) directly from within the measurements palette (Adobe adopted this behavior much later with InDesign). I've probably just become spoiled by having 'dev guides' library to help manually align objects. Maybe the 'alt/option to open-as-editable" method would help there.

The thing that comes to mind though is structuring of this shared folder.
We know it shares a common file structure when you get into the "Toolsets" folder, but MacOS has the "Tools" directory before that. I suspect this is so each modification does not break codesigning, because the codesigning is not interested in this "Tools" folder, so the mac engine is set to go there.
The Linux and Windows engine never looks for a "Tools" / "Toolsets" folder - they are straight into the "Toolsets" folders as default... then onto the similar folder structure.

I don't think LC did that due to code signing because that wasn't a major thing when v6 (and earlier) through v8 were orginally built, I think it's more to do with Apple's .app bundle enforced structure more than codesigning related. MacOS .app bundles must contain certain sub folders and files like info.plist. Those files are only needed for macOS .
Anyesy I would think that symlinking the toolset should work on macOS, that was sort of the opposite of what I was doing having one sort-of merged together Toolset folder on a ExFAT partition I could access from any OS and run the appropriate executable for that platform, that does not work as well as I thought might would due to things like Browser CEF and DataBase externals also needing to be binaries appropriate for the current platform.