Could there be a parallel effort on the Community Edition...

A place to discuss and plan OpenSource xTalk (not exclusively LCC based)
and Community Builds of LCC ...Ask NOT what xTalk can do for you...
Get involved you DO have something to contribute, no matter your skillset!

Forum rules
A place to discuss and plan OpenSource xTalk (not exclusively LCC based) and Community Builds of LCC
Ask NOT what xTalk can do for you... get involved you DO have something to contribute, no matter your skillset!
axwald
Posts: 10
Joined: Mon Sep 27, 2021 1:14 pm
Location: Sol/ Terra/ Europe/ Bavaria
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by axwald »

Hi,
OpenXTalkPaul wrote: Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary [...]
Did you have a look at "TinyDictionary" from Bernd Niggemann et al.? It may contain a solution.
I guess it's hidden somewhere in the "sample stacks".

Have fun!
User avatar
OpenXTalkPaul
Posts: 1485
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by OpenXTalkPaul »

susan wrote: Sat Oct 09, 2021 11:00 pm
susan wrote: Sat Oct 09, 2021 9:35 pm
OpenXTalkPaul wrote: Sat Oct 09, 2021 6:20 pm If someone could figure out the process used to get the individual .lcdoc files that make up the dictionary packed into a SQLite API db file that is what is actually included with the IDE
Am I correct in understanding that there is a sqlite file somewhere, and you are hoping to retrieve the entries in that file that are from individual .lcdoc files?

Is this because the .lcdocs are not part of the github, and so need to be extracted to be able to work with them?
Oh, okay. I re-read your question. I had it back-to-front didn't I!

You want to figure out how to turn the individual .lcdoc files into the sqlite file, right?

Coming to the problem side ways, can the database entries not be packed/encrypted, and instead the OpenXTalk API viewer be altered to take the plain text .lcdoc format, so not need a decoding step?

Or am I still misunderstanding?
You got it the second time, I was looking to convert the loose .lcdocs TO a SQL db.
I believe there are already existing routines in the IDE or maybe in the engine repo, for doing this, I just haven't had time to work out exactly where or how.

I get what you're saying about not using a DB file at all, but I think that there's two issues with that...

For one that could possibly probably be a lot of work, but maybe not if we went back to the previous version of the dictionary, prior to the invention of this .lcdoc markdown, but then the entries would have to be gone through and made up to date. The IDE DOES actually already have the ability to insert new entries from lcdocs or inline documentation (in an .lcb file for example), which happens when you install a new Extension.

And secondly, having all of those loose .lcdoc markdown files would increase the size of the IDE on disk because each would take a block of disk space, which these days means usually at the least 32Kbytes, even though some dictionary entries are more like 32bytes! Personally I think the IDE is already larger, disk space wise, then it needs to be.
axwald wrote: Sun Oct 10, 2021 8:00 am Did you have a look at "TinyDictionary" from Bernd Niggemann et al.? It may contain a solution.
I guess it's hidden somewhere in the "sample stacks".

Have fun!
Yes, Bernd is awesome, he does some great work and he helped me out a bit when I was working on my Piano Widget!
I have a copy of TinyDictionary, and I will be looking at that when I get to it, but I think that probably pulls data FROM the the API DB, but what I want is to build the API DB itself from the lcdoc markdown files.
I'm sure there already exists an automated method for doing this, perhaps using handlers in the revDocsParser IDE file, I just haven't had a chance to look into that yet.
User avatar
OpenXTalkPaul
Posts: 1485
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by OpenXTalkPaul »

OK so I just did a bit more searching around, I think that this script is called by something and used to build the DB file that ships with the IDE:
https://github.com/livecode/livecode/bl ... codescript
axwald
Posts: 10
Joined: Mon Sep 27, 2021 1:14 pm
Location: Sol/ Terra/ Europe/ Bavaria
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by axwald »

Hi,
OpenXTalkPaul wrote: Sun Oct 10, 2021 3:47 pm [...] I think that this script is called by something [...]
Is it just me who's eyes start bleeding looking at this code?

Have fun?
User avatar
OpenXTalkPaul
Posts: 1485
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by OpenXTalkPaul »

Got it, the docs builder scripts are run by a stack called builder.rev in the same folder as docs_builder.livecodescript
had to comment out a couple of things (ensureTitlecase, which was only relevant for certain two word editions such as coumminutyPlus) to get the script to run all the way through. Anyway It built the api.sqlite file.
User avatar
OpenXTalkPaul
Posts: 1485
Joined: Sat Sep 11, 2021 4:19 pm
Contact:

Re: Could there be a parallel effort on the Community Edition...

Post by OpenXTalkPaul »

Hmmm, looks like that docs building script is outdated or something, the API.sqlite file it made was smaller by more then a megabyte, and entries for a bunch of things like most of LCB were missing. So I'll have to take a closer look at that script, in the meantime I tried to come at it a different way. Editing the existing API.sqlite file...

It seems that that db file isn't actually what the Dictionary uses? It seems it actually uses JSON .js files mainly, but still makes use of a db file too? I used an SQL editor to delete some commercial-only entries from the API.sqlite file, and then cleared the Documentation cache folder. When I relaunched the IDE, it rebuilt the dictionary cache and those deleted entries reappeared!?!?! I took the API.sqlite out completely, cleared the cache, relaunched and it rebuilt the cache again, still with the deleted entries, but with a smaller API.sqlite file in the cache folder that it must have built from the JSON files?

Fortunately my eyes aren't bleeding, at least not yet, but it certainly is a confusing setup.
Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests