New dictionary stack

[OpenXTalkPaul]

(edit by tperry2x: these posts originally came from here. - I split this into another topic)

just getting it out of the database

The main sqlite file for dictionary is built from those .lcdoc mark down files in the engines repo, so you could skip the DB all together and just read those into your stack, and then strip the bits of markdown to make them plain text.

[tperry2x]

That's exactly it. I'm a big fan of that approach.
Also, thank you for taking that in jest - as it was intended.

[richmond62]

Strip out the bits of markdown?

< . . . >

presumably.

[tperry2x]

Strip out the bits of markdown?

< . . . >

presumably.

Haha, if only it was that easy. Essentially removing all html tags.
I'm thinking of importing it, setting the htmltext of a field to true, then setting it back to false - or something like that. (sudo-code in my head at the moment), but that might strip out most of the chaff.

[richmond62]

C.F. https://forums.livecode.com/viewtopic.p ... 0&start=15

[tperry2x]

C.F. https://forums.livecode.com/viewtopic.p ... 0&start=15

Ah, don't have my LC forum login to hand at the moment (different computer) - and I never really go there. (mainly because I don't need to)
- says I need to login to see that post (?!)
Any chance you can copy and paste the relevant info here?

[richmond62]

Longish thread:

"Nectar" only:

Richmond:

I am looking, ultimately, for a plain text version of the Dictionary, so I store it as a custom prop in a stack, call bits of it into a field, and edit (add my own guff) using a simple text editor.

TinyDictionary has to option to add user notes to a dictionary entry. Those notes are stored locally and loaded at the start of TinyDictionary. Maybe that does what you want. Bernd

Richmond:

The only snag about this is that I might want to transfer my modded dictionary onto other machines: say the ones in my school.

"unstructured text" would be fairly useless, but structured text would not.

Were the dictionary available as a list of terms, delimited in some way (e.g. '-----'), and it were loaded into a variable when required, searching it should not be unduly difficult.

FourthWorld:

The SQLite DB that contains the data is inherently very nicely delimited.

PDF is a display format, not designed for parsing and notoriously inconsistent in what you get out of it with the clipboard or conversion tools.

I'd go to the source: use the SQLite DB.

[richmond62]

https://brettterpstra.com/2013/10/18/a- ... -markdown/
-

Screenshot 2024-07-18 at 11.57.55.png (108.72 KiB) Viewed 3380 times

-
"The simplest way to install a single Service is to double-click it in Finder. Finder will show a dialog asking if you want to install the Service. Just hit “Install” to add it to your Services menu. You can also move the files into your Services folder manually."

[richmond62]

Well, as usual, "sort of", as using the strip markdown service on the lcdict9.sqlite document extracted from the APK (couldn't make this up even if I wanted to) resulted in things like this:

trtdpResponseCode/tdtd/tdtdpA HTTP response code. See a href="https://www.w3.org/Protocols/rfc2616/rf ... tml"here/a for more detail./p
/td/tr

are still 'what they are'.

I hope you enjoyed your breakfast. ROFL.

[tperry2x]

This is all good, however I need something that works for Windows and Linux too - and as stacks can contain all the formatting we need (and it's a format we obviously want to encourage our xTalk users to use, I see no harm in using that).

Interesting reading though, so I'll give that a ponder.
I should add, I've already extracted it all as text. I wrote a script to do just that.
So, worry you not - that bit has been taken care of. (I left that running over night and just checked in on it)

[richmond62]

Linux:

Code: Select all

$ sudo apt install pandoc

and then:

Code: Select all

$ pandoc  --from markdown --to plain -o output.txt input.md

https://pandoc.org/

Also there are versions for Windows and Macintosh.

Pandoc is in the Debian, Ubuntu, Slackware, Arch, Fedora, NixOS, openSUSE, gentoo and Void repositories.

[tperry2x]

As I say, I've already done this - but they might need some cleaning up. (The text in the database has annoying typos like double spaces). However, this will open it up so ANYONE* can edit the dictionary (as a normal stack). Which is my whole point of doing this (and to make it behave the way everyone wants).

dollar keyword.txt.zip

(981 Bytes) Downloaded 76 times

My example contains no formatting, but that's not a problem. I can style the text as need be with a function - which will be easy.
This will keep me occupied, so work on my revised inspector palette is now paused.

* emphasis mine, because I think it's important that it's back in the hands of the xTalk users to be able to make corrections as needed.

[richmond62]

back in the hands of the xTalk users

IFF xTalk is to be truly 'Open Source' it also needs to be 'Open Access': and a dictionary that is occluded behind what it is currently occluded is not that.

[tperry2x]

IFF xTalk is to be truly 'Open Source' it also needs to be 'Open Access': and a dictionary that is occluded behind what it is currently occluded is not that.

Exactly, I want to make it a stack so that everyone can simply edit a stack to make changes. Then, if it's all okay - I'd simply replace the changed stack (or even just a changed card of the stack) when there are corrections (which we know already there are many of, as there's lots in the dictionary that is either wrong / out of date or certain things are non-existant).

[richmond62]

Rooty, Tooty, and Fruity:

https://github.com/livecode/livecode/tr ... dictionary

Are you proposing to churn through all of those?

[tperry2x]

Yes and no.
Yes - because all of those have been processed.
No - because I didn't have to manually do anything. The script I constructed did it all for me. (even though it took ages, but the extraction process was a one-time thing - won't have to do it again)

The script carried on if there was an error, but did throw up a few errors.
It has shown that searching for any of the following:

Code: Select all

revdb
revXMLRPC
xbrowser

then trying to view those entries in the 'Quick Dictionary' causes a script error.

Not that it bothers me, or won't do shortly - and this is another reason I want to move away from a database approach in favour of a stack/card one.

Now that I have all the data extracted as plain text, I'm just now creating a script to go through and create the cards accordingly. Before doing that though, I'll create a mockup stack with shared groups, so it has something to populate into.

Obviously even though this stack will initially be generated by a repeat loop, I'd like to get the layout (look / feel) of it right to begin with. (So I'm not going back over the same task again).

Am I modelling it on this:

Dictionary-Labels.png (104.69 KiB) Viewed 3327 times

Or, does anyone have a mockup they'd like it to resemble?
Personally, my thoughts on it are that I don't want to completely rip-off LC's effort, I want it to be user friendly - and it has to be clear and concise, while also ensuring people moving from LCC feel like it's not too dissimilar.

[tperry2x]

Thoughts? All mockups welcome.

[richmond62]

I would use the 963 dictionary as your model.

[tperry2x]

Simplistic so far, but it works.

in-light-mode.png (32.94 KiB) Viewed 3277 times

This is just the search field, and the result group (echoing the LCC design).
I'll put in the section listing each property / definition / keyword between the search field and the result.
(This is just a proof of concept at the moment).

More to follow, later - when I have more time...

[tperry2x]

I also wondered if it would be handy to have the option of exporting these dictionary pages as PDFs?
The largest dictionary entry (by far) is the "mobileControlGet" function, so I've tested my dictionary with this - just to see if I could trip it up.

Worked fine (attached) - my only complaint was the icon for the function at the top disappeared, but that's probably some kind of embed issue.

mobileControlGet.pdf

(25.17 KiB) Downloaded 73 times