On-screen keyboards ?

[richmond62]

Somewhere, somewhen, Paul, you were making noises about onscreen keyboards in Linux, Mac and Windows.

I have recently found out that on mobile there is inwith xTalk a keyBoardActivated: I wonder if there would be a way to extend this to the desktop platform?

[richmond62]

Here's my 27" iMac running MacOS 12 and the standard Mac virtual keyboard:
-

keyBoardScreen.jpg (188.36 KiB) Viewed 692 times

-
In 'a perfect world' a stack looking a bit like this:
-

KB1.jpg (190.44 KiB) Viewed 692 times

-
Would NOT look like this when the Virtual keyboard was opened:
-

KB2.jpg (227.63 KiB) Viewed 691 times

-
But would scale so none of the virtual keyboard overlapped it:

[richmond62]

But would scale so none of the virtual keyboard overlapped it:
-

[tperry2x]

Just out of interest, can you try opening a script editor window with that virtual keyboard active, make some changes and close the script editor window.

Does it spontaneously quit the IDE?
https://tsites.co.uk/otherstuff/crash-k ... viewer.mp4

[richmond62]

OXT 0.96, MacOS 12.7.2: does NOT crash OXT Lite:
-

Screenshot 2023-12-16 at 21.27.52.jpg (592.67 KiB) Viewed 675 times

[tperry2x]

OXT 0.96, MacOS 12.7.2: does NOT crash OXT Lite:

That's brilliant news. Just to be sure though, can you try copying some part of the script using the virtual keyboard, pasting in elsewhere, then attempt to close the script editor?

Essentially as I did in the video above to trigger the crash.

[OpenXTalkPaul]

But would scale so none of the virtual keyboard overlapped it:
-

Yes I think that onscreen-keyboard widget could be made to work in this way, you would have to forward the keyUp/Down messages to the target in your script (there's only basic default scripts set up in it). It would also be easy to add a message post 'keyboardDidActivate' back to the script engine when the widget grabs focus in order to match the message sent from the mobile equivalent.

On desktop platforms there really isn't many reasons to use it except for the visual feedback or if there is no physical keyboard attached. But I suppose it could work as a stand-in for the mobile behavior.

It's also set up in a way that I can fairly easily replace or add to the symbols that can be used for each key ( you can use it by passing in rawKeyDown pKeyNum messages, instead of normal keyDown pKeyName messages, both work).

And yes you can scale it to any size, it's resolution independent, the default size is actually 50% of the original vector drawing's size. It may render poorly at super small sizes, but there is a property to turn on/off 'anti-aliasing' render (mostly looks better turned off.

darkMode auto-switch works with its default color props.

https://github.com/OpenXTalk-org/OpenXT ... teMode.jpg
https://github.com/OpenXTalk-org/OpenXT ... rkMode.jpg

[richmond62]

Nothing particularly new in this as I have been farting around with this for going on 14 years:
-

Screenshot 2023-12-17 at 11.09.18.png (589.66 KiB) Viewed 632 times

-
Devawriter Pro.

If you want the source code drop me a private message.

Here's a happy code snippet to either excite you a lot or fill you with revulsion:

Code: Select all

on rawKeyDown RAWK
   if the vis of img "LOKK" is false then
      pass rawKeyDown
   else
      -- deprecated code removed 
      switch RAWK
         case 65288
            if fld "fBUILT" is empty then
               put the word 4 of the selectedChunk into Z4
               
               delete char Z4 of fld "fRESULT"
               set the textColor of fld "fRESULT" to "black"
            else
               delete the last char of fld "fBUILT"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
            end if
            break
         case 65289
            if the vis of img "CONZZ.png" is true then
               send "mouseDown" to img "CONZZ.png"
               send "mouseUp" to img "CONZZ.png"
            else
               send "mouseDown" to img "VWLZZ.png"
               send "mouseUp" to img "VWLZZ.png"
            end if
            break
            --case 65473
            --send "mouseUp" to img "TRANS.png"
            --break
         case 65470
            send "mouseDown" to img "HELPP.png"
            send "mouseUp" to img "HELPP.png"
            break
            ----
         case 65476
            if the vis of img "QWERT" is true then
               set the vis of img "QWERT" to false
               set the vis of img "POXX" to false
            else
               set the vis of img "QWERT" to true
               set the vis of img "POXX" to true
            end if
            break
            --------
            ----Return key---
            --case 65293
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            ---
            --case 65421
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            --------
         case 96
            send "mouseDown" to image "LOTUS"
            wait 5 ticks
            send "mouseUp" to image "LOTUS"
            break
            ----
            --------SHIFT KEY DOWN--------
         case 81 
            send "mouseDown" to img "2325k"
            wait 5 ticks
            send "mouseUp" to img "2325k"
            break

[richmond62]

Here's a stack to help people obtain the raw key code and the key code for any button on their keyboard:
-

Screenshot 2023-12-17 at 11.37.09.png (232.97 KiB) Viewed 631 times

-

https://syndromestore.com/products/pink ... tee-tshirt

[OpenXTalkPaul]

I've been farting around with keyboard (and Gamepads and other HID devices) input for quite a long time myself.

The stack in the following video works on rawKeyDown/Up messages. I had an earlier version of this that used WebMIDI API with the browser external (NOT the Browser Widget) when LC CE was still 32bit and web plug-in friendly, and it worked pretty well. In the scripts I stored all of the rawKey numbers in an array for faster memory access, low-latency being important for musical performance:
https://www.youtube.com/watch?v=s_knX8mJg1c

Interesting to note that these key numbers apparently come from classic macOS, they don't match the key numbers on macOS X and above.

[OpenXTalkPaul]

Nothing particularly new in this as I have been farting around with this for going on 14 years:
-
Screenshot 2023-12-17 at 11.09.18.png
-
Devawriter Pro.

If you want the source code drop me a private message.

Here's a happy code snippet to either excite you a lot or fill you with revulsion:

Code: Select all

on rawKeyDown RAWK
   if the vis of img "LOKK" is false then
      pass rawKeyDown
   else
      -- deprecated code removed 
      switch RAWK
         case 65288
            if fld "fBUILT" is empty then
               put the word 4 of the selectedChunk into Z4
               
               delete char Z4 of fld "fRESULT"
               set the textColor of fld "fRESULT" to "black"
            else
               delete the last char of fld "fBUILT"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
            end if
            break
         case 65289
            if the vis of img "CONZZ.png" is true then
               send "mouseDown" to img "CONZZ.png"
               send "mouseUp" to img "CONZZ.png"
            else
               send "mouseDown" to img "VWLZZ.png"
               send "mouseUp" to img "VWLZZ.png"
            end if
            break
            --case 65473
            --send "mouseUp" to img "TRANS.png"
            --break[i][/i]
         case 65470
            send "mouseDown" to img "HELPP.png"
            send "mouseUp" to img "HELPP.png"
            break
            ----
         case 65476
            if the vis of img "QWERT" is true then
               set the vis of img "QWERT" to false
               set the vis of img "POXX" to false
            else
               set the vis of img "QWERT" to true
               set the vis of img "POXX" to true
            end if
            break
            --------
            ----Return key---
            --case 65293
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            ---
            --case 65421
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            --------
         case 96
            send "mouseDown" to image "LOTUS"
            wait 5 ticks
            send "mouseUp" to image "LOTUS"
            break
            ----
            --------SHIFT KEY DOWN--------
         case 81 
            send "mouseDown" to img "2325k"
            wait 5 ticks
            send "mouseUp" to img "2325k"
            break

Sample code is always welcome, thanks!

What is this about?
----Return key---
--case 65293
---- deprecated code removed
I don't think the returnKey's 'rawKey' value is deprecated.

Anyway here is what is 'particularly new' about the OnScreenKeyboard...

1. The widget is already mostly set up and encapsulated and easily reusable code. I can drag-drop a new virtual keyboard from the tools palette like any other control, click on some of it's 'default scripts' from the side-bar and have some functionality to it in just a few seconds. It is tailored to two or three use-cases that I had in mind.

2. It's SVG/vector based, so it's resolution independent, so it should look good on an 85" 8K TV, and because it is 'line-art" with default properties inherited, it should auto-toggle key labels from dark/light mode.

3. It's also a good (only?) open-source example of another usage for the 'OnDo' message handler in Extension Builder, ...outside of the Browser Widget which uses it for running JavaScript in its Browser instance. Here, instead of JavaScript, we are parsing a subset (only keyboard related messages) of xTalk, which is how you'd expect the do 'command' when no 'as scriptingLanguage' form is used (its actually the Browser Widget that breaks the norm).

[OpenXTalkPaul]

Nothing particularly new in this as I have been farting around with this for going on 14 years:
-
Screenshot 2023-12-17 at 11.09.18.png
-
Devawriter Pro.

If you want the source code drop me a private message.

Here's a happy code snippet to either excite you a lot or fill you with revulsion:

Code: Select all

on rawKeyDown RAWK
   if the vis of img "LOKK" is false then
      pass rawKeyDown
   else
      -- deprecated code removed 
      switch RAWK
         case 65288
            if fld "fBUILT" is empty then
               put the word 4 of the selectedChunk into Z4
               
               delete char Z4 of fld "fRESULT"
               set the textColor of fld "fRESULT" to "black"
            else
               delete the last char of fld "fBUILT"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
               delete the last char of fld "fBUILD"
            end if
            break
         case 65289
            if the vis of img "CONZZ.png" is true then
               send "mouseDown" to img "CONZZ.png"
               send "mouseUp" to img "CONZZ.png"
            else
               send "mouseDown" to img "VWLZZ.png"
               send "mouseUp" to img "VWLZZ.png"
            end if
            break
            --case 65473
            --send "mouseUp" to img "TRANS.png"
            --break[i][/i]
         case 65470
            send "mouseDown" to img "HELPP.png"
            send "mouseUp" to img "HELPP.png"
            break
            ----
         case 65476
            if the vis of img "QWERT" is true then
               set the vis of img "QWERT" to false
               set the vis of img "POXX" to false
            else
               set the vis of img "QWERT" to true
               set the vis of img "POXX" to true
            end if
            break
            --------
            ----Return key---
            --case 65293
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            ---
            --case 65421
            ---- deprecated code removed 
            --set the text of the selectedText to (numToCodePoint(13) & numToCodePoint(10))
            --select after the selectedText --HAPPY--
            --break
            --------
         case 96
            send "mouseDown" to image "LOTUS"
            wait 5 ticks
            send "mouseUp" to image "LOTUS"
            break
            ----
            --------SHIFT KEY DOWN--------
         case 81 
            send "mouseDown" to img "2325k"
            wait 5 ticks
            send "mouseUp" to img "2325k"
            break

Sample code is always welcome, thanks!

What is this about?
----Return key---
--case 65293
---- deprecated code removed
I don't think the returnKey's 'rawKey' value is deprecated.

Anyway here is what is 'particularly new' about the OnScreenKeyboard...

1. The widget is already mostly set up and encapsulated and easily reusable code. I can drag-drop a new virtual keyboard from the tools palette like any other control, click on some of it's 'default scripts' from the side-bar and have some functionality to it in just a few seconds. It is tailored to two or three use-cases that I had in mind.

2. It's SVG/vector based, so it's resolution independent, it should look good on an 85" 8K TV, and because it is 'line-art" with default properties inherited, it should auto-toggle key labels from dark/light mode.

3. It's also a good (only?) open-source example of another usage for the 'OnDo' message handler in Extension Builder, ...outside of the Browser Widget which uses it for running JavaScript in its Browser instance. Here, instead of JavaScript, we are parsing a subset (only keyboard related messages) of xTalk, which is how you'd expect the 'do command' when no 'as scriptingLanguage' form is used (it's actually the Browser Widget that breaks the norm).
Realizing I could use 'OnDo' like this was actually the concept that prompted me to write(/illustrate) this widget to see it working.

[richmond62]

I don't think the returnKey's 'rawKey' value is deprecated.

Not in the greater scheme of things, but in the context of that SWITCH statement inwith my Devawriter Pro it is deprecated.

https://richmondmathewson.owlstown.net/projects

At one point the RETURN key was being used as a modifier key: but end-users were getting their knickers in a twist as they could not out that the RETURN key and the ENTER key are 2 different things (think teachers of Sanskrit in murky places), so I changed things over to the ALT/OPT key.

To illustrate this confusion; the University of Bristol (which really ought to know better) has NO RETURN key, but 2 ENTER keys marked on their diagram to explain something about computer keyboards:

https://www.bristol.ac.uk/is/media/trai ... rd-1r.html
-

SShot 2023-12-19 at 12.09.18.png (71.26 KiB) Viewed 568 times

-
https://en.wikipedia.org/wiki/Enter_key
-

SShot 2023-12-19 at 12.15.29.png (35.02 KiB) Viewed 565 times