AllBASIC

BASIC Developer & Support Resources => Compilers => Topic started by: JRS on September 17, 2010, 04:42:32 AM

Title: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 17, 2010, 04:42:32 AM
Charles,

We may be looking at this from the wrong angle. Embedding SB into O2 would give you much more control and transparency with the interface.

Back to the future (http://www.allbasic.info/mboard/index.php?topic=780.0)

I think showing O2 spawning SB scripts in threads and using shared memory for variable sharing between the two would be a nice example.

You might be thinking,
Quote
Why would that make sense?
and my response is that I could use O2 for framework and OS interfacing SB doesn't provide and use SB for the business logic side of the application.

John
Title: Re: OxygenBasic embedable scripting interpreter
Post by: cevpegge on September 17, 2010, 05:30:19 AM

Yes John, this would be simpler to implement I agree.

The Windows message loop is the dominant issue. Embedding Oxygen in ScriptBasic would require multithreading or some kind of context switching to break out of the loop to hand control back to SB periodically.

With ScriptBasic embedded in Oxygen, this complication is avoided - ScriptBasic functions are called when needed depending on events. This fits in perfectly with the Windows way of doing things.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 17, 2010, 11:30:07 AM
That's great Charles. I agree that this will be a much better way.

The SB embedding API should have everything you need. You may want to have a peek at the SBHTTPD (multi-threaded webserver) as a guide. (ignore CGI & running as a service code) The MT shared memory interface for the script and the embedding API thread functions supporting it should be all that is needed with no new code to write. As I see it, this is Plug N Play.  :)  & you already made the effort to create the .bi include for ScriptBasic embedding.
 

John

FYI The word embeddable ain't in the Microsoft encarta dictionary. My Firefox spell checker and Google doesn't seem to have a problem it being a valid word.
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 17, 2010, 07:16:35 PM
Charles,

I thought I would try and put a set of OxygenBasic scripting extension function concepts together to discuss. Each SB script runs in it's own thread but in the same process as O2. Variable sharing (locking supported) between O2 and scripts running in threads will be handled by the existing SB API.



If you create a SB.DLL interface or SB.LIB interface with the above functionality, I think the application programmer would have enough tools to work with to design a multi-threaded application framework with a high level scripting interface to give it life.

I'm unsure if it makes sense to give SB any direct access to O2 defined controls. If in O2 you define a script function to react to an event, then that function should service any request to the O2 framework within it's definition. SB should be easy to use in O2 but shouldn't steer the ship.

The other question I have is it possible to call a SB function if the script is load as a thread?
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 18, 2010, 04:51:39 PM
Charles,

I have given the SB embedded in O2 concept more thought and this is what I have come up with. I want to use O2 for my application launcher which will be a spiffy navigation system and dashboard front end to SB based master file maintenance, transaction entry, reporting and utilities. I had hope that Armando was going to complete and offer the source to the IUP GUI he was playing with but that looks like a lost effort. Peter's Gtk-Server is a nice second choice and if the Gtk DLLs are preloaded for the process when the O2 based front end is started, SB usage of those DLLs will be more responsive. I was going to use the SB multi-threaded HTTP application server running as a service and start application threads using the CGI protocol for a localhost centric client / server environment until O2 came along.

If I can have something like the SB called functions described above, I should be able to get the best out of both language environments.


John
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 18, 2010, 06:55:49 PM

John,

Threads are not intrinsic in Oxygen. We would use the Windows kernel thread functions directly. Do you want to initiate the threads in Oxygen the get them to do sb_load / sb_call operations? Just to get a clearer picture.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 18, 2010, 07:22:50 PM
SB includes an API for pthread for threading and supports shared/session variable memory management. The MT extension module is the interface for the SB script side and the embedding API has the functions for creating SB aware threads and memory management. (why recreate the wheel?)

The main reason for the request to embed SB in O2 is a host compiler to spawn threads and provided OS based functionality not available at the SB script level.

Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 18, 2010, 08:27:40 PM
Does this mean Oxygen would not need to deal directly with the threading aspect, making normal calls to SB?
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 18, 2010, 08:35:22 PM
The ScriptBasic embedding API has everything you need to do the integration. I thought it would be best to make the SB interface an O2 DLL or linkable library and keep O2 pure.

Quote
You may want to have a peek at the SBHTTPD (multi-threaded webserver) as a guide. (ignore CGI & running as a service code)

The sbhttpd httpd multi-threaded server is a C based host embedding SB in a threaded way. Add running as a service and http/cgi support and there you go, a web server in ScriptBasic. The server application is fast and resource friendly.
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 18, 2010, 10:06:31 PM
Thanks John,

Were you able to make any sense of my source code BTW :)

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 18, 2010, 10:10:13 PM
Quote
Were you able to make any sense of my source code BTW

I'm thinking James Fuller is already converting it to BCX.  ;D
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 19, 2010, 08:38:11 PM
Charles,

Can you offer the option in the SB_Run() and SB_Load() calls to not create a console? I think as part of the embedding interface STDIN/OUT is an argument.

John
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 19, 2010, 10:43:51 PM

John,

I remember working on this before - being able to control the presence or absence of the console. I think the client program has to set up a console before loading Scriba.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 19, 2010, 11:29:53 PM
Charles,

Take a look at these SB embedding function calls.

Quote from: scriba_SetStdout()
You can call this function to define a special standard output function. This pointer should point to a function that accepts a (char, void *) arguments. Whenever the ScriptBasic program tries to send a character to the standard output it calls this function. The first parameter is the character to write, the second is the embedder pointer.

If the standard output function is not defined or the parameter is NULL the interpreter will write the normal stdout stream.

void scriba_SetStdout(pSbProgram pProgram,
                      void *fpStdoutFunction
  ){

Quote from: scriba_SetStdin()
You can call this function to define a special standard input function. This pointer should point to a function that accepts a void * pointer as argument. Whenever the ScriptBasic program tries to read from the standard input it calls this function pasing the embedder pointer as argument.

If the stdin function is not defined or the parameter is NULL the interpreter will read the normal stdin stream.

void scriba_SetStdin(pSbProgram pProgram,
                     void *fpStdinFunction
  ){

Quote from: scriba_SetEmbedPointer()
This function should be used to set the embed pointer.

The embed pointer is a pointer that is not used by ScriptBasic itself. This pointer is remembered by ScriptBasic and is passed to call-back functions. Like the standard input, output and environment functions that the embedding application may provide this pointer is also available to external modules implemented in C or other compiled language in DLL or SO files.

The embedder pointer should usually point to the struct of the thread local data. For example the Windows NT IIS variation of ScriptBasic sets this variable to point to the extension control block.

If this pointer is not set ScriptBasic will pass NULL pointer to the extensions and to the call-back function.

void scriba_SetEmbedPointer(pSbProgram pProgram,
                            void *pEmbedder
  ){

Quote
I think the client program has to set up a console before loading Scriba.

If the Oxygen program is running in a console then SB should use that for STDOUT. I see no reason to mess with STDIN.

If the Oxygen program is running in a GUI window then SB sends STDOUT to null. (unless SB_Debug is used to run the script or load a library of functions and executing through the debug preprocessor)
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 20, 2010, 04:51:49 AM
Good info John,

We could make a GUI console available to communicate with SB. This has the advantage of full integration into a GUI based application. SB could also have more than one console panel in the User Interface.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 20, 2010, 08:35:14 PM
Charles,

If we can get away from having to use a console window at all when the O2 application is declared GUI based than I couldn't be happier.

Peter Verhas alpha released a Windows version of the remote debugger which I got working on Linux and used Telnet for the client. It's still needs a few tweaks but it will be a good starting point for the O2 version of the SB GUI debugger.
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 23, 2010, 02:07:03 PM
Charles,

While your working on O2 getting it ready for beta, I'm going to start building my multi-threaded SB application with sbhttpd. The only twist to this approach is that instead of running a script in a thread with a function call, I'll send a CGI header request to sbhttpd (running as a service on 127.0.0.1) which runs the script. I see some advantages to this approach as I can intermix desktop GUI and HTML in a browser container using SB alone. It gives me shared variable support (with locking) between threads and the main control program. I can also create session variables that remain static between iterations if needed. (MT extension module)

I use sbhttpd as a proxy server for websites I host for clients on a CentOS dedicated server at a data center. (this site is running on that server)

I'll post something on the All Basic interpreter board when I have an example to show.

John
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 24, 2010, 02:28:01 AM

John,

If you are able to use a browser for interfacing your SB applications, what kind of advantages are you looking for in a native MS Windows GUI? I ask this question because the long term trend in computing (if you agree with Google)  is to use a browser exclusively as the front end for everything.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 27, 2010, 04:54:15 PM
John,

First stage demo of SB embedded in Oxygen now done. Its called SBembed1.o2bas and runs SBProg1.bas, both in the SBO2 folder. No need to compile it if you run through the IDE.

The output is very plain, but its one of the biggest steps - demonstrating program control, variable data going both ways and no console :)

I don't think Kaspersky will throw out the new compiler tools EXO2 and GXO2

My bed time.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 27, 2010, 06:00:11 PM
If you are able to use a browser for interfacing your SB applications, what kind of advantages are you looking for in a native MS Windows GUI? I ask this question because the long term trend in computing (if you agree with Google)  is to use a browser exclusively as the front end for everything.

The ScriptBasic httpd application server is nothing more than a C program embedding SB and running the scripts in a thread of the same process. The C host program takes care of installing itself as a service and listening for http/CGI requests that run SB scripts. Session support is done in memory and supports a global variable sharing scheme with locking among threads and the host C program.

I plan to use Gtk-Server for GUI on all platforms. I may use O2 and Gtk API to get the DLLs loaded and create the launcher. The main goal with the O2 front end is to build a multi-threaded  scriba interpreter.
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 27, 2010, 06:28:55 PM
Quote
First stage demo of SB embedded in Oxygen now done. Its called SBembed1.o2bas and runs SBProg1.bas, both in the SBO2 folder. No need to compile it if you run through the IDE.

Thanks Charles!

Works great and a clean example how to embed ScriptBasic. This example shows running a SB script, calling a SB function and get/set a SB global variable from O2. The no-console feature is a nice touch. (8KB .exe generated)

(http://files.allbasic.info/Oxygen/O2embed10.png)

This is the OxygenBasic program that embeds ScriptBasic.

Code: Text
  1. '#file "SBembed1.exe"
  2.  
  3.   include "scriba.inc"
  4.   indexbase 0
  5.  
  6.   dim as long pProgram,iError,cArgs
  7.   dim as long f1,f2,v
  8.   dim as long n,m
  9.   dim as SbData ReturnData,ArgData(3)
  10.   dim as sbdata byref pdat
  11.   dim as long qdat
  12.   '
  13.   cr=chr(13)+chr(10)
  14.   '
  15.   pr="Embedded ScriptBasic Tests" cr cr
  16.  
  17.   function newmem CDECL (ByVal le As Long) As BSTR Export
  18.     return nuls le
  19.   end function
  20.  
  21.   sub freemem CDECL (ByVal p As bstr) Export
  22.     frees p
  23.   end sub
  24.  
  25.   'SetConsoleTitle "FreeBasic ScriptBasic Embedding Test"
  26.  
  27.  
  28.  
  29.   '
  30.   'LOADING AND RUNNING A PROGRAM
  31.   '-----------------------------
  32.   '
  33.   pProgram=scriba_new (& newmem, & freemem)
  34.   'scriba_LoadConfiguration(pProgram,"c:\scriptbasic\bin\scriba.conf")
  35.   '
  36.   finit 'reinitialise FPU
  37.   '
  38.   scriba_SetFileName(pProgram,"SBProg1.bas")
  39.  
  40.   iError=scriba_LoadSourceProgram(pProgram)
  41.   if iError then goto ending
  42.  
  43.   iError=scriba_Run(pProgram,"Hello")
  44.   if iError then goto ending
  45.  
  46.   '
  47.   'ACCESSING GLOBAL DATA
  48.   '---------------------
  49.   '
  50.   v=scriba_LookupVariableByName(pProgram,"main::a")  
  51.   scriba_GetVariable(pProgram,v,& pdat)
  52.   '
  53.   pr+="read A: "str(pdat.lng) cr
  54.   '
  55.   m=pdat.lng+100
  56.   pr+="add 100 to A" cr
  57.   '
  58.   scriba_SetVariable(pProgram,v,2,m,0,"",0)
  59.   scriba_GetVariable(pProgram,v, & pdat)
  60.   '
  61.   pr+="read A: "str(pdat.lng) cr
  62.   '
  63.   'CALLING SIMPLE SUBROUTINE
  64.   '-------------------------
  65.   '
  66.   f1=scriba_LookupFunctionByName(pProgram,"main::dprint")
  67.   if f1=0 then print "Unable to locat Dprint" : goto ending
  68.  
  69.   iError=scriba_Call(pProgram,f1)
  70.   if iError then goto ending
  71.   '
  72.   '
  73.   '
  74.   'CALLING FUNCTION, RETURNING DATA AND GETTING ALTERED PARAMS
  75.   '-----------------------------------------------------------
  76.  
  77.   f2=scriba_LookupFunctionByName(pProgram,"main::eprint")
  78.   if f2=0 then print "Unable to locate Eprint" : goto ending
  79.  
  80.  
  81.   'SETUP ARGUMENTS
  82.   '---------------
  83.  
  84.   'these can be used for both input and output
  85.  
  86.   ArgData(0).typ=SBT_DOUBLE
  87.   ArgData(1).typ=SBT_DOUBLE
  88.   ArgData(2).typ=SBT_DOUBLE
  89.   ArgData(3).typ=SBT_DOUBLE
  90.  
  91.   ArgData(0).siz=0
  92.   ArgData(1).siz=0
  93.   ArgData(2).siz=0
  94.   ArgData(3).siz=0
  95.    
  96.   ArgData(0).dbl=11
  97.   ArgData(1).dbl=12
  98.   ArgData(2).dbl=13
  99.   ArgData(3).dbl=14
  100.  
  101.   cArgs=4
  102.  
  103.   '
  104.   pr+="call F2:" cr
  105.   '
  106.   iError=scriba_CallArgEx(pProgram, f2, ReturnData, cArgs, ArgData)
  107.   if iError then goto ending
  108.  
  109.   sys c=ReturnData.typ
  110.   '
  111.   'READ RETURNED VALUE
  112.   '-------------------
  113.   '
  114.   select case  c
  115.     case SBT_UNDEF  : pr+= "Undefined "
  116.     case SBT_DOUBLE : pr+="double "  ReturnData.dbl
  117.     case SBT_LONG   : pr+="long "    ReturnData.lng
  118.     case SBT_STRING : pr+="zstring " ReturnData.str
  119.     case SBT_ZCHAR  : pr+="zchar "   ReturnData.str
  120.   end select
  121.   '
  122.   pr+=cr
  123.   '
  124.   print pr
  125.  
  126.   '------
  127.   ending:
  128.   '======
  129.  
  130.   scriba_destroy(pProgram)
  131.  
  132.   if iError then print "ERROR:  " + hex(iError)  ' + "  " + hex(pProgram)
  133.  

This is the SB script being run by O2.

Code: ScriptBasic
  1. '
  2. ' SCRIPTBASIC
  3. '
  4.  
  5. ' ----------------
  6. ' GLOBAL VARIABLES
  7. ' ================
  8. '
  9. a=42
  10. b=sqr(2)
  11. s="hello world!"
  12. '
  13.  
  14. ' ---------
  15. ' FUNCTIONS
  16. ' =========
  17.  
  18.  
  19. function dprint
  20.   sp="  "
  21.   cr="\n"
  22.   tab=chr$(9)
  23. end function
  24.  
  25.  
  26. function eprint(a,b,c,d)
  27.   'eprint=a+b+c+d
  28.  'eprint=a+b+c+d+0.5
  29.  eprint="ans " & a+b+c+d
  30. end function
  31.  

(http://files.allbasic.info/Oxygen/O2NoRun.png)

Notice that A isn't preset to 42 this time? That's because I changed Charles's O2 program to call Scriba_NoRun() instead.

Quote from: Scriba_NoRun()
If the embedding program wants to set global variables and execute subroutines without or before starting the main program it has to call this function first. It does all the initializations that are done by scriba_Run() except that it does not actually execute the program. After calling this function the main program may access global variables and call ScriptBasic functions.

That is so cool to be able to RUN an O2 program in the IDE without having to compile it first.

small bug: The IDE creates a .exe executable with no prefix filename. (should be SBembed1.exe)

The attached zip includes the O2 program .exe, the libscriba.dll (SBAPI) and the SB script O2 runs and calls.
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 28, 2010, 12:33:28 AM

Glad it works John.

I have corrected the .exe problem and ensured the co2.exe is not present in the zip as it is no longer needed. In future I will try to avoid dispatching new releases late at night :)

One problem which defeats me utterly: When I changed the email address in my FreeBasic Forum profile, it nobbled my account so I am unable to respond there. I can't even find out who to contact since email to admin@freebasic.net is rejected.


Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 28, 2010, 12:45:22 AM
Can you setup a new FreeBASIC account? Might be faster than trying to fix the old one. The FB board is an old custom phpBB forum. I'm surprised it hasn't been been trashed already by hackers preying on outdated forum software and CMS packages that are full of security holes. (php is a magnet for web hackers) I don't think anyone is steering the ship over there.

What is next on the list? (threading scripts?)
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 28, 2010, 12:57:22 AM

Yes I'll do that. I spent a long time fumbling around but the site offers no clues as to who to contact when things go wrong.

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 28, 2010, 01:17:01 AM
Its no good. When I try to do a new registration, I can't get past the image code test - its case sensitive and the letters are all ambiguous to me. It did have a contact address however.

rubentbstk@gmail.com <rubentbstk@gmail.com>

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 28, 2010, 01:27:47 AM
I posted a request for one of the FreeBASIC admins to fix your account.

You are well liked and they should take care of it. Now if that happened to me, they would write it off as a blessing and I would be left in FreeBASIC limbo.  :D
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: cevpegge on September 28, 2010, 02:10:25 AM
Thanks John!

We will see how long it takes to get a response.

And if it is so difficult to register for the forum, I am surprised they get any new members. I have tried several times to register but now I am denied access to the registration process. (please try later)

Charles
Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on September 28, 2010, 05:13:56 PM
Glad to see you're back online on the FreeBASIC forum. Mathew (counting pine) is one of the good guys and has always been fair with me.

Title: Re: OxygenBasic embeddable scripting interpreter
Post by: JRS on October 16, 2010, 07:38:19 PM
Charles,

Quote
What is next on the list? (threading scripts?)

I know your busy with O2h internals but I was wondering if you could post an example of O2h calling SB in a threaded manor. I can probably figure out how to add the shared variable (MT) support.

I'm sure glad you were able to solve the riddles passing over the virus and FB bridges. It's cool that you are talented enough to drop down to byte/register level and fight back with a vengeance.

John