Author Topic: uCalc Transform Primer  (Read 8437 times)

Daniel Corbier

  • Guest
uCalc Transform Primer
« on: October 25, 2013, 11:07:03 AM »
uCalc Transform Primer for BAS2NIM

The following is a primer for uCalc Transform in the context of the BAS2NIM project.  This program covers uncharted territory and may thus appear daunting before you give it your first try.  The purpose of this primer is to start you off with concepts you're familiar with, and gradually introduce unique uCalc Transform functionality, until you feel like you already knew how to use it all along.


Search feature

If you understand how to use the Ctr-F Find feature in most text editors/viewers, you can understand how to use uCalc Transform.



Chances are you are familiar with the Search feature in Notepad, MS Word, Internet Explorer, programing IDEs, and just about any kind of text editing/viewing program.  Just press Ctrl-F, and you get a text box where you can enter a word, and it will find that word in your document.  Some programs have a better search feature than others.  For instance Notepad finds just one occurrence at a time, while Internet Explorer immediately highlights all occurrences.  Internet Explorer also conveniently tells you how many occurrences there are.  Some programs also let you do advanced searches using "RegEx" patterns.



The above text editors/browsers are apps that happen to have a search feature.  On the other hand, at the most basic level you can think of uCalc Transform as an advanced search/replace program that happens to have a text editor.  The Find and Replace with boxes remain permanently visible in the program.  uCalc Transform combines some of the best search features you'll find in other text editors into one program.  Type in a word in the Find box just as you would in any program.  Instead of Enter, press one of the blue navigation buttons to navigate forward or backward through the highlighted search results.  Or in the box next to the total count, you can type in which occurrence you want to jump to.  You can use RegEx patterns if you like, but uCalc also has its own patterns that are easier to use.  More on that later.



You'll notice an additional feature above that's not typical in other text viewers.  Instead of just one search box, you can have many.  Each pattern is highlighted in a separate color, and each gets its own count.

So uCalc Transform is like Ctrl-F (Find) that you're already familiar with, but with the best features from various programs, plus a few features you won't find elsewhere.


Search/Replace

Think of uCalc Transform as a very flexible Search/Replace tool.

If you're familiar with the Find feature in many programs, you're likely also familiar with the related Find/Replace feature.  Type in a word in the Find box, and another word in the Replace box, and all occurrences of the word in the Find box will be replaced with the word in the second box.  And that's what uCalc Transform lets you do.  But just as it lets you search for more than one word at a time, it also lets you perform Search/Replace on multiple words at a time.



In a way, this is the essence of uCalc Transform.  Basically find occurrences of one or more items, and replace it with something else.  If you stop right here, and learn nothing more, you can already do a great deal with uCalc Transform.  However, a literal replacement may not be sufficient for advanced cases.  uCalc's Find/Replace does not just deal with words.  It works with patterns.


Patterns

Above, we replaced the word PRINT with the word echo.  However, that doesn't go far enough.  Lets say you wanted to replace statements such as:
Code: [Select]
PRINT "Hello!"
A = UCase$("Test")
with:
Code: [Select]
echo("Hello world!")
A = "Test".toUpper

with parenthesis and all.  Pause for a moment.  Don't peak below just yet.  Take a moment to see how to do this either with Notepad, or a more advanced text editor.

You can't?  Maybe not with Notepad, but you can at least try with other programs that support Regular Expressions (RegEx).  RegEx is nice, but it's also notorious for complexity in some cases.  uCalc patterns are generally much easier to deal with (though it supports RegEx as well).  So you might do something like:



Where {data} is just a pattern variable (we could have named it {text}, {Expr}, or anything else) that captures all tokens until it reaches a statement separator such as end of line, or the end of text.  {var}, {start}, and {stop} in the second line are also pattern variables.  The For pattern picks up text with {var} until it reaches "=".  Then it picks up more with {start} until it reaches To, and picks up the rest after that with {stop}.  Notice how corresponding variables in the Find box are featured also in the Replace box.

Here's the converted code:
Code: [Select]
echo("Hello World!")
for x in countup(1, 10):
  A = "Testing".toUpper


Skipping Occurrences

Let's say you wanted to match all occurrences of PRINT, but not when it has a # after it.  Easy!  In the next box after the first PRINT enter PRINT #, and select the Skip Over property for that one.  So when it finds PRINT # that one won't count as a match.  PRINT will be matched only when not followed by #.  By the way, the amount of whitespace between PRINT and # doesn't matter.  Also, when patterns start the same way, the parser checks for patterns furthest towards the bottom first and works its way up until it finds a match.



What if you also wanted to skip TCP PRINT?  Well add a line for that as well.



What if you want to skip COMM PRINT, TCP PRINT, and GRAPHIC PRINT?  Well you could add a separate line for each.  However, you can conveniently combine them in the same line.  There are many ways to do things in UCalc Transform.  The following are all equivalent:



- OR -



- OR -



It's up to you to decide on which style you prefer to use.


Skipping comments

There are various approaches to skipping comments.  One easy way is to do something like what we've done with the skip-over for PRINT.  Instead of using an ordinary pattern variable for this like ' {comment}, we'll use a regular expression.  We don't want any text beyond ' to be parsed in any way.  So we can do it like this:

{Comment:"'.*"}


If you're not familiar with Regular Expressions, then Google RegEx for details.

If you will be defining a transform with patterns broken down into different passes, you may want to instead define a comment as a token, and assign it the whitespace property.  That way, you won't have to define a skip-over comment pattern for each pass.  We won't get into that here.


What next

Basically that's it for the primer.  This is intended just to get you started.  There is so much more you can do.  Here's what to do next:
  • Download uCalc Transform and try it for yourself
  • Play with all the other properties beyond SkipOver
  • Check out examples in the uCalcTransform\Examples\PowerBASIC directory
  • Check out the \PBtoOtherLanguages directory as well
  • Read up on uCalc Patterns
  • Read the uCalc String Library documentation
  • Download and play with this BAS2NIM starter
  • Run Basic.Bat (a basic interpreter) found in the \Language directory
  • Examine *.uc (Basic.uc, uCalc.uc, etc) which contains the full source code for the Basic interpreter
  • Ask me questions
  • Make suggestions (I'm always trying to find ways to make uCalc easier)

IMPORTANT:  Remember to load transforms using the Open Transform button at top:

Transforms (*.uc files) are saved as plain text.  However, opening it as plain text will not be of much use.

kryton9

  • Guest
Re: uCalc Transform Primer
« Reply #1 on: October 26, 2013, 02:48:08 PM »
Wow, thanks Daniel. Am busy now, but will start with this in a couple of days. Thanks for the important post!

kryton9

  • Guest
Re: uCalc Transform Primer
« Reply #2 on: October 29, 2013, 10:25:53 PM »
Daniel, downloaded and started reading your tutorial here. Thanks again.

There is a problem that is my fault, I don't have any .net installed on my system and I don't want to install it. Just me being stupid, but I decided to avoid anything .net

I know there is mono out there for cross-platform .net development, but I have gone away from Microsoft IDE's, Lanuages etc. and trying to stay with gcc, and libraries that are supported on many platforms.

Sorry about that. I can comment however on the download and your instructions, I have read the ReadMe.txt's, that you have in many folders and they are very informative and you did a great job.

This is the second time I read in depth your instructions here and again written very well and explained so that anyone can really use this incredible utility.

You might want to consider future versions written in Nimrod so this wonderful program can be used by all programmers on many platforms.
« Last Edit: October 29, 2013, 11:11:14 PM by kryton9 »

Daniel Corbier

  • Guest
Re: uCalc Transform Primer
« Reply #3 on: October 30, 2013, 08:57:27 AM »
Thanks for the feedback.

.NET is required by the uCalc Transform app, but not by the uCalc DLL.

uCalc Transform started as a kind of demo (formerly as uCalc Search) to show what kind of practical apps can be developed using uCalc technology.  The DLL itself does not require .NET (plus once the C++ conversion is complete, uCalc won't require Windows).  The GUI interface with all the textboxes etc, is what requires .NET.  I envisioned (& still do) people making other kinds of apps that tap into the DLL for search/replace, parsing, scripting, etc, using interfaces that they design themselves, limited only by their imagination.  They wouldn't have to look the same way or use the same subset of uCalc features.  VB.NET just happened to be a convenient tool for developing the GUI interface for uCalc Transform.  But similar programs can be created without .NET.

In fact it would be great to have an online version of something like uCalc Transform.  It wouldn't need all the extra code for batch mode, file selection, etc.  I currently don't have much experience with online apps, but if someone wants to give it a try, I can offer guidance on the uCalc side of it.

Basically the info in the various text boxes are passed to the DLL with calls to ucRun() and uCalc().  I plan to simplify the process even more to make it easier for others to create their own apps.  And I'll need to write documentation about how to do it.

As far writing it in Nimrod, I found Nimrod to actually be quite interesting.  I was about to post a solution to the wordcount challenge that is more succinct using uCalc Transform until I looked at the speed of the Nimrod solution.  Execution of the uCalc Transform solution would be measured in minutes vs Nimrod's fractions of a second (word occurrence tally is the part that took minutes; total word count can already be done in less than 2 seconds).  However, there's room for speed improvement.  I think I already see what I can do to make the same operation take probably 1 or 2 seconds in an upcoming version (with room for even further optimizations later).

BTW, the current uCalc Transform also includes demos for parsing the KJV version of the Bible.  Word count, occurrence tally, etc are included.

Offline John

  • Forum Support / SB Dev
  • Posts: 3510
    • ScriptBasic Open Source Project
Re: uCalc Transform Primer
« Reply #4 on: October 30, 2013, 10:13:30 AM »
Daniel,

Migrating uCalc Transform to Nimrod and using IUP for your GUI would give you a killer cross platform solution. I know the reason you came to the party is to have fun not have it turn into a construction project but something to think about if you're concerned about the viability of the Windows platform.

John
« Last Edit: October 30, 2013, 10:15:48 AM by John »

kryton9

  • Guest
Re: uCalc Transform Primer
« Reply #5 on: October 30, 2013, 11:54:17 AM »
Daniel if you want to port this to Nimrod. I would be a willing person to help. I always think it is great to have a project to help focus on learning a new language.
And uCalc Transform is a great project if you ask me and the more people find out about it the better. Like Nimrod itself.

For a start, I will of course get my Nimrod development environment setup on windows and on linux again and work on the iup or gtk2 interface.

John, why do you prefer IUP over GTK2? I know it is smaller, but is there another reason?

Offline John

  • Forum Support / SB Dev
  • Posts: 3510
    • ScriptBasic Open Source Project
Re: uCalc Transform Primer
« Reply #6 on: October 30, 2013, 12:07:47 PM »
Quote
John, why do you prefer IUP over GTK2? I know it is smaller, but is there another reason?

IUP uses the OS native GUI API. Gtk on Windows is a kludge and not main stream Gtk supported. Just what Windows needs, another layer.  :(


kryton9

  • Guest
Re: uCalc Transform Primer
« Reply #7 on: October 30, 2013, 06:40:17 PM »
I see, I didn't notice that much of a difference when doing the nimrod tests with both before, but I was just looking and testing at a glance and not in depth usage.
Nimrod supports both, so no worries.

Offline John

  • Forum Support / SB Dev
  • Posts: 3510
    • ScriptBasic Open Source Project
Re: uCalc Transform Primer
« Reply #8 on: October 30, 2013, 06:48:57 PM »
Life is good when you're presented with options.


Before you get too excited about this, make sure Daniel is willing to open up his source to you. Doing it blind is nuts.
« Last Edit: October 30, 2013, 06:53:49 PM by John »

kryton9

  • Guest
Re: uCalc Transform Primer
« Reply #9 on: October 30, 2013, 07:26:44 PM »
I decided to focus on the C to Basic, have to make a tough choice. Even beating out my Beaglebone Black in the process. This is a project that is moving quickly and I think the more enthusiasm and participants the quicker we can have a viable alternative for everyone.

Daniel Corbier

  • Guest
Re: uCalc Transform Primer
« Reply #10 on: October 31, 2013, 06:40:27 AM »
When I say online version, I'm thinking about a form done in HTML5, which would send data to (and from) the uCalc DLL on a Windows server.  HTML5 is a widespread standard, and anyone with a tablet, smartphone, or desktop with Internet access regardless of OS, would be able to just go on a certain page and run it like an ordinary web page without installing anything.  And on the server side, you'd just need Windows, the uCalc DLLs, and the HTML5 code.  Probably the hardest part would just be deciding what colors to use, how big to make textboxes, etc., with HTML.  The calls to uCalc routines would be relatively simple.

Offline John

  • Forum Support / SB Dev
  • Posts: 3510
    • ScriptBasic Open Source Project
Re: uCalc Transform Primer
« Reply #11 on: October 31, 2013, 10:43:31 AM »
If you can just get the console mode translator showing enough of a PowerBASIC PBCC5 example being converted to C++, that should attract more folks to the project. I don't see the GUI issues (.NET, VB, ...) as keeping people from trying the tool. You will have to be the driving force with uCalc Transform, posting working conversion scripts you would like others to try with their PBCC5 code. PBWin9 > can come at a later date.