Incursion into "explorable explanations" lands

A few days ago, Nicky Case released an amazing piece of work in form of an interactive guide to alternative voting systems. This work follows the track of Brett Victor's famous research on explorable explanations and immediately caught our eyes because it is a very good match for the capabilities of our reactive native GUI system. We decided to replicate some of Nicky's interactive explanations in Red and see how it goes. The results are pretty amazing, as it was not only relatively easy to design and code (about 8 hours in total and ~280 LOC) but also great fun! Here is the result:

You can, of course, try it yourself by using the latest build from the master (or macGUI) branch or, preferably, one of the prebuilt consoles:

• Ballots demo source on github or zipped archive (171KB).
• Windows prebuilt console (273KB).
• macOS prebuilt console (256KB).
• Original web version from Nicky on github.

Put the console executable in the source code folder, run it and from the prompt, just type:

    red>> do %ballots.red

For Linux folks, sorry guys, the GUI support is not yet ready for prime time, though in the meantime, you could use Wine, Red GUI usually runs just fine on it.

Notes for macOS users:

• You need to `chmod +x` the binary before running it from the terminal.
• When dragging a face quickly, it can lag behind. The macOS GUI backend is still in a development branch, and still need some work to optimize animation latencies.

Implementation notes

The Red version implements only the "model1" and "ballot1" to "ballot4" interactive explanations. The rest could also be easily done in the same way, though we used up all our spare time for that as we are now focussing on the Red 0.6.2 release.

The colored shapes are images in the web version, but instead we chose to draw them using our 2D vector DSL in the Red version. Overall, the Red version is really small, about 25-45 LOC per screen, 85 LOC for the helper code and 2D shapes. All things considered, it is significantly smaller than the web version:

This is an interesting result, as the web version uses only plain JS, no third-party framework (with the exception of the almost weightless minipubsub). The counted JS code could maybe be shortened (excluding minification) though it already looks pretty simple to me, containing very few comments, doing mostly calculations and canvas drawing. I doubt it could be reduced significantly (any thoughts from expert JS readers about that?).

Final thoughts

This fun experiment shows that Red has the potential to be a great match for explorable explanations and other similar reactive interface needs. We hope that this demo will inspire the Red community to dig more in that direction and see what gems are lying there. For example, learning Red using such interactive visual approach would be great, don't you think so? ;-)

Entering the World of Macros

In Rebol languages family, macros have been a topic every now and then, and despite one implementation offered, they never became mainstream. Rebol language, being homoiconic, comes already well-equipped for code transformations at run-time with minimal effort. For example, this short script:

    code: [print "hello"]
    code: reduce [code]
    insert code [loop 2]
    probe code
    do code

when evaluated will output:

    [loop 2 [print "hello"]]
    hello
    hello

For a more sophisticated example, see our GUI live-coding in just a few lines.

Moreover, Rebol being interpreted, the cost of transformations is still paid at run-time.

But with Red, it is a different story. Being compiled, in addition to being interpreted, weighs heavily in favor of a preprocessing facility, in order to leverage compile-time code transformations. It is available now (in the master branch), in the form of a preprocessor with macro capabilities.

Let me restate it for the readers not familiar with macros and source preprocessing:

    The goal is to shift some data and code transformations from run-time to compile-time.

This is the key point of the new features in Red, described below.

Design and implementation

The preprocessing (including macro-time) happens between load and compile phases. It has been designed as a separate phase, with a separate execution context (to the extent allowed by current Red semantics, until we get the module! type implemented). Among other benefits, this enforces hygiene in Red macros. The preprocessor directives syntax relies on the familiar `#` prefixed forms (issue! values), meant to visually stand out from regular code, enforcing the idea of a separate layer of processing.

As Red strives to keep the source code accepted by the compiler and interpreter as interchangeable as possible, the preprocessor can also be run by the interpreter, between load and eval phases, supporting exactly the same features as the compiler version. Actually, they even share the same implementation, a unique file, with a double Rebol/Red header, run by Rebol2 for the toolchain and by Red for the interpreter. Though the preprocessor has to preprocess itself and a few core files used by the compiler setup sequence. In order to solve this chicken and egg problem, some directives have been hardcoded in the preprocessor instead of been implemented as macros.

The preprocessing (including macro expansion) is applied to the whole source fed to the compiler (or interpreter) after the load phase, so it is applied on a parse tree (similar to an AST). This means that no information about objects or other contexts built at run-time is available, it is just a big tree of Red values. Further research is planned in future releases to improve that, if possible. An alternative approach relying on interleaving macro expansion with compile/eval phases (late-time expansion) was considered, but not adopted, due to much higher implementation costs and potentially confusing inter-mixed behaviors, making it hard to grasp and be properly used by a majority of users. As we value simplicity utmostly, we will stick to the current simpler model for now, to ensure the broadest possible usage.

Simple macros

Documentation about preprocessing directives like #if, #either, #switch, #case, ... is available, and should feel familiar to Rebol SDK users, so this article will just focus on the real novel part: macro support.

Macros are basically functions, taking source code references as arguments in order to transform that source code. Most of the time, the returned value from the macro is replacing the macro name and argument at the call site in the source code. Though, this default mode can be overriden, and more control can be given to the macro through pattern-matching macros, explained later below.

For sake of brevity, the Red header is omitted from following code examples. You can copy/paste those code snippets in the console, wrapping them in a do expand [...] call or compile them (just add a Red [] header) using a `red -c -s <script.red>` line.

Simple macros can be defined in a similar way to regular functions, though they need to be preceeded by a #macro directive:

    #macro as-KB: func [n][n * 1024]
    print as-KB 64

When the macro is expanded by the preprocessor, the above source code will result in:

    print 65536

This kind of simple macro is called a named macro in Red's jargon. Once the preprocessor has finished his work, no macro definition or call, nor any preprocessor directive remains in the source code (compiler directives defined as issue! values, are still there though).

Remember that macros take source code values as arguments, without evaluating them, so passing unquoted words or paths is fine:

    #macro capitalize: function [value][
        s: uppercase/part form value 1
        either path? value [to-lit-path s][to-lit-word s]
    ]
    print capitalize hello
    print capitalize hello/macro/world

will result in:

    print 'Hello
    print 'Hello/macro/world

As you can see both func and function constructors are accepted for declaring a macro.

One possible use of macros can then be word-aliasing, for example translating words at compile-time (using French words here):

    #macro si:      func []['if]
    #macro affiche: func []['print]
    #macro vrai:    func []['true]

    si vrai [affiche "Vive Red !"]

will result in:

    if true [print "Vive Red !"]

Now, let's get back to the first example and expand (no pun intended) on it:

    #macro make-KB: func [n][n * 1024]
    #macro make-MB: func [n][make-KB make-KB n]
    print make-MB 1

will result in:

    print 1048576

So macros can freely call other macros, the same way functions would. This is possible because macros are running in a special (hidden) context, which can be accessed and extended (indirectly) by the user using the #do directive:

    #do [kilo: 1024]
    #macro make-KB: func [n][n * kilo]

The #do directive accepts arbitrary code, so local words and functions can be defined there freely, and accessed from macros. This gives a pretty powerful programming layer for the preprocessor, though, it is possible to go even further.

Pattern-matching macros

For Red users, the logical next step once having macros (as defined above), is to be able to combine them with one of the Rebol world's jewels: the Parse DSL. Well that is what pattern-matching macros are. ;-) Instead of matching a word as trigger for the macro, you can replace it with a valid Parse rule (which can call sub-rules defined in #do expressions), defining a pattern that will trigger the macro.

We can now improve our `make-KB` macro with a nicer looking form:

    #macro [number! 'KB] func [s e][to-integer s/1 * 1024]
    print 64 KB
    print 2.5 KB

would result in:

    print 65536
    print 2560

In that example, the macro gets called each time a number! followed by the word! KB is encountered in the source code. They will then both get replaced by the macro returned value. The arguments of pattern-matching macros are always the same: one reference to the starting position (s) of the matched pattern, and one reference to the ending position (e).

Here are a few more examples, showing the true power of Red macros:

Variable-arguments macro

    #macro ['max some [integer!]] func [s e][
        first maximum-of copy/part next s e
    ]
    print max 4 2 3 8 1

would result in:

    print 8

Loop macro

A simple loop macro, extending the existing loop function:

    #macro ['loop [integer! | block!] block!] function [[manual] s e][
        set [spec body] next s
        if integer? spec [return e]    ;-- return position past the pattern
        low: high: none
        step: 1

        unless parse spec [
            word! 
            opt '= set low integer!
            opt [opt 'to set high integer!]
            opt [opt 'step set step integer!]
        ][
            print ["*** LOOP syntax error:" spec]
        ]
        new: reduce either high [
            set-var: to-set-word var: spec/1
            cond: compose [(var) <= (high)]
            repend body [set-var var '+ step]
            [set-var low 'while cond body]
        ][
            ['repeat spec/1 low body]
        ]      
        change/part s new e
    ]

    loop 2 [print "x"]
    loop [i 3][print i]
    loop [i 5 8][print i]
    loop [i = 5 to 10 step 2][print i]

would result in:

    loop 2 [print "x"]
    repeat i 3 [print i]
    i: 5 while [i <= 8] [print i i: i + 1]
    i: 5 while [i <= 10] [print i i: i + 2]

This pattern-matching macro relies on the manual mode ([manual] attribute), where the replacement is done by user code and the returned value needs to be the position in the source code where the expansion process resumes. In this case, loop is also an existing function, so when the argument is an integer, the source is left untouched. With a normal macro, the resuming point would have been the beginning of the pattern, resulting in an infinite loop (again, no pun intended). ;-)


Math expressions folding macro

A powerful way to pre-calculate constant math expressions from your source code could be to use a macro like this one:

    #do [
        p:     none
        op:    ['+ | '- | '* | '** | slash]
        paren: [p: paren! :p into expr]
        fun:   [['sine | 'cosine | 'square-root] expr]
        expr:  [[number! | paren] op expr | number! | paren | fun]
    ]

    #macro [expr] func [[manual] s e][
        if all [e = next s number? s/1][return e]  ;-- skip single number
        change/part s do (copy/part s e) e
        s
    ]

    a: 3 + 2 - 8
    print (3 + 4) * 6
    edge: 100 * cosine 60
    hypotenuse: square-root (3 ** 2) + (4 ** 2)

would result in:

    a: -3
    print 42
    edge: 50.0
    hypotenuse: 5.0

HTML validating macro

HTML tags are a first-class datatype in Red so they can be embedded and manipulated by the core language (like string values). The Red lexer will check the syntax but wouldn't it be nice to also have a minimal check the semantics at compile-time? Let's roll a macro for that:

    #do [
        error: function [pos [block! paren!] msg [block!]][
            print [
                "*** HTML error:" reduce msg lf
                "*** at:" copy/part pos 4
            ]
            halt
        ]
    ]

    #macro tag! function [[manual] s e][
        stack: []
        tag: s/1

        either slash = tag/1 [
            last?: (name: next tag) = last stack
            all [
                not last?
                find stack name
                error s ["overlapping tag" tag]
            ]
            if any [empty? stack not last?][
                error s ["no opening tag for" tag]
            ]
            take/last stack
        ][
            if slash <> last tag [    
                if pos: find tag " " [tag: copy/part tag pos]

                unless find s head insert copy tag slash [
                    error s ["no closing tag for" tag]
                ]            
                append stack tag
            ]
        ]
        next s
    ]

    data: [<html><br/><b>msg<b></html>]
    msg: "Red rocks!"
    print data

The embedded HTML above has an error, the <b> tag is not closed. This macro will catch that error and report it properly during the compilation.


A DSL compiler

Red is already very well-suited for DSL creation, though, the cost of interpreting or compiling DSL has always been paid at run-time so far. With macros, it can be moved to compile-time, when suitable. Here is a simple subset of BASIC language, partially compiled to Red code using a single macro:

    #macro do-basic: function [src /local math][
        output: clear []
        lines:  clear []
        value!: [integer! | string! | word!]
        op:     ['+ | '- | '* | slash]
        comp:   ['= | '<> | '< | '> | '<= | '>=]
        line:   [p: integer! (repend lines [p/1 index? tail output])]
        cmd:    [
            token: 'let word! '= value! opt [copy math [op value!]] (
                emit reduce [to-set-word token/2 token/4]
                if math [emit/part math 2]
            )
            | 'if value! comp value! 'then (
                emit/part token 4
                emit/only make block! 1
                parent: output
                output: last parent
            ) cmd (output: parent)
            
            | 'print value! (emit/part token 2)
            | 'goto integer! (
                line: select/skip lines token/2 2
                emit compose [jump: (line)]
            )
        ]
        emit: function [value /only /part n [integer!]][
            if part [value: copy/part value n]            
            either only [append/only output value][append output value]
        ]
        
        unless parse src [some [line cmd]][
            print ["*** BASIC Syntax error at:" mold token]
        ]
        compose/deep [
            (to-set-word 'eval-basic) function [pc [block!]][
                bind pc 'pc
                while [not tail? pc][
                    do/next pc 'pc
                    if jump [
                        pc: at head pc jump
                        jump: none
                    ]
                ]
            ]
            eval-basic [(output)]
        ]
    ]

    do-basic [
        10 LET A = "hi!"
        20 LET N = 3
        30 PRINT A
        40 LET N = N - 1
        50 IF N > 0 THEN GOTO 30
    ]

will result in:

    eval-basic: function [pc [block!]][
        bind pc 'pc
        while [not tail? pc][
            do/next pc 'pc
            if jump [
                pc: at head pc jump
                jump: none
            ]
        ]
    ]
    eval-basic [
        A: "hi!"
        N: 3
        PRINT A
        N: N - 1
        IF N > 0 [jump: 5]
    ]

Note: I tried to keep this example short, so it ends up as a BASIC source code compiler to Red, but fed to a custom interpreter. Fully compiling that DSL to Red code would be possible, but would require more complex constructs, in order to deal with a GOTO able to arbitrarily jump anywhere, and that is beyond the scope of this article.

Final thoughts

The preprocessor and macros bring great new possibilities to the Red compiler, while still being able to run the same code with the interpreter. Though, as the saying goes, with great power comes great responsibility, so keeping in mind some drawbacks would be wise:

• As there is no difference in Red between code and data, both can be transformed by the same macros, which is not always desirable. Some mechanisms for limiting the application scope exist in the preprocessor, though there is no guarantee they can cover all use-cases. Stay alert, especially with pattern-matching macros.

• It is not always easy to reason about and debug macros, unless you have some existing experience. I would suggest not using them until you have a good grasp of Red's toolchain, semantics and fundamental concepts (like homoiconicity).

• As the Red toolchain is currently run by a Rebol2 interpreter, Rebol is running the compile-time macros, so keep that in mind when writing them. If you want them to run equally well on the interpreter, you need to use only the common subset between Rebol2 and Red. Sooner or later we should move compile-time preprocessing to use a Red engine (thanks to libRed), so this concern is temporary.

Last but not least, for those wondering about syntactic macros (aka readers macros) inclusion in Red, as for AST-macros, they are not strictly necessary, as the Parse DSL already provides us a powerful tool to implement pre-load-time processing. Though they could still bring some extra benefits (like embedding the processing logic within the source code), but could also go against the Red-as-data-format principle, or wreak havoc in IDE support (like messing up syntax coloring and step by step debugging). We need more time to go through each concern and see how to deal with them before adding such a feature.

I hope this long article was useful for those of you who had no past experience with macros and entertaining for those who have. Have fun with this brand new toy and let us know what you think about it on Gitter. Cheers!

See comments on /r/programming.

Eve-style clock demo in Red, livecoded!

Like many others, we are fans of Chris Granger's work on Lighttable and more recently on Eve. The Eve project shares some similar goals with Red, though taking a different road by giving a new try at visual programming, while Red still tries to push further the limits of textual representation. A few days ago, the Eve team tweeted a nice small clock demo using their framework. Here is our own version of that demo using Red and our native reactive GUI (Eve relies on a web engine):

The Eve demo has a "compile and run" button, which we thought was not necessary in our case, as we can easily build a livecoding editor in few lines. ;-)

Here is the full source of our demo. Use our latest Windows build to run it from the GUI console:

    clock-demo: {
    base 200x200 transparent rate 1 now draw [
        scale 2 2
        fill-pen #0B79CE pen off
        circle 50x50 45
        line-width 2
        hour: rotate 0 50x50 [pen #023963 line 50x50 50x20]
        min:  rotate 0 50x50 [pen #023963 line 50x50 50x10]
        sec:  rotate 0 50x50 [pen #CE0B46 line 50x50 50x10]
    ] on-time [
        time: now/time
        hour/2: 30 * time/hour
        min/2:  6  * time/minute
        sec/2:  6  * time/second
    ]
    }

    system/view/silent?: yes

    view [
        title "Eve clock demo"
        backdrop #2C3339
        across

        source: area #13181E 410x300 no-border clock-demo font [
            name: "Consolas"
            size: 9
            color: hex-to-rgb #9EBACB
        ]

        panel 200x300 #2C3339 react [
            attempt/safer [face/pane: layout/tight/only load source/text]
        ]
    ]

As you can see the clock demo code is held in a string, which will be used to feed an area widget that we use as a code editor. Each change in the area triggers a reaction which will try to interpret the code as VID dialect using thelayout function and feed the panel's content with it. We could have used a simple on-change handler here, but react is so simple and versatile, that we can forget about events in many cases.

Yes, livecoding (using native widgets!) in Red can be that simple. As you can see, there's no built-in "livecode" widget or feature, it's an emergent behavior resulting from the combination of existing Red features, homoiconicity being the most fundamental.

As a sidenote, the code above could be improved to avoid some light flickering.

If you want to see more Red feats and get more info about Red, check out:

• our native spreadsheet in 17 LOC
• our reactive framework (not limited to GUI domain)
• our very flexible native GUI system
• our built-in powerful PEG parser DSL
• our built-in system programming DSL

Red is only at Alpha 0.6.1, and there is significant work still to do. Though, you can already imagine the sheer power that will be into your hands by the time Red reaches 1.0. ;-)

Cheers!

Native reactive spreadsheet in 17 LOC

After the release of our reactive framework a few days ago, we though it would be a good idea to implement the, often mentioned, spreadsheet model as a small demo, just to see how much it would take to do it in Red, with its current feature-set. Well, despite not having a grid component, it turns out that 17 LOC (of packed, but still readable code; down to 14 LOC and 1053 bytes if minified) is enough to make a spreadsheet-like demo with native widgets and realtime updating of dependent cells as-you-type! ;-)

Red [] L: charset "ABCDEFGHI" D: union N: charset "123456789" charset "0" 
repeat y 9 [repeat x 9 [col: either x = 1 [#"^(2028)"][#"A" + (x - 2)]
  append p: [] set ref: (to word! rejoin [col y - 1]) make face! [size: 90x24
    type:    pick [text field] header?: (y = 1) or (x = 1)
    offset:  -20x10 + as-pair ((x - 1) * size/x + 2) ((y - 1) * size/y + 1)
    text:    form case [y = 1 [col] x = 1 [y - 1] 'else [copy ""]]
    para:    make para! [align: pick [center right] header?]
    extra:   object [name: form ref formula: old: none]
    actors:  context [on-create: on-unfocus: function [f e][f/color: none
      if rel: f/extra/old [react/unlink rel 'all]
      if #"=" = first f/extra/formula: copy text: copy f/text [parse remove text
          [any [p: L N not ["/" skip not N] insert p " " insert "/data "
          | L skip | p: some D opt [dot some D] insert p " " insert " " | skip]]
        f/text: rejoin [f/extra/name "/data: any [math/safe [" text {] "#UND"]}]
       if f/data [any [react f/extra/old: f/data do f/data]]]]
      on-focus: func [f e][f/text: any [f/extra/formula f/text] f/color: yello]
]]]] view make face! [type: 'window text: "PicoSheet" size: 840x250 pane: p]

You can copy/paste the above code into the Red console for Windows, using the latest toolchain build (950 KB), or, better, using this prebuilt console version (247 KB, requires Windows 7+). Yeah, we still count in KB. ;-)

Features:

• 100% native widgets using our built-in GUI engine (no third-party libraries, Windows only for now, OSX and GTK are coming).
• Support for arbitrary Excel-style formulas (=A1+2*C3).
• Support for arbitrary Red code in formulas.
• Realtime updating of dependent cells as you type.
• While editing a formula, dependent cells will display #UND (for "undefined").
• If a formula is syntactically incorrect, #UND is displayed in the cell.
• Code is packed to reduce the LOC number, but limited to 82 characters per line (could fit on 77 if indentation is removed).
• It takes about 6 LOC to build the spreadsheet and 3 LOC to compile the formulas to Red expressions.
• One expression per line (expressions can have nested expressions), Red's header not counting as an expression for the purpose of this demo, nor the last expression at line 16 for setting the yellow color on focus, it is just there to make the animated captures easier to follow.
• Not using our VID dialect for GUI, such version is left as an exercise to the reader. ;-)

Here is a capture of how it works:

If you want to play with the same dataset, use this script.

This other session shows how to leverage the rich datatypes of Red, to play with, in the spreadsheet. It also shows that you can access the face objects properties from within the cells and modify them directly:

If you want to play with the same dataset, use this script.

Those captures were done on Windows, which is currently the most advanced GUI backend we have, our OSX and GTK backends are still a work in progress.

This demo was inspired by a similar one written in Tcl/tk which weights 30 LOC only, but takes advantage of a built-in grid component, and a C-like expressions parsing/evaluating library called expr. Though, it is still impressive to see what Tcl/tk can achieve. But the real king there, is the JS 220 bytes demo, even if it is more a testimony to the DOM capabilities (with a 100MB+ runtime behind) than JS expressivness. Nevertheless, Red's demo is the smallest one we could find in the native GUI category. Even the executable footprint is minimal. Once compiled (just insert Needs: View in the header in such case), it weights 655 KB, which can be further compressed down to just 221 KB, and as usual, zero dependency.

The above source code is very packed to fit in as less lines as possible, though it is still readable, as it is really hard to obfuscate Red code, even when you want to (mandatory spaces between tokens prevent from reaching C-like extremes). Therefore, you will hardly win a codegolf competition where each byte counts...unless you leverage Red's DSL abilities and write one optimized towards such goal.


How does it work?

It relies on our Red/View GUI engine, the reactive framework, the Parse DSL and the core Red language, which is, for those hearing about it for the first time, a Rebol language descendent, with one of the highest expressiveness among programming languages.

For the ones interested in the details of the above code, you can find a more readable version here and what follows is a detailed explanation. This is actually much simpler than it looks, here we go:

Line 1

 L: charset "ABCDEFGHI" D: union N: charset "123456789" charset "0" 

Skipping the Red [] header, it starts by defining a few bitsets, which will be used for the parsing operations. We create the D charset by combining N and "0", which save space.

Line 2

 repeat y 9 [repeat x 9 [col: either x = 1 [#"^(2028)"][#"A" + (x - 2)] 

A double loop is used to produce all the widgets needed. col is set to a space character if the column is a header, or to a letter starting from A to G. It will be used to create the cell names and the first row labels.

Line 3

 append p: [] set ref: (to word! rejoin [col y - 1]) make face! [size: 90x24 

Here we start building the faces which will be accumulated in p block. p: [] is a static allocation that conveniently avoids using a separate line to define p. The set ref: (to word! rejoin [col y - 1]) part is transparent, and let the face produced by make face! be appended to the p list. That transparent expression creates the cell name (in form of a capital letter denoting the column, and a number for the row), which is converted to a word, that gets set to the newly created face. Those words are necessary for supporting the spreadsheet formulas. Last, the opening block for the face definition leaves an option to append a nested expression, size definition being the shortest of all the other property definitions, is a good fit for that.

Line 4

 type:    pick [text field] header?: (y = 1) or (x = 1)

The face type can be a text for the first row/column and a field otherwise. The header? word will be useful further in the code, to indicate if the cell is a just label or a field. If you wonder why the use of or instead of the idiomatic any, it is to avoid an expensive conversion to logic!, as required by pick in such use-case.

Line 5

 offset:  -20x10 + as-pair ((x - 1) * size/x + 2) ((y - 1) * size/y + 1) 

The face position is calculated using the x and y values to set up a grid, which is sligtly moved to the left for (subjective) minor look improvement.

Line 6

 text:    form case [y = 1 [col] x = 1 [y - 1] 'else [copy ""]] 

The face content is set to col which contains column's label, or row number, or otherwise an empty string for input cells.

Line 7

 para:    make para! [align: pick [center right] header?] 

The face para object is just used there to center the header labels while keeping the cell content right-aligned.

Line 8

 extra:   object [name: form ref formula: old: none] 

The extra field is populated with an object which holds the state of the cell, namely:

• name: name of the cell, in string format for easier usage in the formulas compiler.
• formula: keeps a reference to the last entered formula, in text format, as typed by the user.
• old: keeps a reference of the last reaction set by the cell's formula (or none).

Line 9

 actors:  context [on-create: on-unfocus: function [f e][f/color: none 

The cell definition is almost done, just remain the event handlers, which we start defining from this line. on-create is called when the cell is created, ensuring that the preset content will be properly processed (in case of a formula) before showing it for the first time. on-unfocus is the main way to trigger the user's input processing. on-enter was not used, as the tabbing support is not working currently, so pressing Enter key will keep the focus on the same cell, causing unwanted side-effects which would take several lines to workaround. Once proper tabbing will be there, we could add it too. Last, as the function's body block is opening, we can squeeze in a short expression, which just resets the background color of the cell to its default.

Line 10

 if rel: f/extra/old [react/unlink rel 'all] 

We start with the hot stuff now. If a previous formula did produce a reaction, we first destroy it.

Line 11

 if #"=" = first f/extra/formula: copy text: copy f/text [parse remove text 

If a formula is detected, we copy first the content in text, which will be used for the transformation to a Red expression. As series are owned by deep reactors (a face! object is one), the copy will ensure that no object events are produced during the transformation steps. A second copy creates another instance of the input string to be referenced by extra/formula. In case it is not a formula (all that is done before the test succeeds, it will have no effect on the cell content (just wasting some memory, but that's not what we optimize for, in this exercise). Last, we start the transformation of the input text if it's a formula, using a Parse rule, applied to text with the leading equal sign removed.

Line 12

 [any [p: L N not ["/" skip not N] insert p " " insert "/data " 

The rule starts with a loop, the goal is to spot all the cell names and insert a space before it and /data just after it ("A1" becomes " A1/data "). The not ["/" skip not N] rule is there to avoid transforming cell names followed by a face property (e.g. A1/color). It works by ensuring that the second character after the slash is not a number, allowing to still transform inputs like A1/B2 (A1 divided by B2).

Line 13

 | L skip | p: some D opt [dot some D] insert p " " insert " " | skip]] 

If the input is not a cell name, we search for numbers (some D) including number with decimals (opt [dot some D]), so we can insert a space before and after (e.g "1+2" become " 1 + 2 "), in order to enforce Red's syntactic rules (as we will LOAD that string later). The | L skip part is there to avoid injecting spaces to numbers with leading signs ("-123" would not be touched). The final skip rule just skips every other character we are not interested in.

Line 14

 f/text: rejoin [f/extra/name "/data: any [math/safe [" text {] "#UND"]}] 

The transformation is almost done, the last step is decorating properly the text to generate the Red expression we are aiming for. First we enclose the resulting expression from last step in a math/safe [...] block. The math function just ensures that math precedence rules are enforced, while /safe option evaluates the code using attempt internally, so any error will be returned as a none value (and in such case, the "#UND" string is used). The result of that evaluation is set the the current cell. So for an input formula like: "=A1+B1" in C1 cell, we get as result of the transformation process:
 "C1/data: any [math/safe [ A1/data + B1/data ] "#UND"]", which is a LOADable expression in string format. But LOAD is not used in the demo code? Well, it is, thanks to a new feature in 0.6.1 release: by default the /text property of a field is synchronized in realtime with its /data property, using a LOAD call. If it fails, /data is set to none value. Conversely, setting /data will change /text value at once using a FORM call. Well, that's what the resulting expression is meant to leverage. ;-)

Line 15

 if f/data [any [react f/extra/old: f/data do f/data]]]] 

Now take a deep breath as we reach the crux of the matter. The previous line set f/text, which, at once created a LOADed version of that string, referred by f/data. If the LOADing failed, f/data would be set to none and then we just exit the event handler. Otherwise, we have something we can use as the input to REACT for trying to set up a new reactive relation for that cell. That's where the "/data" injection for the cell names in previous steps, becomes useful. Those path! values are statically analyzed by REACT to determine the reactive sources. Though, if no reactive source has been found in the expression (e.g. "=1+2" which would give [C1/data: any [math/safe [ 1 + 2 ]]] in f/data), REACT returns none and we then can simply evaluate the expression, which would assign the result to the current cell /data (hence to /text, making it visible to the user). If REACT succeeded, we have set a new reactive relation for that cell, and by default, the reaction is run once on creation, ensuring that our cell gets the correct visual value (by indirectly setting /data, as usual now). Moreover, we save in extra/old a reference to the expression we used for creating the reactive relation, as we'll need to destroy if the user inputs a new formula. If you're still following, at this point, congrats, you can consider yourself a master of both View and the reactive framework. ;-)

Line 16

 on-focus: func [f e][f/text: any [f/extra/formula f/text] f/color: yello] 

The second event handler is used to restore the saved formula (if any) in the cell, when the user gives it the focus. We also then set the background color to yellow, which is...well, like yellow color, but a bit less yellow...hence the truncated name for an otherwise anonymous color. (Carl, if you're reading this, I hope you appreciate my tap-dancing around your, sometimes, creative naming schemes. ;-))

Line 17

 ]]]] view make face! [type: 'window text: "PicoSheet" size: 840x250 pane: p] 

This last line is just creating a window, assigning the list of previously created labels and fields to the /pane property (face's children), then displaying it while entering an event loop using view call. That's all folks!

Last thoughts

We hope this demo and explanations were both entertaining and informative. Spreadsheet applications are not your common app, they are special. They are a unique combination of incredibly useful and powerful features, while at the same time being usable by pretty much anyone with basic computer skills. Many consider them as the culmination of our industrial software world, Microsoft's CEO itself declared a few days ago, that Excel was the best product his company ever made.

As Red allows you to create such software in a convenient and straightforward way, using native technologies, we hope this will inspire some of you to invest more time learning Red and to create some amazing software with it!

Beyond the simple fun provided by this demo, it also shows the potential of Red in the native GUI apps domain (we're just at 0.6.1, we have many more features planned and platforms to support). In the big struggle between native vs web solutions, you can expect Red to become, someday, an option to account for.

In the meantime... have fun with Red, as much as we do! ;-)

0.6.1: Reactive Programming

Despite being a minor release, 0.6.1 still weighs a heavy 588 commits with a big number of fixes and many additions, among which the new reactive framework is the most notable.

Last release introduced, for the first time in the Rebol world, a reactive programming framework, as a part of the GUI engine. While working on improving it, we realized that it could actually be easily generalized beyond GUIs, with just minor changes to its design and implementation.

What is reactive programming?

Let me make a short disclaimer first, this is not yet-another-FRP framework relying on event streams. The reactive model we use is known as object-oriented reactive programming (using a "push" model), which is both simple to understand and close to spreadsheet's model (i.e. Excel formulas). That model has often been praised for its simplicity and efficiency. You can now use it directly in Red.

So, in practice, what is it?  It is a way to link one or more object fields to other fields or global words, by specifying relationships in a block of code (can be a single expression or a complex multi-step computation). Each time a source field value changes, the target value is immediatly updated, you don't have to call a function for that, it's pretty much define-and-forget. ;-) Here's a simple example in Red:

    red>> a: make reactor! [x: 1 y: 2 total: is [x + y]]
    == make object! [
        x: 1
        y: 2
        total: 3
    ]
    red>> a/x: 5
    == 5
    red>> a/total
    == 7
    red>> a/y: 10
    == 10
    red>> a/total
    == 15

In that example, the is infix function is used to create a reactive relation between the total field and the x and y ones using a simple formula. Once set, total is refreshed automatically and asynchronously each time the other fields are changed, regardless how, where or when they are changed! It's the same concept as spreadsheet cells and formulas, just applied to object fields.

This reactive programming style belongs to the dataflow programming paradigm. It doesn't enable you to write code, that you wouldn't otherwise be able to write in an imperative style. Though, it helps reduce the size and complexity of your code, by abstracting away the "how" and helping you focusing more on the "what" (not dissimilar to FP). The gains of such approach become significant when you chain together many relations, creating graphs of, more or less complex dependencies. GUI programming is where it shines the most, as nodes are visible objects, and reactions produce visible effects.

Here is a comparative example with a reactive GUI vs the non-reactive version:

Let's make a simple native GUI app using VID, Red's graphic DSL (we call it a dialect). It will just provide 3 sliders, which control the R, G, B components of the box's background color.

The reactive version:

    to-int: function [value [percent!]][to integer! 255 * value]
    
    view [
        below
        r: slider
        g: slider
        b: slider
        base react [
            face/color: as-color to-int r/data to-int g/data to-int b/data
        ]
    ]

The non-reactive version:

    to-int: function [value [percent!]][to integer! 255 * value]
    
    view [
        below
        slider on-change [box/color/1: to-int face/data]
        slider on-change [box/color/2: to-int face/data]
        slider on-change [box/color/3: to-int face/data]
        box: base black
    ]

What can we say about the non-reactive version?

1. Size is pretty much the same, though the non-reactive version has more expressions and code looks denser.
2. The updating code is spread over 3 event handlers.
3. The face word in each handler refers to the widget, so we can remove the slider names (very minor gain though).
4. The box face needs a name (`box`), so it can be referred to, from the event handlers.
5. The box face default color is grey, so it needs a `black` keyword to force it to the right default color (as the sliders are all at position 0 on start). The reactive version sets the right color on start, no need to care about it.

Even in this simple example, we can see that the complexity, and the cognitive load are higher in the non-reactive version. The more relationships can be modeled using reactions in a GUI app, the higher the gains from using the reactive approach.


Red reactive framework

Red's reactive framework is just ~250 LOC long, and written purely in Red (no Red/System). It relies on object events (equivalent to observables in OO languages) and the ownership system (which will be properly documented once completed in one or two releases time). Rebol does not offer any past experience in such domain to guide us, so it should still be considered experimental, and we need to put it to the test in the wild, to study the pros/cons in real-world applications. We are quite excited to see what Red users will create with it.

Full documentation for the reactive framework is available here. It also explains the important difference between static and dynamic relations.

In a nutshell, the reactive API provides 4 functions (quite big API by our standards):

• react to create or remove reactions.
• is infix function for creating reactions which result will be assigned.
• react? to check if an object's field is a reactive source.
• clear-reactions to remove all existing reactions.

Moreover, react is directly supported as a keyword from VID dialect. That's all you need! ;-)

Here is a simple demo linking together a couple dozen balls, following each other. Source code is available here.

Let's now have a look at other features brought by this release.

Time! datatype

A time! datatype is now included in Red, supporting already a broad range of features, like:

• Path accessors: /hour, /minute, /second.
• Math operators, including mixing with other scalar types.
• All comparison operators.
• Actions: negate, remainder, random, pick.

    red>> t: now/time
    == 12:41:52
    red>> t + 0:20:00
    == 13:01:52
    red>> t/second
    == 52.0
    red>> t/hour: t/hour - 5
    == 7
    red>> t
    == 7:41:52

GUI changes

Two main additions to our View engine have enabled the writing, or porting, of some nice graphic demos (thanks to Gregg Irwin for the awesome demos!). Here are a few examples:

Bubble demo

Gradient Lab

Particles demo

View engine changes

• New time event in View, triggered by timers.
• New rate facet in face! objects for setting timers.
• move action allows to move faces between panes in a non-destructive way.
• Adds support for event/window property.
• data syncing with text for field and text faces.
• Add default option for fields (e.g. options: [default 0]).
• at, skip, pick, poke, copy on image! now accept pair! index argument.
• Added /argb refinement for image! datatype.
• Added request-font dialog.
• Improved size-text native.
• GUI console faces are now excluded from the View debug logs.

Draw dialect changes

• fill-pen has been extended to support color gradients.
• pen accepts `off` as argument now, to make the subsequent pen-related operations invisible.
• Allows box to accept edges in reverse order.
• Radius of circle now accepts a float! value.
• Added key-color support for to image command.

VID dialect changes

• Added rate keyword for setting timers.
• do command now support `self` to refer to container face (window or panel).
• Added focus option to faces for presetting focus.
• Added select option support to preselect an item in a list (using an integer index).
• Added default option support for field and text faces' default data facet value.
• Added support for get-words to pass an handler function as an actor.
• Adding glass and transparent color definitions.

The red/code repository has also been filled with more demos using the new features, like color gradients and timers.

Other changes

New actions: change, move

New natives: remove-each, new-line, new-line?, set-env, get-env, list-env, context?, enbase, now/time, browse

New functions: repend, overlap?, offset?, any-list?, number?, react, is, react?, clear-reactions, dump-reactions, make-dir, request-font, time?

Parse improvements

• Added change command.
• remove also accepts, now, a position argument.
• Support for parsing binary! series.
• Several bugs fixed.

Syntax for change command:

• CHANGE rule value
• CHANGE ONLY rule value
• CHANGE rule (expression)
• CHANGE ONLY rule (expression)
• CHANGE pos value
• CHANGE ONLY pos value
• CHANGE pos (expression)
• CHANGE ONLY pos (expression)

Example using rule syntax:

    a: "12abc34"
    alpha: charset [#"a" - #"z"]
    parse a [some [to alpha change [some alpha] dot]]
    a = "12.34"

Example using pos syntax:

    a: "12abc34"
    alpha: charset [#"a" - #"z"]
    parse a [some [to alpha b: some alpha change b dot]]
    a = "12.34"

Console improvements

• Filenames completion using TAB key.
• Font and color settings from new menu bar.
• Ctrl-K will erase to end of line (CLI console).
• Ctrl-D will remove character or exit like Ctrl-C if empty line (CLI console).
• Optimized speed of pasted code in console.

Other improvements

• Allows bitsets to be used as search pattern for find on any-string! series.
• /next refinement support for do and load.
• /seek and /part refinements added to read.
• Added any-list! typeset.
• Added /with refinement to pad function.
• Improved split function (though not final version).
• Added LF <=> CRLF conversions support to UTF-16 codec.
• input can now read from stdin when run from  a child process.
• Added /same refinement into find and select actions.
• Added binary! support for data and HMAC key to checksum.
• Reduced emitted code for setting struct members to float literals on IA-32.
• Allows owned property to be used by modify on objects.
• Compiler now accepts creating global op! values from object's functions.

A big number of tickets have also been processed, 110 bug fixes have been provided since 0.6.0 release. We have about 10% of open tickets which is more than usual, though not surprising after the last huge release, but only 22 are actual bugs. Thanks for all the contributors who reported the issues and helped fix them, Red owes you a lot!

What's next?

On the road to Android support, we need to be able to properly wrap a Red app in a shared library, which is the main focus for the next release. Moreover, being able to build the Red runtime library only once, will greatly reduce compilation time (the runtime library is currently rebuilt on each compilation). As the work on this new feature is already quite advanced, we expect next release to occur during July, even if we always favor quality over arbitrary deadlines. ;-)


In the meantime, enjoy the new release!

0.6.0: Red GUI system

Five years ago, when I started writing the first lines of code of what would become later the Red/System compiler, I had a pretty good picture already of what I wanted to achieve with Red, and all the ideal features that should be included, just not sure how much time and efforts it would require to have them. Two years and half ago, baby Red printed its first output. And today, we celebrate a major step forward with the addition of a brand new GUI system entirely written in Red itself! What a journey!

Here it is, the long awaited 0.6.0 release with its massive 1540 commits! The major new features are:

• View engine, with Windows backend (from XP to 10)
• VID dialect
• Draw dialect
• Reactive GUI programming
• GUI console
• Simple I/O support for files and HTTP(S) queries.
• Codecs system with following codecs available: BMP, GIF, JPEG, PNG
• Objects ownership system

All those additions made our Red executable grow from 767 KB to 910 KB (Windows platform), sorry for the extra 143 KB, we will try to [red]uce that in the future. ;-)

Let's start with the elephant in the room first, the GUI system. Here is an architecture overview:

Only the Windows backend is fully usable for now, Android and OS X are work-in-progress, Linux (using GTK) will follow soon, iOS will come later this year. Also, we have other targets in mind, like JS/HTML which are not scheduled yet, but could come this year too.

Red/View

First let me mention that View, VID and Draw were invented by Carl Sassenrath (of Amiga fame) in the Rebol language, a long time ago. Red's version retains all the best features and pushes the boundaries of simplicity even further. The main features of our View engine are:

• A live updating mode that reduces the need to a single view function in most cases.
• Full abstraction over rendering backends.
• Two-way binding using live objects.
• Event bubbling/capturing stages.
• Built-in drag'n drop support for most face types.
• Gestures support (experimental).
• Native widgets support.
• Full integration with the OS features.
• Flexible backend support that can be mapped to virtually any kind of UI library.

The current list of supported widgets is: base, text, button, check, radio, field, area, text-list, drop-list, drop-down, progress, slider, camera, panel, tab-panel, group-box.

Next releases will bring more widgets, like: table, tree, divider, date/time picker, web-view and many others!

For more info about View, see the View reference document.

Main differences between Red/View and Rebol/View are:

• Red relies on native widgets, Rebol has custom ones only, built over a 2D vector library.
• Red faces are synchronized with their widgets on display in realtime by default, Rebol faces require manual calls to many functions for keeping faces and widget updated.
• Red introduces reactive GUI programming.

Red/View will update both face and graphic objects in realtime as their properties are changed. This is the default behavior, but it can be switched off, when full control over screen updates is desirable. This is achieved by:

    system/view/auto-sync?: off

When automatic syncing is turned off, you need to use show function on faces to get the graphic objects updated on screen.

VID dialect

VID stands for Visual Interface Dialect. It is a dialect of Red which drastically simplifies GUI construction. VID code is dynamically compiled to a tree of faces, feeding the View engine. You can then manipulate the face objects at runtime to achieve dynamic behaviors. VID offers:

• Declarative programming.
• Automatic layout system.
• Cascading styles.
• Default values for...everything.

For more info about VID, see the specification.

In case you are reading about Red or Rebol for the first time, here are a few code demos to show how simple, yet efficient, is our approach to GUI programming:

A GUI Hello word

    view [text "Hello World"]

Greet the name you type in the field

    view [name: field button "Hi" [print ["Hi" name/text]]]

A simple reactive relations demo, drag the logo around to see the effect

   view [
        size 300x300
        img: image loose http://static.red-lang.org/red-logo.png
        return
        shade: slider 0%
        return
        text bold font-size 14 center "000x000" 
            react [face/text: form img/offset]
            react [face/font/color: white * shade/data]
    ]
 

Simple form editing/validating/saving with styles definitions

   digit: charset "0123456789"
    view [
        style label: text bold right 40
        style err-msg: text font-color red hidden
    
        group-box "Person" 2 [
            origin 20x20
            label "Name" name: field 150
            label "Age"  age:  field 40
            label "City" city: field 150
            err-msg "Age needs to be a number!" react later [
                face/visible?: not parse age/text [any digit]
            ]
        ]
        button "Save" [save %person.txt reduce [name/text age/text city/text]]
    ]
    set [name age city] load %person.txt
    ?? name ?? age ?? city

You can run all those examples by copy/pasting them one-by-one into the Red console for Windows. To get the console, just download it and double-click the Red binary, wait ~20 seconds for the console to be compiled for your OS (yes, that little file contains the full Red toolchain, runtime library and console source code), paste the code and have fun. ;-)

Draw dialect

Draw is a 2D vector-drawing dialect which can be used directly, to render on an image, in faces for local rendering, or specified through VID. It is still a work in progress as not all features are there yet. We aim at full Rebol/Draw coverage and full SVG compatibility in the not-too-distant future.

A simple example of Draw usage:

    shield: [
        fill-pen red   circle 50x50 50
        pen gray
        fill-pen white circle 50x50 40
        fill-pen red   circle 50x50 30
        fill-pen blue  circle 50x50 20
        
        pen blue fill-pen white
        polygon 32x44 45x44 50x30 55x44 68x44 57x53 60x66 50x58 39x66 43x53
    ]
    
    ;-- Draw in a draggable face, in realtime.
    view [
        size 300x300
        img: box 101x101 draw shield loose
        at 200x200 base white bold react [
            [img/offset]
            over?: overlap? face img
            face/color: get pick [red white] over?
            face/text: pick ["Hit!" ""] over?
        ]
        button "Hulk-ize!" [replace/all shield 'red 'green]
        button "Restore"   [replace/all shield 'green 'red]
    ]

Copy/paste the above code example in a Red console on Windows, and become an Avenger too! ;-)

For more info about Draw, see the specification.

Main differences between Red/Draw and Rebol/Draw:

• Red does not yet cover all the commands of Rebol/Draw yet.
• Red's version allows commands to be grouped in blocks, ease-ing insertion/removal at run-time.
• Red's version allows commands to be prefixed with a set-word, allowing to save local position in Draw blocks in a word.

Reactive GUI programming

This is a deep topic which should be part of a future separate blog article. So, I will just copy/paste here the little information already in the VID documentation:

Reactions (or reactors, not sure yet which terms is the most accurate) are created using the react keyword, directly from Red code or from VID dialect. The syntax is:

    react [<body>]
    
    <body> : regular Red code (block!).

This creates a new reactor from the body block. When react is used in VID, as a face option, the body can refer to the current face using face word. When react is used globally, target faces need to be accessed using a name.

Reactors are part of the reactive programming support in View, whose documentation is pending. In a nutshell, the body block can describe one or more relations between faces properties using paths. Set-path setting a face property are processed as target of the reactor (the face to update), while path accessing a face property are processed as source of the reactor (a change on a source triggers a refresh of the reactor's code).

Basically, it is about statically-defined relations between faces properties, without caring when or how the reactive expressions will be evaluated. It will happen automatically, when needed. Here are a couple of examples you can copy/paste in the Red console on Windows:

Make a circle size change according to slider's position:

    view [
        sld: slider return
        base 200x200 
            draw  [circle 100x100 5]
            react [face/draw/3: to integer! 100 * sld/data]
    ]

Change the color of a box and a text using 3 sliders:

    to-color: function [r g b][
        color: 0.0.0
        if r [color/1: to integer! 256 * r]
        if g [color/2: to integer! 256 * g]
        if b [color/3: to integer! 256 * b]
        color
    ]

    to-text: function [val][form to integer! 0.5 + 255 * any [val 0]]

    view [
      style txt: text 40 right
      style value: text "0" 30 right bold
    
      across
      txt "Red:"   R: slider 256 value react [face/text: to-text R/data] return
      txt "Green:" G: slider 256 value react [face/text: to-text G/data] return
      txt "Blue:"  B: slider 256 value react [face/text: to-text B/data]
    
      pad 0x-65 box: base react [face/color: to-color R/data G/data B/data]
      return
    
      pad 0x20 text "The quick brown fox jumps over the lazy dog." font-size 14
        react [face/font/color: box/color]
    ]

GUI console

We have a GUI console now, in addition to the existing CLI one!

The GUI console is now the default on Windows platform, and is fully Unicode-aware. The system shell (DOS) console is still available using --cli option:

    red --cli

The GUI console is still in its infancy and will be enhanced a lot in future releases. Anyway, so far, it already supports:

• history of commands
• completion on words and object paths
• multi-line editing for blocks, parens, strings, maps and binaries.
• navigation using HOME and END keys
• select/copy/paste using the mouse and keyboard shortcuts
• auto-scrolling when selecting with the mouse out of the boundaries
• very fast text rendering
• automatic vertical scroll bar
• customizable prompt

Try this cool one-liner for making the prompt more active:

    system/console/prompt: does [append to-local-file system/options/path "> "]

This is how the GUI console looks like:

Simple I/O support

In order to have really some fun with the GUI, we have added some minimal support for blocking IO basic actions covering files and HTTP(S) requests. read and write action are available now. Their /binary, /lines and /info refinement are working. do, load, save have also been extended to work with files and urls.

When not using /binary, read and write are expecting UTF-8 encoded data. Support for ISO8859-1 and other common encoding formats will be available in next release.

The full IO will come in 0.7.0 with ports, full networking, async support and many more features.

Codecs

Codec system support has made his entrance in this release. It is a very thin layer of encoders/decoders for binary data, integrated with load, save and actions which rely on /as refinement. load and save will auto-detect the required encoding format and try to apply the right encoder or decoder on the data.

Currently only image format codecs are provided: BMP, PNG, GIF, JPEG. Any kind of encoding (related to IO) is a good candidate for becoming a codec, so expect a lot of them available in the future (both built-in Red runtime and optionaly installable).

For example, downloading a PNG image in memory, and using it is as simple as:

    logo: load http://static.red-lang.org/red-logo.png
     
    big: make font! [name: "Comic" size: 20 color: black]
    draw logo [font big text 10x30 "Red"]
    view [image logo]

Saving a downloaded file locally:

    write/binary %logo.png read/binary http://static.red-lang.org/red-logo.png

Saving images is not fully functional yet, PNG should be safe though.

Objects ownership system

Red's objects ownership system is an extension of object's event support introduced in previous releases. Now, an object can own series it references, even nested ones. When an owned series is changed, the owner object is notified and its on-deep-change* function will be called if available, allowing the object to react appropriately to any change.

The prototype for on-deep-change* is:

    on-deep-change*: func [owner word target action new index part][...]

The arguments are:

• owner: object receiving the event (object!)
• word: object's word referring to the changed series or nested series (word!)
• target: the changed series (any-series!)
• action: name of the action applied (word!)
• new: new value added to the series (any-type!)
• index: position at which the series is modified (integer!)
• part: number of elements changes in the series (integer!)

Action name can be any of: random, clear, cleared, poke, remove, removed, reverse, sort, insert, take, taken, swap, trim. For actions "destroying" values, two events are generated, one before the "destruction", one after (hence the presence of cleared, removed, taken words).

When modifications affect several non-contiguous or all elements, index will be set to -1.
When modifications affect an undetermined number of elements, part will be set to -1.

Ownership is set automatically on object creation if on-deep-change* is defined, all referenced series (including nested ones), will then become owned by the object. modify action has been also implemented to allow setting/clearing ownership post-creation-time.

As for on-change, on-deep-change* is kept hidden when using mold on object. It is only revealed by mold/all.

Here is a simple usage example of object ownership. The code below will create a numbers object containing an empty list. You can append only integers to that list, if you fail to do so, a message will be displayed and the invalid element removed from the list. Moreover, the list is always sorted, wherever you insert or poke a new value:

    numbers: object [
        list: []
    
        on-deep-change*: function [owner word target action new index part][
            if all [word = 'list find [poke insert] action][
                forall target [
                    unless integer? target/1 [
                        print ["Error: Item" mold target/1 "is invalid!"]
                        remove target
                        target: back target
                    ]
                ]
                sort list
            ]
        ]
    ]
    
    red>> append numbers/list 3
    == [3]
    red>> insert numbers/list 7
    == [3 7]
    red>> append numbers/list 1
    == [1 3 7]
    red>> insert next numbers/list 8
    == [1 3 7 8]
    red>> append numbers/list 4
    == [1 3 4 7 8]
    red>> append numbers/list "hello"
    Error: Item "hello" is invalid!
    == [1 3 4 7 8]
    red>> numbers
    == make object! [
        list: [1 3 4 7 8]
    ]

Object ownership is deeply used in Red/View, in order to achieve the binding between face objects and the widgets on screen, and the automatic "show-less" synchronization. 

The work on this is not yet completed, more object events will be provided in future releases and the ownership support extended to enable objects to own more datatypes. More documentation will be provided once the work on that will be finished. In the future, its use will be extending to other frameworks and interfaces. Such "reactive objects" will be called "live objects" in Red's jargon.

Red/System changes

• Full stack traces in debug mode on runtime errors.
• New compilation directive: #u16 (literal UTF-16LE strings support).
• Added log-b native function for getting the binary logarithm of an integer.
• Added equal-string? runtime function for testing c-string! equality.
• Several improvements to some compiler errors reporting accuracy.
• Improved function! type support.
• New compilation option: debug-safe? (for safer stack traces)
• New --catch command-line option for console to open on script errors.
• Improved compilation speed of variables assignment.
• Fixes for broken exceptions support on ARM backend.

Additions to the Red runtime library

New functions

• show, view, unview, draw, layout, react, size-text, to-image, do-events, dump-face, within?, overlap?, remove-reactor, set-flag, find-flag?, center-face, insert-event-func, remove-event-func.
• event?, image?, binary?.
• debase, wait.
• request-file, request-dir.
• read, write, exists?, to-local-file, to-red-file, dirize, clean-path, split-path.
• what-dir, change-dir, list-dir.
• also, alter, extract, collect, split, checksum, modify, unset.
• as-color, as-rgba, as-ipv4.
• cd, ls, ll, pwd, dir. (console-only)

Use help in the console to get more information about each function.

New datatypes

• binary!
• event! (Windows only for now)
• image! (Windows only for now)

Binary! datatype supports all the series actions. Literal base 2, 16 and 64 encodings are available:

    red>> 2#{11110000}
    == #{F0}
    red>> to string! 64#{SGVsbG8gV29ybGQh}
    == "Hello World!"

Event! and image! are a work-in-progress, though image! is already very capable (documentation will be added soon).

Other changes

o set and get native improvements:

If A and B are object values, set A B will now set the values in A from B, for the fields they have in common (regardless of the fields definition order in the objects).

Added /only and /some refinements:

• /only: set argument block or object as a single value
• /some: `none` values in the argument block or object are not set

o Icons and other "resources" are now supported for inclusion in Windows executables. They can be set from Red's main script header, these are the currently supported options:

• Icon: file! or block! of files
• Title: string!
• Author: string!
• Version: tuple!
• Rights: string!
• Company: string!
• Notes: string!
• Comments: string!
• Trademarks:  string!

If no Icon option is specified, a default Red icon will be provided.

o index? action is now allowed on words. It will return the word's index inside a context or none if the word is unbound. This is a shortcut for the following idiom:
    index? find words-of <object> <word>
 
o Remaining list of changes:

• Implemented type-checking for infix operators in the interpreter.
• Implemented native! functions type-checking support when called by compiled code.
• Added system/state/trace? for enabling/disabling call stack traces on errors.
• system/options/args gets the command-line string.
• Added DO/ARGS support.
• Error report for catchable infinite block rules recursions in Parse.
• Added limits to Parse stack to avoid eating up all the memory.
• Auto-conversion of float values in routines.
• Big series (> 2MB) support enabled.
• Lexer support for base2 and base64 encoding.
• DO and LOAD work on file! and url! values now.
• Added support for cycles detection for MOLD/FORM and comparisons.
• Support for set operations on hash!.
• SORT works on paren! now.
• string! to issue! conversion support.
• file! to string! conversion support.
• Allowed float! values as arguments to AS-PAIR and MAKE pair!.
• Added percent! support in vector! series.
• Added matching typesets support to Parse.
• Added PUT support to object! and any-series!.
• Added support for make bitset! <binary>
• Setting a tuple component to none now eliminates the component.
• Support for HOME and END key in console.
• Multiline editing support for paren! and map! in console.
• Added proper error handling for malformed paths evaluation attemps.
• Scripts using routines will now output a proper error when run from interpreter.
• Better error handling when decoding UTF-8 string.
• Allow PROBE to have an unset! value as argument.
• Support X in addition to x for pair! literal syntax.
• Prevent empty conditions in conditional iterators from entering an infinite loop.
• Improved formatting of error messages arguments.
• Several output improvements to HELP.
• Allow DIR? to take an url!.
• Allow system/console/prompt to be an active value (e.g.: a function).

Ticket processing

We have closed 260+ tickets since last release (0.5.4), among which 54 concern issues in previous releases. We have currently ~92.5% closed tickets overall, which is a bit lower than the usual 95%+, mostly due to the huge amount of new code and feature in this release. So, we will aim at getting back to a lower number of opened tickets for the next release.

I would like to make a big thank to all the contributors who reported issues, tested fixes and sent pull requests. It has been, more than ever given the number of newly implemented features, a huge help in making Red better. I would like to especially thank namely a few people for their outstanding contributions:

• WiseGenius: for helping us solve the epic crush library generation bug, improvement suggestions and huge work on testing/reporting GUI issues! 

• nc-x: for help in testing the GUI, making many useful issue reports and improvement suggestions.

• The "Czech group" (Pekr, Oldes, Rebolek): for their constant support and for taking care of the Red community when I'm not available. ;-)

• PeterWAWood: for bringing us the ~30'000 unit tests, testing framework and constant help and support, since day one!

• Micha: for issues reporting and kindly providing us an online Mac OSX server for our build farm.

What's next?

Our focus for next releases (0.6.x) will be:

• Drastically speed up compilation time by pre-compiling the runtime library.
• Simple garbage collector integration.
• Improvement of our Windows GUI backend.
• First usable versions of MacOSX and Android GUI backends.
• Integration of our Android APK building toolchain in master branch.
• Improvements for reactive GUI programming support.
• Custom widgets creation framework.
• Animation and timers support.
• More documentations and tutorials for beginners.
• More code demos.

See the detail for next releases on our public Trello board and come to our chat room to ask any question.

In the meantime, enjoy the new Red, I hope to see many impressive GUI demos and apps in the next weeks. ;-)