0.6.0: Red GUI system

Five years ago, when I started writing the first lines of code of what would become later the Red/System compiler, I had a pretty good picture already of what I wanted to achieve with Red, and all the ideal features that should be included, just not sure how much time and efforts it would require to have them. Two years and half ago, baby Red printed its first output. And today, we celebrate a major step forward with the addition of a brand new GUI system entirely written in Red itself! What a journey!

Here it is, the long awaited 0.6.0 release with its massive 1540 commits! The major new features are:

• View engine, with Windows backend (from XP to 10)
• VID dialect
• Draw dialect
• Reactive GUI programming
• GUI console
• Simple I/O support for files and HTTP(S) queries.
• Codecs system with following codecs available: BMP, GIF, JPEG, PNG
• Objects ownership system

All those additions made our Red executable grow from 767 KB to 910 KB (Windows platform), sorry for the extra 143 KB, we will try to [red]uce that in the future. ;-)

Let's start with the elephant in the room first, the GUI system. Here is an architecture overview:

Only the Windows backend is fully usable for now, Android and OS X are work-in-progress, Linux (using GTK) will follow soon, iOS will come later this year. Also, we have other targets in mind, like JS/HTML which are not scheduled yet, but could come this year too.

Red/View

First let me mention that View, VID and Draw were invented by Carl Sassenrath (of Amiga fame) in the Rebol language, a long time ago. Red's version retains all the best features and pushes the boundaries of simplicity even further. The main features of our View engine are:

• A live updating mode that reduces the need to a single view function in most cases.
• Full abstraction over rendering backends.
• Two-way binding using live objects.
• Event bubbling/capturing stages.
• Built-in drag'n drop support for most face types.
• Gestures support (experimental).
• Native widgets support.
• Full integration with the OS features.
• Flexible backend support that can be mapped to virtually any kind of UI library.

The current list of supported widgets is: base, text, button, check, radio, field, area, text-list, drop-list, drop-down, progress, slider, camera, panel, tab-panel, group-box.

Next releases will bring more widgets, like: table, tree, divider, date/time picker, web-view and many others!

For more info about View, see the View reference document.

Main differences between Red/View and Rebol/View are:

• Red relies on native widgets, Rebol has custom ones only, built over a 2D vector library.
• Red faces are synchronized with their widgets on display in realtime by default, Rebol faces require manual calls to many functions for keeping faces and widget updated.
• Red introduces reactive GUI programming.

Red/View will update both face and graphic objects in realtime as their properties are changed. This is the default behavior, but it can be switched off, when full control over screen updates is desirable. This is achieved by:

    system/view/auto-sync?: off

When automatic syncing is turned off, you need to use show function on faces to get the graphic objects updated on screen.

VID dialect

VID stands for Visual Interface Dialect. It is a dialect of Red which drastically simplifies GUI construction. VID code is dynamically compiled to a tree of faces, feeding the View engine. You can then manipulate the face objects at runtime to achieve dynamic behaviors. VID offers:

• Declarative programming.
• Automatic layout system.
• Cascading styles.
• Default values for...everything.

For more info about VID, see the specification.

In case you are reading about Red or Rebol for the first time, here are a few code demos to show how simple, yet efficient, is our approach to GUI programming:

A GUI Hello word

    view [text "Hello World"]

Greet the name you type in the field

    view [name: field button "Hi" [print ["Hi" name/text]]]

A simple reactive relations demo, drag the logo around to see the effect

   view [
        size 300x300
        img: image loose http://static.red-lang.org/red-logo.png
        return
        shade: slider 0%
        return
        text bold font-size 14 center "000x000" 
            react [face/text: form img/offset]
            react [face/font/color: white * shade/data]
    ]
 

Simple form editing/validating/saving with styles definitions

   digit: charset "0123456789"
    view [
        style label: text bold right 40
        style err-msg: text font-color red hidden
    
        group-box "Person" 2 [
            origin 20x20
            label "Name" name: field 150
            label "Age"  age:  field 40
            label "City" city: field 150
            err-msg "Age needs to be a number!" react later [
                face/visible?: not parse age/text [any digit]
            ]
        ]
        button "Save" [save %person.txt reduce [name/text age/text city/text]]
    ]
    set [name age city] load %person.txt
    ?? name ?? age ?? city

You can run all those examples by copy/pasting them one-by-one into the Red console for Windows. To get the console, just download it and double-click the Red binary, wait ~20 seconds for the console to be compiled for your OS (yes, that little file contains the full Red toolchain, runtime library and console source code), paste the code and have fun. ;-)

Draw dialect

Draw is a 2D vector-drawing dialect which can be used directly, to render on an image, in faces for local rendering, or specified through VID. It is still a work in progress as not all features are there yet. We aim at full Rebol/Draw coverage and full SVG compatibility in the not-too-distant future.

A simple example of Draw usage:

    shield: [
        fill-pen red   circle 50x50 50
        pen gray
        fill-pen white circle 50x50 40
        fill-pen red   circle 50x50 30
        fill-pen blue  circle 50x50 20
        
        pen blue fill-pen white
        polygon 32x44 45x44 50x30 55x44 68x44 57x53 60x66 50x58 39x66 43x53
    ]
    
    ;-- Draw in a draggable face, in realtime.
    view [
        size 300x300
        img: box 101x101 draw shield loose
        at 200x200 base white bold react [
            [img/offset]
            over?: overlap? face img
            face/color: get pick [red white] over?
            face/text: pick ["Hit!" ""] over?
        ]
        button "Hulk-ize!" [replace/all shield 'red 'green]
        button "Restore"   [replace/all shield 'green 'red]
    ]

Copy/paste the above code example in a Red console on Windows, and become an Avenger too! ;-)

For more info about Draw, see the specification.

Main differences between Red/Draw and Rebol/Draw:

• Red does not yet cover all the commands of Rebol/Draw yet.
• Red's version allows commands to be grouped in blocks, ease-ing insertion/removal at run-time.
• Red's version allows commands to be prefixed with a set-word, allowing to save local position in Draw blocks in a word.

Reactive GUI programming

This is a deep topic which should be part of a future separate blog article. So, I will just copy/paste here the little information already in the VID documentation:

Reactions (or reactors, not sure yet which terms is the most accurate) are created using the react keyword, directly from Red code or from VID dialect. The syntax is:

    react [<body>]
    
    <body> : regular Red code (block!).

This creates a new reactor from the body block. When react is used in VID, as a face option, the body can refer to the current face using face word. When react is used globally, target faces need to be accessed using a name.

Reactors are part of the reactive programming support in View, whose documentation is pending. In a nutshell, the body block can describe one or more relations between faces properties using paths. Set-path setting a face property are processed as target of the reactor (the face to update), while path accessing a face property are processed as source of the reactor (a change on a source triggers a refresh of the reactor's code).

Basically, it is about statically-defined relations between faces properties, without caring when or how the reactive expressions will be evaluated. It will happen automatically, when needed. Here are a couple of examples you can copy/paste in the Red console on Windows:

Make a circle size change according to slider's position:

    view [
        sld: slider return
        base 200x200 
            draw  [circle 100x100 5]
            react [face/draw/3: to integer! 100 * sld/data]
    ]

Change the color of a box and a text using 3 sliders:

    to-color: function [r g b][
        color: 0.0.0
        if r [color/1: to integer! 256 * r]
        if g [color/2: to integer! 256 * g]
        if b [color/3: to integer! 256 * b]
        color
    ]

    to-text: function [val][form to integer! 0.5 + 255 * any [val 0]]

    view [
      style txt: text 40 right
      style value: text "0" 30 right bold
    
      across
      txt "Red:"   R: slider 256 value react [face/text: to-text R/data] return
      txt "Green:" G: slider 256 value react [face/text: to-text G/data] return
      txt "Blue:"  B: slider 256 value react [face/text: to-text B/data]
    
      pad 0x-65 box: base react [face/color: to-color R/data G/data B/data]
      return
    
      pad 0x20 text "The quick brown fox jumps over the lazy dog." font-size 14
        react [face/font/color: box/color]
    ]

GUI console

We have a GUI console now, in addition to the existing CLI one!

The GUI console is now the default on Windows platform, and is fully Unicode-aware. The system shell (DOS) console is still available using --cli option:

    red --cli

The GUI console is still in its infancy and will be enhanced a lot in future releases. Anyway, so far, it already supports:

• history of commands
• completion on words and object paths
• multi-line editing for blocks, parens, strings, maps and binaries.
• navigation using HOME and END keys
• select/copy/paste using the mouse and keyboard shortcuts
• auto-scrolling when selecting with the mouse out of the boundaries
• very fast text rendering
• automatic vertical scroll bar
• customizable prompt

Try this cool one-liner for making the prompt more active:

    system/console/prompt: does [append to-local-file system/options/path "> "]

This is how the GUI console looks like:

Simple I/O support

In order to have really some fun with the GUI, we have added some minimal support for blocking IO basic actions covering files and HTTP(S) requests. read and write action are available now. Their /binary, /lines and /info refinement are working. do, load, save have also been extended to work with files and urls.

When not using /binary, read and write are expecting UTF-8 encoded data. Support for ISO8859-1 and other common encoding formats will be available in next release.

The full IO will come in 0.7.0 with ports, full networking, async support and many more features.

Codecs

Codec system support has made his entrance in this release. It is a very thin layer of encoders/decoders for binary data, integrated with load, save and actions which rely on /as refinement. load and save will auto-detect the required encoding format and try to apply the right encoder or decoder on the data.

Currently only image format codecs are provided: BMP, PNG, GIF, JPEG. Any kind of encoding (related to IO) is a good candidate for becoming a codec, so expect a lot of them available in the future (both built-in Red runtime and optionaly installable).

For example, downloading a PNG image in memory, and using it is as simple as:

    logo: load http://static.red-lang.org/red-logo.png
     
    big: make font! [name: "Comic" size: 20 color: black]
    draw logo [font big text 10x30 "Red"]
    view [image logo]

Saving a downloaded file locally:

    write/binary %logo.png read/binary http://static.red-lang.org/red-logo.png

Saving images is not fully functional yet, PNG should be safe though.

Objects ownership system

Red's objects ownership system is an extension of object's event support introduced in previous releases. Now, an object can own series it references, even nested ones. When an owned series is changed, the owner object is notified and its on-deep-change* function will be called if available, allowing the object to react appropriately to any change.

The prototype for on-deep-change* is:

    on-deep-change*: func [owner word target action new index part][...]

The arguments are:

• owner: object receiving the event (object!)
• word: object's word referring to the changed series or nested series (word!)
• target: the changed series (any-series!)
• action: name of the action applied (word!)
• new: new value added to the series (any-type!)
• index: position at which the series is modified (integer!)
• part: number of elements changes in the series (integer!)

Action name can be any of: random, clear, cleared, poke, remove, removed, reverse, sort, insert, take, taken, swap, trim. For actions "destroying" values, two events are generated, one before the "destruction", one after (hence the presence of cleared, removed, taken words).

When modifications affect several non-contiguous or all elements, index will be set to -1.
When modifications affect an undetermined number of elements, part will be set to -1.

Ownership is set automatically on object creation if on-deep-change* is defined, all referenced series (including nested ones), will then become owned by the object. modify action has been also implemented to allow setting/clearing ownership post-creation-time.

As for on-change, on-deep-change* is kept hidden when using mold on object. It is only revealed by mold/all.

Here is a simple usage example of object ownership. The code below will create a numbers object containing an empty list. You can append only integers to that list, if you fail to do so, a message will be displayed and the invalid element removed from the list. Moreover, the list is always sorted, wherever you insert or poke a new value:

    numbers: object [
        list: []
    
        on-deep-change*: function [owner word target action new index part][
            if all [word = 'list find [poke insert] action][
                forall target [
                    unless integer? target/1 [
                        print ["Error: Item" mold target/1 "is invalid!"]
                        remove target
                        target: back target
                    ]
                ]
                sort list
            ]
        ]
    ]
    
    red>> append numbers/list 3
    == [3]
    red>> insert numbers/list 7
    == [3 7]
    red>> append numbers/list 1
    == [1 3 7]
    red>> insert next numbers/list 8
    == [1 3 7 8]
    red>> append numbers/list 4
    == [1 3 4 7 8]
    red>> append numbers/list "hello"
    Error: Item "hello" is invalid!
    == [1 3 4 7 8]
    red>> numbers
    == make object! [
        list: [1 3 4 7 8]
    ]

Object ownership is deeply used in Red/View, in order to achieve the binding between face objects and the widgets on screen, and the automatic "show-less" synchronization. 

The work on this is not yet completed, more object events will be provided in future releases and the ownership support extended to enable objects to own more datatypes. More documentation will be provided once the work on that will be finished. In the future, its use will be extending to other frameworks and interfaces. Such "reactive objects" will be called "live objects" in Red's jargon.

Red/System changes

• Full stack traces in debug mode on runtime errors.
• New compilation directive: #u16 (literal UTF-16LE strings support).
• Added log-b native function for getting the binary logarithm of an integer.
• Added equal-string? runtime function for testing c-string! equality.
• Several improvements to some compiler errors reporting accuracy.
• Improved function! type support.
• New compilation option: debug-safe? (for safer stack traces)
• New --catch command-line option for console to open on script errors.
• Improved compilation speed of variables assignment.
• Fixes for broken exceptions support on ARM backend.

Additions to the Red runtime library

New functions

• show, view, unview, draw, layout, react, size-text, to-image, do-events, dump-face, within?, overlap?, remove-reactor, set-flag, find-flag?, center-face, insert-event-func, remove-event-func.
• event?, image?, binary?.
• debase, wait.
• request-file, request-dir.
• read, write, exists?, to-local-file, to-red-file, dirize, clean-path, split-path.
• what-dir, change-dir, list-dir.
• also, alter, extract, collect, split, checksum, modify, unset.
• as-color, as-rgba, as-ipv4.
• cd, ls, ll, pwd, dir. (console-only)

Use help in the console to get more information about each function.

New datatypes

• binary!
• event! (Windows only for now)
• image! (Windows only for now)

Binary! datatype supports all the series actions. Literal base 2, 16 and 64 encodings are available:

    red>> 2#{11110000}
    == #{F0}
    red>> to string! 64#{SGVsbG8gV29ybGQh}
    == "Hello World!"

Event! and image! are a work-in-progress, though image! is already very capable (documentation will be added soon).

Other changes

o set and get native improvements:

If A and B are object values, set A B will now set the values in A from B, for the fields they have in common (regardless of the fields definition order in the objects).

Added /only and /some refinements:

• /only: set argument block or object as a single value
• /some: `none` values in the argument block or object are not set

o Icons and other "resources" are now supported for inclusion in Windows executables. They can be set from Red's main script header, these are the currently supported options:

• Icon: file! or block! of files
• Title: string!
• Author: string!
• Version: tuple!
• Rights: string!
• Company: string!
• Notes: string!
• Comments: string!
• Trademarks:  string!

If no Icon option is specified, a default Red icon will be provided.

o index? action is now allowed on words. It will return the word's index inside a context or none if the word is unbound. This is a shortcut for the following idiom:
    index? find words-of <object> <word>
 
o Remaining list of changes:

• Implemented type-checking for infix operators in the interpreter.
• Implemented native! functions type-checking support when called by compiled code.
• Added system/state/trace? for enabling/disabling call stack traces on errors.
• system/options/args gets the command-line string.
• Added DO/ARGS support.
• Error report for catchable infinite block rules recursions in Parse.
• Added limits to Parse stack to avoid eating up all the memory.
• Auto-conversion of float values in routines.
• Big series (> 2MB) support enabled.
• Lexer support for base2 and base64 encoding.
• DO and LOAD work on file! and url! values now.
• Added support for cycles detection for MOLD/FORM and comparisons.
• Support for set operations on hash!.
• SORT works on paren! now.
• string! to issue! conversion support.
• file! to string! conversion support.
• Allowed float! values as arguments to AS-PAIR and MAKE pair!.
• Added percent! support in vector! series.
• Added matching typesets support to Parse.
• Added PUT support to object! and any-series!.
• Added support for make bitset! <binary>
• Setting a tuple component to none now eliminates the component.
• Support for HOME and END key in console.
• Multiline editing support for paren! and map! in console.
• Added proper error handling for malformed paths evaluation attemps.
• Scripts using routines will now output a proper error when run from interpreter.
• Better error handling when decoding UTF-8 string.
• Allow PROBE to have an unset! value as argument.
• Support X in addition to x for pair! literal syntax.
• Prevent empty conditions in conditional iterators from entering an infinite loop.
• Improved formatting of error messages arguments.
• Several output improvements to HELP.
• Allow DIR? to take an url!.
• Allow system/console/prompt to be an active value (e.g.: a function).

Ticket processing

We have closed 260+ tickets since last release (0.5.4), among which 54 concern issues in previous releases. We have currently ~92.5% closed tickets overall, which is a bit lower than the usual 95%+, mostly due to the huge amount of new code and feature in this release. So, we will aim at getting back to a lower number of opened tickets for the next release.

I would like to make a big thank to all the contributors who reported issues, tested fixes and sent pull requests. It has been, more than ever given the number of newly implemented features, a huge help in making Red better. I would like to especially thank namely a few people for their outstanding contributions:

• WiseGenius: for helping us solve the epic crush library generation bug, improvement suggestions and huge work on testing/reporting GUI issues! 

• nc-x: for help in testing the GUI, making many useful issue reports and improvement suggestions.

• The "Czech group" (Pekr, Oldes, Rebolek): for their constant support and for taking care of the Red community when I'm not available. ;-)

• PeterWAWood: for bringing us the ~30'000 unit tests, testing framework and constant help and support, since day one!

• Micha: for issues reporting and kindly providing us an online Mac OSX server for our build farm.

What's next?

Our focus for next releases (0.6.x) will be:

• Drastically speed up compilation time by pre-compiling the runtime library.
• Simple garbage collector integration.
• Improvement of our Windows GUI backend.
• First usable versions of MacOSX and Android GUI backends.
• Integration of our Android APK building toolchain in master branch.
• Improvements for reactive GUI programming support.
• Custom widgets creation framework.
• Animation and timers support.
• More documentations and tutorials for beginners.
• More code demos.

See the detail for next releases on our public Trello board and come to our chat room to ask any question.

In the meantime, enjoy the new Red, I hope to see many impressive GUI demos and apps in the next weeks. ;-)