0.6.5: Changelog

Bumping up the version number was motivated by the breaking syntax change done recently. We do not offer specific builds for a given version number anymore since we provide automatic builds (with builds history) on each new master commit. Though, the changelog for new version number changes will still be provided...and this one is pretty big as it covers about 5000 commits! Hope this will help users who might have missed some changes to catch up.

613 PRs were merged, 2415 fix commits were pushed, among which 902 are closing issues tracked on Github. The most notable new features and changes are listed below with eventual links to docs or previous blog posts describing or mentioning them:

Main new features

• New datatypes: money!, ref!, point2D!, point3D!.
• New codecs: Redbin, JSON, CSV
• New high-performance run-time lexer with instrumentation support.(blog)(doc)
• Interpreter instrumentation support (debugger, tracer, profiler).(doc)
• New powerful APPLY native, with deep interpreter support.(blog)
• Dynamic refinements support.(blog)
• Adds compress and uncompress natives with gzib, zlib and deflate algorithms support.
• Adds gpio:// port with GPIO dialect for RaspberryPi.(blog)
• Adds TAB navigation support to View. (blog)
• Adds raw strings syntax support.(doc)
• Swaps map! and construction syntax. (blog)
• Hashtables are now used for fast lookups in contexts.
• Custom dtoa library implementation to load and form float values.
• Standard library and garbage collector stability vastly improved.

Finished or almost finished features in branches::

• Full IO ports with async support, including new IPv6! datatype.(branch)
• TextUI backend to View.(PR)
• XML codec.(PR)

Other general new features or changes

• New natives: TRANSCODE, SCAN, AS-MONEY, ENHEX.
• New functions: SINGLE?, LAST?, DT, TRANSCODE-TRACE, TRACE, CLOCK, NO-REACT, DO-NO-SYNC
• New routines: SET-TRACE, TRACING?
• Extends EMPTY? to support map! values.
• Allows NONE as value in map!.
• Adds REMOVE/KEY support for removing map! entries.
• Adds FOREACH support for map!.
• Add new-lines automatically when converting maps to blocks.
• Supports issue!, money! and refinement! as key in map
• Allows any-string! series to be matched against binary input in PARSE.
• Adds Accept-Charset and User-agent to HTTP header.
• Update libcrypto version requirement on Linux platforms.
• Adds support for native port actors.
• Extends ROUND/TO to support time!.
• Adds a cross-platform threading library to Red runtime library.
• Adds a FIFO MPMC queue to Red runtime library.
• Adds history support to Red console.
• Extends GET native to accept any-word! as argument.
• FIND is now returning FALSE instead of NONE when used on bitsets.
• Deprecates FIND on objects (redundant with IN).
• Adds ABOUT function to console.
• Preprocessor: fetch-next now supports set-words/paths, get/lit-args, object/series paths.
• COMPLEMENT is now allowed on tuples.
• Adds RANDOM/SECURE.
• SORT support of /skip & /all & /compare integer extended to strings, vectors and binaries.
• Extends FIND/LAST to support hash!.
• Allows FINDing by datatype and typeset on hash! series.
• Removes percent-encoding from files, use double-quotes when needed instead.
• Adds SYSTEM/CATALOG/ACCESSORS.
• Allows pair! for COPY/PART argument on series.
• Allows LOOP and REPEAT to take a float argument as counter.
• Extends REDUCE/INTO to support hash! destination.
• Extends BODY-OF to support action! and native! values.
• REPLACE reimplemented.
• Adds support for REVERSE/SKIP.
• Extends VALUE? native supports any-word! argument.
• Enable image! in red/core on Linux.
• Adds GC support to libRed.
• Extends DISTANCE? to support pair! arguments.
• Adds /KEEP refinement to TRY.
• Implements RENAME action for FILE! datatype.
• Adds routine arguments type-checking support to compiler.
• Accelerates the output speed of LIST-DIR.
• Allows error! values to be modified by users.
• Adds [trace] and [no-trace] function attributs.
• Faster/simpler EMPTY? function implementation.
• Forces the inclusion of Git information in the runtime.
• Optimization for appending values into hashs.
• Extends CHANGE to support image! argument.
• PICK on pair!, date! and time! values now support named accessors as index argument.
• Adds PICK action to event! datatype.
• Makes RECYCLE returns the allocated memory total by default.
• Cleaner implementation of deep reactive paths support.
• Internalizes SYSTEM/STATE/NEAR value.
• Compound scalar datatypes (pair!, date!, time! and tuple!) will now emit ON-CHANGE events when one of their component is changed using an access path (both in compiled and interpreted code).
• Adds reactivity support to bitset! values.
• Adds support for REFLECT action on bitset! values.
• Reports a proper path in compiled path errors.
• Adds memory usage before/after a GC cycle in debug output.
• Measures GC time in debug mode.
• [ARM] Adds "division by zero" and "division overflow" checks in debug mode.

Parse

• KEEP PICK on paren expressions now merges list of values to collected block.
• Optimizes Parse's KEEP memory usage on strings/binaries.
• Removes the end checking in iteration rules.
• Speed optimizations for `TO <token>` rules.
• Adds a fast path for `TO/THRU end` rules.
• New set of optimizations for looping on a char! value in PARSE.

VID

• Smarter merging of actors in a style with custom actors in the instance.
• Added password flag for hidden input
• Adds scrollable flag.
• defines VID origin and spacing values per backend.
• adds next and prev as default options to VID-made faces.

View

• Adds a calendar widget.
• Adds support for tri-state checkboxes.
• Significant GTK backend improvements and upgrades to match other backends.
• Better handling of DPI changes.
• Handles pause and scroll-lock keys.
• Now EVENT/PICKED is a float wheel delta-value in on-wheel event.
• Improves user experience when closing window that contains a large number of faces.
• Scale the font size with Ctrl + mouse wheel in GUI console.
• Adds stop-events function to easily exit a View events loop.
• Adds resize-ns and resize-ew mouse cursors.
• Adds bounds option to /options facet for restricted dragging area.
• Adds new /sync refinement to VIEW function. 
• Adds /color facet info to DUMP-FACE output.
• Improves memory usage when changing /draw facet content.
• Adds support for semi-transparent no-border top-level windows (Windows).
• Minimal dark mode support on GUI console.

Draw

• Switch command parameters to point2D! and float! for subpixel precision.(blog)
• New Direct2D backend for Windows.
• Add line-pattern command for drawing dashed lines.
• Extends scale command to support percent! values.
• Supports image mapping to arbitrary quadrilateral.
• Removes matrix-order command in DRAW.

Red/System

• Adds subroutines to R/S functions.(blog)
• Implements system/io/* instrinsics for CPU I/O read/write instructions.
• Adds system/stack/push-all and system/stack/pop-all intrinsics.(blog)
• Support for atomic operations using system/atomic/* intrinsics.(blog)
• Adds #inline directive to R/S for including assembled binary code.
• Implements support for integer hardware division (ARMv7+).
• Generates optimized code for divisions by a power of 2 literal (ARM).
• Vastly improved loop counter handling robustness.
• Full support for special float values (-0.0, 1.#NaN, 1.#INF, -1.#INF).
• Drops support for % and // operators on float types.
• Adds support for function pointers in literal arrays.
• Switches to 16-bytes stack alignment on Linux.
• Adds log-2 function to standard library.
• Allows cross-referenced aliased fields in structs defined in same context.
• Support multiple variable assignments.
• Allows grouping arguments and local variables type specification.
• Relax lexical format of hexadecimal literals.
• Allows get-paths pointers on function nested in contexts.
• Adds support for simple pointer to pointer type.
• Allows function! type to be specified for local variables.

Toolchain

• Optimizes critical section in linker, twice faster linking time now on average.
• Adds --show-func-map compilation option.
• Various minor improvements to PE format support.
• Adds working Linux-musl target.
• Expands libRedRT to support View backend.
• Adds --no-view option for Red binaries.
• Shows the global words used by the toolchain in the compilation report.
• Adds new Pico compilation target.

Important Change! Switching map and construction syntax.

Sometimes deep changes take a huge amount of code. Sometimes they take a lot of detailed explanation and consideration, leading to long discussions and people taking sides. Rarely does an important syntactic change to a language happen quickly, with universal agreement, simple implementation, and tools to help update scripts in the wild. Today is one of those rare days.

Admittedly, this idea has been discussed for a long time. It would surface, people nodded their virtual heads, and it would submerge again. Today it's ready to deploy. Not only that, but Rebol3 is making the same change, so the two languages will still be compatible in this regard.

What is the change?

It's easy to describe. Today, map! values use this syntax: #(...) and construction syntax (sometimes called serialized form or loadable form) looks like this: #[...]. Going forward, those syntactic forms will be swapped. Why? The answer is easy. In Redbol langs, blocks do not evaluate by default, you have to do or reduce them. Parens, on the other hand, do evaluate by default. Today, maps use paren-like syntax, but they do not evaluate, while construction syntax uses block-like syntax, but does evaluate. This is a carryover from Rebol, so the major concession here is that Red and Rebol3 will no longer be compatible with Rebol2's construction syntax.

If you've never heard of construction syntax, there's a nice explanation of it here. Red only supports a few values via construction syntax today, all datatype literals, true, false, none, and unset; but eventually it will support much more. If you look at the help for mold, you'll see that /all is TBD (very partially implemented for now), and that's how you create loadable, serialized, data that can safely and easily contain any value (like redbin but readable by humans). It helps avoid cases where none or true/false may load as words. This is also why construct evaluates those specific words (including also on/off/yes/no), but not others. When loading untrusted data, we have to strike a balance between ease of use and safety.

What do I have to do?

Not much. There are two tools available, which will convert your scripts automatically. The first is small and simple, showing just how powerful Red is, and leveraging its lexer instrumentation. You can find it here (once merged, that branch may go away and the tool will be in the main branch). The second is a more advanced and standalone tool written by @hiiamboris. You can find that here.

For the simple script it's necessary that you run it under a current version of Red's lexer. Once the change is in place, running it under the new lexer will make the exact opposite change. Of course, you can compile it into a standalone EXE, or use Boris' app, which is already available.

To run the simple script from a Red console, you can just:

  do https://raw.githubusercontent.com/red/red/master/utils/migration/map-conv.red

then you can use help map-conv to see all the available options. By default it runs in preview mode, making no changes, and showing you all the instances it found, which will be changed if you use the /save refinement. A copy of each changed file is created with a .saved extension in the same folder. If you don't want them, you can use the /no-copy refinement.

A word of warning, if you run the conversion tools a second time on the same files, they will convert the data back, because they can't know what you're thinking. On the bright side, this is an effective "undo" feature. Still, it's wise to back up your data before running any tools against them.

Thanks to the power of Red, and these tools, there are already PRs pending for updates to docs and community scripts. But the other thing you can do to help is to let us know when you find things that need to be updated for this change, and especially if you run into any issues when converting your own code.

Conclusion

We know changes like this can be hard, but better now than when there is even more code in the wild that would be affected. If we had been any other language, this long-view improvement might not have happened. Only because Red (and Redbol langs in general) can consume their own code as data and have powerful parse and lexing features, was this change so easy and safe. It's still a code-porting process, and if you run multiple versions of Red you may need to maintain separate versions for a while. The other hard part is retraining your hands and eyes to the new syntax.

Happy Reducing!