Skip to content

Releases: actonlang/acton

v0.15.2

16 Jun 20:36
f984cbd
Compare
Choose a tag to compare

Elevate Acton's build capabilities by completing the adoption of the Zig build
system. Everything, including external library dependencies, builtins, RTS,
stdlib and backend, is now built using build.zig files. A hierarchy of zig
modules are formed, which allow building the entirety of the Acton system with a
single zig build, which is what actonc calls internally. This enables complete
control over all aspects of the low level compilation.

The most striking feature unlocked is likely cross-compilation:

user@machine$ actonc --quiet helloworld.act
user@machine$ file helloworld
helloworld: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.0.0, with debug_info, not stripped
user@machine$ actonc --quiet helloworld.act --target x86_64-macos-none
user@machine$ file helloworld
helloworld: Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE>

Libraries for the local platform are still prebuilt and included in the Acton
distribution, hich are used when using the default target, i.e. targetting the
local machine. For any customization to the target, everything will be built
from source.

Fixed

  • revamped low level build, now potentially rebuilding entire Acton system from
    source code
    • allows cross-compilation and similar advanced features
    • use --target for cross-compilation, e.g. --target aarch64-macos-none to
      target an Apple M1 computer
    • for Acton projects, there is now a build-cache directory that caches all
      built output and speeds things up tremendously
  • json module now correctly encodes and decodes floats [#1345] [#1349]
  • zig build of all external library dependencies
    • gives us much better control over how libraries are compiled
    • much much faster! autoconfig is really slow!
      • e.g. building protobuf-c takes a few seconds on a 2015 MBP while with
        autoconf, it spends > 1 minute trying to figure out capabilities of the
        C++ compiler, despite the C++ parts of libprotobuf-c not being enabled!
      • the debian test job now often build Acton in less than 3 minutes, macos-13
        often run in ~4 minutes - a real improvement!
    • libargp [#1336]
    • libbsdnt [#1337]
    • libgc [#1344]
    • libnetstring [#1325]
    • libpcre2 [#1331]
    • libprotobuf-c [#1341]
    • libutf8proc [#1332]
    • libuuid [#1340]
    • libuv [#1334]
    • libxml2 [#1333]
    • libyyjson [#1323]
    • remove libbsd & libmd as dependency [#1329]
      • only use was of arc4random in numpy, which is now rand using()
  • backend is now built using zig [#1346]
  • use curl instead of git clone to fetch dependencies [#1343]
    • much faster and reduced disk usage

Testing / CI

  • caching has been vastly improved in CI
    • based on the new zig build system, we utilize zigs excellent caching
      capability by using a single zig cache for the entire build of Acton. In CI,
      we cache the zig-cache directory, so we don't actually have to recompile
      files at all. Some CI runs now take less than 2 minutes to build Acton!
    • we should be able to trust Zigs caching to do the right thing, if files are
      modified it will notice!
  • testing of the Homebrew Formula has been removed [#1338]
    • since some time, this test job has been intermittently failing, or rather
      only intermittently passing as most of the time it fails
    • to avoid lots of errors in CI, the job has been removed
    • the failures are likely coming from some change in brew behavior
    • we use a fairly hacked up and not supported workflow
    • several attempts to fix it have been unsuccessful and we're unable to spend
      more time on it right not
  • removed test of compiling Acton on Ubuntu 20.04 due to problems with stack
    • note how executables built by actonc are still compatible with Ubuntu 20.04

v0.15.1

02 Jun 19:47
d7c088d
Compare
Choose a tag to compare

A better strategy for constraint solver results in vastly lower constraint
solving times and thus reduced overall compilation times. Large improvements in
code generation resulting in less memory allocations and thus a much lower load
on the GC, yielding much better runtime performance.

The internal compilation of generation C code has been revamped, now more
structured and simplified in many ways, internally making use of the Zig buils
system which in turn unleashes a lot of benefits.

Changed

  • release builds now use -O3 and dev builds use -Og [#1270]
  • Collection overloading has been removed [#1316]
    • It was previously possible to overload collections like dict / Mapping
    • {} now always means the builtin dict

Fixed

  • Faster constraint solver through new strategy [#1316]
    • Evaluates vastly fewer possibilities
    • One application used to take over 10 mins for constraint solving which now
      runs in roughly 10 seconds - woohoo
  • New second stage compilation of generated C code [#1249] [#1304]
    • new builder performs "second stage" compilation of the C code that
      actonc generates into object files, archives and executable binaries
    • Built on Zig's Build System, gaining all that it offers
    • Very good caching, dependency detection, up-to-date checking and concurrency
    • Custom compilation to make builder itself portable
    • Compilation can now be more clearly separated in two steps:
      • step 1: actonc compiles .act to .c & .h
      • step 2: builder compiles .c & .h to .o & .a & executables
    • Up-to-date checking in actonc adjusted to only compare .act with .c &
      .h output
    • builder does a fantastic job of following dependencies (looking at
      #include statements etc) to figure out when things need to be recompiled
    • All of this is internal, actonc calls builder under the hood, much like
      it earlier called cc under the hood
    • --no-zigbuild can be used to force old compilation method
      • --no-zigbuild will be removed in the future
  • Much faster build of Acton itself using prebuilt libActonDeps [#1279] [#1282]
    • Uses a checksum of git refs of all lib that constitutes libActonDeps
      • If any dep lib is updated to use new version, libActonDeps will be rebuilt
    • Uploaded to github.com/actonlang/libactondeps and downloaded on demand from
      normal Makefile, thus used for local builds and in CI
    • Falls back to local compilation if prebuilt version is not available
  • builtins, RTS & stlib now organized as an Acton project called base [#1309]
    • More streamlined towards imagined design of how project dependencies will
      work in the future, meaning less special kludges
    • Cleans up include paths and similar, avoiding potential naming conflicts
  • builtin code is now generated [#1256] [#1260]
    • __builtin__.c and __builtin__.h are now generated from __builtin__.ty
      at build time, whereas they were earlier checked in to git, having once been
      generated but heavily modified by hand
    • given complexity of these files, it is difficult to keep the files up to
      date when there are changes to the CodeGen pass in actonc
    • actonc CodeGen pass is now infused with some of the bespoke customizations
      done for builtins, other customizations have been removed to streamline the
      whole thing or inserted into .ext.c file to use the "standard way of
      customization" (what an oxymoron)
  • Better performance through code generation improvements [#1263] [#1274]
    • Code generation has been improved, primarily to avoid unnecessary
      allocations, thus reducing pressure on GC, leading to improved performance
    • One application that ran in > 12 minutes now run in < 2 minutes
  • xml module now correctly parses default namespace [#1262]
  • address xml bugs with more null checks [#126]
  • Switched to a nightly version of zig [#1264]
    • pre-release version of zig v0.11
    • we still pin to this particular zig version which goes through the Acton CI
      testing so there is relatively low risk of this silently breaking something
  • Document after and encourage not using sleep [#1269]
  • actonc now transparently passes through zig output [#1277]
    • treats subprocess output as bytestring instead of trying to decode it
    • we only pass it through anyway, there is no analysis done on it
    • avoids errors seen when not using UTF-8 locale, like when LANG is unset
      (effectively defaulting to C)
  • Remove bunch of old code, irrelevant tests, old gen env scripts ect [#1278]
  • math module switched from stub style to .ext.c style [#1309]
  • Add missing MacOS headers [#1309]
  • Remove note on segfault in AbE that's no longer valid [#1303]

Testing / CI

  • Added macos-13 as test target in CI [#1258]
  • New CI job to build libactondeps and upload to "cache" repository [#1283] [#1284]
    • Also using cached for CI builds (as well as local)
    • MacOS CI builds 12 -> 6 minutes, Linux ~8 -> 4 minutes
      • brew-macos is still slow, but we get quicker feedback of failures looking
        at test-macos & test-linux jobs
  • Fixed Homebrew build by applying workaround
    • Broke due to upstream changes and is still broken with default suggested
      config in brew new-tap

v0.15.0

26 Apr 18:35
5057264
Compare
Choose a tag to compare

Added

  • actonc is now statically linked in the Acton APT repo packages [#1148]
    • enables development using Acton on Ubuntu 18.04, Debian 10 and distributions
      of similar age
    • static linking only works when the build platform is a slightly older Linux,
      it works on Debian 11 which we are using, while on a newer Debian (testing
      as of December 2022) the compilation fails
    • our static compilation is conditioned on STATIC_ACTONC=true, which is set
      in our build-debs CI job which builds the .deb packages used in the Acton
      APT repo
    • local compilation still defaults to a dynamically linked actonc
  • Parallel GC marking, speeding up mark phase significantly on computers with
    many cores [#]
    • Up to 16 parallel threads are used for marking
  • Add sorted() function [#1211]
    • Uses timsort, which is the same sorting algorithm used by Python and Java's
      Arrays.sort()
  • Add re module for regular expression matching [#1208]
    • Offers match() function to match on text,
      • Unanchored pattern matching, unlike the Python equivalent which is
        anchored at the start per default (seems highly unintuitive)
  • Completely revamped time module [#1188]
    • The high level interface is now oriented based on the two common usage
      patterns of measuring elapsed time or telling the current date and time.
    • Stopwatch measure elapsed time
    • time.now() now returns a DateTime object, which is a higher level object
  • New fixed width integers: i16, i32, i64, u16, u32, u64 [#1202] [#1202] [#1203]
  • Backend queue subscriptions now done in bulk [#1127]
    • An actor mailbox is mapped to a queue in the backend
    • For every actor creation, the RTS would create and subscribe to the
      corresponding queue.
    • There's a new interface for queue subscriptions where individual queues are
      grouped together into queue groups, so the RTS only needs to subscribe to a
      group once
  • New acton projects are now git initialized [#1185]
    • Using actonc new foo to create a new Acton project, we now check if git is
      available and if so, initialize the repository. Also include a .gitignore
  • Allow overriding number of worker threads [#1199]
    • It's been possible to set but we always used a minimum of 4, so even setting
      threads to 2 you would end up with 4. It can be really useful for debugging
      to run on a single WT, so honoring this is useful.
  • Add list methods clear, extend & pop [#1206]
  • New --rts-no-bt option to disable automatic backtrace printing [#1226]
    • Automatic backtrace printing is nice for the normal case where one might run
      a program and do not have core dumps enabled
    • However, when doing actual debugging, one usually enables core dumps and
      analysis it using a debugger interactively, in which case it's only annoying
      to get a couple of screenfuls of backtrace in the terminal on every
      iteration
  • Laid foundation for exception handling [#1228]
    • Handles exceptions for CPS converted code
    • Proc style functions do not currently handle exceptions properly
    • Not fully enabled just yet, as it requires some more work
  • int literals between 0 and 256 are now statically allocated [#1235]
    • Python uses the same trick for -5 to 255 for speedup for commonly used
      integers
    • Acton literal negative integers are really not negative integers but an
      expression of "minus" and the integer 5 so we cannot currently do the same
      trick for common negative values, like -1 etc
  • lists can now shrink to reduce memory usage [#1237]
    • When list length goes below half, the list will be reallocated and copied
      over to a new memory area

Changed

  • bash completion for actonc now completes .act files [#1246]
  • TCPIPListener interface has changed for callback registration [#1181]
    • The on_receive and on_error handler for individual client connections
      (TCPListenConnection) are now registered after instantiation of each
      TCPListenConnection actor by calling cb_install().
      • This is most appropriately done in the on_accept callback.
    • Unlike the old pattern, where we provided these callbacks up front to the
      TCPIPListener actor, having them registered per client connection
      naturally opens up to having different handlers per client.

Fixed

  • Fix @property handling & inference [#1207]
  • Fix bug in dict __eq__, so equality check now works [#1144]
  • min() & max() now work as they should [#1150]
  • isinstance now works correctly [#1124]
  • Fix compilation with return in nested branches [#1162]
  • Signal interrupts are now properly handled in acton.rts.sleep() [#1172]
    • Previously, a signal would interrupt the sleep and it would simply not sleep
      long enough. Now, if interrupted, the sleep is called with the remaining
      time until the full duration has passed.
  • Fix worker thread indexes in WTS rtsmon interface [#1176]
  • bool([]) now correctly returns False [#1193]
  • Correct hex, bin and ascii functions [#1196]
  • actonc docs --signs now works [#1197]
  • Correct pid is now reported in rtsmon [#1177]
  • Atomic mallocs are now used where possible to speed up GC [#1225]
    • The GC does not need to scan the value of a str or bytearray, so they
      are now atomically allocated which means the GC will skip scanning them
  • Document more list operations [#1136]
  • Explain named tuples [#1192]
  • Improved some parser errors [#1198]
  • Now using zig v0.10.1 [#1174]
  • Update to LTS 18.28 [#1141]
  • Internal name mangling has been changed [#1160]
  • builtins has been reorganized [#1169]
  • New make clean-all target to clean move stuff [#1210]

Testing / CI

  • Add Ubuntu 18.04 as test platform in CI [#1149]
    • It's not possible to build Acton on Ubuntu 18.04
    • As actonc is now statically built, the version built on Debian now also runs
      on Ubuntu 18.04!
  • Test all available operations on lists [#1137]
  • Test all available operations on dicts [#1140]
  • Concurrent builds disabled on MacOS due to intermittent failures [#1145]
  • web page rebuild now triggered on changes to docs/acton-by-example [#1216]
  • Deb package now also gets a deterministic name [#1217] [#1219] [#1220]
    • Makes it much easier to install the pre-built tip packages for testing / CI
      etc on other repos

v0.14.2

27 Nov 16:59
9f2123a
Compare
Choose a tag to compare

Fixed

  • Improve actonc performance [#1119]
    • The constraint solver can have a massive performance impact on the
      compilation process, in particular if it has a very large amount of
      solutions to evaluate.
    • The number of alternative solutions had too little weight during constraint
      sorting which would result in a very large amount of potential solutions to
      evaluate. An example program took 31 minutes to compile, which after the fix
      compiles in milliseconds.
    • For a 100 constraints, with the wrong strategy we might need to evaluate
      5^100 solutions (heat death of universe etc) whereas if we do things
      correctly we can solve all constraints in perhaps 100*3 tries. Exponential
      is exponential.
  • Fixed size i64 integer type is now instantiable [#1118]
  • Improved str to int conversion [#1115]
  • Corrected str .split and .splitlines()
  • Correct DB server & client comm loop select handling [#1111]
    • Ignores EBADF and have ensured this design is correct with regards to timing
      of invalid fds.
  • Drop explicit gcc dependency for Debian package [#1110]
    • Haskell GHC still depends on it though, so it still gets installed.
  • Use slightly newer bsdgc / libgc version [#1112]
    • Slight build simplification as we've upstreamed some modifications.

v0.14.1

15 Nov 07:53
0ea2f39
Compare
Choose a tag to compare

Added

  • actonc --cc to specify which C compiler to use to compile a module and
    binary executables [#1103]
    • Note that the Acton system is still compiled by zig cc. On Linux we are
      targeting GNU libc 2.28, which might affect compilation of individual
      modules or executables.

Fixed

  • float.__ge__ is now working correctly [#1105]
  • Removed incremental operations from integers [#1106]
  • Improved int to str conversion [#1107]

v0.14.0

11 Nov 08:26
a4b8936
Compare
Choose a tag to compare

Acton RTS now does garbage collection!

Added

  • The Acton RTS now does garbage collection [#1091]
    • Using libgc a.k.a the Boehm-Demers-Weiser garbage collector
    • The GC stops the world to perform garbage collection during which all Acton
      RTS worker threads are paused. It is likely this will be visible as pauses
      of the whole system.
    • Performance appears to be largely on par with and without GC but this is
      based on small artificial programs.
    • It is possible to disable the GC at run time by setting the environment
      variable export GC_DONT_GC=1 before starting the Acton application
      • This will be removed in the future when the GC has been field proven.
    • Long term is to implement an Acton specific GC that can, for example, avoid
      stop the world events by doing collection per actor. This is quite far into
      the future. Until then, this will have to do!

v0.13.1

10 Nov 14:49
ba3e511
Compare
Choose a tag to compare

Changed

  • Allow _ as a dummy variable name [#1020] [#1061]
    • _ can be used in an assignment to effectively throw away a result
    • Unlike using a variable like dummy, _ acts as a wildcard from a type
      perspective, so that we can do _ = 3 and _ = "a", while if we attempt to
      use another dummy variable name we will get a type error as we try to assign
      it values with different types
  • __self__ has been renamed to self [#1056]
    • it is a reference to the own actor and holds the "external" view, i.e. it
      can be passed to another actor as a reference to the local actor

Fixed

  • Correct DB client comm thread to avoid potential deadlock [#1088]
    • Incorrect handling of error return status from select meant we could attempt
      to read on fds that had no data and thus the comm thread would deadlock.
    • Now we do proper error handling, always continuing for another spin &
      attempt at select in case we get an error back.
    • The (under development) garbage collector (GC) uses signals to instruct
      threads to pause for stop the world events and thus we end up interrupting
      the select loop a lot more frequently than before, thus surfacing this bug.
  • Correct DB client comm thread to avoid busy waiting [#1089]
    • On Linux, using select() with a timeout, if the select is interrupted the
      timeout value will be modified to reflect the time not slept. Thus the next
      select in our comm thread loop would sleep for a shorter period of time.
      Depending on the timing of the interrupt, the sleep might be shortened to
      effectively form a busy wait.
    • The (under development) garbage collector (GC) uses signals to instruct
      threads to pause for stop the world events and thus we end up interrupting
      the select loop a lot more frequently than before, thus surfacing this bug.
    • Fixed by always resetting the timeout value in the loop.
  • Make RTS DB clients stick to one partition [#1087]
    • Make sure RTS (a DB client) does not hop between different DB server
      partitions even if they learn about disjoint gossip views.
    • Prevents incorrect operation when DB servers are not aware of each other but
      are "joined" by a RTS that see all the servers.
    • This scenario is most easily reproduced by starting DB servers without
      specifying a seed, and so the DB servers won't see each other, and then let
      a RTS connect to the DB servers, which then sees a view of all 3 servers.
      • This is now rejected as invalid.
    • Using vector clocks to determine and reject invalid views. Very elegant =)
  • Correct time.monotonic() [#1097]
    • It returned a wildly incorrect results as the nanoseconds part was not
      properly added up with seconds.
  • Include argp-standalone library in libActonDeps [#1058]
    • No more external dependencies!
    • argp is available as part of the system on GNU/Linux but on MacOS we have
      relied on the argp-standalone package installed via brew. We now prefer to
      use our own on both Linux and MacOS.
  • Acton system is now compiled with -Werror to improve code quality [#1060]
    • There are some exceptions to this but the overall goal is to be essentially
      free of compilation warnings
  • Fix bug in truediv for int [#1076]
  • Fix bad codegen of classname argument to isinstance() [#1055]
  • Correct effect declaration to mut for some builtin protocols [#1053]
  • Method decorators like @staticmethod now work [#1054]
  • Added lost constraints inferred on target expressions [#1050]
  • Avoid undefined behavior in builtin object hash [#1065]
  • Clean up library include paths etc [#1068] [#1077] [#1080] [#1092]
    • Made possible by including all of our external dependencies in libActonDeps

Testing / CI

  • Add back testing on Ubuntu 20.04 [#1093]
    • libxml2 requires a newer automake (1.16.3) than is available on Ubuntu 20.04
    • We fix this by hacking the configure.ac file to require the version
      available on Ubuntu 20.04 (1.16.1)

v0.13.0

04 Nov 14:58
874550a
Compare
Choose a tag to compare

New "deactorizer", which unlocks proper async / sync actor method calls.

Added

  • Added new "deactorizer" pass in compiler [#374]
    • No real user visible change, like no change in syntax, but we now properly
      compile programs with regards to async / sync calling behavior of methods.
    • Briefly, an actor method called from the local method is called directly.
      This effect is called "proc". Remote actor methods are normally called
      asynchronously and these are called "action". If we assign the return value
      of an action, we are locking for a synchronous behavior which is achieved by
      an await. These semantics are now correctly implemented.
    • In particular, passing methods as arguments, a method might not know whether
      it is passed an action or proc and thus needs to handle this in a generic
      way. This is particularly tricky as we don't want to to any run time
      inspection of arguments and thus need to have it all figured out at compile
      time.
    • Many many many other things are fixed through the merge of the new
      deactorizer. There are improvements and fixes to various passes in the
      compiler. This has been in the works for almost a year.
  • Added --auto-stub to actonc [#1047]
    • Enables automatic detection of stub mode compilation
  • Extended actor argument pruning analysis to honour NotImplemented [#524]
    • Pruning analysis prunes away arguments that are not used under the lifetime
      of an actor, e.g. an argument only used for actor body initialization code.
      Pruned arguments are not persisted.
    • Pruning analysis does not cover C code, so when one or more methods are
      implemented in C and defined as NotImplemented in the Acton module, we
      cannot reliably determine what arguments are used or unused.
    • The safe choice is to assume all arguments are used, which is what we are
      now doing.
    • This removes a bunch of _force_persistance methods in stdlib.

Changed

  • Default is now to not automatically detect stub mode [#1047]
    • Use --auto-stub to enable automatic stub mode detection

Fixed

  • Now using zig v0.10.0, which was recently released [#1029]
    • Previously using a nightly build of v0.10
  • Correct arithmetic operations using hexadecimal literals [#1027]
  • Build actondb using zig with -target [#1003]

v0.12.0

27 Oct 16:36
cf9d322
Compare
Choose a tag to compare

Edvin's second birthday, only 10 minor releases behind Acton ;)

Added

  • Zig is now used as the C compiler for Acton [#972]
    • Many parts of Acton, like the run time system, builtins and parts of the
      standard library are written in C
    • actonc compiles Acton programs into C which are then compiled into binary
      executables using a C compiler
    • zig is now used for both of these use cases
    • zig is bundled with Acton so that we know what version of zig we get, which
      is the same across Linux and MacOS
  • Acton programs on Linux are now backwards compatible to GNU libc v2.28
    • Previously, acton programs would be built for the GNU libc version on the
      compiling computer, making it impossible to run on older distributions
    • Now compatible with for example Ubuntu 20.04 LTS & Debian 10
    • This is made possible by zigs incredible cross-compilation functionality,
      which isn't just about targetting other OS, CPUs but also older libc
      versions
    • v2.28 appears to be the oldest version we can practically target without
      code changes, even earlier versions fail compilation
    • We could reduce backwards compatible, for example by targetting glibc 2.31,
      if we find things that we would like access to, i.e. don't regard glibc 2.28
      compatibility as a hard promise
  • Added --always-build to actonc [#988]
    • Used in test suite so that we always force compilation in order to ensure
      valid test results
  • actonc argument parsing now done using sub-parsers, allowing greater freedom
    in constructing more complex commands [#976]
  • All external library dependencies are combined in one .a archive [#849]
    • Decouples library dependency in builtins, RTS & stdlib from actonc compiler
      arguments
  • All external library dependencies are now built from source [#984]
    • Allows us to compile them with zig, thus power to select target libc
  • Add xml module [#835]
    • Offers simple encode() and decode() operations
    • Based on libxml2
  • The int type now supports arbitrary precision [#146]
    • Add some big numbers!
    • The previous implementation of int was as a signed 64 bit integer, this is
      now available as the i64 type in Acton, although its use is currently
      quite limited.
    • Longer term, fixed size integers like i64 or u64 (for an unsigned) are
      available and can be chosen for high performance use cases. Developers
      familiar with this level of coding are likely able to make an informed
      choice about what integer type to choose. Using int for the arbitrary
      precision type is the safer choice which will make most users happy.
    • Using BSDNT library, which is a reasonably fast C implementation of
      arbitrary precision types and arithmetic operations. It is not as fast as
      MPL but has a more liberal license.

Fixed

  • Fixed seed arg parsing in actondb [#900]
  • Avoid crash during TCP client resume
  • DB now using logging rather fprintf(stderr, ...)
  • Reduce unprovoked CPS & 'rtail' bugs [#949]
  • Update docs for --root & runacton [#950]
  • file.ReadFile can now read arbitrarily large files [#955]
  • RTS does proper error handling of DB interaction [#957]
    • This is a huge improvement!!!
    • The run time system now properly reads the return code from all DB query
      operations and acts appropriately. For example, lacking a quorum, the RTS
      will continuously retry an operation until quorum can be reached.
    • For some failures a larger chunk of operations needs to be retried, i.e. we
      can't just wrap up a small function in a retry loop but need a larger retry
      loop around a group of operations.
    • DB API now returns a minority status which the RTS can and does react on
      • Typical example is with 3 DB nodes and a commit goes through on 2, i.e.
        with a quorum and is thus considered a success, however the 3rd node
        rejected it because of a schema mismatch. The RTS can now be notified
        through the minority status and attempt to repair the schema.
        • A typical example of schema repairs necessary are for the queues, which
          are dynamic. Every actor gets a queue in the database, so when new
          actors are created, new queues also need to be created and if a
          particular DB server is not up when the queue is created, it will be
          missing and needs to be repaired later on.
        • We should have proper sync up between DB servers, so that they query
          each other and converge on the latest state. Until then, repair
          initiated from RTS is our way of fixing it.
  • Notify gossip view to clients (RTS) from agreement round leader [#951]
    • For example, in this scenario:
      • db1 is started
      • RTS is started & connected to db1 but unable to progress due to no quorum
      • db2 is connected, gossips with db1 and we now have quorum
      • RTS continues to be stalled as it is never notified about db2 and thus
        unable to reach a quorum
  • Avoid MacOS quarantine shenanigans that kills actonc [#971]
    • MacOS has some quarantine concept, so like if a file is downloaded from the
      Internet, it is marked as quarantine and the user is asked "Are you sure you
      want to run this" after which the quarantine flag, which is stored as a file
      system extended attribute, is cleared.
    • Somehow we trigger this quarantine, presumably by that we overwrite, through
      cp, a binary executable that macos has run and this triggers it to set the
      quarantine flag. The effect is that whenever we execute actonc, MacOS kills
      -9 it, which is obviously rather annoying.
    • By copying to a temporary file and then moving it in place, we avoid MacOS
      setting the quarantine flag. We already did this, though for other reasons,
      for actondb, so in a way this is a unification for the files in bin/

Testing / CI

  • Simplify and clean up RTS DB test orchestrator [#941] [#973]
  • Stop testing on Ubuntu 20.04
    • It's not possible to compile libxml2 on a stock Ubuntu 20.04 as a newer
      version of automake is required than is shipped
    • We mainly want to uphold run time compatibility with Ubuntu 20.04, like it
      should be possible to run Acton applications but utilizing Ubuntu 20.04 as a
      development platform for Acton is not a high priority target, thus dropping
      it is quite fine.
  • Revamp Increase timeout
  • Avoid kill -9 on macos [#969]
    • MacOS quarantine functionality thinks we are doing something fishy and kill
      -9 our process
    • Worked around by doing cp & mv instead of just a cp

v0.11.7

03 Oct 17:02
6fb049f
Compare
Choose a tag to compare

Many important fixes for RTS/DB and the language in general!

Added

  • Bash completion is now part of the Debian packages & brew formula

Changed

  • actondb now uses a default value for gossip port of RPC port +1 [#913]
    • The gossip protocol only propagates the RPC port & parts of the
      implementation has a hard-coded assumption that the gossip port has a +1
      offset
    • In order to avoid configuration errors, the default gossip port is now RPC
      port + 1 and if another gossip port is explicitly configured, an error log
      message is emitted on startup.
    • While this is marked as a change, it could really be considered a fix as any
      other configuration of the system was invalid anyway.

Fixed

  • Fixed include path for M1
    • /opt/homebrew/include added to header include path [#892]
    • Actually fixes builds on M1!
    • This has "worked" because the only M2 where Acton was tested also had header
      files in /usr/local/include but on a fresh install it errored out.
  • Fix up-to-date check in compiler for imported modules from stdlib [#890]
  • Fix seed arg parsing in actondb that lead to "Illegal instruction" error
  • Fix nested dicts definitions [#869]
    • Now possible to directly define nested dicts
  • Avoid inconsistent view between RTS & DB in certain situations [#788]
    • If an RTS node was stopped & quickly rejoins or if a transient partition
      happens and the gossip round does not complete before the partition heals.
    • We now wait for gossip round to complete.
    • This ensures that local actor placement doesn't fail during such events.
  • Fix handling of missed timer events [#907]
    • Circumstances such as suspending the Acton RTS or resuming a system from the
      database could lead to negative timeout, i.e. sleep for less than 0 seconds.
    • The libuv timeout argument is an uint64 and feeding in a negative signed
      integer results in a value like 18446744073709550271, which roughly meant
      sleeping for 584 million years, i.e. effectively blocking the RTS timerQ.
    • It's now fixed by treating negative timeouts as 0, so we immediately wake up
      to handle the event, however late we might be.
  • Timer events now wake up WT threads after system resumption [#907]
    • Worker Threads (WT) are created in NoExist state and should transition
      into Idle once initiated, however that was missing leading to a deadlock.
    • This was masked as in most cases, a WT and will transition into Working
      once they've carried out some work and then back into Idle
    • wake_wt function, which is called to wake up a WT after a timer event is
      triggered, wakes up threads that are currently in Idle state, if they are
      in NoExist, it will do nothing.
    • If there is no work, such as the case after system resumption from the DB,
      WTs will stay in the NoExist state and then wake_wt will do nothing, so
      the system is blocked.
    • WT now properly transition into Idle.
  • Only communicate with live DB nodes from RTS DB client [#910] [#916]
    • When the RTS communicates with the DB nodes, we've broadcast messages to all
      servers we know about. If they are down, they've had their socket fd set to
      0 to signal that the server is down. However, fd=0 is not invalid, it is
      stdin, so we ended up sending data to stdin creating lots of garbage output
      on the terminal.
    • fd -1 is used to signal an invalid fd, which prevents similar mistakes.
    • The DB node status is inspected and messages are only sent to live servers.
  • Avoid segfault on resuming TCP listener & TCP listener connection [#922]
    • Invalidate fds on actor resumption [#917]
  • Remove remaining ending new lines from RTS log messages [#926]
  • Remove ending new lines from DB log messages [#932]

Testing / CI

  • Rewritten RTS / DB tests [#925] [#929]
    • More robust event handling, directly reacting when something happens, for
      example if a DB server segfaults or we see unexpected output we can abort
      the test
    • Now has much better combined output of DB & app output for simple
      correlation during failures
    • Test orchestrator now written in Acton (previously Python), at least async
      IO callback style is better supported to directly react to events...