v1.8.0b

This release contains many workflow improvements, error-handling improvements,
and bug fixes in preparation for Crystal to exit beta status.

It is faster than ever before to define the structure of a site using the UI,
with support for loose browser-style URL entry, better guesses for names and
sources of entities, and the ability to rename entities after creation.

• First-time-run experience improvements

  • New Root URL Dialog:
    • Accept URLs in loose format, similar to what regular web browsers accept.
    • Better error message when try to create duplicate root URL.
    • Disallow create of empty root URL.
  • New Group Dialog:
    • Improve suggested source when creating a new group.
  • New Root URL and New Group Dialogs:
    • Improve suggested name when creating a new root URL or group.
    • Make it optional to provide a name.
    • Rearrange fields to deemphasize the name field.
    • Update the selected node in the Entity Tree intelligently after
      creating or forgetting a root URL or a group.
    • Allow resizing.
    • Ignore leading and trailing whitespace in URLs and URL patterns.
  • Main Window
    • Prevent resizing the window to be too small.
  • Use ⚓️ and 📁 icons consistently in the UI to refer to
    Root URLs and Groups respectively.

• Workflow improvements

  • Can now edit the name and source of Root URLs and Groups after creation.
  • Can mark resource group as "do not download" to prevent their members
    from being downloaded when in an embedded context.
  • By default the Default Domain will be set to match the first Root URL
    created, enabling reliable serving of more-modern websites with
    client-side URL routing.

• Improved support for Default URL Prefixes

  • Can now set a Default Domain when serving a downloaded project.
    Previously only a Default Directory could be set.
  • The Default Domain/Directory can be set to match a Root URL that
    is being created or edited.
  • A top-level menuitem can now be used to set the Default Domain/Directory
    to match an existing Root URL. Previously it was necessary to use a
    right-click menuitem instead.
  • Hovering over a URL or Group in the Entity Tree always shows the
    full URL or URL Pattern for the entity, even if a Default Domain
    or Default Directory is set.

• Downloading improvements

  • Remaining time is now reported while downloading groups whose members
    are slow to download, taking >7 seconds each.
  • Don't crash when downloading a group that already contains some member
    URLs that were already downloaded.
    • This crash bug was introduced in v1.7.0b with the new strategy of
      creating member download tasks on demand rather than upfront.
  • Don't crash when try to download a URL that is already downloading.
  • Don't crash when try to download a group that has no member URLs.

• Crawling improvements

  • Download the implicit favicon referenced by the root page of any domain.

• Parsing improvements

  • Can identify URL references to images inside <source srcset="...">.
  • Gracefully handle references to invalid URLs like "//*[@id='"
    rather than crashing.
  • Parse links from RSS and Atom feeds advertised with a specialized XML MIME type
    like application/rss+xml or application/atom+xml.

• Serving improvements

  • XML files like Atom feeds and RSS feeds are now served correctly,
    without introducing an invalid <script> tag.
  • The log showing HTTP requests made to the served project now always
    displays inside the main window, rather than attempting to appear as an
    attached drawer on macOS and Windows.
    • Drawers are not a concept not natively supported by any OS except
      macOS, and even there they are deprecated.
    • Drawers have never worked properly on Linux, due to Wayland not
      providing APIs to position windows precisely relative to each other.
    • The old drawer mode didn't stay attached to the main window properly
      when using Mission Control on macOS.

• Error handling improvements

  • If a task crashes, show it as crashed in the UI and allow it to be dismissed.
  • If the scheduler thread crashes, show it as crashed in the UI and allow it to be restarted.
  • If an update to the entity tree crashes,
    show the crash in the UI and allow the entity tree to be refreshed.

• Testing improvements

  • Waits now use a soft timeout in addition to a hard timeout,
    which makes it easier to tune/bump timeout durations as needed.
    • Triggering a soft timeout causes a warning to be logged.
  • Warnings logged during a test run are collected and reported
    at the end of the test run.
  • Warnings logged during a test run are reported to GitHub Actions
    as warning annotations.
  • A screenshot is taken automatically whenever a timeout error occurs
    and whenever a rich assertion method (from asserts.py) fails.
  • A terminal bell sound is played automatically when tests finish running.
  • When an abort() or SIGABRT occurs while running tests during continuous integration,
    print a stack trace using faulthandler.

• Major fixes

  • Fixed multiple cases where code updating the task tree accessed
    the task hierarchy without synchronizing with the scheduler thread,
    which could cause crashes when downloading groups containing members
    that were already downloaded in the same session.
    • Access to the task hierarchy is now protected with @scheduler_affinity
      and explicit is_synced_with_scheduler_thread() checks.
    • This issue was first introduced in v1.7.0b and is now fixed.
  • For projects on a non-SSD drive, fix issue where newly created groups
    did not find any member URLs that were discovered since the project was opened.
    It was previously necessary to reopen a project to reliably find all
    members of a recently created group.
    • This issue was first introduced in v1.7.0b and is now fixed.

• Minor fixes

  • Prevent system idle sleep while tasks are running,
    in more situations on macOS.
  • Fix disappearance of error nodes when new root URL or group is added.
  • If try to create group with empty URL pattern, show error dialog
    rather than silently failing.