Version 583

docs/PTR.md

@@ -16,7 +16,7 @@ The easiest method is to use the built in function, found under `help -> add the
 Once you are connected, Hydrus will proceed to download and then process the update files. The progress of this can be seen under `services -> review services -> remote -> tag repositories -> public tag repository`. Here you can view its status, your account (the default account is a shared public account. Currently only janitors and the administrator have personal accounts), tag status, and how synced you are. Being behind on the sync by a certain amount makes you unable to push tags and petitions until you are caught up again.
 
 !!! note "QuickSync 2"
-    If you are starting out with a completely fresh client, you can instead download a fully pre-synced client [here](https://breadthread.duckdns.org/) Though a little out of date, it will nonetheless save time. Some settings may differ from the defaults of an official installation.
+    If you are starting out with a completely fresh client (i.e. you have not imported any files yet), you can instead download a fully pre-synced client [here](https://breadthread.duckdns.org/) ([overview](https://cuddlebear92.github.io/Quicksync/)) Though a little out of date, it will nonetheless save processing time. Some settings may differ from the defaults of an official installation.
 
 ## How does it work?
 For something to end up on the PTR it has to be pushed there. Tags can either be entered into the tag service manually by the user through the `manage tags` window, or be routed there by a parser when downloading files. See [parsing tags](getting_started_downloading.md). Once tags have been entered into the PTR tag service they are pending until pushed. This is indicated by the `pending ()` that will appear between `tags` and `help` in the menu bar. Here you can chose to either push your changes to the PTR or discard them.

docs/access_keys.md

@@ -66,6 +66,6 @@ Then you can check your PTR at any time under _services->review services_, under
 
 ## jump-starting an install { id="quicksync" }
 
-A user kindly manages a store of update files and pre-processed empty client databases to get your synced quicker. This is generally recommended for advanced users or those following a guide, but if you are otherwise interested, please check it out:
+A user kindly manages a store of update files and pre-processed empty client databases. If you want to start a new client that syncs with the PTR (i.e. you have not started this new client and not imported any files yet), this will get you going quicker. This is generally recommended for advanced users or those following a guide, but if you are otherwise interested, please check it out:
 
 [https://cuddlebear92.github.io/Quicksync/](https://cuddlebear92.github.io/Quicksync/)

docs/changelog.md

@@ -7,6 +7,31 @@ title: Changelog
 !!! note
     This is the new changelog, only the most recent builds. For all versions, see the [old changelog](old_changelog.html).
 
+## [Version 583](https://github.com/hydrusnetwork/hydrus/releases/tag/v583)
+
+### new
+
+* added a 'command palette' options page. the ability to show menubar and media actions is now set here with a couple of checkboxes (previously these were hidden behind advanced mode)
+* you can also now search for 'page of pages' with the command palette. this is turned on in the same new options page
+* the new sidecar test panel now only shows its notebook tabs if there is more than one source. the tab labels, when shown, are now '1st' ... '2nd' ordinal strings respective to the source list on the left
+* added some explanation text, a button to the help docs, and a bit of better layout to the sidecars UI
+* the file import report mode now writes the 'nice human description' for the file import to the log. this will expose the source URL or local file path for better context than just the random temp path
+* in the 'manage times' dialog, you can now set the same timestamp (including cascading timestamps) to multiple domains or file services at once. the logic is a little delicate where some files are in one domain in your selection and others are only in another, but I think I got it working ok. if you have a complicated setup, let me know how you get on!
+* the 'Dark Blue' stylesheets and the new 'Purple' stylesheet have updated custom colours
+
+### fixes
+
+* fixed a typo in the 'fetch a mappings petition' server code
+* when the client is called to render an image that was just this moment removed from the file store, the image renderer should now give you a nicer error image with an appropriate messare rather than throwing an error popup
+* hydrus no longer applies EXIF Orientation (rotation) to PNG files, which it recently started doing automatically when we started scanning PNGs for EXIF. EXIF is not well-defined and supported for PNG, and if an Orientation row exists, it is likely a false positive mistake of some encoder. hydrus will show this data but it will not apply it, and the metadata review UI now also notes this
+* fixed the 'has exif?' file maintenance job, which I think was false-negativing!
+
+### code and help
+
+* the `psutil` library is now technically optional. it is still needed for a bunch of normal operations like 'is the client currently running?' and 'how much free space is there on this drive?', but if it is missing the client will now boot and try to muddle through anyway
+* did more multi-column list 'select, sort, and scroll' tech for: the url class parameters list; the url class links list; the external launch paths list; the tag suggestion related tags namespace lists; import folder filename tagging options; manage custom network context headers; the string to string match widget list; the string match to string match widget list; the edit subscription queries list; and more of the edit subscriptions subscription list, including the merge and separate actions
+* updated the QuickSync help a little, clarifying it is only useful for _new, empty_ clients
+
 ## [Version 582](https://github.com/hydrusnetwork/hydrus/releases/tag/v582)
 
 ### fixes
@@ -328,45 +353,3 @@ title: Changelog
 * deleted the local booru service db/init code
 * deleted the local booru service network code
 * on update, the local booru service will be deleted from the database
-
-## [Version 573](https://github.com/hydrusnetwork/hydrus/releases/tag/v573)
-
-### new autocomplete tab, children
-
-* **this is an experiment. it is jank in form and workflow and may be buggy**
-* the search/edit tag autocomplete dropdowns now have a third tab, 'children', which shows the tag children of the current tag context, whether that is the current search tags or what you are editing
-* the idea is you type 'series:evangelion' but can't remember the character names; now you have a nice list of a bunch of stuff related to what was already entered
-* note you can select this tab real quick just by hitting 'left arrow' on an empty text input
-* this is a first draft, and I would like feedback and ideas, mostly around workflow improvement ideas. it seems to work ok if you have one or two tags with interesting children, but against a big list of stuff, it just becomes another multi-hundred list of spam blah that is difficult to navigate. maybe I could filter it to (and sort by?) the top n most count-heavy results?
-* I wonder if it could also show children on the same level, so if you have 'shinji', it'll also show 'rei' and 'asuka'. I would call this relationship 'siblings', but then we'd be in an even bigger semantic mess
-* also obviously please let me know if this fails anywhere. I think I have it hooked up correct, but some of the code around here is a bit old/messy so some scenario may not update properly
-* don't worry about background lag if you regularly manage lots of tags--it only actually fetches the list of children when you switch to the tab, so you're only spending CPU if you actively engage with it
-
-### misc
-
-* a user and I figured out a new twitter tweet downloader using the excellent fxtwitter mirror service. it doesn't do search, but dropping a tweet URL on the client should work again. should handle quoted media and works for multi/mixed-image/video posts, too. note it will nest-pursue quoted tweets, so if there's like fifty in the nested chain, it'll get them all--let me know if this is a big pain and I'll figure out a different solution. I learned that there is another twitter downloader made by a different user on the discord; I have made the update code check for this and not replace it with this if you have it already, and I expect I'll integrate what that can do into these defaults next week
-* the archive/delete and duplicate filters now yes/no confirm when you say to 'forget' at the end of a filtering run
-* the duplicate filter page now only allows you to set the search location to local file domains--so it'll only ever try to search and show pairs for files you actually have
-* fixed the system predicate parsing of `system:duration: has duration` and `system:duration: no duration` when entered by hand, and added a unit test to catch it in future
-* the manage siblings/parents dialogs now have a little shorter minimum height
-* updated some text around the PTR processing in the help--it is only the database proper, the .db files normally in `install_dir/db`, that needs to be on an SSD, and temporary processing slowdowns to 1 row/s are normal
-* touched up some of the 'installing' and 'running from source' help, particularly for some Linux vagaries
-
-### some build stuff
-
-* all the builds and the setup_venv scripts are moved from 'python-mpv' to 'mpv', the new name for this library, and the version is updated to 1.0.6, which supports libmpv version >=0.38.x. if you are a windows user and want to live on the edge, feel free to try out this very new libmpv2.dll here, which I have been testing and seems to work well: https://sourceforge.net/projects/mpv-player-windows/files/libmpv/mpv-dev-x86_64-20240421-git-b364e4a.7z/download
-* updated the setup_venv scripts' Qt step to better talk about which Qt version to use for which Python version. it turns out Python 3.12 cannot run something I was recommending for >=3.11, so the whole thing is a lot clearer now
-
-### boring stuff
-
-* refactored some question/button dialog stuff
-* fixed up some file domain filtering code in the autocomplete filter and variable names to better specify what is being filtered where
-
-### local booru deconstruction
-
-* _reminder: I am removing the local booru, an ancient, mostly undocumented experiment._ if you used it, please check out https://github.com/floogulinc/hyshare for a replacement!
-* the local booru service no longer boots as a server
-* deleted the local booru share cache
-* the local booru review services panel no longer shows nor allows management of its shares
-* deleted the local booru unit tests
-* deleted the local booru help and ancient screenshots

docs/developer_api.md

@@ -742,7 +742,7 @@ Response:
 
 The `url_file_statuses` is a list of zero-to-n JSON Objects, each representing a file match the client found in its database for the URL. Typically, it will be of length 0 (for as-yet-unvisited URLs or Gallery/Watchable URLs that are not attached to files) or 1, but sometimes multiple files are given the same URL (sometimes by mistaken misattribution, sometimes by design, such as pixiv manga pages). Handling n files per URL is a pain but an unavoidable issue you should account for.
 
-`status` is the same as for `/add_files/add_file`:
+`status` mas the same mapping as for `/add_files/add_file`, but the possible results are different:
 
   *   0 - File not in database, ready for import (you will only see this very rarely--usually in this case you will just get no matches)
   *   2 - File already in database

docs/old_changelog.html

@@ -34,6 +34,28 @@
         <div class="content">
             <h1 id="changelog"><a href="#changelog">changelog</a></h1>
             <ul>
+                <li>
+                    <h2 id="version_583"><a href="#version_583">version 583</a></h2>
+                    <ul>
+                        <li><h3>new</h3></li>
+                        <li>added a 'command palette' options page. the ability to show menubar and media actions is now set here with a couple of checkboxes (previously these were hidden behind advanced mode)</li>
+                        <li>you can also now search for 'page of pages' with the command palette. this is turned on in the same new options page</li>
+                        <li>the new sidecar test panel now only shows its notebook tabs if there is more than one source. the tab labels, when shown, are now '1st' ... '2nd' ordinal strings respective to the source list on the left</li>
+                        <li>added some explanation text, a button to the help docs, and a bit of better layout to the sidecars UI</li>
+                        <li>the file import report mode now writes the 'nice human description' for the file import to the log. this will expose the source URL or local file path for better context than just the random temp path</li>
+                        <li>in the 'manage times' dialog, you can now set the same timestamp (including cascading timestamps) to multiple domains or file services at once. the logic is a little delicate where some files are in one domain in your selection and others are only in another, but I think I got it working ok. if you have a complicated setup, let me know how you get on!</li>
+                        <li>the 'Dark Blue' stylesheets and the new 'Purple' stylesheet have updated custom colours</li>
+                        <li><h3>fixes</h3></li>
+                        <li>fixed a typo in the 'fetch a mappings petition' server code</li>
+                        <li>when the client is called to render an image that was just this moment removed from the file store, the image renderer should now give you a nicer error image with an appropriate messare rather than throwing an error popup</li>
+                        <li>hydrus no longer applies EXIF Orientation (rotation) to PNG files, which it recently started doing automatically when we started scanning PNGs for EXIF. EXIF is not well-defined and supported for PNG, and if an Orientation row exists, it is likely a false positive mistake of some encoder. hydrus will show this data but it will not apply it, and the metadata review UI now also notes this</li>
+                        <li>fixed the 'has exif?' file maintenance job, which I think was false-negativing!</li>
+                        <li><h3>code and help</h3></li>
+                        <li>the `psutil` library is now technically optional. it is still needed for a bunch of normal operations like 'is the client currently running?' and 'how much free space is there on this drive?', but if it is missing the client will now boot and try to muddle through anyway</li>
+                        <li>did more multi-column list 'select, sort, and scroll' tech for: the url class parameters list; the url class links list; the external launch paths list; the tag suggestion related tags namespace lists; import folder filename tagging options; manage custom network context headers; the string to string match widget list; the string match to string match widget list; the edit subscription queries list; and more of the edit subscriptions subscription list, including the merge and separate actions</li>
+                        <li>updated the QuickSync help a little, clarifying it is only useful for _new, empty_ clients</li>
+                    </ul>
+                </li>
                 <li>
                     <h2 id="version_582"><a href="#version_582">version 582</a></h2>
                     <ul>

hydrus/client/ClientController.py

@@ -1,8 +1,6 @@
 import hashlib
 import os
-import typing
 
-import psutil
 import signal
 import sys
 import threading
@@ -19,6 +17,7 @@
 from hydrus.core import HydrusExceptions
 from hydrus.core import HydrusGlobals as HG
 from hydrus.core import HydrusNumbers
+from hydrus.core import HydrusPSUtil
 from hydrus.core import HydrusSerialisable
 from hydrus.core import HydrusThreading
 from hydrus.core import HydrusTime
@@ -2250,15 +2249,18 @@ def SystemBusy( self ):
 
             if HydrusTime.TimeHasPassedMS( self.GetTimestampMS( 'last_cpu_check' ) + 60000 ):
 
-                cpu_times = psutil.cpu_percent( percpu = True )
-
-                if len( [ 1 for cpu_time in cpu_times if cpu_time > system_busy_cpu_percent ] ) >= system_busy_cpu_count:
-
-                    self._system_busy = True
+                if HydrusPSUtil.PSUTIL_OK:
 
-                else:
+                    cpu_times = HydrusPSUtil.psutil.cpu_percent( percpu = True )
 
-                    self._system_busy = False
+                    if len( [ 1 for cpu_time in cpu_times if cpu_time > system_busy_cpu_percent ] ) >= system_busy_cpu_count:
+
+                        self._system_busy = True
+
+                    else:
+
+                        self._system_busy = False
+
 
 
                 self.TouchTime( 'last_cpu_check' )

hydrus/client/ClientFiles.py

@@ -693,13 +693,23 @@ def _GetFileStorageFreeSpace( self, hash: bytes ) -> int:
 
                 free_space = HydrusPaths.GetFreeSpace( base_location.path )
 
+                if free_space is None:
+
+                    free_space = 0
+
+
                 self._locations_to_free_space[ base_location ] = ( free_space, HydrusTime.GetNow() )
 
 
         else:
 
             free_space = HydrusPaths.GetFreeSpace( base_location.path )
 
+            if free_space is None:
+
+                free_space = 0
+
+
             self._locations_to_free_space[ base_location ] = ( free_space, HydrusTime.GetNow() )
 
 
@@ -2420,7 +2430,7 @@ def _HasEXIF( self, media_result ):
 
                 raw_pil_image = HydrusImageOpening.RawOpenPILImage( path )
 
-                has_exif = HydrusImageMetadata.HasEXIF( path )
+                has_exif = HydrusImageMetadata.HasEXIF( raw_pil_image )
 
             except:
 

hydrus/client/ClientOptions.py

@@ -257,7 +257,10 @@ def _InitialiseDefaults( self ):
             'do_icc_profile_normalisation' : True,
             'mpv_available_at_start' : ClientGUIMPV.MPV_IS_AVAILABLE,
             'do_sleep_check' : True,
-            'override_stylesheet_colours' : False
+            'override_stylesheet_colours' : False,
+            'command_palette_show_page_of_pages' : False,
+            'command_palette_show_main_menu' : False,
+            'command_palette_show_media_menu' : False
         }
 
         #

hydrus/client/ClientRendering.py

@@ -245,10 +245,10 @@ def _Initialise( self ):
         # do this here so we are off the main thread and can wait
         client_files_manager = CG.client_controller.client_files_manager
 
-        self._path = client_files_manager.GetFilePath( self._hash, self._mime )
-
         try:
 
+            self._path = client_files_manager.GetFilePath( self._hash, self._mime )
+
             self._numpy_image = HydrusImageHandling.GenerateNumPyImage( self._path, self._mime )
 
         except Exception as e: