Version 592

misc

• the 'read' autocomplete dropdown has a new one-click 'clear search' button, just beside the favourites 'star' menu button. the 'empty page' favourite is removed from new users' defaults
• in an alteration to the recent Autocomplete key processing, Ctrl+c/Ctrl+Insert will now propagate to the results list if you currently have none of the text input selected (i.e. if it would have been a no-op on the text input, we assume you wanted whatever is selected in the list)
• in the normal thumbnail/viewer menu and review services, the 'files' entry is renamed to 'locations'. this continues work in the left hand button of the autocomplete dropdown where you set the 'location', which can be all sorts of complicated things these days, rather than just 'file service key selector'. I don't think I'll rename 'my files' or anything, but I will try to emphasise this 'locations' idea more when I am talking about local file domains etc.. in other places going forward; what I often think of as 'oh yeah the files bit' isn't actually referring to the files themselves, but where they are located, so let's be precise
• last week's tag pair filtering in tags->migrate tags now has 'if either the left or right of the pair have count', and when you hit 'Go' with any of the new count filter checkboxes hit, the preview summary on the yes/no confirmation dialog talks about it
• any time a watcher subject is parsed, if the text contains non-decoded html entities (like >), they are now auto-converted to normal chars. these strings are often ripped from odd places and are only used for user display, so this just makes that simpler
• if you are set to remove trashed files from view, this now works when the files are in multpile local file domains, and you choose 'delete from all local file services', and you are looking at 'all my files' or a subset of your local file domains
• we now log any time (when the client is non-idle) that a database job's work inside the transaction wrapper takes more than 15 seconds to complete
• fixed an issue caused by the sibling or parents system doing some regen work at an unlucky time

default downloaders

• thanks to user help, the derpibooru post parser now additionally grabs the raw markdown of a description as a second note. this catches links and images better than the html string parse. if you strictly only want one of these notes, please feel free to dive into network->downloaders->defailt import options for your derpi downloader and try to navigate the 'note import options' hell I designed and let me know how it could be more user friendly

parsing system

• added a new NESTED formula type. this guy holds two formulae of any type internally, parsing the document with the first and passing those results on to the second. it is designed to solve the problem of 'how do I parse this JSON tucked inside HTML' and vice versa. various encoding stuff all seems to be handled, no extra work needed
• added Nested formula stuff to the 'how to make a downloader' help
• made all the screenshot in the parsing formula help clickable
• renamed the COMPOUND formula to ZIPPER formula
• all the 'String Processor' buttons across the program now have copy and paste buttons, so it is now easy to duplicate some rules you set up
• in the parsing system, sidecar importer, and clipboard watcher, all strings are now cleansed of errant 'surrogate' characters caused by the source incorrectly providing utf-16 garbage in a utf-8 stream. fingers crossed, the cleansing here will actually fix problem characters by converting them to utf-8, but we'll see
• thanks to a user, the JSON parsing system has a new 'de-minify json' parsing rule, which decompresses a particular sort of minified JSON that expresses multiply-referenced values using list positions. as it happened that I added NESTED formulae this week, I wonder if we will migrate this capability to the string processing system, but let's give it time to breathe

client api

• fixed the permission check on the new 'get file/thumbnail local path' commands--due to me copy/pasting stupidly, they were still just checking 'search files' perm
• added /get_files/local_file_storage_locations, which spits out the stuff in database->move media files and lets you do local file access en masse
• added help and a unit test for this new command
• the client api version is now 72

some security/library updates

• the 'old' OpenCV version in the (a)dvanced setup, which pointed to version 4.5.3.56, which had the webp vulnerability, is no longer an option. I believe this means that the program will no longer run on python 3.7. I understad Win 7 can run python 3.8 at the latest, so we are nearing the end of the line on that front
• the old/new Pillow choice in (a)dvanced setup, which offered support for python 3.7, is removed
• I have added a new question to the (a)dvanced venv setup to handle misc 'future' tests better, and I added a new future test for two security patches for setuptools and requests:
• A) setuptools is updated to 70.3.0 (from 69.1.1) to resolve a security issue related to downloading packages from bad places (don't think this would ever affect us, but we'll be good)
• B) requests is updated to 2.32.3 (from 2.31.0) to resolve a security issue with verify=False (the specific problem doesn't matter for us, but we'll be good)
• if you run from source and want to help me test, you might like to rebuild your venv this week and choose the new future choice. these version increments do not appear to be a big deal, so assuming no problems I will roll these new libraries into a 'future' test build next week, and then into the normal builds a week after

boring code cleanup

• did a bunch more super() refactoring. I think all __init__ is now converted across the program, and I cleared all the normal calls in the canvas and media results panel code too
• refactored ClientGUIResults into four files for the core class, the loading, the thumbnails, and some menu gubbins. also unified the mish-mash of Results and MediaPanel nomenclature to MediaResultsPanel