API Proposals to Remove Frame Rate Limits #165
Comments
|
Thank you for doing this due diligence. I offered to do this proposal too -- but now that you've started this (and reused some of my writings too -- make sure you list me as a coauthor). I'll get behind this effort too! Any assistance I can offer in furthering this proposal, let me know. |
|
Thanks @JeffreyBlanz. Would you mind generalizing the proposal? Talking about "Apple's sales" etc. is probably not appropriate so it would be good to drop that. As you noted, this is not OS dependent and the mode could switch based on environmental conditions (e.g., the user could plug their device into a power source). For the proposal itself, I think it's presumptuous to ask for permission. As with Apple's Game Mode, the user has the ability to turn it off without needing to ask for permission: it's something that happens automatically upon request. Further, the refresh rate can change for whatever reason, so staying at the high frame rate is never guaranteed. As such, it might be good to look at making this an option when requesting fullscreen. As such, I'd suggest looking at That would allow the user agent to opt into the higher modes whenever requested, while leaving the user and the user agent to decide if/when it's enabled without exposing if the hardware supports such as mode. As you pointed out, this is also not about games... this has a lot of use cases beyond gaming, so a generalized opt in for high refresh rate would be appropriate. Given the above, can you refactor the proposal to take the above into consideration? |
IntroductionSince some here are not familiar with me, I would like to introduce myself better. I am the creator of Blur Busters and a popular motion testing website, testufo.com and HDR version at beta.testufo.com. I'm in over 35 research papers (See Google Scholar), and the best learning point about me is to read Blur Busters Area 51 Display Science, Research & Engineering. About TestUFO and Raison d'Etre of Easy Opt-In HFRTestUFO, at 8 figure count of unique visitors per year, is now the world's most popular high-framerate-educating website. It has many tests for teaching, for demos, for scientific research, etc. Some TestUFO's only require human eyes, and some require equipment (camera, high speed camera, oscilloscope). More interesting animation examples are Frame Rates Demo, Game Panning Demo, Black Frame Insertion Demo, Ghosting Test for Cameras, Local Dimming Test, Visual MPRT Test, Stutters-to-Blur Continuum Demo, and dozens others. I have been noticing that 120Hz is becoming gradually more pro-mainstream, and TestUFO has become more popular for users to see for themselves the benefits of high refresh rates. TestUFO was made 10 years ago when 120 Hz was still a much more luxury niche. Inconvenience of Low Frame Rate CapSome motion education pages are only properly educational at certain refresh rate and frame rate minimums, for certain physics & science reasons (e.g. Comparing two frame rates that are BOTH well above flicker fusion thresholds, e.g. 120fps versus 240fps). There are mudane physics reasons why I want the world's most popular Hz/framerate "education-in-a-link" to properly teach a user with minimum effort. Pick Poison Choice
Besides, no permission request is needed for Chrome, Edge, Safari, FireFox. Safari is the lone holdout, and the permission request is optional; only needed if not defaulting to uncapped.
A permission request is no longer sounding onerous anymore; given Windows Management API already does a permission API request too (to enumate the resolutions of all displays; that naturally can be done to include refresh rates and frame rates). So there are existing APIs that are congruent and consistent with the theory of doing a permission request.
(that begrudgingly added a way to override only via a hidden advanced setting). I provide a suggestion of permission request to give them the fewest excuses not to implement an easier way than the current status quo. Imagine, I'm going to display the popup. Many are going to fiddle with their unknown browser setting. But that also defeats a quick 5-second "education in a link". Where, on OLED where ergonomic benefits of refresh rate sare more massively visible, with 240Hz even mainstreaming for office use cases, and no longer just for gaming. A lot more inexperienced users are going to be presented with a daunting message, just for a webpage to teach a simple motion concept. To help sidestep internal bias of a large number of non-game developers on here who are unaware of the benefits of 240Hz for office/productivity/development situations (and OLED refresh rate improvements are massively more visible than LCD refresh rate improvements): We already know the boom of 120Hz in the premium non-gaming arena (e.g. Androids, iPhones, iPads, MacBook, Microsoft Surface, DELL, not just Xbox, PS2, VR), but that even goes beyond. TechSpot said in their 240Hz OLED review during office use:
So due to the boom of mainstream use of 120Hz (and beyond), with even Dell starting to sell 120Hz office monitors (e.g. Dell UltraSharp U2724DE), pushing 120Hz and 240Hz out of the gaming arena and slowly into the premium mainstream.... After all, 4K was a $10K luxury in 2001 (IBM T221) but a $299 walmart or amazon special nowadays. Same thing is (slowly) happening to refresh rates and frame rates. It doesn't always get used (e.g. we still view 1080p content on 4K, we still view 24fps hollywod), but the capability becomes there, including for mere office smooth scrolling or panning, which creates an ergonomic benefit that many Apple users now enjoy. Explainer: 120 Hz and 240 Hz Is No Longer For GamesUnfortunately, I have to add this section because a lot of developers are unfamiliar wit the ergonomic benefits of high-Hz, even outside gaming. The ergonomic benefit is much more dramatic on the 60-vs-120 OLED (2.0x human visibiility difference) than LCD (sometimes only 1.1x difference due to slow GtG). And even more dramatic for 120Hz vs 480Hz OLED, than 60Hz vs 120Hz due to the 4x geometric. The diminishing returns goes a long way, but it's based on geometric and not a cutoff. From moving map test www.testufo.com/map is this amount of motion blur, depending on which refresh rate you are at: But static images can only teach so much. I need to teach it en motion, and some educational motions are only properly educational at "Hz > X" where X may be a triple digit number. Easy Opt-In HFR...I just find an easy way for me to teach high-framerate-mandatory concepts to the millions of existing 120Hz Apple displays in an easy link click. How? It's impossible without fiddling an Advanced developer setting on many of them. |
EDIT Shorter Reply:
I volunteer to do it within this month!
Other browsers don't need to ask for permission (only specific browsers like Safari that choose to cap at 60fps). And you see the additional mainstream use cases beyond gaming. Apple is a privacy sensitive company. Plus, other display-related APIs ask for permission (Windows Management API) and this proposal could be rolled into that in theory (recycle the same permission).
It's not exclusive to fullscreen or gaming. 240-480Hz is good for office productivity and ergonomics too (scrolling, panning). My MacBook M1 scrolls fine on my 480Hz and 240Hz monitor, and looks miraculously CRT clear. 120 versus 480 at GtG=0 (OLED) has a bigger ergonomic scrolling difference than between 60Hz and 120Hz. TechSpot and other mainstream reviewers even said 240Hz was much better than expected for productivity (scrolling that looked better than even an OLED 120Hz iPad. 480Hz and 1000Hz, ultimately not for gaming anymore (in ten years), much like 4K is no longer a luxury product.
*** GENIE ALREADY OUT OF BOTTLE ALARM *** All major browsers (Chrome, FireFox, Edge, etc) do it on all platforms (Mac, Windows, Linux Wayland, Android), except for Safari. This ended up being kind of a defacto industry standard. It simply offers a simple solution for Apple & Safari that satisfies Apple's concern about privacy & battery (but can be open to any browser vendor). Imagine, a link that does a 5-second teaching demo, whether to educate about display physics, or simply to sell the benefits of high Hz. Why should users have to be instructed to go through a Developer menu for this? Old Long Reply:This was my original oversized reply, now hidden in `
` upon request by @marcoscaceres
That reference was because Apple was, at the time, the lone standout on violating section 7.1.4.2 of the older HTML 5.2 standard, in running TestUFO at an incorrect framerate, unlke Chrome, FireFox, Opera, Edge, and other browsers tested (Oculus in-VR browser, etc). See the explanation of my git commit I made in 2017. All browsers complied, except for Apple. That was before the move to WHATWG. They fixed it with (references need to be updated, I'll edit with link), but still requires manually configuring a setting "Prefer Page Rendering Updates Near 60fps" setting to be disabled. Scientific testing websites (of which some of my peer papers at www.blurbusters.com/research-papers ...) based on research at the various multiple different tests (upper-right corner) at www.testufo.com -- only run at 60fps. That site works reliably on Windows at any odd refresh rates (75Hz, 85Hz, 120Hz, 144Hz, 240Hz, 480Hz) and less so on Mac. However, now that it's been moved here, I'll review and remove references. I have a sense that (at least historically) there's an internal holy war between certain Apple employees on the benefits of refresh rates (power consumption, human visibility of 480Hz, etc), so my wording is intentionally constructured to prod the lone standout. I love Apple products, but they do need to fall in line with the modern science & research. NOTE: The diminishing curve of returns go a long way. Retina refresh rates is not until approximately 20,000fps 20,000Hz when the variables are set to the most extreme situations. Although you need major framerate=Hz differences, 4x geometric differences, to really tell apart, you need to temporally VHS-vs-8K, not incrementally 720p-vs-1080p, to fix the stroboscopics/persistence blurs when well beyond flicker fusion threshold. For the Hz/framerate-inexperienced people not aware of newer research and VR research, or never having seen framerates at geometric multiples beyond 60-120, see textbook articles that I've written, as well as links to peer reviewed research.
It's not presumptous. Various different Apple employees has already explicitly told me they are prioritizing on power. But people who wish to have better performance, or need to access research-quality motion-testing pages that demand exact framerates beyond 60. There are over 500 content creators (cherrypicked examples, some with multimillion viewership including RTINGS and LinusTechTips) using my free display testing inventions, some of which are driven by ....a HTML engine. One example is the sync track on this pursuit camera test at www.testufo.com/ghosting which is also used by multiple explained in extreme detail on this RTINGs page (with frequent mentions of Blur Busters and my name). Accurate enough to be research quality. However, on Apple, it throttles to 60fps and it's also harder to teach many display concepts at less than 120-240Hz. For example, see TestUFO Variable-Persistence Black Frame Insertion Demo. That annoyingly flickers at 60fps but much more educational at 120fps and 240fps. If I can't temporarily get permission from user (without forcing the user to manually configure the damn browsers' advanced settings) to run the animation at the full Apple 120Hz refresh rate... ...Then the animations' educational teaching value is lost. (BTW, I teach classrooms too) TestUFO was created because it is instant education-in-a-link. I can trust it to work on a Windows system in any browser running on it, for example. Why can't I conveniently do the same for a visiting Apple user running an Apple 120Hz display? So, I admonish, and turn the tables back: Why can't I ask the user temporarily for permission to run a specific animation (even in windowed mode) at 120 frames per second? It's not presumptous; and is consistent with other permissions that is also available. So, to allay Apple's concerns, I may have to ask permission, in order to avoid being forced to explain how to users to manually reconfigure their Safari browser to override a default setting. I don't care if permission is revoked. Sometimes the high-framerate permission is only temporarily needed, for the duration of a visit to a research-quality motion testing website. Some demos are for teaching, some demos are for reseach, some are for playing games, some are for enjoying HFR content, etc. But I cherrypick the teaching use case as very important, as some motion concepts are easier to teach at specific frame rates.
Why can't I automatically have high frame rates such as 120fps in non-fullscreen-mode, like I can do with any browser on a Windows system, on an Android device, on a VR headset, or on a recent Wayland Linux box?
*** BARN DOOR AND ESCAPED HORSES ALERT ****** CART BEFORE HORSE ALARM ****** GENIE ALREADY OUT OF BOTTLE ALARM ***We're not putting the genie back in the bottle. Let's not forget the thing about the barn door and horse thing: Native refresh rates are now easy to detect on all platforms except for Apple. See www.testufo.com/refreshrate ... Also, resolution-detecting APIs (doesn't need permission) and multidisplay-management APIs (Windows Mangement API, which prompts a permission request) are widespread. Since I want to show off teaching materials in a windowed manner, I think the 120Hz permission request could easily be part of the Windows Management API permission request, rather than a Game Mode. But there's overlap/congruencies, so we have to think this through. If Apple does not want to report the real refresh rate to me immediately, at least let me ask for permission without forcing me to display a popup message telling users how to manually change an advanced setting. I simply suggest an OPTIONAL permission request (that doesn't have to be done by all browser vendors, only vendors that are more jealously guarding the refresh rate) for vendors not already doing it unconditionally like Chrome, FireFox, Edge, Opera, etc. If browsers eventually want to put the genie back in the bottle (add frame rate caps), that interferes with display ergonomics, since there's newly discovered office-productivity ergonomics to having 2x-4x-8x more framerate & refreshrates now. Chrome is not going to do that. ' But I'd like to avoid useragent detection (display a complex message to Safari visitors).
Correct. Jeff Blanz, what do you think about me scrap #165 and reopen a brand new copy paste of this thread? Or maybe copy and paste my edits, as we can't share the same co-authoring account for a WICG proposal.
Sure; let me give it a go. |
@JeffreyBlanz Retitle Required"
|
JeffreyBlanz
changed the title
Extra Note to Apple EmployeesKudos on the MacBook M1, my main TestUFO development machine, in Visual Studio Code now. Watching for OLED MacBooks, hopefully 240Hz+
If you want to read more: BTW, @marcoscaceres I met a few Apple employees at DisplayWeek 2024 from the displays department (120hz screens for tablets/laptops/vision pro). You may have seen my UFO sticker on one of their MacBooks! That department is big fans of TestUFO. One of them helped me with a project years before he was hired by Apple. I do observe that a lot of Apple employees in the web department are less interested in 120Hz than the Apple display department, due to the prioritization of battery life. I understand. I love the battery life of my 120 Hz MacBook M1 during TestUFO development. A 120Hz MacBook is actually now my main portable development machine for TestUFO, although it's not the best 120Hz display (Being LCD rather than OLED, it's not as human visible 60-vs-120 difference as on the OLED Apple iPad). I look forward to upgrading to an OLED MacBook, and also look forward to the time when Apple introduces 240 Hz OLEDs in at least one product in two years from rumors I hear. Also, 120Hz Apple LCD is pretty slow compared to a 120Hz desktop gaming monitor, since power efficiency also slowed the LCD pixel response (lack of overdrive algorithm). That's worth it for a 10-hour MacBook battery life, and I understand Apple's rationale. That being said, slowing down the pixel response diminished 60-vs-120 human visibility differences, but on the new 120Hz OLED iPad, running iOS 18 with 120fps, TestUFO 60-vs-120 looks simply stunning amazing. Try www.testufo.com/framerates-versus on the OLED iPad Pro that is manually uncapped (using the setting and the beta iOS 18). Instead of 60-vs-120 being a mere 1.1x human visibility difference, it's now suddenly a 2.0x human visibility difference during fast smooth scrolling/panning situations. There are some debates between me and @smfr of Apple in the past. But perhaps talk to that person in the displays department who has UFO sticker on their MacBook or iPhone instead (there's a few), as the UFO is like a university graduation symbol to them in the temporal sphere. They can show how 120Hz is crap or good. Plus also for competitor-research, perhaps witness a Windows running framerate=Hz on a 480Hz OLED (even on an Intel internal GPU) more smoothly than MacBook running on a 120Hz LCD, since prioritization has been on power management over stutter-free TestUFO. But easy activation of a framerate uncap would be nice, even if behind a Game Mode. But the new M1-M4 chips are quite amazingly performant, and my MacBook even runs fine on an external 240Hz monitor now too (Safari now can be forced to do 240fps, thankfully, at least with the laptop lid closed and only external OLED active). Some people inside Apple formerly thought there wasn't performance on Macs to run at 480Hz. This is false. There is framepacing jitter because sometimes executions are coalesced at less precise times to maximize sleeping between refresh cycles, which can jitter a bit, because sometimes the ideal-power requires a bit of aberration from executing at refresh cycle timestamps to merge all the CPU/GPU workloads that needs to be executed in a surge. It's a common engineering power management optimization, which may have partially helped (in addition to slow 120Hz LCDs) make some Apple employees blind to the benefits of 120Hz+. So TestUFO does understandably stutter more on MacBooks than on Windows. However, with the M4, it's much more reliable, and perhaps more testing is needed. Thanks to OLED, we can have "great battery life" + "great color" + "fast pixel response", with more organic 120fps 120Hz improvements visible to humankind. Also, I want to point out TechSpot about why Apple should add 240Hz to their OLEDs: TechSpot wrote as part of their OLED burnin-test review:
4x geometrics produce a much more stunning wow effect (60-vs-240) than (60-vs-120), which can help sell Apple product. When I show off 60-vs-120 on an OLED (especially bigger screens like OLED tablets, OLED laptops, and OLED desktops), people suddenly want to buy a 120Hz product. So yes, it helps sell. Much more visibly than a 120Hz LCD Apple product. You want to VHS-vs-8K it, not 720p-vs-1080p it. Big jumps + GtG=0 is key. Now it's possible at much lower low power levels thanks to OLED and the M-series, and the latest iOS 18 beta version of TestUFO uncapped to 120fps looks absolutely gorgeous on the 120Hz OLED iPad Pro. Massively more visible than on my 120Hz LCD MacBook M1 that's my TestUFO development machine. But I had to jump through a lot of hoops to do that (install iOS 18 + also change a setting), so it's not an easy successful "5 second educate in a link" to default unconfigured 120Hz Windows systems. However, it's become clear (on Windows side) that 60-vs-240 sells even more powerfully, as a 4x improvement (like 1/60sec vs 1/240sec camera shutter). OLED uncaps the humankind benefits of refresh rates to the mainstream. Especially with the emerging anecdotes of 240Hz OLED ergonomics for productivity for non-gaming use cases. But a baby step of easy 60-vs-120 comparisions in a browser on an iPad people can spontaneously show off to others. Today, that's not easy with the mandatory need to fiddle with settings (at least there's now that -- formerly, it was impossible with a hard 60fps cap that formerly couldn't be overriden at all). As an outside smart analyst (and Apple is notoriously secretive), I know other companies though. It requires coordination between many departments:
So, an orchestra between multiple departments regardless of platform. Apple controls all of them internally and I have no idea how Apple does it, but I know there's been suggestive public hints over the last 10 years of 120Hz disagreements (very pro versus very anti). But bottom line, M1-M4 framepaced TestUFO fine at 480fps 480Hz on an external monitor in my tests (once the OS compositing power-optimizing jitter was solved -- not Safari's fault!). The nice thing is that OLED 60-vs-120 quited a lot of the dissenting voices... So, the weak link, Apple employees, was actually in a different Apple department!! The operating system compositing (in my microsecond-accurate analysis), has intentional jitter that sometimes accidentally adds stutters to TestUFO despite using only 10% of an Apple GPU, due to all the execution-coalescing and microsleeps-between-refreshcycles. So it's potentially just a timing issue, not a performance issue. However, granting Game Mode API, means you could switch to a more performance mode that also includes more accurate graphics compositing timings (When prodded as such with hacks, M4 easily did 480fps 480Hz, in my tests on an external OLED). Yes, generic industry proposals must avoid these kind of details. For this proposal I will remove the apple-sell portion, but I had to explain why I added that to the earlier proposal (that Jeff Blanz did). Yes, there's a 10-year history between me and various Apple employees (on public venues like these) trying to advocate their framerate uncapping, and it's a long slog. It's a breakthrough to finally see Apple add a flag override ("Prefer Page Updates Near 60fps" that can be disabled to uncap frame rates). But that's still not easy enough for a quick way to market 60-vs-120 to end users to help sell the benefits of mainstream non-gaming use cases of 120fps and 240fps. Just explaining to add more detail why it slipped into this venue from a paragraph copy-and-pasted from somewhere else I posted. I actually caused some people to buy 120Hz products (including Apple products) showing them 60-vs-120 on an OLED iPad, so, there is a genuine, actual "sell" factor, that portions of Apple may have apparently overlooked. Earlier, the power concerns and pixel response limitations meant power management outweighed the ability to sell 60-vs-120. But with the new OLED 120Hz iPad Pro, it has fantastic battery life while still doing 120fps 120Hz, while still being much more human-visible than previous LCDs. Now the sell-factor outweighs the power-management-concern factor of Safari having a permanent 60fps cap out-of-box by default. It's time for Apple to internally re-evaluate why they're not uncapping Safari by default (like everyone else now), but I just wanted to offer an olive branch of an alternative quick mechansim (Permission request for uncapping www.testufo.com framerates without me displaying instructions) that allows Apple to cap by default, but makes it easy for webpages to uncap in less than 2 seconds (one-time permission request). I am open to ideas of alternatives that Apple may prefer, while helping sell 120Hz product. Remember, I am a big-time Apple customer, and I am communicating how I've sometimes sold 120Hz to other Apple users. Summary of Weak Links Killing 120 Hz Benefits
All fixable, and voila -- 480fps 480Hz TestUFO works fine on some Apple M-series with zero TestUFO stutter! (But lots of manual configuring, unlike automatigically works on Windows). You do have to purchase a third party 480Hz OLED, and close the laptop lid (to prevent a 60Hz-120Hz clock interference), and do a bunch of power management disable tweaks (something that automatically happens when you are playing video games, anyway). The interdepartmental orchestra inside Apple, not visible to public, helped keep Safari keep a permanent 60fps cap for much longer than competitors. It's justified -- my MacBook is my main TestUFO development machine, and can do 10 hours at a cafe just fine. Wonderful battery life, despite me overriding Safari to 120fps 120Hz during TestUFO development. I want a "easy sell 60-vs-120 in a single web link" mechanism, Apple, that I can tell anybody to do... (without asking them to surf through the Advanced settings). Closeout: Yes, I'll remove Apple references from the proposal. But you can clearly see where I'm coming from; this proposal is also trying to satisfy a lone holdout's privacy & power management needs. |
|
I'm partially rewriting because I hate the Game Mode focus, actually. I want to delete the Introduction and replace it with something broader to better introduce: API Proposals to Remove Frame Rate Limits<Rewrite>IntroductionFor a long time, multiple web browsers had frame rate caps for various reasons, such as simplicity (e.g. 60Hz timer event) or power management (e.g. battery consumption) or arbitrary (e.g. decision made by a single developer or team). These frame rate caps can be less than the display refresh rate, which can affect user experience. This specification aims to standardize a simplified method that websites can use to access higher frame rates for many use cases, in situations where the browser frame rate is capped by the browser engine. Today, only a few browsers still have frame rate caps outside of low-battery mode. HistoryBefore around 2012, some browsers ran off an internal 60 Hz timer event, but that changed. Around this time, both Chrome and FireFox uncapped their frame rates, and Mark of Blur Busters did 120 Hz Browser Tests in year 2013 and discovered that most web browsers had uncapped frame rates that worked fine on early 120 Hz gaming monitors over a decade ago. At the time, Internet Explorer and Safari became the holdouts. Internet Explorer had an internal 105 fps frame rate cap that automatically halved frame rates if the refresh rate was above 105 Hz. This was established all the way back to the CRT days when CRT refresh rates were generally only up to about 85 Hz to 100 Hz, and the number was just above the 100 Hz CRT number. Now that Internet Explorer is replaced by Edge, Internet Explorer is no longer relevant to this discussion. All Chromium browsers have the same uncapped framerate behavior - Chrome, Brave, Vivaldi, etc. These browsers have been successfully tested to 540 Hz on commercial displays, and tested to 1000 Hz on laboratory displays (Windows 10 supports up to 500 Hz, and Windows 11 supports up to 1000 Hz). Since 120Hz is much more common and premium-mainstream than it was in 2012, this has become a more pressing issue than back then. Originally, high native refresh rates were only on specialized gaming displays of lower quality (e.g. TN panels), while now high refresh rates have become inclusions on more mainstream and higher quality displays (e.g. HDR OLED panels) than back then. Current State of Browser Frame Rate LimitsToday, this is the state of frame rate capping:
This proposal aims to find a way to make it easier for the user to uncap their web browser for any purpose, within the OS currently allowed frame rates and refresh rates. Possible Venues For Proposal
Hybrid approaches are possible, e.g. Game Mode automatically grants permission to uncap frame rate, while you could need a different API to uncap frame rate outside of Game Mode. Browsers that already uncap by default (e.g. Chromium) can automatically grant permission. Discussion is welcome. TBD - Further Sections To Be Rewritten</Rewrite> |
|
@mdrejhon, appreciate the updates. Can I kindly ask to try to keep replies succinct 🙏 (try to limit to 500 words max). Otherwise this issue will become unwieldy and difficult to follow. If you find that you want to add background details, please use ...lots of details here...
About game mode, it's not exclusively about uncapping frame rates (which may end up breaking some games that are expecting to run at ~60hz), it also impacts CPU/GPU prioritization, etc. |
Done! I've slimmed my previous text and used
|
Quick Testing Of Smooth Safari >60fpsUpdate, I was experimenting the easiest way to have users reliably demo high-framerate TestUFO in Mac Safari. With these steps, 240fps 240Hz and 480fps 480Hz Safari works jitter-free in TestUFO on MacBook Pro M1, the very first M1. (cc:@smfr) It works now [PHOTO], but it's still convoluted, but this is the simplest instructions: Reproduction Instructions for Jitter-Free 120fps, 240fps and 480fps on any M-series Mac
These instructions Apple Safari/Chrome surprisingly does concurrent multi-Hz support better than Microsoft's Edge/Chrome does. Photos of 120fps-120Hz TestUFO and 240fps-240Hz TestUFO running concurrently on MacBook M1: Both in Chrome and Safari. (This one is Chrome), photo of MacBook Pro M1 connected to external Corsair 240Hz ultrawide OLED gaming monitor:
And Safari:
And even inside Parallels VM:
It goes smooth if I rapidly circle the mouse arrow pointer on the touchpad, since that forces the M1 CPU into a higher power state (mouse movements temporarily disables power management jitter from execution-coalescing). This is only the main TestUFO demo, others are pretty useful for showing off too. My MacBook Pro M1 (TestUFO 120fps) running concurrently with Corsair Xenon Flex Ultrawide 240Hz (TestUFO 240fps). The two TestUFO windows run at independent framerates much better than on a Windows platform. Perhaps there should be a browser API to request a high-performance high-precision mode (even if permission based), even for a very brief period (e.g. 60 seconds), for the duration of a critically framerate=Hz motion educational demo. Or maybe as part of a Game Mode API system. That would prevent users from having to rapidly circle mouse pointer to de-jitter 240fps-480fps on a Mac. Long battery life is important, but also needs to be weighed by other needs such as briefly showing a teaching-animation demo, so TestUFO generally usually does not need perpetual access to this higher power state. The easy out-of-the-box tip (without using external p-state forcing utilities) for testing 480fps 480Hz on Safari with an external monitor connected to Mac; it works if you continually move the mouse cursor in tiny circles. This also makes any refresh rate such as 240fps 240Hz and 120fps 120Hz work smoother too! This makes TestUFO run at ~0.2ms jitter, accurate enough for having successfully tested jitter-free 480fps 480Hz in TestUFO Safari on external 480Hz OLED without jitters or stutters! In an era where it was discovered that 120fps-vs-480fps OLED is more human visible than 60fps-vs-120fps LCD. (Surprisingly, MacBook M1 automatically did 480Hz too; there's no refresh rate limitation even on the older MacBook M1's, it's more a USB-C cable bandwidth limitation in number of pixels/sec for display connectivity) Browser engine developer departments may wish to procure a 240Hz and 480Hz display, to show how ergonomic scrolling can become as a Hz-mainstream feature (much better than 120Hz OLED iPad). Internal company demos may help sidestep some Hz holy wars, when tested on external-vendor 240-to-480Hz OLEDs for refresh rate experimentation. But this is too many steps. Impressing people who owns 120Hz displays for 15 seconds with display elements (retina feature, or HDR feature, or 120 Hz feature, or 480 Hz feature etc), requires easy access that doesn't require going to a Developer menu. I'd be okay with a temporary time-limited permission, as long as it was a one-step permission request. The power management execution-coalescing jitter disappears when I spin my mouse arrow in tight circles, since mouse movements forces the M1 into a higher P-state, and framerates stop stuttering in the browsers. If you want to test the latest version of TestUFO with browser engines for standardization purposes, try testing https://beta.testufo.com -- much more Apple mobile friendly (better full screen mode on iPhone and iPad). I will soon be coding a notiuce message for all MacOS users visiting TestUFO (soon production) to tell users how to remove the frame rate cap. TestUFO gets 8 figure visits per year (over 1M unique visitors per month), an increasing proportion of which are Mac visitors. Which means I'm telling a lot of Mac users to fiddle a developer setting.... |
Comments Welcome@JeffreyBlanz and @marcoscaceres any comments before I continue editing? Also, beta.testufo.com is finally public to the whole world, if you want to test high Hz+HDR simultaneously.
Privacy ConsiderationsApple is a very privacy-sensitive company, so if Apple does not want sites to know the refresh rate of their display, it can be put as part of a permission request, or rolled into an existing one like Display Management API. This could be a compromise that @smfr likes. I already pop up a small warning to all 60fps Apple users at beta.testufo.com, and will be expanding that to display a detailed step by step guide to users on how to disable the 60fps flag. Users can choose to do so, but I'd like a more proper way than inexperienced users fiddling a dev flag. Opt-In Site-Specific Global UncapIntent of this proposal is to uncap everything at the browser compositor level all the way to the OS-allowed limit: HFR video (120fps realtime), scrolling, animations, 2D, WebGL, CANVAS, Game APIs, Canvas 2D, requestAnimationFrame, and any web animation related APIs. >60fps Now Works in All APIs in All Major Browsers / OSesAll of that already works fine >60fps in all current major browsers (Safari, FireFox, Edge, Chrome, etc) on all major platforms (PC, Mac, Linux Wayland, Android, iPhone iOS 18+, iPad, 120Hz PlayStation, 120Hz XBox, 90Hz+ VR browsers, etc). All prior art, all defacto standardized. It's merely simply a hoop-jumping manoeuvre for Apple Safari users where I have to teach inexperienced users how to fiddle a developer flag (Prefer Page Rendering Updates Near 60fps). K.I.S.S.Doing >60fps already works in all HTML5.2+ APIs in all major browsers on all platforms. The last holdout, 120Hz iPhones does 120fps TestUFO as of iOS 18+ via byzantine flag change. This WICG proposal is simply to find a privacy-friendly opt-in method uncap single variable flag (LimitFramerateTo=60). FireFox has a framerate cap setting too (turned off by default). Other browsers can decide to enable it by default, and use this WICG proposal to uncap. So this is not Apple-exclusive. This is simple. There are no other dependencies. All HTML5 features already work fine at 60fps, 90fps, 120fps, 240fps in all major browsers on all major platforms, as of year 2024. We do not need to committee-overcomplicate what already works. Sites that specifically request uncap (by any mechanism) means they're >60fps-aware. |
|
@mdrejhon I think the rewrite above is good and you should continue. I would like to know more about why @marcoscaceres thinks it might not need to be a permission prompt. More specifically; what are Apple's views on privacy with proposal, and what is Apple's stance on the "prefer page rendering updates near 60 fps" feature flag being default off instead of on? Assuming this is a privacy concern and the feature flag is staying default on, why would a permission prompt not be ideal? |
|
Ideally, I'd love Apple to a mostly-unconditional API call to uncap upon a click (like Pointer Capture or Fullscreen requires). It doesn't have to be permission-based, as long as it's an easy global-all-APIs uncap for the specific site requesting it (not other sites). But indeed want to accomodate privacy concerns. Open to ideas for global uncap! |
Introduction
On iOS, iPadOS, MacOS, and Android there is a game mode that can be enabled with fullscreen apps that are designated as games. There is currently no way to enable the system game mode in a PWA or normal web engine. On iOS, iPadOS, and MacOS users also have to manually turn off "Prefer Page Rendering Updates Near 60fps" in Developer Feature Flags. This feature flag also does not apply to PWAs, meaning PWAs are stuck at 60fps (see webkit bug 272226). This API intends to add the system Game Mode support as well as enabling native screen refresh rates through a permission prompt for web apps and browsers. Native screen refresh rate can be enabled independently from game mode as well.
Why High Frame Rate is Important Outside of Gaming
120fps was less visible on LCD so this a low priority, due to poor performance of LCD. However, with the dramatic visibility increase of 60-vs-120 on OLED displays (even TechSpot discovered 240Hz OLEDs was much better than expected for office productivity). Even 120Hz-vs-480Hz on OLED was recently scientifically discovered to be more human-visible than 60-vs-120 on LCD, due to a geometrics-style effect (4x vs 2x) during fast motion.
Use Cases
Solution: Game Mode Permission API
In this API there will be two types of permission prompts. One for just native screen refresh rate, and one for enabling the system game mode as well as the native screen refresh rate.
When a website asks for Game Mode permission, the browser will popup a message asking the user for permission to use Game Mode. Upon approval the browser will enable the system game mode and disable any limitations on screen refresh rate.
When a website asks for High Refresh Rate permission, the browser will popup a message asking the user for permission to use High Refresh Rate. Upon approval the browser will disable any limitations on screen refresh rate. CPU and GPU prioritization will not be enabled.
Note: Both permission prompts will only work if it's a same-origin or permitted-origin (so ad banners can never request it).
Solution Sequence
Examples
A possible permission-request message could be:
"Webpage is requesting permission to enable Game Mode. This may consume more battery power. Allow? Yes/No"
Another possible permission-request message could be:
"Webpage is requesting permission to enable High Refresh Rate. This may consume more battery power. Allow? Yes/No"
Privacy & Security Considerations
Note: By default, webpage MAY NOT discover max Hz or if Game Mode API is enabled if the page is currently capped (e.g. Limit to 60fps), this may resolve privacy concerns if a company is concerned about webpages discovering refresh rates of visitors. However, implementors MAY provide a separate discovery mechanism for refresh rate, much like existing discovery mechanisms for screen resolutions. Another possible Implementation would be for a webpage to return is "60 Hz" versus "Above 60 Hz", via some discovery mechanism whether or not the browser is framerate capped or uncapped. Even if it's simply detecting it simply via a non-immediate-fail on a Game Mode Permission Request.
References
Coauthor: @mdrejhon (Creator of TestUFO)
mdrejhon Similar Bugzilla Proposal: https://bugs.webkit.org/show_bug.cgi?id=274372
Apple: https://support.apple.com/en-us/105118
Android: https://developer.android.com/games/optimize/adpf/gamemode/gamemode-api
PWA Bug: https://bugs.webkit.org/show_bug.cgi?id=274372
The text was updated successfully, but these errors were encountered: