MEDIUM: [Tracking] Implement more E2E tests and add new metrics

[mountiny]

The E2E tests are now set up and working again, lets expand on their options and add more flows and metrics:

• chat switching/opening for the first time
• typing
• scrolling chat messages
• number of renders when we open chat (it should be 1-2)
• searching
• signing in
• scrolling list of chats
• onyx tests

@Szymon20000 @hannojg

[hannojg]

👋

[hannojg]

@perunt

[perunt]

👋

[mountiny]

Also Assigning @AndrewGable as he will most likely help us too 🤝 feel free to unassign if you prefer

[hannojg]

Update:

There was again a false positive on the e2e test runs. Added this PR trying to minimise those errors:

• [NoQA] [DEV] E2e/increase reliability #30484

[hannojg]

Update:

First PR for adding tests for typing is up:

• [DEV] E2e/add typing tests #30502

[melvin-bot]

⚠️ Looks like this issue was linked to a Deploy Blocker here

If you are the assigned CME please investigate whether the linked PR caused a regression and leave a comment with the results.

If a regression has occurred and you are the assigned CM follow the instructions here.

If this regression could have been avoided please consider also proposing a recommendation to the PR checklist so that we can avoid it in the future.

[mountiny]

Lies, it was a different pr causing the issue

[melvin-bot]

This issue has not been updated in over 15 days. @AndrewGable, @hannojg, @mountiny, @perunt eroding to Monthly issue.

P.S. Is everyone reading this sure this is really a near-term priority? Be brave: if you disagree, go ahead and close it out. If someone disagrees, they'll reopen it, and if they don't: one less thing to do!

[mountiny]

@hannojg @perunt how is it going?

[hannojg]

• The typing test is implemented and merged!
• For Chat switching we have a PR which should be done latest by end of this week: E2E chat opening #30634
• For the remaining tests we will put again more focus on that to finish it off soon!

(Note: we are internally working on a system to make the e2e tests even better, more news on that soon! ™️ )

[mountiny]

Thanks!

[dangrous]

adding myself here for reviewing the chat switching one!

[hannojg]

Chat switching is merged 🎊 !

@perunt I think we can add "number of renders when we open chat (it should be 1-2)" to the same test as second metric?

Also next up would be the search page test!

[hannojg]

Next is to allocate time for this task in our team and move forward! Nothing blocking as far as I can see. We'll pick this up again next week!

[melvin-bot]

This issue has not been updated in over 15 days. @dangrous, @hannojg, @perunt eroding to Monthly issue.

P.S. Is everyone reading this sure this is really a near-term priority? Be brave: if you disagree, go ahead and close it out. If someone disagrees, they'll reopen it, and if they don't: one less thing to do!

[roryabraham]

We were chatting in slack about pointing these E2E tests at staging and using them to help figure out our "high water point" at which the performance degrades critically with an increasing number of reports across various devices.

[hannojg]

@roryabraham I remember we discussed doing a one-off report for those "high water points" here

Is this something we should look at?

cc @kirillzyusko

[mountiny]

I believe @rinej is working on that

[hannojg]

@kirillzyusko lets finish this one!

Which tests are currently missing? I see that we are still missing:

• number of renders when we open chat (it should be 1-2)
• scrolling list of chats
• scrolling chat messages
• Onyx tests

I think for the onyx tests we wanted some tests where we don't load the full app, but just run some performance tests with mock data in onyx to catch performance regressions in onyx.

[kirillzyusko]

@hannojg yes, you are right. I think for number of renders we only track amount of re-renders in Composer.

Also I think we'll need to revise the approach for tracking number of re-renders, because now we rely on the fact, that if parent was re-rendered, then children that are not wrapped in memo will be re-rendered. But with react-compiler it doesn't work anymore, because it'll memoize everything automatically 👀

[kirillzyusko]

Also I think we'll need to revise the approach for tracking number of re-renders, because now we rely on the fact, that if parent was re-rendered, then children that are not wrapped in memo will be re-rendered. But with react-compiler it doesn't work anymore, because it'll memoize everything automatically 👀

Just as a proof of my words - below I attached a report (after react-compiler was merged). And as we can see (first of all it really works 😅) that amount of renders went down from 2 to 0. I assume we have the same amount of re-renders, but since we were relying on an approach described above -> we have modified numbers now, so eventually at some point of time we'll need to revisit that.

---

Performance Comparison Report 📊

Significant Changes To Duration

There are no entries

Meaningless Changes To Duration

Show entries

Name	Duration
App start nativeLaunch	46.421 ms
App start nativeLaunchEnd_To_appCreationStart	50.533 ms
App start appCreationEnd_To_contentAppeared	590.857 ms
App start appCreation	57.070 ms
App start contentAppeared_To_screenTTI	985.122 ms
App start TTI	1727.189 ms
App start runJsBundle	441.492 ms
App start regularAppStart	0.022 ms
Load Search Options	213.100 ms
Open Chat Finder Page TTI	1237.410 ms
Composer typing rerender count	0.000 ms
Chat opening	166.850 ms
Chat TTI	565.710 ms
Comment linking	480.110 ms

Show details

Name	Duration
App start nativeLaunch	Baseline Mean: 46.474 ms Stdev: 4.519 ms (9.7%) Runs: 39 39 41 41 41 41 41 42 42 42 43 43 43 43 43 44 44 44 44 44 44 44 44 44 45 45 45 45 46 46 46 46 47 47 47 47 47 48 48 48 48 49 49 50 50 50 51 51 51 51 52 52 54 54 57 58 59 Current Mean: 46.421 ms Stdev: 5.231 ms (11.3%) Runs: 39 40 40 40 40 40 41 41 42 42 42 42 42 42 42 42 43 43 43 43 43 43 44 44 45 45 45 46 46 46 46 46 46 46 46 47 47 47 48 48 48 49 50 50 51 51 52 52 52 52 53 54 54 55 59 60 61
App start nativeLaunchEnd_To_appCreationStart	Baseline Mean: 49.867 ms Stdev: 8.401 ms (16.8%) Runs: 36 37 37 38 39 40 40 41 41 42 42 42 43 43 44 44 44 44 44 45 45 45 46 46 46 47 48 48 48 49 49 49 50 51 51 51 52 52 52 52 52 53 53 54 54 56 57 57 57 58 59 59 60 60 63 63 64 66 69 75 Current Mean: 50.533 ms Stdev: 8.401 ms (16.6%) Runs: 38 39 39 40 40 41 41 41 42 42 42 43 43 43 44 44 44 45 45 45 45 47 47 47 47 47 47 48 48 49 49 50 51 51 52 52 52 53 53 53 53 54 54 55 55 56 57 57 57 57 57 60 60 61 62 67 69 70 70 72
App start appCreationEnd_To_contentAppeared	Baseline Mean: 586.533 ms Stdev: 43.644 ms (7.4%) Runs: 497 506 514 516 520 524 529 529 534 537 538 539 540 545 551 554 561 564 575 576 576 578 580 581 582 582 582 586 587 588 589 589 590 590 591 595 596 596 602 603 610 611 612 613 617 617 617 618 622 622 627 627 639 642 645 648 656 662 685 690 Current Mean: 590.857 ms Stdev: 31.269 ms (5.3%) Runs: 519 524 530 537 538 543 544 546 553 555 565 574 577 577 577 579 580 580 582 582 584 586 587 587 587 588 588 592 592 593 594 596 597 597 598 598 602 610 610 611 612 615 616 618 620 621 621 623 626 627 628 629 631 637 650 655
App start appCreation	Baseline Mean: 61.390 ms Stdev: 10.279 ms (16.7%) Runs: 46 48 49 50 51 51 51 51 51 52 52 52 53 53 53 54 54 54 54 54 54 55 55 55 55 56 58 59 59 60 60 60 61 61 62 62 62 62 63 64 65 66 68 69 69 69 71 71 72 72 72 74 74 75 75 80 85 87 92 Current Mean: 57.070 ms Stdev: 6.948 ms (12.2%) Runs: 44 45 46 47 48 48 48 50 50 50 51 51 51 51 51 52 52 54 54 54 55 55 55 55 56 56 56 57 57 57 58 58 58 59 59 59 59 59 59 59 60 60 61 61 62 62 62 63 63 64 66 69 69 71 71 72 74
App start contentAppeared_To_screenTTI	Baseline Mean: 1065.342 ms Stdev: 76.461 ms (7.2%) Runs: 907.8019419999328 915.2966060000472 922.8915379999671 923.1415490000509 936.7570619999897 941.0060360000934 949.8935799999163 982.3591420000885 983.5623409999534 990.5694039999507 998.6525149999652 999.3633300000802 999.6344999999274 1006.9684609998949 1007.717413000064 1016.4667450000998 1020.0574990001041 1022.4106940000784 1023.4612030000426 1023.9040620001033 1033.2889250000007 1041.210217999993 1044.919617000036 1048.5714539999608 1050.9262329998892 1058.0397880000528 1064.8353329999372 1064.9087469999213 1065.360864999937 1065.9842719999142 1073.4153340000194 1081.0659119999036 1081.4825420000125 1081.8841289998963 1082.1053369999863 1082.308960000053 1082.649822999956 1090.1506209999789 1090.231924999971 1090.3712689999957 1106.3483090000227 1107.3801849999 1114.1298309999984 1115.2328929998912 1115.4641899999697 1116.5955270000268 1124.0819810000248 1124.3340419998858 1128.1248780000024 1131.7707030000165 1140.8266129998956 1165.3738959999755 1165.5893920001108 1169.9072310000192 1173.0982860000804 1173.8252739999443 1182.3338279998861 1197.9031980000436 1206.981495999964 1215.5781890000217 Current Mean: 985.122 ms Stdev: 82.303 ms (8.4%) Runs: 816.1936719999649 828.1428680000827 842.0098999999464 849.1310709998943 854.7382259999868 864.8342579999007 890.2964689999353 897.9933609999716 899.5491200000979 900.2879840000533 903.3480809999164 907.226885000011 907.6650960000698 923.9573609998915 924.5785950000864 924.6644840000663 940.2367819999345 940.8640999998897 947.6720690000802 949.1422260000836 949.1836399999447 949.1871100000571 949.3551290000323 954.6597519998904 956.0930820000358 956.447179999901 958.2914809999056 967.0230159999337 972.9453010000288 973.3050220001023 977.215649999911 982.6516629999969 998.3009860001039 1005.8315520000178 1006.9743359999266 1007.1177580000367 1007.7846750000026 1015.1390990000218 1015.2986280000769 1015.7307490000967 1016.4739329998847 1022.6975750001147 1023.8839889999945 1024.0584040000103 1032.2667839999776 1056.6662920000963 1073.7304310000036 1074.429194000084 1082.1811830000952 1082.3930500000715 1082.4438249999657 1098.0842269998975 1098.9252840001136 1099.4649529999588 1107.1244930000976 1107.71744700009 1113.516067000106 1114.0042310000863 1132.7914040000178 1133.428477999987
App start TTI	Baseline Mean: 1811.642 ms Stdev: 110.432 ms (6.1%) Runs: 1570.1415490000509 1605.7570619999897 1610.891537999967 1612.8935799999163 1613.2966060000472 1631.8019419999328 1643.5623409999534 1648.3591420000885 1649.3633300000802 1650.0060360000934 1677.717413000064 1692.9040620001033 1702.6344999999274 1716.5694039999507 1759.4667450000998 1761.1053369999863 1764.0574990001041 1769.4825420000125 1771.0659119999036 1774.0397880000528 1778.4106940000784 1779.968460999895 1791.210217999993 1800.5714539999608 1803.4612030000426 1808.360864999937 1823.649822999956 1824.6525149999652 1828.9087469999213 1830.3712689999957 1836.7707030000165 1840.919617000036 1843.8841289998963 1847.4153340000194 1848.8353329999372 1851.1506209999789 1854.308960000053 1856.9842719999142 1862.2328929998912 1871.2889250000007 1875.1248780000024 1887.4641899999697 1891.1298309999984 1892.0819810000248 1894.3340419998858 1894.9262329998892 1895.9072310000192 1899.3483090000227 1914.231924999971 1914.8266129998956 1915.3801849999 1917.5955270000268 1922.3738959999755 1929.9031980000436 1934.8252739999443 1953.0982860000804 1953.5893920001108 1963.3338279998861 2005.5781890000217 2034.981495999964 Current Mean: 1727.189 ms Stdev: 107.662 ms (6.2%) Runs: 1503.1428680000827 1507.1936719999649 1533.1310709998943 1538.0098999999464 1550.2879840000533 1555.8342579999007 1556.3480809999164 1589.9933609999716 1594.5785950000864 1596.2367819999345 1601.1871100000571 1624.0930820000358 1626.226885000011 1626.7382259999868 1627.6644840000663 1654.549120000098 1668.2964689999353 1671.9573609998915 1687.6650960000698 1695.447179999901 1705.0230159999337 1713.3050220001023 1724.1836399999447 1726.3551290000323 1728.2914809999056 1728.8640999998897 1730.7846750000026 1732.1422260000836 1735.1177580000367 1745.651662999997 1747.9743359999266 1749.8315520000178 1750.215649999911 1750.9453010000288 1753.6720690000802 1755.6975750001147 1762.7307490000967 1765.4739329998847 1766.6597519998904 1770.2986280000769 1776.3009860001039 1777.1390990000218 1778.8839889999945 1779.6662920000963 1789.2667839999776 1797.71744700009 1801.0584040000103 1803.7304310000036 1831.1811830000952 1839.3930500000715 1844.9252840001136 1846.0842269998975 1864.0042310000863 1864.1244930000976 1864.429194000084 1864.4649529999588 1889.4438249999657 1910.428477999987 1925.516067000106 1931.7914040000178
App start runJsBundle	Baseline Mean: 440.915 ms Stdev: 31.211 ms (7.1%) Runs: 368 379 387 395 395 397 402 403 406 406 409 409 414 414 419 420 423 425 425 427 431 433 433 434 435 436 440 442 443 444 444 445 447 447 447 448 448 449 454 455 456 458 458 460 463 468 468 469 471 473 473 475 478 481 482 489 493 508 513 Current Mean: 441.492 ms Stdev: 28.374 ms (6.4%) Runs: 384 385 386 387 397 399 400 402 408 409 412 416 420 420 425 426 427 430 430 432 432 432 434 437 440 440 441 445 446 447 447 448 448 448 449 450 452 452 453 454 454 455 455 455 457 459 465 467 469 470 470 472 475 476 481 487 489 498 504
App start regularAppStart	Baseline Mean: 0.023 ms Stdev: 0.002 ms (10.3%) Runs: 0.017089999979361892 0.017699999967589974 0.01818799995817244 0.01827000011689961 0.018554000183939934 0.018718000035732985 0.01977499993517995 0.019896999932825565 0.01989799994044006 0.02030499977990985 0.02087399992160499 0.02123999991454184 0.021321999840438366 0.02140299999155104 0.021403000224381685 0.021646999986842275 0.021727999905124307 0.021769999992102385 0.021849999902769923 0.021850999910384417 0.022013999987393618 0.022461000131443143 0.02254199981689453 0.022582999896258116 0.02258300012908876 0.02266400004737079 0.022704999893903732 0.022745999973267317 0.022786000045016408 0.022948999889194965 0.02294900012202561 0.02298999996855855 0.023194000124931335 0.02327399980276823 0.023314999882131815 0.0235190000385046 0.023558999877423048 0.023600999964401126 0.023640999803319573 0.023680999875068665 0.023681999882683158 0.024252000031992793 0.024373000022023916 0.024495000019669533 0.02474000002257526 0.02482099994085729 0.024983000010252 0.024984000017866492 0.02518699993379414 0.02547200000844896 0.025594000006094575 0.0258390000090003 0.025919999927282333 0.025959999999031425 0.026286000153049827 0.026448999997228384 0.027180999983102083 Current Mean: 0.022 ms Stdev: 0.003 ms (12.8%) Runs: 0.01745599997229874 0.017577999969944358 0.0176189998164773 0.018229999812319875 0.01831099996343255 0.018514000112190843 0.01871700002811849 0.019287000177428126 0.019328000023961067 0.019328000023961067 0.019368999870494008 0.01977499993517995 0.019816000014543533 0.019896999932825565 0.01989799994044006 0.020141000160947442 0.020141999935731292 0.020183000015094876 0.02034499985165894 0.020345000084489584 0.020385999931022525 0.02050799992866814 0.020751999923959374 0.02079300000332296 0.020913999993354082 0.021036999998614192 0.02123999991454184 0.021280999993905425 0.021484999917447567 0.02148500015027821 0.021607000147923827 0.02168800006620586 0.02172800013795495 0.021850999910384417 0.021890999982133508 0.021932000061497092 0.022012999979779124 0.022827000124379992 0.022867999970912933 0.023275000043213367 0.023641000036150217 0.024088000180199742 0.024088999954983592 0.024373000022023916 0.024536000099033117 0.02474000002257526 0.025228000013157725 0.025430999929085374 0.0256759999319911 0.026203999994322658 0.026285999920219183 0.02636700007133186 0.026814999990165234 0.027180999983102083 0.027222000062465668 0.029541000025346875
Load Search Options	Baseline Mean: 215.810 ms Stdev: 15.900 ms (7.4%) Runs: 187.6241859998554 188.15079699992202 189.3496509999968 190.11763500003144 190.5600179999601 191.2069500000216 191.7418619999662 192.7706299999263 193.29142200015485 199.33841999992728 199.9901129999198 201.3704830002971 202.27295000012964 205.46675600018352 205.87142000021413 207.3771569998935 207.7867439999245 208.11596699990332 209.21459899982437 209.2482089996338 210.2288000001572 210.27315300004557 212.57914200006053 212.72977699991316 212.9648849996738 213.21862800000235 213.52095600031316 214.11641399981454 215.16345199989155 215.73836300009862 217.7987069999799 218.44311500014737 218.49833199987188 218.6028639995493 218.76013199985027 219.09814399993047 221.77832000004128 224.21106000011787 225.00195300020278 225.20874000014737 225.68843599967659 226.3619389999658 227.2277020001784 227.28361000027508 227.28812699997798 227.8553059999831 227.86226400034502 228.46329799992964 230.8099360000342 230.8671059999615 230.94376600021496 233.64123499998823 236.1510820002295 236.4513339996338 241.75793500011787 247.57438199967146 249.14916999964043 252.78511600010097 Current Mean: 213.100 ms Stdev: 15.295 ms (7.2%) Runs: 179.6796059999615 186.46903500007465 190.09228500002064 190.5006510000676 192.99190200003795 193.0804440001957 193.30700699985027 194.13647399982437 194.97448699967936 195.29024200001732 195.31978299980983 195.3685710001737 195.54191099992022 197.5196940000169 199.6716310000047 201.90002500033006 203.11031099967659 204.49642000021413 204.76607300015166 206.48881000000983 207.07328300038353 208.9291179999709 209.98046899959445 210.40722699975595 212.0319409999065 212.41699200030416 212.6582430000417 216.0651039998047 216.43379699997604 216.7900390001014 216.89347400004044 217.84602899989113 218.96207699971274 219.21671500010416 219.50752800004557 220.14009599993005 221.03352800011635 221.03857500012964 221.14009599993005 221.41162099968642 221.6957610002719 222.19934000028297 222.6646729996428 222.98185299988836 223.0439039999619 224.09293700009584 225.1695559998043 227.60632300004363 229.25406899955124 230.33272300008684 230.39489699993283 233.8977049998939 235.01298000011593 235.75081399967894 238.08630400011316 238.8903810000047 250.9480789997615
Open Chat Finder Page TTI	Baseline Mean: 1183.998 ms Stdev: 57.718 ms (4.9%) Runs: 1049.1873379999306 1070.6500249998644 1091.3407799997367 1093.3906260000076 1100.8560799998231 1106.7984219999053 1111.3887950000353 1114.4760750001296 1125.1623539999127 1134.828614000231 1137.7478849999607 1138.5943610002287 1139.1834720000625 1139.372802999802 1139.5280770002864 1139.805136000039 1141.2639580001123 1146.276041999925 1146.4478360000066 1151.9972340003587 1152.890585000161 1155.1714679999277 1155.269328000024 1156.8361410000362 1157.2795820003375 1157.4157310002483 1158.2241620002314 1169.0699880002066 1174.953288000077 1181.7361249998212 1184.7868249998428 1186.482340999879 1191.9135340000503 1199.40730800014 1203.1869720001705 1214.8201910001226 1215.7211919999681 1218.0220139999874 1218.7476820000447 1220.501221000217 1225.2565520000644 1226.3272310001776 1228.6660569999367 1228.710694000125 1229.2508960003033 1229.7907730001025 1232.983196000103 1240.5893149999902 1240.7472740001976 1242.1815189998597 1245.8743489999324 1248.7130950000137 1248.7187099996954 1250.643962000031 1251.3100589998066 1252.8673109998927 1255.250326000154 1255.9399010003544 1270.9145919997245 1344.4232180002145 Current Mean: 1237.410 ms Stdev: 85.441 ms (6.9%) Runs: 1056.287558000069 1062.01550400001 1098.9885260001756 1108.5623380001634 1112.0209559998475 1116.9572350000963 1124.5742600001395 1130.574137999909 1138.5173349999823 1143.43933199998 1157.0199390002526 1159.6073809999507 1160.4798590000719 1166.092244999949 1169.2841799999587 1171.2209479999729 1183.5891929999925 1186.5815029996447 1187.3382979999296 1197.3414309998043 1208.0832519996911 1210.9396979999729 1212.4313570000231 1213.7568359998986 1219.5778410001658 1220.9313159999438 1226.4167490000837 1229.743530999869 1231.2180590000935 1232.7742509997915 1241.372844000347 1242.3741460000165 1243.3283700002357 1245.799682999961 1253.8372810003348 1255.6403000000864 1260.2985029998235 1266.7641209997237 1271.83203099994 1273.852295000106 1278.9971929998137 1282.7675789999776 1287.2347420002334 1289.0062259999104 1299.5423999996856 1306.2535410001874 1306.8204350001179 1312.210367999971 1312.2255859998986 1318.6031909999438 1326.724731999915 1334.040934999939 1346.6018480001949 1346.867065999657 1349.7342129996978 1349.9047860000283 1373.926270000171 1403.1051030000672 1409.0791429998353 1419.4740809998475
Composer typing rerender count	Baseline Mean: 2.000 ms Stdev: 0.000 ms (0.0%) Runs: 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 Current Mean: 0.000 ms Stdev: 0.000 ms (NaN%) Runs: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
Chat opening	Baseline Mean: 144.959 ms Stdev: 6.599 ms (4.6%) Runs: 134.66532400017604 135.12951700016856 135.66825400013477 136.48018399998546 137.52022300008684 138.20890300022438 138.2253020000644 139.0252680000849 139.7583010001108 139.76721200020984 139.94966599997133 140.440510999877 140.51761899981648 140.68684900039807 140.8257650001906 140.87076900014654 141.05513500003144 141.07238799985498 141.22603400005028 141.54764899984002 141.60290500009432 141.6551510002464 141.7930899998173 142.13582399999723 142.19893399998546 142.75113999983296 143.33548999996856 143.66524200001732 143.85380100039765 145.12675000028685 145.1649989997968 145.6087239999324 145.70882199984044 145.71130400011316 145.83064700011164 146.2416179999709 146.3520510001108 146.36348400032148 147.00370300002396 147.28881900012493 147.55061899963766 147.90820399997756 147.92561799986288 149.02136199967936 150.31978400005028 151.28027400001884 153.69934100005776 153.95800799969584 154.4168289997615 156.9649659995921 157.28751600021496 160.70922800013795 161.990233999677 162.7268880000338 Current Mean: 166.850 ms Stdev: 12.657 ms (7.6%) Runs: 141.99723300011829 144.2677810001187 144.35876500001177 145.1138510000892 145.99772100010887 149.32535899989307 149.53910399973392 150.03788299998268 152.4448239998892 152.45634000003338 152.56005899980664 153.10286499978974 153.11889600008726 159.0053309998475 160.40144800022244 161.34920200007036 161.5302320001647 161.79292799998075 162.3262120001018 162.61356600001454 164.56384300021455 165.2255859998986 165.512940000277 165.82426000013947 166.455851000268 167.01375399995595 167.04829900013283 167.40071600023657 169.56388300005347 169.5749109997414 170.3244630000554 170.60412599984556 170.78129100007936 170.88480600016192 171.63403300009668 171.98120100004598 172.35278399987146 172.98783400002867 173.89680999983102 174.24361200025305 174.73632800020278 175.04858400020748 175.34790099970996 176.57291600015014 176.94393000006676 177.11149099981412 177.92614699993283 180.98140500020236 182.7631829995662 183.49198399996385 184.63033000007272 187.30403700005263 188.45646200003102 188.98754899995402 195.2517499998212
Chat TTI	Baseline Mean: 476.442 ms Stdev: 30.748 ms (6.5%) Runs: 434.2028810000047 437.42981000011787 440.26863600034267 441.4828290003352 441.9324139999226 446.1614179997705 446.4310710001737 446.7055259998888 447.65071700001135 448.716349999886 449.24047899991274 449.6853030002676 449.82076000003144 450.2500409996137 450.9113360000774 450.969034999609 451.24296100018546 452.19966599997133 452.81559300003573 454.2888600002043 454.53576700016856 456.12874399963766 457.02852400019765 457.63704500021413 458.65694200014696 459.0605069999583 462.0355630000122 463.3727220003493 466.6648770002648 466.9698080001399 467.61844900017604 472.4313559997827 475.7039799997583 479.0117190000601 479.1413169996813 479.52579799992964 481.0275070001371 481.9576820000075 482.4648040002212 483.312663000077 484.4969489998184 485.2662769998424 485.91410299995914 492.203572999686 495.68957499973476 500.75052900006995 501.087728000246 503.28108800016344 504.24967499962077 504.2858070000075 509.18249499984086 509.9790040003136 512.5047209998593 518.9646809999831 532.6081550000235 532.9685880001634 534.7239990001544 545.7838129997253 548.5006919996813 557.3652350003831 Current Mean: 565.710 ms Stdev: 30.276 ms (5.4%) Runs: 496.25329599995166 510.42346200020984 510.9249669997953 512.8595790001564 513.2970790001564 518.8798019997776 520.0266920002177 536.0394290001132 536.6150309997611 538.2088629999198 538.3254800001159 540.3033849997446 542.1627200003713 542.3945309999399 543.8304039998911 545.8713389998302 546.221923999954 550.6606040000916 550.6827389998361 552.5664880000986 553.3953860001639 554.043864000123 557.8120929999277 558.5373129998334 561.6619469998404 563.2617600001395 566.0299889999442 567.0897220000625 567.4525150000118 568.374307999853 569.0174569999799 570.2483320003375 572.0125739998184 572.2180989999324 572.6911220001057 573.6181649998762 575.0176190002821 576.9523119996302 577.2944739996456 578.8898519999348 580.112548999954 580.962849999778 581.153239000123 581.2606609999202 582.7671309998259 583.2261969996616 585.7374269999564 593.535116000101 594.7001950000413 596.4340010001324 601.5576980002224 606.8865559999831 608.2360849999823 609.4044590000995 609.8971350002103 610.0624190000817 613.2183429999277 615.7701820000075 639.7746989997104
Comment linking	Baseline Mean: 479.941 ms Stdev: 94.333 ms (19.7%) Runs: 329.9139000000432 343.6412349999882 345.00870699994266 353.8523350004107 356.29203299991786 365.00968400016427 368.8347169999033 369.1029059998691 369.19584200065583 370.2426760001108 377.52616299968213 378.4616290004924 381.8841559998691 387.89355499949306 391.8345539998263 394.7826339993626 401.44868999999017 415.91259799990803 419.47282000025734 419.6573490006849 420.69120300002396 433.2966719996184 437.5493980003521 441.165731000714 449.463175999932 454.1970620006323 455.46671599987894 463.6658939998597 469.843913000077 470.22635999973863 470.660442000255 477.4665130004287 481.33044399973005 483.81404699990526 492.84175700042397 503.6965330000967 507.8522140001878 508.1464029997587 510.19746899977326 512.2441819999367 522.2788909999654 533.435344000347 535.5654710000381 537.1874599996954 540.5981049994007 548.4115810003132 551.3695480003953 564.5918780001812 568.8046059999615 571.7102049998939 580.5857340004295 600.5196130005643 607.0284420000389 630.4750570002943 634.9811599999666 637.326253999956 639.7915850002319 658.9739180002362 663.3828539997339 685.6864419998601 Current Mean: 480.110 ms Stdev: 129.942 ms (27.1%) Runs: 290.7557779997587 298.0888269999996 301.43457099981606 307.4532060008496 318.6726479995996 319.74788499996066 323.0387369999662 326.82503299973905 328.23693799972534 337.19824200030416 338.0709229996428 350.3486730000004 359.3664959995076 360.7155770007521 362.8916020002216 363.6289880000986 370.4261879995465 377.2214360004291 380.35819499986246 380.4551599994302 382.2853199997917 385.06620300002396 385.9578049997799 388.3621829999611 402.1499429997057 427.3040360007435 452.9881600001827 463.32287599984556 475.5500900000334 478.95011400058866 482.47977699991316 485.61649600043893 490.5386150004342 496.2018639994785 509.42675900040194 518.3033050000668 521.200154999271 522.1745200008154 541.8848069999367 552.8354080002755 554.4312340002507 558.0227460004389 567.8896490000188 568.1798509992659 568.9352619992569 580.8712160000578 587.284831000492 604.4516600007191 605.9324139999226 615.2051189998165 624.5551350004971 633.5082599995658 650.5862229997292 665.2394620003179 671.7214359994978 686.9343670001253 687.4185389997438 704.3919680006802 721.1540120001882 792.3304449999705

[dangrous]

hi friends! just checking in - where are we on this one? let me know if I can help in any way!

[kirillzyusko]

@dangrous I think we are currently (still) trying to fix e2e tests. And only after that we can start to write new e2e tests 👀

[dangrous]

how are we looking?

[kirillzyusko]

@dangrous Still fixing e2e tests 😅 Recent RN 0.75 broke them 😔

[hannojg]

Okay, so the tests are stable again, and we are collecting additional metrics.
Now we'd like to add more e2e performance regression tests. What other flows do you think are important to cover?
I feel like we currently don't really cover any money request flows?

We will add other tests like chat scrolling etc as mentioned in the issue's description soon!

[mountiny]

Yes, create money request flow - a manual would be good. Not sure if its possible to test the one with a scan, you might have to upload a file

[dangrous]

How are things going here?

[hannojg]

cc @kirillzyusko is working on that (please assign him as well). We've overhauled the e2e pipeline to be more robust. We have to fix one failing test, but are otherwise ready to write some more e2e tests this week (finally!!)

[dangrous]

awesome!

[kirillzyusko]

Yes, I should get some time to work on new e2e tests this week!