improve performance issue fetching all the states

[Mariusthvdb]

Hi Bram,
after inspecting my HA instance, Powercalc integration turned out to be one of the most intense of the system... Please check what Nick (BDraco) mentions about the way it gets its data and manipulates that. Would be very interesting to see (and experience) if that could be changed for the better.

key performance issue with the powercalc integration is its ignoring the state updates and fetching them from the state machine for every entity in the group every time it changes in

homeassistant-powercalc/custom_components/powercalc/sensors/group.py

Line 573 in 0dc3ad4

def on_state_change(self, _: Any) -> None: # noqa

so every state change ends up fetching all the states in the group back from the state machine every time

graphical:

[bramstroker]

I don't see how this can be optimized, except introducing some throtteling, i.e. only handle update ones every X seconds, and not on every state change of a group member entity.

This is similar as all the core group helpers are implemented, they also fetch all individual member states on every state change of one of the group entities.
Also when you use template sensors with expand / sum etc, this is essentially doing the same. Fetching all states from the state machine and doing the aggregation.

State fetches are also very cheap as they are just getting a single key from a dictionary, which is a highly optimized data structure in python.

When you have a lot of group sensors, which in turn consists of many member entities it's expected to react to a lot of state changes, which cause all the get calls on dictionary.

[Mariusthvdb]

thats impressive!
can I test run it, or is it not yet production proof yet?

[bramstroker]

Should be production ready as all tests are passing and there are a lot of them.
But would be great if you can give it a test run.
Only file which is changed is group.py, so you could get that one from the branch and override in your custom_components dir.

https://github.com/bramstroker/homeassistant-powercalc/blob/fix/group-state-retrieval-performance/custom_components/powercalc/sensors/group.py

[Mariusthvdb]

ok will do!
I'll report back, after Nick has had a chance to judge the new Profile, and added his comments.

[bramstroker]

I had some further conversations with Nick on discord, and we also found some further improvements.
Have merged the MR this morning.
Will release new beta v1.13 soon which you can use to test.

[Mariusthvdb]

Magic, you 2 are the best?

[Mariusthvdb]

Nick checked my profiler files, and reported this is much better indeed, so mission accomplished I would say.
If we get green light from the Master in HA speed and efficiency, there is no better thing than that!

thank you for taking the time to do this.
shall we close?

[bramstroker]

That's great to hear, also thanks for your time on reporting this and testing!
Also massive thanks to @bdraco for your guidance on this one.