Global configurable data folders

[DellanX]

I'm trying to think far ahead with this one.

While I think the current LUT data folder works great for now, I don't think it is very scalable for the long term.

1. An Artifact build of powercalc is required each time new models are added, even though the processing logic hasn't changed.
2. The current folder structure is specialized for Light Models
3. Similar to 1, it'll be difficult to stabilize PowerCalc into a low iteration cycle with the data directory tied to powercalc
   (For example, HomeAssistant has monthly builds, that'd be a significant slowdown for this project)

The solution I am proposing, is the ability to define data folders inside powercalc's global config, and the eventuality of switching the data folder into another repository. This has a few advantages:

1. Separation of source code and LUT data, which can simplify the git history to the logic changes.
2. Ability to break up model repos by type, each with a simpler specialized validation pipeline.
   (Validation pipeline by type, resulting in a lower complexity, and fewer chances for bugs)
3. If ever this project is to be moved into HomeAssistant, it'll still be easy to streamline the addition of new models

Some Disadvantages:

1. There would need to be model.json version numbering, in case the schema changes with time.
2. If this project went so far as to separate into multiple Repos, the HACs setup would be more complicated.

With knowledge of the global_config was done, this would probably be easy to implement; however, I wanted to propose it here first. Any ideas to this?
(I did see #466; however, this isn't for the addition of custom models, but ones from a secondary repository)

[bramstroker]

I don't plan in overhauling the library architecture short term, as there are no real issues with it right now, and it needs a significant amount of work to facilitate this. It's not worth the effort for me atm.

• Need to write some downloader component to download the library from another location (git repo), maybe per manufacturer, and incoorporate that in the library loading code.
• Also custom model directories still need to work, so we need to add some kind of abstraction there (to deal with internal / external lib equally)
• The home assistant component needs to know when something has changed in the library code, because than it needs to download again (and not each time HA starts, when nothing has changed). So we need some kind of hashing logic.
• In the new repo all the github actions needs to be ported / added.
• Generator of supported_models needs to be refactored.
• All readme's/WIKI need to be changed.
• etc. etc.

It's not hundreds of new additions which are done right now. Maybe 1 or 2 a week. Which is perfectly managable with the current system.

It's definitely something I have on the roadmap When the library grows significantly and contribution frequency increases it's definitely something to consider.

Btw what made you think the current folder structure is specialized to light models? It can also contain fixed / linear configurations etc. in model.json. And we just started adding standby usage of smart switches to the library.
We are also looking into adding google home / alexa etc. etc.

[DellanX]

I see, and I am glad this is something on your roadmap. As to what you said, it's not an issue at the moment. I think this topic is very far-forward looking.

An internal downloader component makes a lot of sense compared to what I was thinking. And if bundled with the .storage directory with version/hash information, then Update needs could be detected and automated similar to how HACS works.

I'll tinker with the downloader approach and see what I can come up with in the next few months. If I can develop something that can give a smooth transition with regards to the bullet points above, I'll create a draft PR for a Proof of Concept.

• Feel free to addon requirements or nice-to-haves to this, or to object my development on this
• Don't wait on me to develop this. As I also don't see it as a priority at the moment.
  While I don't have a lot of free time to tinker, this wouldn't be my first time making such a feature. I just need to get better at non-C based languages to help.

As to the light models. I've only seen a few models on there that weren't lights, and I've seen a lot of discussion on new model type additions. (anecdotal logic on my part, which is/was error prone)