Variables from codes - too many codes

[AlexOrlek]

I want to create variables from a codelist using the loop_over_codes function (https://docs.opensafely.org/study-def-tricks/#create-a-variable-for-each-code-in-a-codelist)
But the codelist (dm+d antibacterial codes) I planned to use has >2500 codes (https://www.opencodelists.org/codelist/opensafely/antibacterials/27089eaa/) - I've been told to aim for <100 codes to keep things computationally manageable.
To get round this, I wondered whether it's possible to use a hierarchical codelist and just restrict to codes for broader concepts e.g. amoxicillin rather than including all descendants (as indicated below for pseudo BNF codes). I understand I have to convert to dm+d which is non-hierarchical (#343) - could I just convert and include codes for broader concepts in a dm+d codelist?

[iaindillingham]

Hello @AlexOrlek, thanks for your question.

You could use "parent" codes, as you describe in your question, which would require a hierarchical codelist. I'm not an expert in codelists, but are you comfortable converting between codelists of different types?

Alternatively, you could split the codes across multiple study definitions. The study definitions would be similar; you could share common variables between them, to reduce error. When splitting the codes, you could group by "parent" codes, which would require a hierarchical codelist. Alternatively, you could create groups of arbitrary size (e.g. 100 codes).

[AlexOrlek]

Hi Iain, thanks for your reply.
I decided to split the original codelist into meaningful sub-groups and then use these codelists to define variables in a single study definition (I think this is more or less what you were suggesting).
Since the antibacterials codelist has a bnf_code column, I split into the broad antibacterial categories (13 subsections) defined in BNF antibacterials section 5.1 (https://openprescribing.net/bnf/0501/).
For example, I created a penicillin codelist by extracting codes from the antibacterials codelist with the penicillin bnf_code prefix 050101
Below is the link to my penicillin codelist. I've created codelists for the other BNF antibacterial categories.
(https://www.opencodelists.org/codelist/user/alexorlek/penicillins-antibacterials/324f0a29/)

[iaindillingham]

Great! Thank you for reporting back.