Support for index sorted files

[jainaditi0204]

Hi Scott,
I hope you're doing well!

I wanted to ask you about the support for FACs Diva index sorted files, and if there are ways to do analyses with them, along with the general FCS files. For example, if there's way for the user to see which events (gate specific, from the respective GatingML) from the index sorted files lie in the main FCS files.
I would like to hear your thoughts on it!

Best,
Aditi

[whitews]

Hi Aditi,

I don't have any examples of these types of files, so there are no current plans to add support for them. If you have examples that you are able to make publicly available, let me know and I will take a look.

Thanks,
Scott

[jainaditi0204]

Hi Scott,

Thank you for your answer.
I'm sending you the file, as I type this :)

Best,
Aditi

[whitews]

Hi Aditi,

I looked at the files you sent. The FCS file parses fine for me. The GatingML document parses ok but there is a parent gate 'root' that is referenced in several gates, yet there is no such 'root' gate. If a gate should be applied to the whole population of events then the parent gate should be omitted, so the fix is simply removing those gating:parent_id="root" attributes. Also, FlowKit reserves the term 'root' for the root of the gating tree so there may be issues if you tried to create such a gate, but I haven't tested that.

Hope this helps,
Scott

[jainaditi0204]

I have removed the top-level 'root' parent gates, and now it throws a KeyError.

file_path = '/.../output_file.xml'
g_strat = fk.parse_gating_xml(file_path)
sample = '/.../sample.fcs'

Here, for now, I am using a normal FCS file, and not the index-sorted one.

sample_ = fk.Sample(sample)
gs_results = g_strat.gate_sample(sample_, verbose=True)

KeyError: 'fcs'

[whitews]

Can't say for sure what is going on there, but the 'fcs' string would be used to indicate that a channel should be compensated using the spill matrix within the FCS file. I looked again at the original XML you sent and another odd thing is that it specifies 'fcs' for the compensation for scatter channels, but scatter channels do not require compensation. Very odd!

[jainaditi0204]

I'll check the file again, and then get back to you.
Also, as I mentioned earlier, if you have any example data set (a GatingML and a FCS file) to make comparisons, and understand why it's breaking, it would be a huge help.

[whitews]

There are several examples in the repo here:

https://github.com/whitews/FlowKit/tree/master/examples/data/gate_ref

[jainaditi0204]

I modified the file again, and added ‘FCS’ in uppercase for the compensation-ref (as mentioned in the gating standards document), and it works for some gates, and throws the same key error for others. To test, I also changed it to ‘uncompensated’ and that seems to work just fine. The only issue is that the values are in fact compensated, so keeping the ref as uncompensated doesn’t make any sense. I am still looking into it, and if I find need some guidance, I’ll let you know.
The example files helped a lot, so thank you again!

Best,
Aditi

[whitews]

And regarding the index sorted file, I am not very familiar with these, but it looks like the file corresponds to a 384 well plate (I'm guessing 16 x 24 ?) since there are 384 events. Also, there are extra metadata keys that contain what look like coordinates. For example, there's a keyword : value:

"index sorting locations_1": '0,0;0,1;0,2;0,3;0,4;0,5;0,6;0,7;0,8;0,9;0,10;0,11;0,12;0,13;0,14;0,15;0,16;0,17;0,18;0,19;0,20;0,21;0,22;0,23;1,23;1,22;1,21;'

There are 16 of these keywords (1..16) with a total of 384 of these coordinate values. The values do seem to correspond to a 16 x 24 matrix, but they just snake across the rows / columns in order so I don't think there's any real information in those coordinates. Would someone using these index sorted files just want this list of well locations in the order they are stored in these metadata values? If so, I could provide a simple Sample method that returns this if it is available in the metadata (i.e. the Sample is an index sorted file).

Thanks,
Scott

[jainaditi0204]

Yes, that can be helpful.. In general, for reading-in the index-sorted files, and retrieving the stored information.

Best,
Aditi