You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Not sure if this is a bug or not. If I run fastfec 878160 and I look at the resulting output/878160/SA11D.csv, then I see this:
form_type,filer_committee_id_number,transaction_id,back_reference_tran_id_number,back_reference_sched_name,entity_type,contributor_organization_name,contributor_last_name,contributor_first_name,contributor_middle_name,contributor_prefix,contributor_suffix,contributor_street_1,contributor_street_2,contributor_city,contributor_state,contributor_zip_code,election_code,election_other_description,contribution_date,contribution_amount,contribution_aggregate,contribution_purpose_descrip,contributor_employer,contributor_occupation,donor_committee_fec_id,donor_committee_name,donor_candidate_fec_id,donor_candidate_last_name,donor_candidate_first_name,donor_candidate_middle_name,donor_candidate_prefix,donor_candidate_suffix,donor_candidate_office,donor_candidate_state,donor_candidate_district,conduit_name,conduit_street1,conduit_street2,conduit_city,conduit_state,conduit_zip_code,memo_code,memo_text_description,reference_code
SA11D,C00477828,C7168136,,,CAN,,Clarke,Hansen,,,,2900 E Jefferson Ave,Apt C4,Detroit,MI,482074242,P2012,,2013-06-30,565.73,565.73,,,,,,H0MI13398,Clarke,Hansen,,,,H,MI,13,,,,,,,,"* In-Kind: In-kind, web hosting and phone services, to be reimbursed"
It looks to me that this is missing the required trailing comma that separates the memo_text_description and (the missing) reference_code value. If I try to load this with a pyarrow csv reader with the given 45 column names, it gets mad because it only sees 44 values in the row. You can replicate with pd.read_csv(path, engine="pyarrow"). Other CSV parsers such as vanilla pandas (pd.read_csv(path)) and vaex are more forgiving and just fill in NA for the missing reference_code values, so perhaps that is why this hasn't been caught before.
If I look at at the resulting output/878160/SB17.csv, it's a similar story: there is one less trailing comma than there should be to separate the missing last value.
However, if I look at output/878160/F3S.csv, then this looks correct. I'd guess this is because the last value in that row are non-missing:
Not sure if this is a bug or not. If I run
fastfec 878160
and I look at the resultingoutput/878160/SA11D.csv
, then I see this:It looks to me that this is missing the required trailing comma that separates the
memo_text_description
and (the missing)reference_code
value. If I try to load this with a pyarrow csv reader with the given 45 column names, it gets mad because it only sees 44 values in the row. You can replicate withpd.read_csv(path, engine="pyarrow")
. Other CSV parsers such as vanilla pandas (pd.read_csv(path)
) and vaex are more forgiving and just fill in NA for the missingreference_code
values, so perhaps that is why this hasn't been caught before.If I look at at the resulting
output/878160/SB17.csv
, it's a similar story: there is one less trailing comma than there should be to separate the missing last value.However, if I look at
output/878160/F3S.csv
, then this looks correct. I'd guess this is because the last value in that row are non-missing:The text was updated successfully, but these errors were encountered: