Issue with QRZ.com and Chinese infos

[OH1KH]

FYI

There is an issue with Chinese information (name qth etc written with Chinese symbols) returned from QRZ.com
QRZ says it's xml code is utf-8 and Cqrlog should now support utf-8 in NewQSO columns and database records.

How ever some Chinese information causes database writing errors.
You can test this with QRZ callbook and for example callsign BG4RJT. Without understanding of Chinese printing it looks ok when seen in NewQSO. But saving causes error.
More careful look to NewQSO columns show that Chinese marks have also a plain square as one of marks. I believe (not sure) that this is not Chinese writing symbol, but comes because Utils.GetQRZInfo does not parse utf-8 string properly leaving one 8 bit part away from string.

That is just a guess now and is under investigation

It can also be that QRZ sends and odd 16bit Chinese mark (who knows...)
Something there is because names including Scandinavian letters öäåÖÄÅ are saved without errors.
And also that Chinese "test callsign", BG4RJT, if "squares" are removed from NewQSO columns before saving qso.

(Problem reported by OH1MRR)

[OH1KH]

It seems that there is some problem with FPC/Lazarus, or fonts used. There you see how the first mark of two shown propely in Leafpad text editor converts to "square" in TEdit. Effect is same if marks are copied and pasted to TEdit or TEdit.Text is initiated with hex byte string (like in picture).
When pulling out TEdit.Text the hex byte string remains same, there are no changes compared to what pushed in.

Saving of this qso fails with mysql error.

Column "state" is varchar(4), but error does not say string is too long. It complains the first character. Editing state string shows that it has two marks, but when looking with hex editor there are 6 bytes E6 B1 9F E8 8B 8F , mc editor shows mark one as 0x6C5F and mark two 0x82CF

Seems that the easiest fix is to wipe out non alphabet characters and save qso ;-)

[ok2cqr]

This problem will be in whole CQRLOG. As far as I know, the default charset is utf8 which is alias for utf8-mb3 - 3 bytes UTF8. All the varchar or text blob fields should have utf8-mb4 as default. I'll fix it.