TX DMA Current skip due to TO buffers

[sam-dee]

I'm trying to transmit some data, but DMA buffers seems to overflow under used data rates. I think this is kinda similar to myriadrf/xtrx_linux_pcie_drv#7, I tried to increase DMA buffers myself, but failed with

13:22:34.756898 ERROR:  [PCIE] PCI:/dev/xtrx0: DMA TX mmap*() failed: Unknown error -22

It seems to work for a while and then start failing, here's log

linux; GNU C++ version 5.4.0 20160609; Boost_105800; UHD_003.009.007-5-ge8202128

13:26:53.137199 DEBUG:  xtrxllpciev0_discovery:264 [PCIE] pcie: Found `pcie:///dev/xtrx0`
Found device #0: addr=pcie:///dev/xtrx0, dev=pcie:///dev/xtrx0, driver=xtrx, label=XTRX: pcie:///dev/xtrx0 (10Gbit), media=PCIe, module=SoapyXTRX, name=XTRX, serial=, type=xtrx, 
13:26:53.615690 DEBUG:  xtrxllpciev0_discovery:264 [PCIE] pcie: Found `pcie:///dev/xtrx0`
[INFO] Make connection: 'pcie:///dev/xtrx0'
13:26:53.620728 INFO:   [CTRL] PCI:/dev/xtrx0: XTRX Rev4 (04000113)
13:26:53.620760 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA STOP MIMO (BLK:0 TS:0); TX DMA STOP MIMO @0.0
13:26:53.620783 INFO:   [PCIE] PCI:/dev/xtrx0: Device `pcie:///dev/xtrx0` was opened
CPU Features: SSE2+ SSE4.1+ AVX+ FMA+
13:26:53.736923 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x000304
13:26:53.837081 INFO:   [CTRL] PCI:/dev/xtrx0: FPGA V_GPIO set to 3280mV
13:26:53.837114 INFO:   [CTRL] PCI:/dev/xtrx0: LMS PMIC DCDC out set to VA18=1880mV VA14=1480mV VA12=1340mV
13:26:53.840488 INFO:   [CTRL] PCI:/dev/xtrx0: FPGA V_IO set to 1800mV
13:26:53.850632 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x000306
13:26:53.861074 INFO:   [LSM7] PCI:/dev/xtrx0: LMS VER:7 REV:1 MASK:1 (3841)
13:26:53.861101 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x00031e
13:26:53.861191 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x00031e
13:26:53.861207 INFO:   [DBGP] Starting XTRX debug thread
13:26:53.876551 INFO:   [XTRX] PCI:/dev/xtrx0: Set INT RefClk to 26000000 based on 26060640 measurement
13:26:53.876656 INFO:   [LMSF] PCI:/dev/xtrx0: Increase RXdiv= 0 TXdiv= 8 => CGEN 2.4 Mhz
13:26:53.876677 INFO:   [LMSF] PCI:/dev/xtrx0: Increase RXdiv= 0 TXdiv=16 => CGEN 4.8 Mhz
13:26:53.876687 INFO:   [LMSF] PCI:/dev/xtrx0: Increase RXdiv= 0 TXdiv=32 => CGEN 9.6 Mhz
13:26:53.876711 INFO:   [LSM7] PCI:/dev/xtrx0: CGEN: VCO/2=1171200000 k/2=122 int=90 frac=96791
13:26:53.877320 INFO:   [LSM7] PCI:/dev/xtrx0: CGEN: binary result: 153
13:26:53.877665 INFO:   [LSM7] PCI:/dev/xtrx0: CGEN: Retuned [153:159] -> 156
13:26:53.877878 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x00031e
13:26:53.877895 INFO:   [CTRL] PCI:/dev/xtrx0: FPGA V_IO set to 1800mV
13:26:53.877905 INFO:   [LMSF] PCI:/dev/xtrx0: rxrate=0.000MHz txrate=0.300MHz actual_master=9.600MHz rxdecim=1(h_1) txinterp=32(h_1) RX_ADC=2.400MHz TX_DAC=9.600MHz hintr=0 hdecim=0 delay=0 NRXFWD=0 LML1HID=3 LML2HID=1 RX_div=0 TX_div=0 RX_TSP_div=1 TX_TSP_div=16 FclkRX=0.000 (PHS=0) RXx2=0
Actual TX Rate: 300000.000000...
13:26:53.877982 INFO:   [LMSF] PCI:/dev/xtrx0: Set gain -12.0 to 3 on 1 channel
Actual TX Gain: 40.000000...
13:26:53.878076 INFO:   [LMSF] PCI:/dev/xtrx0: FE_FREQ rx=0 type=1 freq=1000000000.000000 ch=3
13:26:53.878221 INFO:   [LSM7] PCI:/dev/xtrx0: SXT: initial VCO=2000000000 DIVH=1 VCOs:1000
13:26:53.878237 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: VCO0 N=76 frac=967916
13:26:53.878608 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: binary result: 255
13:26:53.878662 INFO:   [LSM7] PCI:/dev/xtrx0: SXT: VCO0 [0;0] -> 0
13:26:53.879740 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: VCO0 N=76 frac=967916
13:26:53.880104 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: binary result: 43
13:26:53.880484 INFO:   [LSM7] PCI:/dev/xtrx0: SXT: VCO0 [43;50] -> 46
13:26:53.880498 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: VCO0 N=76 frac=967916
13:26:53.880863 INFO:   [LSM7] PCI:/dev/xtrx0: SXX: binary result: 45
13:26:53.881205 INFO:   [LSM7] PCI:/dev/xtrx0: SXT: VCO0 [45;51] -> 48
13:26:53.881260 INFO:   [LMSF] PCI:/dev/xtrx0: NCO ch=1 type=4 freq=0
Actual TX frequency: 1000.000000 MHz...
13:26:53.881332 INFO:   [LSM7] PCI:/dev/xtrx0: 0x0124[02, 10]
13:26:53.881445 INFO:   [LSM7] PCI:/dev/xtrx0: TBB: path 4
Actual BW: 20.000000 MHz...
13:26:53.881563 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA SKIP MIMO (BLK:0 TS:0); TX DMA STOP MIMO @0.0
13:26:53.881614 INFO:   [LSM7] PCI:/dev/xtrx0: 0x0124[02, 10]
13:26:53.881744 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA SKIP MIMO (BLK:0 TS:0); TX DMA STOP MIMO @0.0
13:26:53.881932 INFO:   [LMSF] PCI:/dev/xtrx0: AFE TX=[1;0] RX=[0;0]
13:26:53.882013 INFO:   [LSM7] PCI:/dev/xtrx0: TBB: path 0
13:26:53.882067 INFO:   [LSM7] PCI:/dev/xtrx0: 0x0124[03, 10]
13:26:53.882280 INFO:   [LMSF] PCI:/dev/xtrx0: TBB Restore BW[0]=20000000
13:26:53.882295 INFO:   [LSM7] PCI:/dev/xtrx0: TBB: path 4
13:26:53.882392 INFO:   [LMSF] PCI:/dev/xtrx0: TBB Restore DSP[0]=0
13:26:53.882410 INFO:   [LSM7] PCI:/dev/xtrx0: TXTSP CMIX=0
13:26:53.882600 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA SKIP MIMO (BLK:0 TS:0); TX DMA 16 bit SISO @0.0
[INFO] SoapyXTRX::activateStream(TX) 0 Samples per packet; res = 0
Press Ctrl+C to stop streaming...
Using avx for xtrxdsp_sc32_iq16
13:27:08.272963 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current delayed buffers: 9, prev: 0 TS:4562496
13:27:08.300232 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current skip due to TO buffers: 10, prev: 9 TS:4571136 STPS:4574016
13:27:08.573529 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current delayed buffers: 19, prev: 10 TS:4654656
13:27:08.600678 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current skip due to TO buffers: 20, prev: 19 TS:4663296 STPS:4666176
13:27:08.873946 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current delayed buffers: 29, prev: 20 TS:4746816
13:27:08.901423 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current skip due to TO buffers: 30, prev: 29 TS:4755456 STPS:4758336
13:27:09.347530 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current delayed buffers: 39, prev: 30 TS:4887936
13:27:09.374836 ERROR:  [XTRX] PCI:/dev/xtrx0: TX DMA Current skip due to TO buffers: 40, prev: 39 TS:4896576 STPS:4899456
^C13:27:09.709339 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA SKIP MIMO (BLK:0 TS:0); TX DMA STOP MIMO @0.0
13:27:09.709421 INFO:   [LSM7] PCI:/dev/xtrx0: 0x0124[02, 10]
13:27:09.709560 INFO:   [BPCI] PCI:/dev/xtrx0: RX DMA SKIP MIMO (BLK:0 TS:0); TX DMA STOP MIMO @0.0
13:27:09.709921 INFO:   [CTRL] PCI:/dev/xtrx0: RFIC_GPIO 0x000300
13:27:09.711086 INFO:   [PCIE] PCI:/dev/xtrx0: Device closing

Here's program I use, any ideas how to fix?
main.zip

p.s This one seems to work with uhd B210 without failing, ofc using SoapySDR

[sam-dee]

it has been discovered that using buffer with it's size equal to a power of 2 partially solves the problem. It's a lack of performance for me and I can't afford splitting my buffers into powers of 2, do you have any idea how to set it up for using non-power of 2 buffers?
(I've experimented with my test program pinned above, in that case changing buffer size to 2^n does it work, but I have not succeeded in other cases)