Provider Fails to Withdraw Funds from Lease

[chainzero]

Provider version - v0.2.1

Summary

Providers failing to initiate wihdraw lease transaction on small percentage of active leases.

Example of Issue

• Provider - akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc
• DSEQ - 10561974
• Owner - akash1qyfg4zl2dku8ry7gjkhf88vnc3zrn6vmnzlvr9

When the lease state of this example, problem deployment is queried the escrow_payment > withdrawn > amount field is 0 despite the lease being active for several months.

provider-services query market lease get --dseq 10561974 --provider akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc --owner akash1qyfg4zl2dku8ry7gjkhf88vnc3zrn6vmnzlvr9
escrow_payment:
  account_id:
    scope: deployment
    xid: akash1qyfg4zl2dku8ry7gjkhf88vnc3zrn6vmnzlvr9/10561974
  balance:
    amount: "0.000000000000000000"
    denom: uakt
  owner: akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc
  payment_id: 1/1/akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc
  rate:
    amount: "17.000000000000000000"
    denom: uakt
  state: open
  withdrawn:
    amount: "0"
    denom: uakt
lease:
  closed_on: "0"
  created_at: "10561981"
  lease_id:
    dseq: "10561974"
    gseq: 1
    oseq: 1
    owner: akash1qyfg4zl2dku8ry7gjkhf88vnc3zrn6vmnzlvr9
    provider: akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc
  price:
    amount: "17.000000000000000000"
    denom: uakt
  state: active

Majority of leases on this example provider are being withdrawn from properly and this issue only appears to occur on a very small subset of active leases.

Have asked provider to collect logs from time period in which lease withdraws occur daily and will attach to issue once available.

Additional Examples

• Additional examples of zero withdraw issue can be found via:

provider-services query market lease list --provider akash1p5vh8dsck7mnvc5zha6u3aes0fya3lq4ec9dpy --state active

provider-services query market lease get --dseq 12305281 --provider akash1lwzjtsqkuytusa98py77cflhqxl3z7v7ur5632 --owner akash1722lgxkjze280zv2gvhr05as9vsfm704yswj2l

provider-services query market lease get --dseq 11447603 --provider akash1tweev0k42guyv3a2jtgphmgfrl2h5y2884vh9d --owner akash1srujzhj2v9fkzhnn635udlczyhdpetuh34mhad

[chainzero]

Attaching log provider logs for following scenario:

• Akash Provider - akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc
• Based on 12 hour withdraw period - provider began issuing send withdraw to provoke withdraw lease transactions at 2023-08-17T17:47:21.518309326Z
• Beginning at 2023-08-17T17:47:51.617786486Z we begin to see failed to do lease withdrawal messages for all send withdraw attempts
  -However withdraw lease TXs were indeed successful for a majority of leases as proven by this transaction for DSEQ 12161021 as example. Yet we find no new send withdraw messages in either log file 1 or 2 for this DSEQ. It appears the TX actually was successful despite the failed to do lease withdrawal message. Proof of successful TX:
  https://www.mintscan.io/akash/transactions/8CBAE4B07566A4F491582FB8C9C3E08AE6F654144387EE4BC1D8CFB2C02828B1
• But we find no send withdraw message in the provider logs for this lease that has been active for several months during this withdraw lease period:
  provider-services query market lease get --dseq 10561974 --provider akash18ga02jzaq8cw52anyhzkwta5wygufgu6zsz6xc --owner akash1qyfg4zl2dku8ry7gjkhf88vnc3zrn6vmnzlvr9
• It appears that for problem leases - those that have 0 withdrawn from lease over time - no send withdraw is initiated by the provider

europlots_08172023_logs_1.txt
europlots_08172023_logs_2.txt