List topic messages is extreamly slow after upgrading from 0.25.0 to 0.25.1

[djeanprost]

Hello

As a follow up of this conversation #1880, I'm raising this issue because I couldn't find any issue related to my problem.

Since upgrade to 0.25.1, this API became very slow. The topic I'm listing the content is empty.
2024-09-12 14:27:22,946 INFO io-executor-thread-6 org.akhq.log.access [Date: 2024-09-12T14:25:21.823535418Z] [Duration: 121123 ms] [Url: GET /api/dev/topic/facture.hospi.create/data] [Status: 200]

Same with 0.25.0 : Still the same empty topic.
2024-09-12 15:45:15,605 INFO io-executor-thread-13 org.akhq.log.access [Date: 2024-09-12T15:45:14.191436025Z] [Duration: 1415 ms] [Url: GET /api/dev/topic/facture.hospi.create/data] [Status: 200]

[AlexisSouquiere]

Can you post your application.yml please ? I suspect that the topic-data.poll-timeout value is high, so the consumer waits this poll-timeout for the topic-data.size number of messages before returning a incomplete response (empty in your case)

[lathil]

Hello,

I work with djeanprost, I reply while he is away. We use a basic application.yaml for akhq, with just a list of kafka connection and ldap server for authentication. We have no specific setting for topic-data.poll-timeout, so I guess it must use the default value, I see this in version 25.1:

  topic-data:
    size: 50
    poll-timeout: 10000
    kafka-max-message-length: 1000000

do you suggess we should lower this value from default 10000 ,

Anyway, here below is ou application.yml:

akhq:
  connections:
    dev:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9093,srv-int-misc002.XXX.local:9093,srv-int-misc003.XXX.local:9093
    dev-build:
      properties:
        bootstrap.servers: srv-int-misc001.XXXX.local:9098,srv-int-misc002.XXX.local:9098,srv-int-misc003.XXX.local:9098
    int:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9092,srv-int-misc002.XXX.local:9092,srv-int-misc003.XXX.local:9092
    int-build:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9099,srv-int-misc002.XXX.local:9099,srv-int-misc003.XXX.local:9099
    learn:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9097,srv-int-misc002.XXX.local:9097,srv-int-misc003.XXX.local:9097
    mh:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9094,srv-int-misc002.XXX.local:9094,srv-int-misc003.XXX.local:9094
    mut:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9101,srv-int-misc002.XXX.local:9101,srv-int-misc003.XXX.local:9101
    rec:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9095,srv-int-misc002.XXX.local:9095,srv-int-misc003.XXX.local:9095
    rec-build:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9100,srv-int-misc002.XXX.local:9100,srv-int-misc003.XXX.local:9100
    sbx:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9102,srv-int-misc002.XXX.local:9102,srv-int-misc003.XXX.local:9102
    vyv:
      properties:
        bootstrap.servers: srv-int-misc001.XXX.local:9096,srv-int-misc002.XXX.local:9096,srv-int-misc003.XXX.local:9096
  security:
    basic-auth:
    - groups:
      - admin
      password: XXXX
      username: admin
    - groups:
      - reader
      password: XXXX
      username: reader
    default-group: no-roles
    ldap:
      groups:
      - groups:
        - admin
        name: grp-prem-stg-adm-akhq
      - groups:
        - reader
        name: grp-prem-stg-usr-akhq
  server:
    access-log:
      enabled: true
      format: '[Date: {}] [Duration: {} ms] [Url: {} {}] [Status: {}] [Ip: {}] [User:
        {}]'
      name: org.akhq.log.access
micronaut:
  security:
    enabled: true
    ldap:
      default:
        context:
          managerDn: CN=svc-app-stag-akhq,OU=AKHQ,OU=Comptes Services,OU=Horizon,DC=XXXX,DC=LOCAL
          managerPassword: XXXXXX
          properties:
          - java.naming.referral: follow
          server: ldap://XXXX:389
        enabled: true
        groups:
          base: DC=XXXX,DC=LOCAL
          enabled: true
          filter: member={0}
          subtree: true
        search:
          base: DC=XXXX,DC=LOCAL
          filter: userPrincipalName={0}
          subtree: true
    token:
      jwt:
        signatures:
          secret:
            generator:
              secret: XXXXX

[lathil]

I just noticed that pool-timeout was set to 1000 in 25.0 and is now 10000 in 25.1, as you suggest that must be the probleme.

[AlexisSouquiere]

Please confirm it after testing with a lower poll-timeout and close the issue if it's ok for you

[djeanprost]

@AlexisSouquiere seems ok now with this configuration :

ahhq:
   topic-data:
    poll-timeout: 1000  

Anyway, I would have expected to have a request duration around 1s instead of 13s. Could it be related to the number of partition we have to go through go fetch data (twelve in my case, so 12*1000=~13000) ?

2024-09-16 09:19:57,457 INFO io-executor-thread-5 org.akhq.log.access [Date: 2024-09-16T09:19:44.421205431Z] [Duration: 13038 ms] [Url: GET /api/dev/topic/facture.hospi.create/data] [Status: 200]

[AlexisSouquiere]

Yes you are right. We had previously a single consumer for all the partitions and we switched in 0.25.0 to a consumer by partition. But I'm seeing that we are doing a classic sequential loop so AKHQ waits for an answer from the 1st partition before requesting the 2nd partition, etc. We should probably do a parallelStream instead.

Labelling this issue as enhancement