Browsing a large number of nodes, abnormally slow, and will report a warning

[mayongbin]

Hey Team
I have some questions. Currently, I want to browse nodes. If the number of nodes is relatively small, there is no problem. If the data volume of a node is large, it will be abnormally slow and start to issue warnings. What I mean by abnormally slow is that if I have 5000 nodes, I can browse for about 2-3 seconds. If it is 10000 nodes, it may take more than 2 minutes, and then various warnings will start to be issued, Here are some of my code and error messages

public void run(OpcUaClient client, CompletableFuture<OpcUaClient> future) throws Exception {


        // start browsing at root folder
        UaNode rootNode = client.getAddressSpace().getNode(Identifiers.RootFolder);

        Tree<UaNode> tree = new Tree<>(rootNode);

        long startTime = System.nanoTime();
        browseRecursive(client, tree).get();
        long endTime = System.nanoTime();
        log.info("excute {}ms", TimeUnit.MILLISECONDS.convert(endTime - startTime, TimeUnit.NANOSECONDS));
        System.out.println(tree);


        future.complete(client);
    }

    private CompletableFuture<Void> browseRecursive(OpcUaClient client, Tree<UaNode> tree) {
        return client.getAddressSpace().browseNodesAsync(tree.node).thenCompose(nodes -> {
            // Add each child node to the tree
            nodes.forEach(tree::addChild);

            // For each child node browse for its children
            Stream<CompletableFuture<Void>> futures =
                    tree.children.stream().map(child -> browseRecursive(client, child));

            // Return a CompletableFuture that completes when the child browses complete
            return CompletableFuture.allOf(futures.toArray(CompletableFuture[]::new));
        });
    }

Warning and error messages

20:35:54.523 [milo-shared-thread-pool-3] WARN org.eclipse.milo.opcua.sdk.client.AddressSpace - Failed to create Node from Reference to ExpandedNodeId{ns=2, id=9001, serverIndex=0}
java.util.concurrent.CompletionException: UaServiceFaultException: status=Bad_SessionIdInvalid, message=The session id is not valid.
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.eclipse.milo.opcua.stack.client.UaStackClient.lambda$deliverResponse$5(UaStackClient.java:337)
at org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$Task.run(ExecutionQueue.java:119)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.milo.opcua.stack.core.UaServiceFaultException: status=Bad_SessionIdInvalid, description=The session id is not valid.
... 5 common frames omitted
20:35:54.523 [milo-shared-thread-pool-3] WARN org.eclipse.milo.opcua.sdk.client.AddressSpace - Failed to create Node from Reference to ExpandedNodeId{ns=2, id=9000, serverIndex=0}
java.util.concurrent.CompletionException: UaServiceFaultException: status=Bad_SessionIdInvalid, message=The session id is not valid.
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:593)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977)
at org.eclipse.milo.opcua.stack.client.UaStackClient.lambda$deliverResponse$5(UaStackClient.java:337)
at org.eclipse.milo.opcua.stack.core.util.ExecutionQueue$Task.run(ExecutionQueue.java:119)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.milo.opcua.stack.core.UaServiceFaultException: status=Bad_SessionIdInvalid, description=The session id is not valid.
... 5 common frames omitted
java.util.concurrent.ExecutionException: UaServiceFaultException: status=Bad_SessionIdInvalid, message=The session id is not valid.
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)

Thank you for your help

[kevinherron]

What server are you connected to?

[kevinherron]

Not sure it actually matters much - what's probably happening is that you are simply overwhelming the server until it can't respond or your session times out.

The async browse example you've copied has no limit on its concurrency. At each level the browse is sent async/concurrent, and the number of requests likely continues to grow near exponentially depending on the structure of the address space you are connected to.

Recursively browsing an address space is not something to be done lightly. Consider using a synchronous/serial version of or adding your own concurrency limits.

[mayongbin]

Thank you for your answer. I will try using serial communication to see if I can meet my reading speed requirements