SVT: Application keeps on restarting in Dev mode without making any changes to the app #1755
Comments
|
The prev errors were on Eclipse. |
|
Could be related to #699? Although Rumana did say she did not see the same behavior from the command line. Need to try and recreate from the cmd line ourselves to see. |
|
Update about command line. I just retried, and I see the same behavior from the command line as well. The issue is where I run the However - if I run within the project AcmeWebEjbEar |
|
I've been experimenting with the CargoTracker Jakarta sample: https://github.com/eclipse-ee4j/cargotracker/ There seems to be a similar "thrashing" behavior where the app continually restarts upon update. I accounted for the issue: #699 by building a local snapshot and hard-coding disabling the initial compile (the one we do AFTER the compile mojo runs). While this simplified the initial startup, I still experienced the same thing after later making an update once the app had been up and started. There seems to be something app-related in play here. The cargo tracker uses EJB persistent timers and its JPA runs a startup script... not sure if either of those are related. |
|
@rumanaHaque can you please try to recreate with this trace:
I saw something similar w/ the cargotracker app...there I suspect it is some failures related to messaging & MDB init causing the thrashing..combined with the fact that restarting some of these messaging components forces an app recycle. I'm not too familiar with this from the runtime perspective.. so possibly we'd need a couple rounds of tracing to get to the bottom of this. Thank you ! |
|
I added the server trace string as you mentioned in my server.xml file- I did an mvn clean, and started my server using the Liberty Tools dashboard. These are the logs for the server after I started, and it also has the console.txt file where I tried to capture most of the console output. I also got the server dump as you requested - by running this command. Here is the server dump. |
|
It looks to me like the |
|
Hi @scottkurz - Yes - sorry I had put the trace string in a diff server.xml in my workspace. And the server dump again. |
|
Discussed this more in DXDI call today: https://github.com/orgs/OpenLiberty/projects/23/views/3?pane=issue&itemId=50131096 I'm going to share some sketched out, incomplete thoughts here, though this doesn't even include detailed analysis of the captured trace, just some guesses based on a preliminary look at the logs captured back in Dec. THOUGHTSI wonder if this issue raises a weakness with the design assumption that we don't need an mbean notification on app update, (like we used in WDT/LDT). This would potentially allow us to do a single app restart rather than multiple ones. That said, I wouldn't think the app would just keep restarting if this were the core issue. I'd think it'd eventually settle down...but this is just a guess. Looking into it a bit, it seems WDT builds a set of changed files and invokes the FileNotificationMBean passing this set, after which the Liberty runtime then (re)starts the app. I also wonder if bumping up the polling rate could reduce the chance for this kind of situation. E.g. if we switched from 500ms to 5s would the mbean not be as important?
CONCLUSIONThat's all for now, I'm not asking anyone to rerun anything, just sharing some of my own notes on the subject here. |
|
We encountered the same problem with our apps going through several restarts at the initial liberty:dev execution (@scottkurz please note that Rumana does not speak of continually restarting but instead says "It goes through this cycle a couple of times before the application is finally available"). I took the suggestion from your previous comment and experimented with the applicationMonitor pollingRate. Setting it to more than what is required for the application startup time plus all the module's 'source compilation was successful/tests compilation was successful' calls indeed prevents the multiple restarts. This however will make the liberty:dev experience a lot less responsive to code changes. The mbean notification mechanism would be my preferred solution, but perhaps an extra one letter command to restart the application is an easy to implement alternative. Ofcourse, getting rid of the initial compile calls as described in #699 would also help. |
|
@m-schutte-ohra-nl, though I don't have any further updates on this issue, I just wanted to thank you for sharing your experience here. |
|
@scottkurz This is still a real problem for us, can we expect progress to be made on this issue ? |
I have the ACME application - that I imported in my workspace and start using the Start menu of the Liberty Dashboard.
After starting the application - I see that the app was started, but it keeps on restarting and updating, even though I do not make any changes to my application or the configuration.
c:\eclipse-workspace\acme-ee10\acme-ee\AcmeWebEjbEar>mvn liberty:devI get some of the following output:
However, if I use the Liberty Tools to do the same thing - I see that the app doesn't start, and immediately, and it keeps on restarting
It goes through this cycle a couple of times before the application is finally available. During these times - if I try to access the app url - I get Context Root not found.
Finally - when I see this message:
then the app is finally stable and I can access the app url.
The text was updated successfully, but these errors were encountered: