Machine freezes

[pastaclub]

I occasionally get the problem that the machine freezes and doesn't react anymore. This happened in the past when the machine had been on for a while, but now it happens everytime during the reflow process.

I suspect that ESP32 runs out of memory, since mine was almost full before, and now with the code improvements even more memory is used up. Needs to be examined further.

I also suspect that continuosly adding elements to the list temp_points as it's currently done, could lead to memory fragmentation. Would it be possible to replace this by an array of fixed size (and still be compatible with the chart drawing)?

[dukeduck1984]

Indeed it could be the memory that caused the issue, because what I'm using is an ESP32 Wrover which has 4MB spiram, the freeze never happened.

Do you think adding gc.collect() to measure_temp() will help?

I'm not sure about the array, the docs of Lvgl is mostly based on C which I'm not familiar with at all. But if you look at the gui.py, it creates a null data list called null_chart_point_list when the chart is initialized, then when the chart updates with new temp data points, that null data list is copied to contain the new data points. This is where I think it can be optimized.

[BG7YWL]

我使用ESP-WROOM-32模块遇到了同样的问题，运行一段时间自动停止。

[dukeduck1984]

@BG7YWL 你的开发板是不带外部4MB spiram的对吧？我的板子是ESP32 WROVER，自带SPIRAM，没有遇到过死机，估计的确是内存的关系。

[BG7YWL]

@BG7YWL 你的开发板是不带外部4MB spiram的对吧？我的板子是ESP32 WROVER，自带SPIRAM，没有遇到过死机，估计的确是内存的关系。

是的，我现在用的是四博智联的ESP-32开发板，上面的模块型号是ESP-WROOM-32。
我已经从淘宝下单LILYGO T8 V1.8开发板了。

[dukeduck1984]

ok，你换上新的开发板再试下，如果不再死机，说明是内存问题，多数是绘图的部分导致，我尝试做些优化，但不确定是否可以真的解决

[dukeduck1984]

@pastaclub I have updated the branch beta2 in main.py, gui.py & oven_control.py to optimize the memory usage to hopefully solve the crash issue. Pls help test it if you have time. Thanks.

[dukeduck1984]

@BG7YWL 你换了开发板之后还死机吗？我在beta2分支里更新了代码，试图解决死机的问题，有空请帮忙测试下，谢谢

[ryanjAA]

I’m having the crash problem almost every time when using a wroom32 board. I’ll try the beta2 branch and report back. If that doesn’t solve it I’ll get a wrover board with the spiram and see if that fixes it.

Awesome work by the way! Mine follows the curve almost perfectly with no tuning of the pid settings.

Thanks for this! Much better than even some of the purpose built reflow ovens!

[dukeduck1984]

Thanks for the feedback. Pls let me know if the version in branch beta2 works out the issue. Also disable the FTP server in config.json will probably help save some memory too.

[ryanjAA]

Thanks. Will try that as well. Have one of the new wrover units with 8mb of psram (spiram) coming as well so will try first and switch if it doesn’t work. Either way will report back findings.

[pastaclub]

Yes, please report your findings.

Regarding the PID working well: does it work well with all profiles? And what kind of oven or plate are you using?

[ryanjAA]

Will do.

I’ve only tested the sm63/pb47 and there’s a big difference between having it start from ambient (20 deg c) and in the 40-60 deg range. It did freeze every time so I’ll need to do some more testing. One time it froze while on full heat and it just kept going up in temp so I was going to put a separate temp override to cut power but I’m going to first get the software running on the correct esp32 or not freezing with the beta branch before looking at adding more to it.

When it’s cold it tends to jump too high on heat up but when already heated some it works better. Also the cool down is too fast with the door open and too slow with it closed but that will just be a product of finding the right amount to keep it partially opened.

I’m using BLACK+DECKER TO1700SG. It’s the one recommended from the Adafruit site. No increase to power or extra heat generating mods were added or insulation. I did however remove all the electronics, timer and thermostat and wire it so when the ssr enables it, it has full power. The same as if it was on Toast. Just removed everything to mount the lcd on the front.

I found your repo by trying to first buy a 926a reflow oven but it was out of stock, then found the adafruit guide but their screen was out of stock which led me to you and I was able to buy all the components the same day and get this built in a couple of hours.

Definitely great! Going to do some temp tests with an ir thermometer and potentially do a boiling water and/or ice water calibration on the thermocouple to get it perfect. It jumps a little but I need some more data points from testing first to get things squared away with best practices.
👍👍👍

[dukeduck1984]

I wrapped the heating chamber of the oven with some thick ceramic fiber cotton sheets, which (I think) will reduce the thermal inertia to make the temperature regulating more responsive and probably less affected by the starting temperature.

[ryanjAA]

Beta 2 branch fixes everything. No freezing at all. Ran it several times.

I was trying to avoid the ceramic fiber initially just because that stuff is a mess to deal from a safety perspective. It is cool how you have it be 1300deg on one side and 100 on the other and I bet it does help with regulating temp up and down.

Here is a photo of my untuned curve (your settings) and no other changes. Ther thermocouple is taped to a pcb in the middle of the oven. The dips actually I think are errors and have to do with sensor read errors and not temp errors.

[pastaclub]

That's great News, Ryan!

It seems that your oven (just like my hot plate) needs a few extra seconds to go from flat temperature into heating. You may want to try a curve with a pre-delay that reflects this, to make the curve fit better. I did that in this commit in my experimental branch and it would be interesting to know how it works out for you. There's also a discussion about it in this issue.

One more thing I notice about your curve is the high level of noise. It's most apparent in the cooling stage (which is probably related to cold air flow with the open door, and can safely be disregarded). But also in the heating stages, your curve has a bit more noise than Kaiyuan's or mine. Maybe check your temperature sensor and it's connection - it might bring some improvement as well.

[dukeduck1984]

Good to know! 👍 I will then merge beta2 into master.

[ryanjAA]

Thanks @pastaclub that definitely helped. The extra 15 seconds makes a big difference from a cold start.

As for the sensor noise I changed the thermocouple but that didn’t fix it (I didn’t have any more sensors on hand) but adding a 10nf capacitor to the thermocouple +/- at the sensor smoothed the readings right out.

I’ll prob need to increase ki a little since I’m getting to 223-225 now (when it’s already hot and I run it I hit the desired temps though) but overall this works great.

I could add a small fan to increase cooling a little but this is by far more than good enough, an extra minute on the cooling side is not a problem.

My buzzer however doesn’t work. When is it supposed to go off?

@dukeduck1984 sounds good! I didn’t even end up testing the esp with the spiram since this is rock solid.

Photos are both with the extra 15 seconds, before the capacitor and after. Almost exact same curve just much more noise.

Awesome project!

[dukeduck1984]

The buzzer goes off at the start, the transition between stages and the end of reflow process. Whenever you see the status changing it should be making a sound.

[ryanjAA]

Thanks. I’ll double check what pin I have it set to but of all the things I have not working that is the least of my worries! I wired the existing light on the unit in the circuit so it’s on when it’s cooking to show it’s hot.

I have a hot plate sitting by this I might try and set it up but I know for a fact it has some hot/cold spots. A pan though helps with that but I might just make another of the same exact type as this one since this works so well.

[dukeduck1984]

I'm closing this issue as it's been solved.

[pastaclub]

I cannot confirm it's solved. (sorry, I wanted to test this much earlier but after an OS update I was unable to use the USB-to-serial and only fixed that today)

My reflower still freezes. I turned off the FTP to save some RAM (which used to work in an earlier stage of development) but I'm still getting the freezes.

What kinds of boards are you using? Mine is a WROOM-32.

[dukeduck1984]

Oops, sorry, I should have asked you explicitly.

My board is a TTGO T8 v1.8 Esp32-Wrover, but I think @ryanjAA tested beta2 using a Wroom-32.

Any error message showed in the REPL when the machine freezed?

I just found another place where more memory could be saved and pushed a new commit to beta2. Pls take a look and if possible pls help test it. Thanks.

[ryanjAA]

@dukeduck1984 yes, I tested the beta2 branch pretty extensively on a wroom-32. Didn’t freeze once and froze almost every time on the master branch.

I’m going to use it quite a bit soon so will put it through its paces and see if anything comes up but so far it’s been fine.

Did you make any other code changes?

[pastaclub]

Mine does not freeze as long as I don't start the reflowing process. Interestingly, it also doesn't freeze if I do start the reflow but I don't power the heater, so it never gets ouf of the pre-heat stage. If the heater is working, it freezes at 0:51. On the console it says "Traceback (most recent call last):" but then nothing anymore (the console also freezes).

Nothing special is supposed to happen at 0:51. I configured an additional no-heat phase at the beginning and the next change of target temperature is supposed to happen at 1:00.

I checked gc.mem_free() manually via the REPL during operation and it varies somewhere around 40k. At least I don't see it decreasing dramatically towards 0.

Another thing I noticed is that I once declared a function in boot.py that imports network (to connect to my router). I'm not calling that function, but I'm not sure if it imports network anyways. I will try removing this and see if it has any effect on memory.

One annoying thing with using a hot plate instead of an oven is that it always takes forever to cool so I have to wait a long time before I can test again.

[pastaclub]

Just tried again: it froze at 1:02min and 74.4 deg with the same Traceback message. The last value mem_free() returned before was 42352, so it doesn't look like a memory issue. By the way, removing that boot code did not change anything.

[dukeduck1984]

Did you test the most current 'beta2', or you own modified version?

What firmware was it?

As I recall the freeze didn't happen that frenquently and you had a few good runs didn't you? Now it seems to be worse and freezes every single time?

Anything inside 'boot.py' will be executed.

I used a small USB fan to speed up the cooling of my oven during testing.

[dukeduck1984]

I fixed a bug which will cause crash at the very end of the reflow process.

Also added the pre-delay warm-up prior to the start of the reflow process, so that manually editing profile is not required.

This picture shows the effect: it was a cold start from 20 degree, preheat & soak curves were good.

Update has been pushed to beta2.