Docker Crashing ROC-RK3328-CC

[bigjohnson]

Hey, having a problem running docker on my RK3328. I currently only have docker-compose running Octoprint. Upon starting docker and immediately using the top command I get:

r

oot@renegade:~# sensors
soc_thermal-virtual-0
Adapter: Virtual device
temp1:        +47.3°C  (crit = +95.0°C)

root@renegade:~# service start docker
start: unrecognized service
root@renegade:~# service docker start
top
root@renegade:~# top
top - 19:32:08 up 2 min,  1 user,  load average: 0.58, 0.51, 0.21
Tasks: 145 total,   2 running, 143 sleeping,   0 stopped,   0 zombie
%Cpu(s): 22.3 us,  4.5 sy,  0.0 ni, 73.2 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   3973.6 total,   3434.5 free,    211.7 used,    327.5 buff/cache
MiB Swap:   1986.8 total,   1986.8 free,      0.0 used.   3680.9 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
   3385 root      20   0  262984  35420   8424 R  96.0   0.9   0:02.91 octoprint
   3208 root      20   0    9844   3196   2724 R   1.0   0.1   0:00.07 top
   3386 systemd+  20   0   12772   7524   6120 S   1.0   0.2   0:00.03 haproxy
     15 root      20   0       0      0      0 I   0.3   0.0   0:00.23 rcu_preempt
    207 root      20   0       0      0      0 I   0.3   0.0   0:00.18 kworker/u8:3-writeback
    346 root      20   0       0      0      0 I   0.3   0.0   0:00.05 kworker/3:2-mm_percpu_wq

Jumps to 99-103% CPU utilization and then the board locks and needs unplugged/plugged back in (Sometimes red led is stuck on sometimes stuck off). I have passive cooling on the RK chip. Figured I would ask here first before somewhere else because I’ve never had a problem with docker running octoprint on any other SBC’s with docker. I am using the armbian bullseye cli image. I was previously having problems with what I think is my SD card, but I don’t think this would be related. Thanks.

[librecomputer]

@bigjohnson

CPU utilization and then the board locks and needs unplugged/plugged back in (Sometimes red led is stuck on sometimes stuck off). I have passive cooling on the RK chip. Figured I would ask here first before somewhere else because I’ve never had a problem with docker running octoprin

We don’t maintain the kernels for Armbian so you need to ask on their forum.

It might be a MicroSD card problem. sudo iostat -dxm 1 or check dmesg for MMC problems.

[bigjohnson]

Completely forgot about that, sorry. Unfortunately I am gonna close up on this thread because Ubuntu is now working and I’d much rather use it. More support, and I’m not developing any arm things.

[librecomputer]

@librecomputer

We don’t maintain the kernels for Armbian so you need to ask on their forum.

It might be a MicroSD card problem. sudo iostat -dxm 1 or check dmesg for MMC problems.

Usually freezing is due to power problems if it suddenly shutdown or freezes. If it stalls certain programs and dies a slow death, then it is usually the MicroSD card. If those two are ruled out, then its the board.

[bigjohnson]

Since Linux server is now running, I will do some testing to rule out the board. What testing can I do to rule either or out?

[librecomputer]

If you want to test power supply:

git clone https://github.com/ssvb/cpuburn-arm.git
cd cpuburn-arm
gcc cpuburn-a53.S -o cpuburn-a53
./cpuburn-a53

[bigjohnson]

Board gets much hotter than I’m comfortable with running it with just passive alum. fin cooling so I guess I can rule that out. Abt 70c.

[librecomputer]

cpuburn stresses the NEON FPU in the chips. You can get it to generate even more heat by running a GPU load and a VPU load simultaneously. 70C is the throttling temperature we set as to not damage the SoC but it is tolerant of much more. Raspberry Pi for examples allows for a maximum of 85C.

[bigjohnson]

I printed an rpi3b case with room for a 40mm fan. Gotta work out how to control a pwm fan now… Currently stable around 40c just running home assistant but was much warmer running with a display attached (60ish). I have it headless now.

[librecomputer]

How to Read and Control PWM Fan Speed on AML-S905X-CC

PWM fans come with 4 wires: Ground Power (3.3V, 5V, 12V) Tachometer PWM speed control Per the GPIO map, there is a convenient header 9J1: Ground GPIO (for tach) 5V On the 40-pin header, there are 5 hardware PWMs we can use per our PWM guide. We do not want to waste CPU cycles to generate inaccurate PWM via software GPIO bitbanging. We can hook these up to the fan via female to male jumper wires. pwm-a - Pin 33 pwm-ao-a - Pin 11 pwm-ao-b-6 - Pin 12 pwm-ao-b-9 - Pin 13 pwm-e - Pin 32 pwm-…

[dalbert]

I ran the same test with the custom heat sink and as @bigjohnson said, it reaches 70C (and throttles) almost instantly. I’d like to monitor my board to see how often it is throttling; is there a way to monitor the CPU frequency (using raspberry pi os aka raspbian)?

[librecomputer]

The test is suppose to reach 70C without a fan very quickly. This is normal. It’s a theoretical FPU burn test.

For CPU frequency, you can use cpupower or use Google. Our boards use standard Linux interfaces and utilities.

[bigjohnson]

Fwiw I am monitoring the board temp via home assistant.

[dalbert]

@librecomputer thanks for the quick reply! You’ve solved half of my problem (how to check the CPU frequency). Is there a way to check whether there is thermal throttling?

For anyone following this thread and having trouble finding it, the cpupower utility is not part of the stock raspbian installation; you must install the package linux-cpupower: sudo apt install linux-cpupower. Then you can run cpupower frequency-info to see the current CPU frequency.
All cores run at the same frequency.

[librecomputer]

Just check the temps in /sys/class/thermal. Per https://boot.libre.computer/ci/roc-rk3328-cc.dts, trip point 1 is set to 0x11170 (70C) for reliability although the SoC can handle higher.