arm_hardfault: PANIC!!

[eden-desta]

So I was able to build and such. Now I am on the nsh terminal and get this massive stack dump

nsh> microros
arm_hardfault: PANIC!!! Hard fault: 40000000
up_assert: Assertion failed at file:armv7-m/arm_hardfault.c line: 135
up_registerdump: R0: 00000010 38002810 00000010 00000000 24002798 00000010 0000000a 38002878
up_registerdump: R8: 00000000 00000000 00000000 00000000 0000000c 38002800 0801faf3 0801fb0a
up_registerdump: xPSR: 21000000 PRIMASK: 00000000 CONTROL: 00000000
up_registerdump: EXC_RETURN: ffffffe9
up_dumpstate: sp:         38002678
up_dumpstate: stack base: 38001450
up_dumpstate: stack size: 00001758
up_stackdump: 38002660: 38002ba8 38002678 38002660 38002660 38002678 3800272c 00000010 0000000a
up_stackdump: 38002680: 38002688 08008639 00001758 38001450 38002678 38001040 380026a8 08008705
up_stackdump: 380026a0: 00000087 00000000 00000087 080327b4 380026b8 080061db 00000087 080327b4
up_stackdump: 380026c0: 380026c8 08001efd deadbeef 00000000 3800272c 00000003 00000001 a9040000
up_stackdump: 380026e0: 0801fb08 3800272c 380026f0 08002621 3800272c 00000003 38002700 00000003
up_stackdump: 38002700: 00000000 08001e99 38002710 08001e77 3800272c 00000003 deadbeef 00000000
up_stackdump: 38002720: 38002878 08001759 deadbeef 38002800 00000000 24002798 00000010 0000000a
up_stackdump: 38002740: 38002878 00000000 00000000 00000000 00000000 ffffffe9 00000000 00000000
up_stackdump: 38002760: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 38002780: 00000000 00000000 00000000 00000000 00000000 00000000 00000010 38002810
up_stackdump: 380027a0: 00000010 00000000 0000000c 0801faf3 0801fb0a 21000000 7951843f 3fabda49
up_stackdump: 380027c0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 380027e0: 7951843f 3fabda49 ffc00000 41dfffff 7919cfac 06f6925e 00000010 0801faed
up_stackdump: 38002800: 00000008 24003e90 38002cc8 00000000 00000000 00000000 00040103 feff0200
up_stackdump: 38002820: 38002810 38002820 38002820 00000000 00000010 38000101 00000000 00000000
up_stackdump: 38002840: 38002848 08007919 00000000 38002cd0 38002858 080193fb 00000000 38002cd0
up_stackdump: 38002860: 38002868 380028cc 38002c60 00000000 38002878 0801ae71 08019403 38002c78
up_stackdump: 38002880: 00000000 00000000 00000000 00000000 380028a0 080265dd 00000005 38002d30
up_stackdump: 380028a0: 00000000 00000000 38002d30 240024f8 00000000 38002ad8 24002450 080193d1
up_stackdump: 380028c0: 080193eb 08019403 08019421 00000000 00000000 00000000 ffffffff 38002d30
up_stackdump: 380028e0: 00000000 38002ae0 38002c78 38002ce8 38002c50 00000000 380029a4 38002c60
up_stackdump: 38002900: 38002918 00000000 00000000 08026d39 00000005 3800299c 00000000 00000000
up_stackdump: 38002920: 00000000 08032663 00000000 00000000 00000000 ffffffff 38002800 00000000
up_stackdump: 38002940: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
up_stackdump: 38002960: 00000000 00000000 38002ad8 38002ad0 00000000 00000000 00000001 00000000
up_stackdump: 38002980: 00000000 240009c8 00000000 00000005 00000000 080193d1 080193eb 08019403
up_stackdump: 380029a0: 08019421 00000000 380029b0 00000001 38002cf8 38002d00 38002cf4 38002ae0
up_stackdump: 380029c0: 00000001 00000000 00000000 00000000 deadbeef 380029f8 00000000 00000001
up_stackdump: 380029e0: 38002ad8 00000000 380029f8 00000000 00000000 08019085 00000000 00000000
up_stackdump: 38002a00: 00000000 00000000 38002acc 00000000 00000000 38002ad0 08035dc0 00000000
up_stackdump: 38002a20: 38002bc8 38002acc 38002a38 08015d39 38002b18 00000000 38001438 00000001
up_stackdump: 38002a40: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
up_stackdump: 38002a60: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
up_stackdump: 38002a80: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
up_stackdump: 38002aa0: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
up_stackdump: 38002ac0: deadbeef deadbeef deadbeef 38002bb0 38002bb0 080105df 38002c50 00000000
up_stackdump: 38002ae0: 00000000 00000000 38003070 00000001 38003070 38002c50 00000001 38003070
up_stackdump: 38002b00: 38002b53 00000004 38002bc8 38002c50 00000001 08011421 080193d1 080193eb
up_stackdump: 38002b20: 08019403 08019421 00000000 0801ec5b 380030f0 0801ec33 38002f80 0800faf5
up_stackdump: 38002b40: 00000000 00000000 00000000 00000000 00000000 38002bc8 00000000 08033d2c
up_stackdump: 38002b60: 00000000 00000000 38002b78 00000000 00000000 08006605 00000000 38001438
up_stackdump: 38002b80: 00000001 08015c75 38002b90 08002e77 00000000 38001040 00000001 00000001
up_stackdump: 38002ba0: 00000000 00000000 00000060 80001790 080193d1 080193eb 08019403 08019421

I have not started to run the agent though I anticipated being able to start, exiting out the port and running the docker command.

sudo docker run -it --rm -v /dev:/dev --privileged --net=host microros/micro-ros-agent:galactic serial --dev /dev/ttyACM0 -v6

[pablogs9]

Is possible to check in which line is this happening?

[eden-desta]

It looks like it is failing:
RCCHECK(rclc_support_init_with_options(&support, 0, NULL, &init_options, &allocator));
At this point in the main.
I ran it in chunks Essentially ensuring each line didnt cause the stack dump. This is the location of failure.

[pablogs9]

Which board are you using?
Does micro-ROS outputs something in the used serial port?

[pablogs9]

Are you doing this configuration for your board?

kconfig-tweak --enable CONFIG_MICROROSLIB
kconfig-tweak --enable CONFIG_SERIAL_TERMIOS
kconfig-tweak --enable CONFIG_STM32L4_UART4
kconfig-tweak --enable CONFIG_STM32L4_UART4_SERIALDRIVER
kconfig-tweak --enable CONFIG_UART4_SERIALDRIVER
kconfig-tweak --set-val CONFIG_UART4_RXBUFSIZE 256
kconfig-tweak --set-val CONFIG_UART4_TXBUFSIZE 256
kconfig-tweak --set-val CONFIG_UART4_BAUD 115200
kconfig-tweak --set-val CONFIG_UART4_BITS 8
kconfig-tweak --set-val CONFIG_UART4_PARITY 0
kconfig-tweak --set-val CONFIG_UART4_2STOP 0

[pablogs9]

Have you check that you are using the correct serial port here?

[eden-desta]

so i do have all those configurations @pablogs9 the only difference is i replaced L4 with H7 and replaced UART4 with USART3.

And what should I be replacing the /dev/ttyS1 with? My STM board is currently registered as dev/ttyACM0

[eden-desta]

ls /dev produced ttyS0 so I replaced it with that and it looks like it is getting past the initial fail point. Rebuilding and rerunning to make sure everything is good!

WIll run the agent as well sudo docker run -it --rm -v /dev:/dev --privileged --net=host microros/micro-ros-agent:galactic serial --dev /dev/ttyACM0 -v6

And see if any errors occur! Thank you @pablogs9

[eden-desta]

@pablogs9 Is it a problem that I am flashing with usart3 and also communicating over it as well for the agent?

Because this output that you see

NuttShell (NSH) NuttX-10.1.0-RC1
nsh> microros /dev/ttyS1
micro-ROS transport: connected using serial mode, dev: '/dev/ttyS1'
INFO: rcl_wait timeout 0 ms
Sent: 0
Sent: 1
Sent: 2

Does not show up on my end. Instead this does:

nsh> microros
micro-ROS transport: connected using serial mode, dev: '/dev/ttyS0'
���~�XRCEvF                                                        ~�XRCEvF

Which I have seen before when trying to run Zephyr or FreeRTOS while the board was trying to communicate to the agent.
It would be fine if when running the agent there was more of an output than:

eden@eden-stealth:~$ sudo docker run -it --rm -v /dev:/dev --privileged --net=host microros/micro-ros-agent:galactic serial --dev /dev/ttyACM0 -v6
[1626880465.989759] info     | TermiosAgentLinux.cpp | init                     | running...             | fd: 3
[1626880465.989964] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 6

or when i do ros2 topic list it only returns:

/parameter_events
/rosout

PS: if you deem this a separate issue i can definitely make a new ticket instead

[eden-desta]

So it definitely suggests there is still a problem at RCCHECK(rclc_support_init_with_options(&support, 0, NULL, &init_options, &allocator)); because through the messages i can sort of see that is outputed, it says problem at line 47, which happens to be that function.

Changed the serial channel just to see the full message Failed status on line 47: 1. Aborting. which is coming from the RCCHECK function defined on line 14.

nsh> microros
micro-ROS transport: connected using serial mode, dev: '/dev/ttyS1'
Failed status on line 47: 1. Aborting.

[eden-desta]

Additionally, upon changing the serial channel, I used a serial to usb adapter in order to pull that into my agent PC and run the agent from that port.
Commanded the agent using sudo docker run -it --rm -v /dev:/dev --privileged --net=host microros/micro-ros-agent:galactic serial --dev /dev/ttyUSB0 -v6 and still only receive what I mentioned above:

[1626894085.415025] info     | TermiosAgentLinux.cpp | init                     | running...             | fd: 3
[1626894085.415299] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 6

Additionally, for more information, I did reconfigure USART6 similar to what I had done for USART3 with the kconfig-tweaks as you have recommended above

[pablogs9]

Your micro-ROS client is trying to connect to the micro-ROS Agent with the ���~�XRCEvF messages... Is it possible for you to connect this port to the micro-ROS Agent? It should work, it seems that you have the same port for console and for micro-ROS.

[eden-desta]

Hi @pablogs9 so I did try to connect the uROS Agent to that port but I did not get the expected behaviour

eden@eden-stealth:~$ sudo docker run -it --rm -v /dev:/dev --privileged --net=host microros/micro-ros-agent:galactic serial --dev /dev/ttyACM0 -v6
[1626880465.989759] info     | TermiosAgentLinux.cpp | init                     | running...             | fd: 3
[1626880465.989964] info     | Root.cpp           | set_verbose_level        | logger setup           | verbose_level: 6

It gets hung up at this point and does not move forward. I also attempted using it through micro_ros_setup instead of the docker images and it had the same behaviour. My attemptto use another port was to see if the board was getting confused with the agent trying to communicate with it whilst also having the nsh terminal up. The behaviour was repeated in both instances.

Is there potentially a board dependency that is associated with the agent? I dont remember there being one but I could be mistaken. Like potentially having to specify somewhere which terminal exactly we are using not via the command line but through like on of the config files or something.

[pablogs9]

In general, your board should have one serial port for NSH and another for micro-ROS. Does your board have two serial ports?

Once you have configured two serial ports, make sure that you have access to NSH in one of them and run microros [the other serial port] and check if you have binary traces (���~�XRCEvF) on the other serial port.

Are you in this point?

[eden-desta]

Yes, so I did do this. Where I used USART3 as my serial port of nsh and I used USART6 for my serial port for uROS. I used a USB to serial adapter to pull out the pins out of my board and connect them directly to my PC running the agent. The behaviour I mentioned above with the agent not passing past the first two lines occur. With this message being sent Failed status on line 47: 1. Aborting.1 and line 47 for me is RCCHECK(rclc_support_init_with_options(&support, 0, NULL, &init_options, &allocator));

[pablogs9]

Ok, if you open your uROS port using minicom or miniterm instead of the agent are you seeing anything being wrote?