Multi-monitor Buildroot x64 target

I am working on an application SW that displays to multiple monitors (somewhere between 2 to 4).  Eventually, I want to drive the multi-monitor display from an SoC, but working out the SW architecture with a multi-monitor GPU plugged into a PCIe slot of a modern PC is an excellent way to understand and derisk problems.  Before diving into a multi-monitor GPU, I can experiment with software supported multi-monitor setup in virtualbox.

NFS booting the virtual multi-monitor x64 target from a virtual Ubuntu server

In a previous blog entry, I PXE-booted a Buildroot x64 target (Dell Optiplex 755) from an Ubuntu server.  If I run both the target and the server as Virtualbox guests, I can use the Virtualbox internal network, which is completely a software network stack.  This is convenient when studying the kernel and software on a laptop, while away from the desktop server.

Setup virtualbox target

Created a virtualbox x64 guest with these settings:

• General, Basic
• Name: Target
• Type: Linux
• Version: Other Linux (64 bit)
• System
• Base memory: 512 MB
• Boot order: Network ONLY
• Display
• Video memory: 64 MB
• Monitor count: 4
• Enable 3D acceleration
• No storage, no audio, no serial port, no USB, no shared folder
• Network:
• 1st Adapter: NAT
• 2nd adapter: internal network "intnet", which should MATCH the name of the 2nd adapter's internal network for the server.

Cross-compiling the target on the virtual Ubuntu server

WARNING: Buildroot must be extracted and then built on a filesystem that supports softlinks (so don't do this on an NTFS!).

After getting the latest stable Buildroot as shown in a previous blog entry, I configure Buildroot with the following options.

• Target options: x86_64, corei2 (core7 selects SSE4 features, which Virtualbox does NOT support yet)
• Build options:
• enable ccache, but change the cache location to within the Buildroot directory (to avoid saving to the virtual HDD, and keep all work in the Vbox shared folder): $(TOPDIR)/.buildroot-ccache
• Optimization level 3
• Toolchain
• glibc
• Enable C++ support, necessary for Qt5
• Build cross gdb for the host: does NOT build on the vbox server for some reason.  Besides, the native gdb should just work
• Register toolchain within Eclipse Buildroot plug-in
• System configuration
• /dev management: eudev
• Kernel
• Using a custom--as supposed to in-tree--(def)config file: I need to add a couple of options for the NFS rootfs. I changed x86_64_defconfig from the kernel.org to create /home/henry/x64/BR2/kernel.config
• Target packages
• Debugging
• gdb: only the gdbserver
• Graphics
• mesa3d
• Gallium swrast (software OpenGL) driver
• OpenGL EGL
• OpenGL ES
• Qt5
• Approve free license
• Compile and install examples
• gui module
• widgets
• OpenGL: OpenGL ES 2.0 and opengl module
• linuxfb support
• eglfs support
• Default platform: linuxfb
• GIF, JPEG, PNG support
• Hardware handling
• lshw (does NOT even build!)
• pciutils (lspci)
• Networking applications
• openssh: necessary to connect to the target from gdb on the server

Kernel config for NFS rootfs

The x86_64_defconfig alsready has a few options I needed, so I just added the following:

CONFIG_NETWORK_FILESYSTEMS=y

CONFIG_NFS_USE_KERNEL_DNS=y
CONFIG_E1000E=y

The stock x86_64_defconfig has CONFIG_E1000 but does NOT contain CONFIG_E1000E, so this is just a safety measure.

Build and deploy the binaries

After Buildroot make runs at the top level, its output/images will contain bzImage.  Copy this to the tftp download on the Virtualbox host.  On Windows, this is in C:\Users\<your uid>\.VirtualBox\TFTP folder, as shown here:

The rootfs images need to be extracted to the NFS export on the server:

~/o755/buildroot$ sudo tar -C /export/root/o755/ -xf ~/o755/buildroot/output/images/rootfs.tar

Setup the Virtualbox NFS server

Create a virtualbox x64 guest, with at least 2 CPUs, as much memory as possible, and the network with 2 adapters:

• 1st adapter: NAT: for general Internet access (I am writing this blog on the host)
• 2nd adapter: internal network: to communicate with the target.  Assign a static IP address 192.168.2.1 in Ubuntu Unity network settings, so that the target can refer to the NFS server with a static IP address (see below).

Virtualbox NAT network already has a PXE enabled DHCP server.  On Windows, the folder C:\Users\<your account>\.VirtualBox\TFTP plays the role of /var/lib/tftpboot (by default) for the Ubuntu hpa-tftp server.  So the following files should be put into that folder:

• bzImage: compressed Linux kernel built by Buildroot, in its output/images (see above)
• Target.pxe: I copied this file from Ubuntu desktop's /usr/lib/syslinux/pxelinux.0.  VirtualBox DHCP server has a rule for mapping the PXE image for each virtual box by its <name>.pxe.  Since my target's name is "Target", the PXE binary should be named Target.pxe.
• menu.c32: This is the PXE menu program, copied verbatim from Ubuntu desktop's /usr/lib/syslinux/ folder.
• pxelinux.cfg/ folder, which will contain the PXE menu entry
• default: the catch-all menu.  PXE has lots of rules for matching the menu by the target's IP address, netmask, etc.  But default is the final fallback.  This file should point to the NFS rootfs the virtual server will host (see below).  My "default" file therefore looks like this:

DEFAULT menu.c32

PROMPT 0

MENU TITLE PXE Boot Menu

TIMEOUT 50 #This means 5 seconds

LABEL buildroot

 MENU LABEL buildroot kernel

 kernel o755Image

 append ip=192.168.2.3:192.168.2.1:192.168.2.1:255.255.255.0:o755:eth1:off root=/dev/nfs nfsroot=192.168.2.1:/export/root/o755 rw earlyprintk

The "ip" (used to be called nfsaddr) syntax is ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:<hostname>:<device>:<autoconf>

Setup the NFS server

The NFS server should get a static IP address.  I prefer to use the Ubuntu Unity tool, as you can see below.

Install the NFS server:

$ sudo apt-get install nfs-kernel-server

I configured the NFS server in /etc/exports, to serve any target in the "intnet":

/export         192.168.2.0/24(rw,fsid=0,insecure,no_subtree_check,async)

/export/root    192.168.2.0/24(rw,no_root_squash,no_subtree_check)

Remember to restart the NFS server after saving this file.  The root file system Buildroot generated should be expanded to the /export/root folder just mentioned, like this:

$ sudo mkdir /export/root/o755

$ sudo tar -C /export/root/o755/ -xf ~/o755/buildroot/output/images/rootfs.tar

nVidia Quadro multi-monitor video card

I bought an nVidia Quadro NVS 420 (Dell PN K722J)--I wanted an nVidia card because IMHO nVidia has the best Linux driver support--from eBay.  As you can see below, there was even a driver update earlier this year.

Now to the fine prints, from the driver README file: X is required, and glibc >= 2.0 is required.  The X server requirement is a deal-breaker for me: I want to keep the embedded distribution small.  Maybe a better alternative is to pick up an open source driver from the noveau project.  Quadro NVS 420 is supported under the NV50 Tesla family, code name NV98 (G98), as you can see on this page.

Buildroot config for Qt5, OpenCV, and nVidia Quadro 420 GPU

Since I am an embedded SW engineer, I treat even the PCs like targets (rather than desktops).  In a previous blog, I demonstrated an NFS booted Buildroot distribution for this Intel Core2 Duo Dell PC.  Except for updating Buildroot to the latest stable release (2015.02), I'll pick up from where I left off.

$ cd buildroot
$ git checkout 2015.02
$ git pull . 2015.02

The only difference in the Buildroot config between the virtual target and this real target is the Gallium nouveau driver (which supports all nVidia cards), under Target packages --> Graphics libraries and applications --> mesa3d.

To pick up the nouveau device driver, I added CONFIG_DRM_NOVEAU=y to the kernel defconfig file.

Also, I can connect over serial to the real target, by adding console=ttyS0,115200 to the kernel parameters in the pxelinux.cfg/default's "append" line.  During boot, the GPU is probed:

...
[    1.198462] nouveau  [  DEVICE][0000:03:00.0] BOOT0  : 0x298c00a2
[    1.204534] nouveau  [  DEVICE][0000:03:00.0] Chipset: G98 (NV98)
[    1.210605] nouveau  [  DEVICE][0000:03:00.0] Family : NV50
[    1.216169] nouveau  [   VBIOS][0000:03:00.0] checking PRAMIN for image...
[    1.233709] Console: switching to colour frame buffer device 160x64
[    1.245164] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[    1.251332] i915 0000:00:02.0: registered panic notifier
[    1.307274] nouveau  [   VBIOS][0000:03:00.0] ... appears to be valid
[    1.313692] nouveau  [   VBIOS][0000:03:00.0] using image from PRAMIN
[    1.320231] nouveau  [   VBIOS][0000:03:00.0] BIT signature found
[    1.326303] nouveau  [   VBIOS][0000:03:00.0] version 62.98.6f.00.07
[    1.352876] nouveau 0000:03:00.0: irq 27 for MSI/MSI-X
[    1.352886] nouveau  [     PMC][0000:03:00.0] MSI interrupts enabled
[    1.359245] nouveau  [     PFB][0000:03:00.0] RAM type: GDDR3
[    1.364969] nouveau  [     PFB][0000:03:00.0] RAM size: 256 MiB
[    1.370868] nouveau  [     PFB][0000:03:00.0]    ZCOMP: 960 tags
[    1.378477] nouveau  [    VOLT][0000:03:00.0] GPU voltage: 1110000uv
[    1.915015] tsc: Refined TSC clocksource calibration: 2992.481 MHz
[    2.024020] nouveau  [  PTHERM][0000:03:00.0] FAN control: PWM
[    2.029844] nouveau  [  PTHERM][0000:03:00.0] fan management: automatic
[    2.036484] nouveau  [  PTHERM][0000:03:00.0] internal sensor: yes
[    2.062678] nouveau  [     CLK][0000:03:00.0] 03: core 169 MHz shader 358 MHz memory 100 MHz
[    2.071087] nouveau  [     CLK][0000:03:00.0] 0f: core 550 MHz shader 1400 MHz memory 700 MHz
[    2.079645] nouveau  [     CLK][0000:03:00.0] --: core 550 MHz shader 1400 MHz memory 702 MHz
[    2.088296] [TTM] Zone  kernel: Available graphics memory: 1001774 kiB
[    2.094802] [TTM] Initializing pool allocator
[    2.099147] [TTM] Initializing DMA pool allocator
[    2.103841] nouveau  [     DRM] VRAM: 256 MiB
[    2.108181] nouveau  [     DRM] GART: 1048576 MiB
[    2.112869] nouveau  [     DRM] TMDS table version 2.0
[    2.117987] nouveau  [     DRM] DCB version 4.0
[    2.122500] nouveau  [     DRM] DCB outp 00: 02000386 0f220010
[    2.128312] nouveau  [     DRM] DCB outp 01: 02000302 00020010
[    2.134124] nouveau  [     DRM] DCB outp 02: 040113a6 0f220010
[    2.139935] nouveau  [     DRM] DCB outp 03: 04011312 00020010
[    2.145747] nouveau  [     DRM] DCB conn 00: 00005046
[    2.150791] nouveau  [     DRM] DCB conn 01: 00006146
[    2.182206] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[    2.188798] [drm] Driver supports precise vblank timestamp query.
[    2.247926] nouveau  [     DRM] MM: using M2MF for buffer copies

[    2.287545] nouveau 0000:03:00.0: No connectors reported connected with modes

[    2.294654] [drm] Cannot find any crtc or sizes - going 1024x768

[    2.302510] nouveau  [     DRM] allocated 1024x768 fb: 0x60000, bo ffff88007a126c00

[    2.310181] fbcon: nouveaufb (fb1) is primary device

[    2.310182] fbcon: Remapping primary device, fb1, to tty 1-63

[    2.453168] nouveau 0000:03:00.0: fb1: nouveaufb frame buffer device

[    2.459500] [drm] Initialized nouveau 1.2.1 20120801 for 0000:03:00.0 on minor 1

The GPU shows up as another framebuffer device /dev/fb1 (/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:00.0/0000:03:00.0/graphics/fb1).  The multiple folder level corresponds to the PCI switches built into the card (apparently), as can be seen from the lspci output:

01:00.0 PCI bridge: NVIDIA Corporation NF200 PCIe 2.0 switch for Quadro Plex S4 / Tesla S870 / Tesla S1070 / Tesla S2050 (rev a3)
02:00.0 PCI bridge: NVIDIA Corporation NF200 PCIe 2.0 switch for Quadro Plex S4 / Tesla S870 / Tesla S1070 / Tesla S2050 (rev a3)
02:02.0 PCI bridge: NVIDIA Corporation NF200 PCIe 2.0 switch for Quadro Plex S4 / Tesla S870 / Tesla S1070 / Tesla S2050 (rev a3)
03:00.0 VGA compatible controller: NVIDIA Corporation G98 [Quadro NVS 420] (rev a1)

I can see Qt GUI drawing to fb1 by running Qt examples like these:

/usr/lib/qt/examples/opengl/2dpainting/2dpainting -platform linuxfb:fb=/dev/fb1

Of course, OpenGL display doesn't work against the linuxfb platform.  But there seems to be a problem with the mesa3d nouveau platform I compiled, because gbm cycles through a few Gallium drivers EXCEPT the one I actually built: nouveau_dri, which is even in the search path (/usr/lib/dri).

# ./application -platform eglfs
gbm: failed to open any driver (search paths /usr/lib/dri)
gbm: Last dlopen error: /usr/lib/dri/i915_dri.so: cannot open shared object file: No such file or directory
...
Could not initialize egl display

I realized that eglfs platform is querying the framebuffer property from /dev/fb0 even though I set the environment variable QT_QPA_EGLFS_FB to /dev/fb1.  So I removed the Intel GPU from the picture by commenting out the Intel GPU drivers from the kernel config.  And now I see this message:

# ./openglwindow -platform eglfs
Unable to query physical screen size, defaulting to 100 dpi.
To override, set QT_QPA_EGLFS_PHYSICAL_WIDTH and QT_QPA_EGLFS_PHYSICAL_HEIGHT (in millimeters).
EGL Error : Could not create the egl surface: error = 0x300b
Aborted

The error (EGL_BAD_NATIVE_WINDOW) is logged in src/plugins/platforms/eglfs/qeglfswindow.cpp, QEglFSWindow::resetSurface(), but thrown in eglCreateWindowSurface:

    if (!rx::IsValidEGLNativeWindowType(win))
    {
        recordError(egl::Error(EGL_BAD_NATIVE_WINDOW));
        return EGL_NO_SURFACE;
    }

This begs the question: what is a Qt native window?  I decided that I still do NOT know the low level graphics SW stack, and just stick to either linuxfb or directfb.

directfb: the lowest level display abstraction in userspace

DirectFB (Direct Frame Buffer) is a software library with a small memory footprint that provides graphics acceleration, input device handling and abstraction layer, and integrated windowing system with support for translucent windows and multiple display layers on top of the Linux framebuffer without requiring any kernel modifications.  DirectFB allows applications to talk directly to video hardware through a direct API, speeding up and simplifying graphic operations.

But directfb does not support Quadro (noveau) drivers?  See directfb/gfxdrivers/nvidia/nvidia.c  Platform independent examples src is in buildroot/output/build/directfb-examples-1.6.0/srcles

Each GPU detected by DRM is referred as a DRM device, and a device file /dev/dri/cardX (where X is a sequential number) is created to interface with it, as in this example for the NVS420 card in a PC:

# ls /dev/dri
card0       controlD64  renderD128

Note that on Zedboard which lacks a GPU, Lars Clausen (of ADI)'s adv7511 DRM driver does not offer the renderD128 file:

# ls /dev/dri

card0       controlD64

User space programs that want to talk to the GPU must open the file and use ioctl calls to communicate with DRM. Different ioctls correspond to different functions of the DRM API.  A library called libdrm was created to facilitate the interface of user space programs with the DRM subsystem, as shown here:

This library is merely a wrapper that provides a function written in C for every ioctl of the DRM API, as well as constants, structures and other helper elements.  DRM consists of two parts: a generic "DRM core" and a specific one ("DRM Driver") for each type of supported hardware.  DRM driver, on the other hand, implements the hardware-dependent part of the API, specific to the type of GPU it supports; it should provide the implementation to the remainder ioctls not covered by DRM core, but it may also extend the API offering additional ioctls with extra functionality, for which extra userspace library is offered.  For the nVidia card, we therefore see:

# ls /usr/lib/libdrm*

/usr/lib/libdrm.so.2.4.0   /usr/lib/libdrm_nouveau.so.2.0.0

But strangely, Zedboard has enumerated a non-existent card (Adreno)?

# ls -Lh /usr/lib/libdrm*

/usr/lib/libdrm.so.2.4.0   /usr/lib/libdrm_freedreno.so.1.0.0

GEM (graphics executaion manager) manages graphics buffers.  Through GEM, a user space program can create, handle and destroy memory objects living in the GPU's video memory.  Confusingly, there are mesa3d userspace drivers that use the kernel drivers, as you can see in the AMD example below:

As the demand for better graphics increased, hardware manufacturers created a way to decrease the amount of CPU time required to fill the framebuffer. This is commonly called "graphics accelerating".  Common graphics drawing commands (many of them geometric) are sent to the graphics accelerator in their raw form. The accelerator then rasterizes the results of the command to the framebuffer.

Debugging a full screen directfb application

The df_fire example did NOT run against the nouveau driver, so let's debug into it.  Following relevant build variables are in the Makefile:

CFLAGS = -Wall -O3 -pipe -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -pipe -O3  -Werror-implicit-function-declaration

CPPFLAGS = -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64

DIRECTFB_CFLAGS = -D_REENTRANT -I/home/henry/work/o755/buildroot/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/include/directfb  

DIRECTFB_LIBS = -ldirectfb -lfusion -L/home/henry/work/o755/buildroot/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/lib -ldirect -lpthread  

AM_CFLAGS = -D_REENTRANT -I/home/henry/work/o755/buildroot/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/include/directfb   -D_GNU_SOURCE

LIBADDS = \

        -ldirectfb -lfusion -L/home/henry/work/o755/buildroot/output/host/usr/x86_64-buildroot-linux-gnu/sysroot/usr/lib -ldirect -lpthread  

AM_CPPFLAGS = \

        -DDATADIR=\"${datarootdir}/directfb-examples\" \

        -DFONT=\"$(fontsdatadir)/decker.ttf\"

Trying the Quadro NVS 420 nVidia card (model G98) on Ubuntu desktop

I could not get Ubuntu to see the DLP 3010 evaluation module, so I checked whether the nVidia proprietary drivers could The list of driver updates can be queried:

henry@o755:~$ sudo ubuntu-drivers devices

== /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:00.0/0000:03:00.0 ==

model    : G98 [Quadro NVS 420]

modalias : pci:v000010DEd000006F8sv000010DEsd0000057Ebc03sc00i00

vendor   : NVIDIA Corporation

driver   : nvidia-304-updates - distro non-free

driver   : xserver-xorg-video-nouveau - distro free builtin

driver   : nvidia-331 - distro non-free recommended

driver   : nvidia-331-updates - distro non-free

driver   : nvidia-304 - distro non-free

driver   : nvidia-173 - distro non-free

An easy way to install the proprietary device drivers is through System Settings (unity-control-center) --> System --> Software & Updates --> Additional Drivers.  Even after installing the nVidia proprietary driver, the 2nd monitor would not enumerate; I had to run the nVidia Xserver settings tool, and explicitly detect the display for the Xserver settings to update.  I think this means that nouveau driver cannot drive NVS420 card in multi-monitor mode.

Indeed, the computer only sees 1 framebuffer device:

$ ls -lhg /sys/class/graphics/
total 0
lrwxrwxrwx 1 root 0 Apr 25 12:06 fb0 -> ../../devices/pci0000:00/0000:00:02.0/graphics/fb0
lrwxrwxrwx 1 root 0 Apr 25 11:50 fbcon -> ../../devices/virtual/graphics/fbcon

As this nouveau multi-monitor setup explanation shows, it is the X server that lays out the multi-monitor on the desktop.  Multiple monitors has to be organized as 1 logical screen, because an X application can only display to 1 screen (i.e. application cannot choose to run on multiple screens, nor change from 1 screen to another dynamically).

AMP state machine applications on Zynq

In a previous blog entry, I demonstrated how to read/write OCM from a Linux userspace application.   And in another, I ran an LED blink bare metal application on Zynq CPU1, launched from Linux running on CPU0.  In this article, I take the next step and demonstrate a shared memory message queue in OCM.  This is an indispensable part of a practical AMP computing architecture, because a hard real-time software needs high level control (some UI) running on Linux (or Windows if Xilinx ever supports Windows CE).

Dining Homer demo

The DPP problem is the demo of choice when porting the QP framework.  I showed a working DPP demo when porting QP to the bare metal, and on Linux, integrated with Qt.  Here is the same DPP GUI containing only 1 active object (the table object) running on desktop-less, Buildroot Linux distribution I built, cooperating with 5 instances of the philosopher (Homer) active objects.

Making CPU1 hot-pluggable: 3-step startup/shutdown

In a previous blog, I started/stopped bare metal application from Linux shell command.  This makes the bare metal app on CPU1 hot pluggable for any userspace Linux application.   Conversely, a putative Linux userspace application may start and stop at any time, so the CPU1 application cannot count on its CPU0 counterpart being always there.  So a question naturally arises: how to remove the assumption of the other side still running?  Here are the techniques I used in the past:

• Master slave: communication is initiated by the master.  If the slave does not respond, master considers the slave dead.  The slave may also consider the master to be dead if it does not hear from the master within a watchdog timeout.  In practice, the slave runs mostly independently of the master, so whether the master is alive is usually irrelevant.
• Fault tolerant networking: there are many networking options available to a Linux application, while the choice is more limited for the bare metal application.  The key for a networked application is to be tolerant of the connection going down silently.

The OCM is essentially an error-free communication channel between the 2 CPUs.  A circular queue on top of OCM would yield a reliable single-direction communication channel.

Without using a mutex, there is no way to protect against the reader from reading garbage head index.  So one side has to initialize the queue first; I will require the bare metal to start and initialize the queue, BEFORE the Linux app starts.

Startup sequence

1. zynq_remoteproc module is loaded on Linux boot, but if it has been unloaded (see the shutdown sequence below), the user can load it as root, with a modprobe command. The zynq_remoteproc module I modified (originally TI/Xilinx code) does NOT kick-off the CPU1 application on module load; it is in a ready state.
2. The Linux startup script can then start the CPU1 application by writing "1" to the zynq_remoteproc module's "up" attribute.  The CPU1 application then initializes the inter-AMP message queue, and starts its state machines.  It CAN potentially start writing to the CPU1 --> CPU0 queue.
3. The same Linux startup script can then start a privileged (because it needs to perform mmap to the OCM memory) userspace application that OVERLAYS the message queue on the OCM, without initializing the queue.  It can write to the CPU0 --> CPU1 queue.

Shutdown sequence

1. Quit the userspace app, WITHOUT touching the inter-AMP message queue.  This is sufficient for the usual (and more frequent) case of developing the userspace application.
2. If the real-time application has to be restarted for some reason, writing "0" to the zynq_remoteproc module's "up" attribute will merely STOP (but it does not call the destructors) the CPU1 application.
3. Optionally, rmmod the zynq_remoteproc module, to release the CPU1 firmware code (ELF file), to write a new FW.

Learning about zero-copy event queue from the QP framework

QP framework offers lots of idea on how to implement a zero-copy event queue.  For the zero copy to work safely, memory pool for the events are created by the application, but registered to the framework for management.  When an event is needed, it can be loaned from the pool and then cast to QEvt, which is the parent of all QP events.

#define QF_EPOOL_GET_(p_, e_, m_) \
        ((e_) = static_cast<QEvt *>((p_).get((m_))))

When loaning an element from its pool, QMPool searches for the next free block within a critical section, which has to be effective across multiple CPUs that are part of the QP framework.  This event has to be returned back to the framework when the event is not used by any active objects.  ONE of the places this garbage collection takes place is right after the event has been handled by all possible receivers ("act" in the POSIX port below):

// loop until m_thread is cleared in QActive::stop()
do {
    QEvt const *e = act->get_(); // wait for event
    act->dispatch(e); // dispatch to the active object's state machine
    gc(e); // check if the event is garbage, and collect it if so
} while (act->m_thread != static_cast<uint8_t>(0));

An event has a reference count that must be protected by both the incrementers (those situations when it is USED) and decrementers (situations when it is no longer used).  In QP, there is no assumption about which thread may use an event; it may be chained (receive an event and send the event to some other place right away) for example.

An event queue just POINTS at these events (whether static or dynamic--and therefore garbage collected as shown above).  But the framework still needs an array of QEvt children POINTERS, like this example:

   static QP::QEvt const *tableQueueSto[N_PHILO];

Inside, an event queue is initialized to manage this raw pointer arrays.

    m_eQueue.init(qSto, qLen);

An event queue belongs to only 1 thread.  The owner of the event queue waits for an event to be inserted into the queue if the event queue is empty

    QACTIVE_EQUEUE_WAIT_(this); // wait for event to arrive directly

which on POSIX is just a condition variable wait

while ((me_)->m_eQueue.m_frontEvt == static_cast<QEvt const *>(0)) \

    pthread_cond_wait(&(me_)->m_osObject, &QF_pThreadMutex_)

Note that IF there is already a pending event (such as in QK_sched_ called in QK_irq interrupt handler), the owner does NOT wait, in order to suck up the events as quickly as possible.  Conversely, an event sender signals the queue owner when it inserts an event into an empty queue

if (m_eQueue.m_frontEvt == null_evt) {

    m_eQueue.m_frontEvt = e;      // deliver event directly

    QACTIVE_EQUEUE_SIGNAL_(this); // signal the event queue

}

Since there are usually multiple event senders (but 1 receiver), above code must be protected with a mutex.

Adaptation of the QP event queue to AMP

In a previous blog, I worked out how to wake up the bare metal code running on CPU1 from Linux userspace application on CPU0.  That code can be used in EQUEUE_SIGNAL pseudo code above.  I also know how to wake up a Linux kernel module, but I have not yet written a code to chain that to a userspace application.  At any rate, waking up a Linux userspace application with minimal latency is not all that important.  There are 2 bigger problems: protecting access to the queue, and protecting the reference count in an event.  In both cases, a mutex is necessary.  But there is no general purpose mutex that can work in both Linux and bare metal code.  I could copy Linux mutex code--which uses atomic_t under the hood, but that seems like a long row to hoe.  QP's event queue locks the queue on both insert and removal, so cannot be used across inter-AMP.  But if I constrain the AMP inter-process messaging problem, cross OS mutex becomes unnecessary: Bare-metal code runs on within RTC (run-to-completion) single-threaded framework (such as QK or QVanilla).  

A lockless queue is a special case of a circular queue for a single-writer and single-reader constraint; the writer only increments the tail, and the reader only increments the head.  If all writers are on the Linux side, a POSIX mutex can protect the tail.  I can use this queue to send events from Linux userspace app to a bare metal software ISR, which will then turn around and forward the events to an RTC application.  When the RTC framework handles a message, I know it is completely done, so I can garbage collect the event right there.  This allows an important simplification: an interrupt lock can sufficiently protect access to a bare metal side resource, compared to the required use of a full-blown mutex on the Linux side.  I drew up the Linux to the bare metal scenario and the opposite in 2 diagrams below.

I had to make 1 modification to QP (in qep.h), to put QEvt (its children, actually) in OCM.  Zero out the poolId_ and refCtr_ in the constructor.  This is necessary because the bare metal side uses placement new, instead of the more commonly used version that calls calloc:

        QEvt(QSignal const s) // poolId_/refCtr_ intentionally uninitialized

          : sig(s) {

        poolId_ = refCtr_ = 0;//good hygene for overlaying on memory

        }

Turn off cacheing in OCM

As discussed in a previous blog, the Linux userspace application accesses the OCM through the Xilinx proprietary device driver which exposes the 256 KB of OCM to /proc/iomem.  As shown in that blog entry, the userspace application can mmap the address 0xFFFC0000.  Man page on mmap says:

A mapping created using /dev/mem will be uncached if it's above the top of RAM.

So the Linux side already turns off cache on OCM.  But to ensure the same for the bare metal, I need to manually change the master MMU table (AKA L1 lookup table).  When porting QP to Zynq CPU1, I left the MMU table as default.  After debugging the inter-CPU OCM messaging for a couple of days, I discovered that it's because cacheing was turned on for the OCM range.  I was forced modify the translation_table.s:

//.word SECT + 0x4c0e /* S=b0 TEX=b100 AP=b11, Domain=b0, C=b1, B=b1 */

.word SECT + 0x4c02 /* S=b0 TEX=b100 AP=b11, Domain=b0, C=b0, B=b0 */

.set SECT, SECT+0x100000

The only change is turning off the C and B bits of the L1 policy.

Circular event queue between Linux and bare metal

Like the QP event queue, my first event queue implementation stored pointer (to the events) in the array.  But unlike the bare metal side, where the pointer is pointing to the physical memory, the Linux side points to VIRTUAL address.  Since the 2 sides use a different memory offset even for the SAME physical location, I wrote 2 different versions of the queue.  The bare metal side stores OFFSET from the beginning of the physical OCM in the event queue, while the Linux side uses the address returned from mmap.  Examine the bare metal side first:

Bare metal circular event queue

class AMP_CEQ {
    uint8_t m_head;
    volatile uint8_t m_tail;
    uint8_t m_mask, dummy;//to align to word
    //16 bit for [word] offsets are enough for 256 KB OCM
    volatile uint16_t m_offset_array[1<<8];//Keep everything real simple

public:
inline void push(QP::QEvt* e) {
uint16_t e_offset = (uint16_t)(((uint32_t)e - OCM_LOC) >> 2);
m_offset_array[m_tail++] = e_offset;
}
inline const QP::QEvt* pop() {
uint16_t e_offset = m_offset_array[m_head++];
const QP::QEvt* e = (const QP::QEvt*)
(((uint32_t)e_offset << 2) + OCM_LOC);
return e;
}
inline bool empty() { return m_tail == m_head; }
inline void init() {
m_tail = m_head = 0;
m_mask = 0xFF;
}
private://don't allow ctor
    AMP_CEQ() {}
};

Since QK is a single-task scheduling system, I initially thought that I can let the state machines send messages directly to Linux without any concurrency protection, like this:

            BSP_send2Linux(&BSP_staticHungryEvts[PHILO_ID(me)]);

which just shoves the event to the m2lQ; the function is helpful in hiding the circular queue from the rest of the SW--they just deal with a BSP service to push events to Linux.

void BSP_send2Linux(QP::QEvt* evt) {

QF_INT_DISABLE();

m2lQ->push(evt);//push atomically into L1 cache (NOT to the memory yet)

QF_INT_ENABLE();

}

But when I thought more, I realized that QK's PREEMPTIVE nature breaks this assumption that would otherwise hold in the QVanilla (no premption; run to completion) scheduling.  So the quick interrupt locking and unlocking was required for concurrency protection.

You might ask: why don't I raise a SW interrupt to the Linux, so that it can start servicing the message as soon as possible?  Please remember that the whole point of the AMP architecture is that the non-real-time Linux side and the hard-real-time bare metal side are now loosely coupled.  There is no real-time guarantees about when the messages will be communicated.  So it's perfectly fine for the Linux Qt GUI to check for pending messages every system tick (as slow as 2 Hz, but can easily run at 100 Hz or faster):

void QP::QF_onClockTick(void) {

    QP::QF::TICK_X(0U, &l_time_tick);

    while(!m2lQ->empty()) {

        //In this application, there is only 1 active object: DPP:AO_Table

        const DPP::TableEvt* e = static_cast<const DPP::TableEvt*>

                        (m2lQ->pop());

        DPP::AO_Table->POST(e, &l_dummyOnClockTick);

    }

}

In this simple AMP DPP application, table is the only active object, so I can post directly to it.  But in general, I will have to PUBLISH the received event to the QP framework.

Linux side circular event queue

This code is the same as above, except for using the globally stored l_ocm pointer, returned from mmap() during BSP_init():

static char* l_ocm = NULL;//(virtual) IO memory mapped OCM section

static int l_memf = -1;//and the Linux /dev/mem file backing it up

class AMP_CEQ {

    uint8_t m_head;

    volatile uint8_t m_tail;

    uint8_t m_mask, dummy;//to align to word

    //16 bit for [word] offsets are enough for 256 KB OCM

    volatile uint16_t m_offset_array[1<<8];//Keep everything real simple

public:

    inline void push(const QP::QEvt* e) {

        uint16_t e_offset = (uint16_t)(((uint32_t)e - (uint32_t)l_ocm) >> 2);

        m_offset_array[m_tail++] = e_offset;

        //m_tail &= m_mask;

    }

    inline const QP::QEvt* pop() {

        uint16_t e_offset = m_offset_array[m_head++];

        const QP::QEvt* e = (const QP::QEvt*)

                (((uint32_t)e_offset << 2) + (uint32_t)l_ocm);

        //m_head &= m_mask;

        return e;

    }

    inline bool empty() { return m_tail == m_head; }

private:

    AMP_CEQ() {}

};

static AMP_CEQ* l2mQ = NULL;

static AMP_CEQ* m2lQ = NULL;

Note that the 2 queues are owned by the bare metal side, so the Linux side just gets the pointer to the agreed-upon location in BSP_init():

    //mmap OCM.  man mem says: "Byte addresses in /dev/mem are

    //interpreted as physical memory addresses."

    l_memf = open("/dev/mem"

            , O_RDWR /*| O_SYNC*/); //do I want cacheing?

    Q_ASSERT(l_memf > 0);

    //A mapping created using /dev/mem will be uncached if it's above

    //the top of RAM.  Also, Zynq OCM driver mapped this memory as non-cached

#define OCM_LOC 0xFFFC0000

#define OCM_SIZE (4*64*1024)

    l_ocm = (char*)mmap(NULL,//Tried specifying OCM_LOC, no luck

                        OCM_SIZE, PROT_READ | PROT_WRITE,

                        MAP_SHARED /*| MAP_LOCKED*/,

                        l_memf, OCM_LOC);//0xe0080000);//

    char* pocm = l_ocm;

    l2mQ = (AMP_CEQ*)pocm; pocm += sizeof(AMP_CEQ);

    m2lQ = (AMP_CEQ*)pocm; pocm += sizeof(AMP_CEQ);

On the Linux side, there are MT protection is required, so I use the QP's mutex:

void BSP_send2Metal(const QP::QEvt* evt) {

    //Multiple AO can send to Linux, so the write must be serialized

    QF_CRIT_ENTRY(dummy);

    l2mQ->push(evt);//push atomically

    QF_CRIT_EXIT(dummy);

    //Explicit DMB not required because mutex unlock should already do that

    //asm("DMB": : : "memory");//flush L1 for CPU0

}

Like the Linux reader, the bare metal BSP pops the messages from its timer tick handler.

...

QP::QF::TICK(&l_ISR_tick);

//This interrupt will NOT nest, so NO need to lock interrupt

while(!l2mQ->empty()) {

const DPP::TableEvt* e = static_cast<const DPP::TableEvt*> (l2mQ->pop());

AO_Philo[e->philoNum]->POST(e, &l_ISR_sw2);

}

And like this Linux side, this POST() will have to change to PUBLISH() soon.

Finding static events preallocated in OCM

Let's do the easy case first: static event.  A static event lives for the duration of the application--without changing its content (events are read-only to the event receivers).  Such event can be overlaid on OCM, along with the event queues, like this:

#define BARE_METAL
#define OCM_SIZE (4*64*1024) //256KB too big?
#define OCM_LOC 0xFFFC0000


//Don't try be too fancy in the global initializers; just pin the pointers to
//the right place; place new will be called later on the instances in BSP_init
#define L2M_Q_LOC OCM_LOC //(M2L_Q_ARRAY_LOC + sizeof(QP::QEvt*) * (1<<M2L_Q_SIZE_EXP))
#define M2L_Q_LOC (L2M_Q_LOC      + sizeof(AMP_CEQ))
#define EAT_EVT_LOC (M2L_Q_LOC      + sizeof(AMP_CEQ))
#define HUNGRY_EVT_LOC (EAT_EVT_LOC    + sizeof(TableEvt) * N_PHILO)
#define DONE_EVT_LOC (HUNGRY_EVT_LOC + sizeof(TableEvt) * N_PHILO)

static AMP_CEQ *const l2mQ = (AMP_CEQ*)L2M_Q_LOC;
static AMP_CEQ *const m2lQ = (AMP_CEQ*)M2L_Q_LOC;

DPP::TableEvt* const BSP_staticEatEvts    = (DPP::TableEvt*)EAT_EVT_LOC;

DPP::TableEvt* const BSP_staticHungryEvts = (DPP::TableEvt*)HUNGRY_EVT_LOC;

DPP::TableEvt* const BSP_staticDoneEvts   = (DPP::TableEvt*)DONE_EVT_LOC;

Note that this just establishes the pointers.  The queue initialization and the static event placement new ctors are called in BSP_init():

    l2mQ->init();//new(l2mQ) AMP_CEQ();

    m2lQ->init();//new(m2lQ) AMP_CEQ();

    for(i=0; i < 128; ++i) {//For sanity test, shove fake addresses

    l2mQ->push((QP::QEvt*)(OCM_LOC + 4*i));

    m2lQ->push((QP::QEvt*)(OCM_LOC + 4*i));

    }

    for(i=0; i < N_PHILO; ++i) { //call the constructors for each event

     new(&BSP_staticEatEvts[i])    TableEvt(EAT_SIG, i);

     new(&BSP_staticHungryEvts[i]) TableEvt(HUNGRY_SIG, i);

     new(&BSP_staticDoneEvts[i])   TableEvt(DONE_SIG, i);

    }

Note the placement new semantics, to avoid calling malloc().

Memory pool in OCM

Since AMP events will NOT be contained within 1 QP system, I cannot blindly entrust some OCM block to the QP's memory pool--not unless I come up with an AMP mutex.  An alternative is to make the memory pool circular, just like the event queue.  As long as the event is not (re)used for longer than it takes for the memory pool free pointer to wrap around, the event can be used as if it is a static event.  In fact, to ensure that QP framework does NOT try to reclaim the memory on its own, the AMP event will be declared as static to QP (by leaving the poolId to 0, just like the static event).

Without the QP framework managing the reference count, I cannot think of a good way to indicate that an event is completely done being used.  But let me come back to this later.

Bare metal side memory pool

If I enforce that the memory to be loaned is word aligned, the circular memory pool is quite simple on the bare metal side, because can hard code where to place the memory pool:

//The pool should be word aligned
static uint32_t* BSP_m2lPool = (uint32_t*)(OCM_LOC + 1*64*1024);
static uint32_t* BSP_l2mPool = (uint32_t*)(OCM_LOC + 2*64*1024);
uint32_t* BSP_loanMemory(uint8_t wordSize) {
static uint32_t* p = (uint32_t*)(OCM_LOC + 64*1024);
QF_INT_DISABLE();//vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
uint32_t* next = p + wordSize;
if(next >= BSP_l2mPool) {//wrap
p = BSP_m2lPool;
next = p + wordSize;
}
uint32_t* e = p;
p = next;//move the pointer
QF_INT_ENABLE();//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
return e;
}

A convenience macro can then just call the placement new constructor on any type that has a constructor.  Note that I divide the size of any type by 4 to get the number of words for a type:

#define BSP_loanEvent(evtT_, sig_, ...) (new(BSP_loanMemory(sizeof(evtT_)>>2)) \

evtT_((sig_), ##__VA_ARGS__))

An active object can then loan an event from the event pool and just toss the event to the Linux side, like this example:

            TableEvt *pe = BSP_loanEvent(TableEvt, HUNGRY_SIG, PHILO_ID(me));

            BSP_send2Linux(pe);//&BSP_staticHungryEvts[PHILO_ID(me)]);

Linux side memory pool

The Linux side closely mirrors the bare metal side, except for once again not knowing priori, the base address of the OCM in virtual memory.  So the global memory are initially NULL, but then given proper address in BSP_init().  Once initialized properly, the load function works entirely within a critical section:

static uint32_t* BSP_l2mPool = NULL, *BSP_l2mPoolEnd = NULL, *BSP_l2mPoolPtr = NULL;

uint32_t* BSP_loanMemory(uint8_t wordSize) {

    QF_CRIT_ENTRY(dummy);//vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv

    uint32_t* next = BSP_l2mPoolPtr + wordSize;

    if(next >= BSP_l2mPoolEnd) {//wrap

        BSP_l2mPoolPtr = BSP_l2mPool;

        next = BSP_l2mPoolPtr + wordSize;

    }

    uint32_t* e = BSP_l2mPoolPtr;

    //qDebug("BSP_l2mPoolPtr = %p -> %p", BSP_l2mPoolPtr, next);

    BSP_l2mPoolPtr = next;//move the pointer

    QF_CRIT_EXIT(dummy);//^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    return e;

}

The helper macro and its usage is the same as the bare metal side.

State machine based Qt5 GUI on Zedboard

In a previous blog entry, I explored creating a minimal embedded Linux distribution containing the Qt5 framework, and writing and debugging a "Hello world" Qt GUI application.  Whenever possible, I write all my SW within an event-driven, hierarchical state machine framework called QP.  But since Qt is also an event-driven framwork in its own right, meshing the 2 together is not straight-forward.  When creating a WPF MVVM (model-view-view model) GUI application with state machines, I could update the WPF view model from a special active object (I called it the GuiStateMachine) in response to any update events (of interest to the GUI) from ALL other active objects.  Apparently, you cannot do that in Qt, because in the official Qt-QP integration example, the singleton GUI state machine runs in Qt context.  So unlike in my WPF-QP integration, the events delivered to the GUI state machine (active object, really) are transformed into a Qt event and shoved into the Qt's event delivery mechanism.  The Qt-QP reference application is available for mingw, but I cross-compile for the Zynq (ARM Cortex A9), so I am going to modify the reference application for my situation.

Create the DPP Qt Widgets project

The reference application creates the QP Qt library first.  But on my system, one Qt GUI is the only application (I am an embedded SW engineer, not a desktop SW engineer), so I will not bother with a separate library, and just put all code in 1 Qt widgets application, in the qpcpp/example/qt/arm/buildroot folder.

~/work/Dorking/QP/qpcpp/examples/qt$ mkdir -p arm/buildroot

Then in Qt Creator (the previous blog entry discussed how to get and install the Qt Creator FROM qt.io rather than as a Debian package)

1. Click "New Project" button, and then choose the "Qt Widgets Application" template.
2. Following the reference application example, I create a project called "dpp-gui" in the /mnt/work/Dorking/QP/qpcpp/examples/qt/arm/buildroot folder just created.
3. Next, I choose the zedbr2 kit I created in the  previous blog entry.
4. In a departure from the example, I create my GUI as a QMainWindow (vs. QDialog).  Also unlike the example, I WILL use the form.  But I will still call the main class "Gui", to follow the example.

Qt Creator can ready build this empty main class, which is always a good first step.

Preprocessor include path and defines in qmake project file

At minimum, the project must include the QP include/, qep/source/, qf/source/, and  the QP port folders.  Unlike other IDEs, the build variables like include paths are NOT a project property; I write these are directly into the project (.pro) file in a text editor, using a qmake variable, like this:

QP_ROOT = ../../../../..

INCLUDEPATH += $$QP_ROOT/include $$QP_ROOT/qep/source \$$QP_ROOT/qep/source \ $$QP_ROOT/qf/source \$$QP_ROOT/qf/source \ $$QP_ROOT/ports/qt

Qt itself has a state machine infrastructure, which is redundant for a QP state machine application, so I turn off the Qt's state machine feature in the qmake .pro file:

DEFINES += QT_NO_STATEMACHINE

Add sources to the project and tailor to my needs

QP platform independent sources

In Qt Creator, right click on Sources --> Add Existing Directory --> Browse to the qpcpp/qep/source/ folder --> Start Parsing, to expand the folder and unselect the unnecessary files, as shown below (I do not use FSM, only HSM):

I later learned that you can also include the header files, and Qt Creator will correctly pull them into the HEADERS variable, so qep_pkg.h should have been checked in the above screenshot.

I add qpcpp/qf/source folder similarly, without leaving out any files this time.

Note on updating to the QP 5 API

When copying examples written for QP API 4.5 or earlier, the following changes are required:

• Delete the deprecated call to QS_RESET()
• QTimeEvt ctor now takes the owning active object as the 1st argument.  In C++, that would show up as the "this" pointer if the timer belongs to an active object.  In exchange, the armX method of the QTimerEvt--which should be used instead of postIn() method--now does NOT take an active object.
• Q_NEW now takes ctor arguments, to call the PLACEMENT new operator (i.e. unlike the new does NOT hit the heap) of the type being created.  While this is great for a single process usage of the memory pool, the virtual table you get with the new operator is dangerous when the memory pool spans multiple processes (through shared memory)--as will be the case for me.  The danger lies in the possibility for different compiler versions laying out the virtual table differently (C++ compilers are notorious for this, even among different versions).  I decide to play it safe here, turn off QEvent's CTOR and VIRTUAL features in qep_port.h, as shown below (and pay the price of having to initialize the memory pool objects myself):

// don't define QEvent to avoid conflict with Qt

#define Q_NQEVENT    1

// provide QEvt constructors

#undef Q_EVT_CTOR

// provide QEvt virtual destructor

#undef Q_EVT_VIRTUAL

QP Qt port sources

Because Qt is a multi-platform code, the example QP port to mingw Qt still works for embedded ARM.  I just have to include the qpcpp/ports/qt/ folder, like I have done for the qep/ and qf/ folders above.  But since the PixelLabel is only necessary for the fly-and-shoot example, I excluded them.

SOURCES += \...
$$QP_ROOT/ports/qt/guiapp.cpp \
$$QP_ROOT/ports/qt/qf_port.cpp

HEADERS += gui.h \
$$QP_ROOT/ports/qt/qep_port.h \
$$QP_ROOT/ports/qt/qf_port.h \
$$QP_ROOT/ports/qt/tickerthread.h \
$$QP_ROOT/ports/qt/aothread.h \
$$QP_ROOT/ports/qt/guiapp.h \
$$QP_ROOT/ports/qt/guiactive.h

Unlike the example Qt integration on mingw, setting a stack size to 4 KB is preventing QThread start, so I commented them out and let QThread use the default thread stack size for now.


   //thread->setStackSize(stkSize);


Application support files

The final step in mating QP to an application is to specify functions that QP calls for certain events (startup, onClockTick, onAssert, etc) and the application state machine calls (like updating the philosopher stats from the Table state machine).  Unlike the port files, which can theoretically be shared between different QP-Qt projects (again, I will only have 1), the application specific files are coupled to the application logic.  For the DPP application, dpp.h and the bsp header/source files are such files, so I add them to the first lines of SOURCES and HEADERS in the qmake pro file:

SOURCES += main.cpp gui.cpp bsp.cpp philo.cpp table.cpp \
...

HEADERS += gui.h bsp.h dpp.h \
...

dpp.h contains the application specific event class TableEvt.  To turn off the event polymorphism feature, I take in only the signal number in the TableEvt constructor.

When I examine bsp.cpp, I see that the philosopher states (THINKING/HUNGRY/EATING) are displayed with QPixmaps showing 3 different PNG files, and the table state (PAUSED/SERVING) is displayed with a text on a button.  The images for the philosopher states are in res folder,  pointed to by the gui.qrc (Qt resource) file.  So I add this file to the project (Add Existing File).  I also copied the entire res/ folder from the mingw example folder, so that when I click on one of the PNG files in the resource, I see the image in the Qt Creator, like this:

In the qmake pro file, the resource shows up like this:

RESOURCES += gui.qrc

To update the files to the latest QP API, I make the changes discussed above, in "Note on updating to the QP 5 API" section.

UI

Instead of just blindly copying the QDialog based UI from the example, I went through the trouble of copying the buttons and labels from the example UI to the QMainWindow based UI, all to preserve the possibility of using the top menu and the bottom status bars in the future.  In Qt Creator's Designer View, the UI looks like this:

Note that all widgets I copied are in the central widget; that is, the north, south, east, west widget areas do not exist.

I wire the signals emitted from the widgets to the 3 slots defined in gui.cpp constructor:

...

    QObject::connect(m_quitButton, SIGNAL(clicked()), this, SLOT(onQuit()));

    QObject::connect(m_pauseButton, SIGNAL(pressed()), this, SLOT(onPausePressed()));

    QObject::connect(m_pauseButton, SIGNAL(released()), this, SLOT(onPauseReleased()));

    QObject::connect(this, SIGNAL(finished(int)), this, SLOT(onQuit()));

    } // setupUi

The UI designer just lays out the widgets (and possibly statically connects signals to slots).  The code behind the UI is in gui.cpp, which I copied from the example.  After this step, my gui.cpp code is the same as the example, except for Gui parent being QMainWindow instead of QDialog.

State machines

The philosopher and the table state machines drive the application logic.  The Qt integration example has the 2 state machine implementations generated by the QM state charting tool, but I do NOT want to generate my code, so I copy philo.cpp and table.cpp from another example (examples/arm/vanilla/gnu/dpp-at91sam7s-ek) that does not yet use the new style of coding the state transition.  I also added these 2 files to the project.  But I later found out that weird crash can occur if I update the GUI in a non-GUI thread.  Examples of the crash:

QObject::startTimer: Timers cannot be started from another thread

QBasicTimer::stop: Failed. Possibly trying to stop from a different thread

QObject::connect: Cannot queue arguments of type 'QTextBlock'

(Make sure 'QTextBlock' is registered using qRegisterMetaType().)

valgrind  --undef-value-errors=no --leak-check=yes dpp-gui > dpp_valgrind.txt 2>&1

I added the Desktop kit to the project, in the Projects toolbar icon, and reproduced the problem even on Ubuntu.  More errors:

QApplication: Object event filter cannot be in a different thread.

QWidget::repaint: Recursive repaint detected

This is why in the Qt integration example, the table active object it the ONLY active object that derives from GuiQActive class, which is supplied in the port.

class Table : public QP::GuiQActive {

...

Application main

I copied main.cpp verbatim from the example, which gives the table GuiQActive object NO event queue (because events to the GUI go through the Qt event delivery mechanism).  So the following code snippet is correct:

    DPP::AO_Table->start((uint_fast8_t)(N_PHILO + 1),
                         //GuiQActive does not need event queue
                         //&l_tableQueueSto[0], Q_DIM(l_tableQueueSto),
                         (QP::QEvt const **)0, (uint32_t)0,
                         (void *)0, (uint_fast16_t)0);

Build and debug on the target

1. Leveraging the hard work of setting up the cross-compile in the previous blog entry, I build the target ELF file easily by clicking on the build icon (the hammer).  The debug target is still only 2.3 MB on the disk.
2. Following the workaround for the cross-debug not working, I copy the ELF file to the target's /root folder.
3. I start the gdbserver on the copied app, specifying the mouse device (note that this application does NOT use the keyboard, but the keyboard device is event1)
   
   gdbserver localhost:1234 /root/dpp-gui -plugin evdevmouse:/dev/input/event0
4. In Qt Creator, attach to the remote gdbserver (menu --> Debug --> Start Debugging --> Attach to Remote Debug Server), specifying the port and the ELF file, as you can see in this example:

I see 5 Homer icons happily taking turns eating, thinking, being hungry!