In this article we walk through creating an Internet enabled clock using htcw_uix (UIX), my user interface library.
Introduction
In this article I will endeavor to show you the ropes in terms of using UIX, my user interface library and GFX, my graphics library which it builds on. We will be creating a simple clock on an M5 Stack Core2. The clock uses NTP, and IP location services to get the time information. Everything is asynchronous, and the radio is only turned on while in use.
Prerequisites
- One M5 Stack Core2 and a USB data cable (cable included)
- Visual Studio Code /w PlatformIO
- A WiFi router that can connect at 2.4Ghz
Background
Documentation and tools for the htcw_gfx graphics library can be found here: https://honeythecodewitch.com/gfx
Documentation for the htcw_uix user interface library can be found here: https://honeythecodewitch.com/uix
I didn't want to write UIX, as LVGL already exists and I have no desire to directly compete with it. That said, Espressif forced my hand by breaking SDA reads over SPI under Arduino on the ESP32. Long story short, I needed a draw-on-demand framework to work around the lack of being able to read the frame buffer back off the display. It was the only way I was going to be able to do things like alpha-blend and anti-alias with my graphics library. UIX is my draw-on-demand framework.
UIX is simpler than LVGL, with far fewer widgets. It's also far more conservative with memory. Furthermore, since it builds on my graphics library, it can support things like SVG and Truetype. LVGL is C while UIX is C++14 and C++17.
Essentially you create screens, and then controls which you lay out on those screens. UIX handles redrawing the controls as necessary, and only updates the portions of the display that have changed. UIX can also handle touch feedback for things like buttons. In this way it works similarly to LVGL. I did not derive from LVGL, however. There's really only one way to do draw-on-demand correctly, and so both LVGL and my code follow that pattern.
Here we will be using the built in analog clock, a label and a couple of canvas controls to construct our display.
Using the code
The first thing we need to do is set up our platformio.ini file:
[env:m5stack-core2]
platform = espressif32
board = m5stack-core2
framework = arduino
upload_speed=921600
monitor_speed=115200
monitor_filters = esp32_exception_decoder
lib_deps = codewitch-honey-crisis/htcw_m5core2_power ; AXP192 power chip
codewitch-honey-crisis/htcw_ili9341 ; screen
codewitch-honey-crisis/htcw_ft6336 ; touch screen panel
codewitch-honey-crisis/htcw_bm8563 ; real time clock
codewitch-honey-crisis/htcw_uix ; UI and Graphics
build_unflags = -std=gnu++11
build_flags= -std=gnu++17
-DBOARD_HAS_PSRAM
upload_port = COM10 ; change this to your configuration
monitor_port = COM10 ; change this to your configuration
The first 3 lines are boilerplate for the Core 2 under Arduino. After that we set the upload and monitor speeds. I've added the esp32 exception decoder filter in case of a crash so you can get decent error messages out of it.
After that is a whole lot of dependencies to code of mine. I have a whole ecosystem for IoT and embedded.
After those we change the compiler's C++ version to GNU C++17 and indicate that this device has PSRAM even though we're not using it currently.
Finally, we have the COM ports which you should change to match your operating system and hardware configuration.
config.hpp
Let's look at include/config.hpp first since you may want or need to change it:
#pragma once
static constexpr const char* wifi_ssid = nullptr;
static constexpr const char* wifi_pass = nullptr;
static constexpr const unsigned int wifi_fetch_timeout = 30;
static constexpr const char* time_server_domain = "pool.ntp.org";
static constexpr const unsigned int time_refresh_interval = 10*60;
static const gfx::open_font& text_font = OpenSans_Regular;
Here we have a null SSID which indicates that we'll be using whatever the previous WiFi connection the ESP32 remembers was. You can change this to reflect your network and credentials.
Next we have the timeout for when we're fetching Internet data and it doesn't arrive. This is in seconds.
After that we have the NTP server address. We use pool.ntp.org.
After that, we have the time refresh interval in seconds. This is the frequency that the clock is synced to Internet time.
Finally, we set the font. This font was downloaded from fontsquirrel.com and converted to a header with my generator tool. GFX and UIX support most Truetype or Opentype fonts, Win3.1 FON files, or VLW font files.
ui.hpp
include/ui.hpp is where we declare our user interface elements. I typically break them off into their own header so they can be accessed in more than one place.
#pragma once
#include <gfx.hpp>
#include <uix.hpp>
using color_t = gfx::color<gfx::rgb_pixel<16>>; using color32_t = gfx::color<gfx::rgba_pixel<32>>;
using screen_t = uix::screen<gfx::rgb_pixel<16>>;
using surface_t = screen_t::control_surface_type;
using svg_clock_t = uix::svg_clock<surface_t>;
using label_t = uix::label<surface_t>;
using canvas_t = uix::canvas<surface_t>;
extern screen_t main_screen;
extern svg_clock_t ana_clock;
extern label_t dig_clock;
extern canvas_t wifi_icon;
extern canvas_t battery_icon;
First we include the GFX and UIX libraries. Next come a couple of color definitions. The color<>
template is a pseudo-enumeration of all the colors in the X11 standard represented in the specified pixel type. We've declared one of these for the native display type (rgb_pixel<16>
) and one for UIX's intrinsic pixel type, which is always rgba_pixel<32>
. You can use these to retrieve named colors like color_t::purple
.
Now we create an alias screen_t
which instantiates a screen<>
template with rgb_pixel<16>
in it. Let me explain. Screens produce bitmaps, which are then sent to the display. We want that to match our screen's native pixel type for best performance, so by passing in rgb_pixel<16>
we are telling it to generate bitmaps in that format.
Next we create an alias surface_t
that maps to the screen's control_surface_type
. A control surface acts as a kind of "canvas" for drawing operations. In GFX this is referred to as a kind of "draw destination". Drawing operations can target this type. It's not a simple bitmap, but it is backed by one. It also maps logical coordinates to physical coordinates, and potentially does post processing or different pixel layouts (but not in this or most cases). Each control needs to know what type of surface it will be drawing to, so we'll be giving them surface_t
instantiations. This alias just makes things easier to type.
Now we can make aliases to instantiate our control templates. We create a label_t
, a canvas_t
and an svg_clock_t
.
Finally, we declare instantiations of each of them (extern).
For a larger project, I would also make a ui.cpp and break off the initialization for the UI into that file, but for this project, with only a few controls, it didn't add a lot of value, so it's in main.cpp which we'll cover now.
main.cpp
Now we'll cover src/main.cpp where all the magic happens.
First up, quite a few #include
s:
#include <Arduino.h>
#include <m5core2_power.hpp>
#include <bm8563.hpp>
#include <ft6336.hpp>
#include <tft_io.hpp>
#include <ili9341.hpp>
#include <uix.hpp>
#include <gfx.hpp>
#include <WiFi.h>
#include <ip_loc.hpp>
#include <ntp_time.hpp>
#define OPENSANS_REGULAR_IMPLEMENTATION
#include <assets/OpenSans_Regular.hpp>
#define ICONS_IMPLEMENTATION
#include <assets/icons.hpp>
#include <config.hpp>
#include <ui.hpp>
Of note are the assets. These were pulled off the net, and generated with online tools I provide at my website. They've been converted to headers in order to embed the content in the firmware. Particularly with fonts, this is important. Attempting to read them off of flash or SD, while supported, tends to be prohibitively slow in terms of rendering, although they can be loaded into PSRAM from a source like that, and then used from there.
Moving on, we import a few frequently used namespaces:
using namespace arduino;
using namespace gfx;
using namespace uix;
Next we declare the power management. This is necessary for the Core 2 to function properly:
static m5core2_power power;
Now things get a little complicated, as we're declaring hardware bus connection - in this case SPI, and then passing that instantiation of bus to our LCD controller driver - in this case an ILI9342c, which is what the Core 2 uses. This sets the pins for the bus, the DMA size and the SPI host to use, which it then passes to the driver. All the pins are hard coded because they'll never change. Finally we instantiate the lcd_t
driver.
using tft_bus_t = tft_spi_ex<VSPI,5,23,-1,18,0,false,32*1024+8>;
using lcd_t = ili9342c<15,-1,-1,tft_bus_t,1>;
static lcd_t lcd;
Next we have two 32KB byte arrays which are used by UIX to refresh the display:
static uint8_t lcd_transfer_buffer1[32*1024];
static uint8_t lcd_transfer_buffer2[32*1024];
UIX uses these to create the backing bitmaps for those control surfaces I mentioned earlier. The reason there are two is because we are using DMA, and so UIX will write to one while the other is being transmitted. DMA isn't required, and if not used, we could have one buffer that's twice as large. However, there's better throughput using DMA. On the other hand, a smaller transfer buffer can mean some controls have to be redrawn multiple times in order to fully render. It's robbing Peter to pay Paul. Here however, our largest control is 128x128 at 16-bit color, and that is 32KB, so we shouldn't need anything larger than that.
We don't use the touch for anything in this project, but since it's such an integral part of UIX I felt I should at least give an example of wiring it up. The first step is declaring it. In the Core 2 it's on the 2nd I2C bus (Wire1
).
using touch_t = ft6336<280,320>;
static touch_t touch(Wire1);
Now onto all the mess we use for keeping time, and related tasks:
static bm8563 time_rtc(Wire1);
static char time_buffer[32];
static long time_offset = 0;
static ntp_time time_server;
static bool time_fetching=false;
The first line defines the real time clock hardware driver. The second line we use to hold a string for our digital clock. UIX does not copy strings. It uses the pointer you gave it so it's your job to allocate the space for those strings as needed.
The time_offset
is the UTC offset in seconds. Essentially this is our time zone.
The time_server
holds the NTP service manager.
time_fetching
indicates whether we're currently gathering time information from the Internet. This is on whenever the radio is.
Now onto our UI screen and control definitions:
screen_t main_screen(
{320,240},
sizeof(lcd_transfer_buffer1),
lcd_transfer_buffer1,
lcd_transfer_buffer2);
svg_clock_t ana_clock(main_screen);
label_t dig_clock(main_screen);
canvas_t wifi_icon(main_screen);
canvas_t battery_icon(main_screen);
Here we pass the size of the screen, the size of the transfer buffer(s) - two in this case, but each must be the same size, and then one or two transfer buffers - here we use two.
The reason again, for two buffers is to fully utilize DMA such that UIX can draw to one buffer while the MCU is transmitting the other.
The rest are just our controls, tied to the main_screen
.
Next, we have our routines that essentially plug UIX into our hardware drivers:
static void lcd_flush(const rect16& bounds,const void* bmp,void* state) {
const const_bitmap<decltype(lcd)::pixel_type> cbmp(bounds.dimensions(),bmp);
draw::bitmap_async(lcd,bounds,cbmp,cbmp.bounds());
}
static void lcd_wait_flush(void* state) {
lcd.wait_all_async();
}
static void lcd_touch(point16* out_locations,size_t* in_out_locations_size,void* state) {
*in_out_locations_size = 0;
uint16_t x,y;
if(touch.xy(&x,&y)) {
Serial.printf("xy: (%d,%d)\n",x,y);
out_locations[0]=point16(x,y);
++*in_out_locations_size;
if(touch.xy2(&x,&y)) {
Serial.printf("xy2: (%d,%d)\n",x,y);
out_locations[1]=point16(x,y);
++*in_out_locations_size;
}
}
}
Basically the first function is responsible for sending the bitmap data we generated to the display. The second one is responsible for handling waiting on a DMA transfer to complete. The third routine plugs our touch device into UIX's touch system. We're not using that for this project but I decided to include it since it's so fundamental.
Now onto more application specific, rather than hardware specific methods:
static void update_time_buffer(time_t time) {
tm tim = *localtime(&time);
strftime(time_buffer, sizeof(time_buffer), "%I:%M %p", &tim);
if(*time_buffer=='0') {
*time_buffer=' ';
}
}
This routine basically formats a time string for hour 12-hour clock. It uses strftime()
but hacks it to remove any leading zero. Note that time_buffer
is global. UIX is very stingy about memory, and like GFX, it usually expects you to provide your own. Text is no exception. When you set the text of a UIX control it will use the pointer you gave it, as is. It does not copy. Ergo, you need to keep that pointer valid for as long as you might be refreshing the screen.
The next two routines draw our canvases, which are basically just icons. UIX doesn't have an icon control so we just use a canvas and draw it ourselves:
static void wifi_icon_paint(surface_t& destination, const srect16& clip, void* state) {
if(time_fetching) {
draw::icon(destination,point16::zero(),faWifi,color_t::light_gray);
}
}
static void battery_icon_paint(surface_t& destination, const srect16& clip, void* state) {
int pct = power.battery_level();
auto px = power.ac_in()?color_t::green:color_t::white;
const const_bitmap<alpha_pixel<8>>* ico;
if(pct<25) {
ico = &faBatteryEmpty;
if(!power.ac_in()) {
px=color_t::red;
}
} else if(pct<50) {
ico = &faBatteryQuarter;
} else if(pct<75) {
ico = &faBatteryHalf;
} else if(pct<100) {
ico = &faBatteryThreeQuarters;
} else {
ico = &faBatteryFull;
}
draw::icon(destination,point16::zero(),*ico,px);
}
The main complication is just choosing the right icon based on the battery level. Otherwise we're just drawing icons from include/assets/icons.hpp. That file was generated using my icon pack tool.
setup()
This routine is long so we'll cover it in sections:
Serial.begin(115200);
power.initialize(); lcd.initialize(); touch.initialize();
touch.rotation(0);
time_rtc.initialize();
Serial.println("Clock booted");
We're basically just initializing devices and then giving an indication that we booted.
main_screen.background_color(color_t::black);
main_screen.on_flush_callback(lcd_flush);
main_screen.wait_flush_callback(lcd_wait_flush);
main_screen.on_touch_callback(lcd_touch);
This is where we set our callbacks. It's important to hook these all up properly if you want UIX to function. It doesn't need the touch necessarily, but if you're using DMA be absolutely careful to set your callbacks and implement them appropriately as we did. Failure to do so can cause hang ups, blank screens, and basically ugliness all around.
Next we initialize our controls, starting with the analog clock. Note that each control must be explicitly registered with its screen using register_control()
.
ana_clock.bounds(srect16(0,0,127,127).center_horizontal(main_screen.bounds()));
ana_clock.face_color(color32_t::light_gray);
auto px = ana_clock.second_color();
px.template channel<channel_name::A>(
decltype(px)::channel_by_name<channel_name::A>::max/2);
ana_clock.second_color(px);
px = ana_clock.minute_color();
px.template channelr<channel_name::A>(0.5f);
ana_clock.minute_color(px);
main_screen.register_control(ana_clock);
Mostly we're just setting properties. bounds()
is important, and takes an srect16
(signed rectangle with 16-bit coordinate values) which tells UIX where on the screen the control belongs. You typically always want to set this when you initialize the control, even if you plan on moving it later.
One thing we're careful about here is the clock is 128x128 at 16-bit color, which is exactly 32KB, which by no coincidence is the same size as each of our transfer buffers. If it was any larger, the clock would have to draw more than once in order to render the entire clock. It's SVG. SVG is not lightweight. It's fluffy and fancy and it costs. Keeping the clock to the size of the transfer buffer increases performance quite a bit.
Otherwise we're getting and setting colors. Note that regardless of your display's native pixel type, UIX uses 32-bit RGBA pixels (RGBA8888) so we're using the color32_t
pseudo-enumeration to retrieve colors in rgba_pixel<32>
format.
One thing to note is that we're alpha-blending in two places. This means we're making parts of the clock semi-transparent. This is as simple as setting the alpha (A) channel of an RGBA pixel (or any pixel format with an alpha channel). In the first case we're retrieving the second_color()
which defaults to red, for the second hand. We're then setting the alpha channel. We want to set it to half of the maximum which would be 127, but rather than use a hard coded value we're querying the pixel's metadata for the alpha channel (A) and getting its maximum value, which we then divide by two to get our result which we then use to set the alpha channel of px
. We could have just used 127 but it wasn't as illustrative. Essentially an alpha channel indicates the opacity, where 0 is completely transparent and the bit depth of the channel indicates the value for fully opaque, which is also the default. For our pixels here the alpha channel is 8-bits (1/4 of 32) which is 0-255. Setting it to 127 essentially makes it only half opaque. This means colors below the second hand will partially bleed through.
In the second place we're alpha blending the minute hand. We're using a negligibly less perfomant (but much easier to type!) variation of setting the pixel's alpha channel to half. We use channelr<>()
instead of channel<>()
to access the value as a scaled real number between 0 (minimum) and 1 (maximum - resolves to 255 scaled in this case).
These both do the same alpha blending but in two slightly different ways, the latter being far more brief.
Finally we register the control. If you forget this step it will never show up on the screen, and won't respond to touch inputs.
Next we initialize the digital clock portion, which is just a label:
dig_clock.bounds(
srect16(0,0,127,39)
.center_horizontal(main_screen.bounds())
.offset(0,128));
update_time_buffer(time_rtc.now()); dig_clock.text(time_buffer);
dig_clock.text_open_font(&text_font);
dig_clock.text_line_height(35);
dig_clock.text_color(color32_t::white);
dig_clock.text_justify(uix_justify::top_middle);
main_screen.register_control(dig_clock);
We just give it our buffer and a bunch of the information we need to display text. Note that we updated the time buffer here. This is so when the control is first displayed it has something meaningful to present.
Now we set up our canvas icon controls. These are trivial so we'll cover them together:
wifi_icon.bounds(
srect16(spoint16(0,0),(ssize16)wifi_icon.dimensions())
.offset(main_screen.dimensions().width-
wifi_icon.dimensions().width,0));
wifi_icon.on_paint_callback(wifi_icon_paint);
main_screen.register_control(wifi_icon);
battery_icon.bounds(
(srect16)faBatteryEmpty.dimensions().bounds());
battery_icon.on_paint_callback(battery_icon_paint);
main_screen.register_control(battery_icon);
Basically we're just setting the bounds, callback and then registering. The bounds are computed in part based on the width and height of the icons they hold.
That's it for setup()
.
loop()
Discounting the connection and fetching loop() is fairly simple, but first let's cover the part that isn't - the connection and fetching business:
static int connection_state=0;
static uint32_t connection_refresh_ts = 0;
static uint32_t time_ts = 0;
IPAddress time_server_ip;
switch(connection_state) {
case 0: if(connection_refresh_ts==0 || millis() >
(connection_refresh_ts+(time_refresh_interval*1000))) {
connection_refresh_ts = millis();
connection_state = 1;
time_ts = 0;
}
break;
case 1: time_fetching = true;
wifi_icon.invalidate();
if(WiFi.status()!=WL_CONNECTED) {
Serial.println("Connecting to network...");
if(wifi_ssid==nullptr) {
WiFi.begin();
} else {
WiFi.begin(wifi_ssid,wifi_pass);
}
connection_state =2;
} else if(WiFi.status()==WL_CONNECTED) {
connection_state = 2;
}
break;
case 2: if(WiFi.status()==WL_CONNECTED) {
Serial.println("Connected.");
connection_state = 3;
} else if(WiFi.status()==WL_CONNECT_FAILED) {
connection_refresh_ts = 0; connection_state = 0;
time_fetching = false;
}
break;
case 3: Serial.println("Retrieving time info...");
connection_refresh_ts = millis();
ip_loc::fetch(nullptr,nullptr,&time_offset,nullptr,0,nullptr,0);
WiFi.hostByName(time_server_domain,time_server_ip);
connection_state = 4;
time_ts = millis(); time_server.begin_request(time_server_ip);
break;
case 4: if(time_server.request_received()) {
const int latency_offset = (millis()-time_ts)/1000;
time_rtc.set((time_t)(time_server.request_result()+
time_offset+latency_offset));
Serial.println("Clock set.");
update_time_buffer(time_rtc.now());
dig_clock.invalidate();
connection_state = 0;
Serial.println("Turning WiFi off.");
WiFi.disconnect(true,false);
time_fetching = false;
wifi_icon.invalidate();
} else if(millis()>time_ts+(wifi_fetch_timeout*1000)) {
Serial.println("Retrieval timed out. Retrying.");
connection_state = 3;
}
break;
}
If we weren't handling this in an asynchronous fashion it would be a lot simpler to code, but it would not remain responsive during the connect and fetch process, which can take several seconds. To keep things happening we break the process into steps and run those steps using a simple state machine driven by connection_state
. Note the static
variables, which is important as they maintain their state between successive calls to loop()
.
The first step/state is the idle state 0. When we're in this state we're just checking to see if our time refresh interval has elapsed. If so we move to state 1.
State 1 kicks off trying to connect to the WiFi network, after which it moves to state 2.
In state 2 we wait for the connection to either succeed or fail, and move on to state 3, or back to state 0 accordingly.
In state 3 we fetch our time zone based on our current IP, and then begin the query to the NTP server. Note that we keep a timestamp here because we need it later. We move on to state 4.
In state 4 we wait for an NTP response or a time out. If it times out we move to state 3, retrying the fetch. If it succeeded we set the RTC clock and update our digital clock display, plus turn off the WiFi.
The entire time we track time_fetching
so we tie its state to the state of the radio - whether it's off or on.
Now on to the simpler stuff - updating our UI information:
time_t time = time_rtc.now();
ana_clock.time(time);
if(0==(time%60)) {
update_time_buffer(time);
dig_clock.invalidate();
}
static int bat_level = power.battery_level();
if((int)power.battery_level()!=bat_level) {
bat_level = power.battery_level();
battery_icon.invalidate();
}
static bool ac_in = power.ac_in();
if((int)power.battery_level()!=ac_in) {
ac_in = power.ac_in();
battery_icon.invalidate();
}
Here several things are happening. First we take the current time and use it to update the analog clock. It only redraws if it actually changed, so doing this won't trigger a repaint unless necessary. For the items that follow it's not as simple.
The label doesn't know when the contents of time_buffer
have changed. We have to invalidate()
the label ourselves to force a repaint, but we only need to do that every 60 seconds on the minute. This is far preferable to forcing a repaint on every loop()
iteration as that would make things terribly slow. Truetype fonts are not cheap to render.
With the battery level we only want to trigger a redraw once in awhile - when it has changed. Actually we could do it only when it has changed significantly enough to change the icon, but that complicates things and doesn't add a ton of value.
We also need to trigger a redraw of the battery icon if the Core 2 was plugged in or unplugged from USB power. There are some cases when this leads to a second invalidation, but it doesn't matter because UIX combines overlapping dirty rectangles, so this only leads to one invalidation.
The last bits of code are pretty simple, but demand a bit of explanation. Everything in this code is "cooperatively threaded" which means each thing does a little bit of work before passing control flow on to the next task, rather than doing long running tasks all at once. This leads to a smoother running application. The various objects however, need to be "pumped" in loop()
so that they can process the next unit of work. We do that here for the NTP server, the main screen, and the touch panel driver. The touch panel requires some special consideration since it doesn't like to be updated too quickly or it causes I2C read errors and false touches. To that end we only read it once every 50ms.
time_server.update();
main_screen.update();
static uint32_t touch_ts = 0;
if(millis()>touch_ts+50) {
touch_ts = millis();
touch.update();
}
I'm not going to cover the IP location or NTP client code, as I've covered it in a previous article.
History
- 8th May, 2024 - Initial submission