STM32 Graphic capabilities

I had a great opportunity yesterday, the kind of opportunities I usually try not to waste: I was offered to learn something!
Never pass an opportunity like that!

But that was just the beginning of my luck.
What if I tell you the topic was incredibly cool, that the instructor definitely knew his stuff, the same stuff I have been struggling around for a while, before getting invited at this free 1 day training?

Does it get any better? Actually it does, the whole thing came with free pizza (and a STM32F746 discovery board too!).

(Sorry, don't have an image of the pizza)

Jokes aside, thanks ST Microelectronics for this opportunity.

As usual, one of my preferred ways of writing down notes about something I learnt or I am learning is to write a blog post, it definitely helps me in the process of digesting the information and, from some feedback I received, it turns sometimes helpful to others too.

Ok, enough blah blah, down to business.
Graphics, oh yeah!

Let's start from the basics.

We live in a world where even toilets might have some GUI (not joking, see yourself..)

..so, adding graphics capabilities to MCUs seems just natural.
You can find in the web several projects even with Arduino unos and some SPI screen, some are pretty damn cool too.
However if you want something a bit more performant, you need horse power.
Cortex cores (from M3 upwards) start to get you in the ballpark, but for complex applications, you may need to scale up to M4s and M7s.
A 32 bit risc processor running at 216MHz such as the STM32F746 has the needed computational power to support a modern interactive GUI.

Turns out the horsepower is not all, there is more because we need more.

Ideally you want your gui to
1) run on a screen with decent definition (the bigger the screen, the higher the definition in order to mantain an acceptable DPI resolution)
2) Have a high refresh rate (60Hz would do)
3) Have decent color depth, such as 16 or 24 bits
4) Avoid to drain your CPU as it usually needs to do other suff in the meanwhile, like reacting to your inputs.

If you put all that together you discover that you need a fast processor, with quite a bit of ram and maybe some easy way to expand it with fast external SDRAM, you need maybe some fast storage for your images, and ideally something that deals with the complexity of the screen interface for you.

The thing is, it is not enough to solve ONE problem, you need to solve them all... and that's the kind of thing where MCUs excell, they provide you with a set of integrated peripherals designed specifically to tackle all the needs for an application.
I believe the STM32F7 is one remarkable example of that (Note : I am not sponsored by ST, although, techncially... they gave me a free pizza so... I might be biased :) ).

To explain why I believe the F4 and F7 families have very good features to support a gui, let's analyze what are the things that are needed.

If you are interested in getting the details, ST has great documents such as the Application Note AN4861, which I strongly encourage you to check.

When you interface a screen you can chose different devices, but there are mainly two (more actually, read the AN4861 for that) kind of screens :
- Those that have memory and timing controller onboard
- Those that don't

The TFT screens you see around connected with arduinos or even Cortex M3s in projects are usually of the first kind.
You can find them on ebay for few bucks so, why would you even consider the second kind?
Typically, if you need a higher resolution than 320x240 with a decent color depth and refresh rate, you need to manage yourself the controller and that adds quite a bit of complexity.
You need to manage the timing for the sync signals (see my posts here and here if you understand a bit of Verilog and FPGAs) and a 24 bit parallel interface for the colors, but the worse part is that the two must be precisely synchronized.
Say that you are refreshing a VGA screen at 60Hz, that means that at any specific instant you have to send the exact RGB values to correctly draw the current pixel, over and over.
If you CPU is dealing with that, probably it is not going to be able to do much more.
A cortex M7 @216Mhz might be able to do that, but actually it does not have to because you want to save your CPU clocks for your application, besides the GUI.

ST added to some F4 F7 and L4 devices a peripheral called LCD - TFT Display Controller (LTDC).
It can interface with several kind of displays (note : depending on the device itself, some recent interface technologies might or might not be supported), it is integrated in the MCU and can access the MCUs DMA channels.
Basically you define a buffer in memory (internal or external memory, works just the same) and your code populates this buffer with your graphics, the LTDC reads it authonomously and provides the needed signals for the display, no CPU needed once you started it.
And that's your RAM you are dealing with, the fastest thing you can read and write to from your MCU, no match for a SPI interface.

During a hands-on lab in the training I was debugging the code, the execution was halted at a breakpoint so the CPU was actually waiting, not doing anything. Still the internal LTDC was refreshing the screen, completely authonomously, just like if the screen had its own controller and ram buffer onboard (and it did not, the screen is a RK043FN48H-CT672B, it has an RGB parallel interface, no controller).

So, the LTDC plays an important role in enabling graphics capabilities and it does it allowing quite a bit of flexibility.

A second key component for the solution is RAM : GUIs in high resolution are memory hungry and this requires two things :
1) A decent amount of internal memory and / or an easy way to integrate inexpensive and powerful external memories
2) An efficient high bandwidth DMA

The F7 shines there, I suggest you check the schematics for the F7Discovery board, it is surprisingly (to me) quite readable and extremely informative.
The beauty is that the LTDC can tap with no issues in that DMA, in fact it is a master device in the AHB

 from the AN4861 @ST Microelectronics

If you want to learn more about the F7  architecture, ST has a nice MOOC training about it, you can search it on their website.

The potential draw-back of this kind of solution is that you have a high pin count to connect to the display (24 for the RGB, 3 for the sync signals, something to control the backlight, an I2c eventually for the touch panel...) wich calls for some "interesting" PCB layout and also you should expect to deal with high pin count devices, typically BGAs... maybe not all of you -nor me anyways- will be able to solder those in your kitchen.
Some new technologies such as MIPI-DSI , supported in some STM32 devices, solve that problem, I will not enter in details here.

So, now we can update the screen and we have a fast access to the frame buffer, being internal or external, that's a lot already, but it's not all.

Chrom-ART, aka DMA2D.
This is another important component for graphics of the STM32 architecture, it actually helps in populating your frame buffer.
Imagine your gui has a background image and some buttons with icons.
What you do is you get the background image maybe from a QSPI ram, copy it on the frame buffer, then you draw the buttons, back in the QSPI to fetch tthe icons, and finally copy them on the framebuffer as well.
It works, but requires wuite some effort to the CPU to handle all those memory transfers... unless you have a DMA2D which gladly takes care of that for you.
It's pretty cool, read more about it in the AN4943 application note document :)

Finally, all this is indeed amazing, but how does it come all together when you are writing an application?
The keyword here is : Software libraries.
By any means, you can fire up you STM32CubeMX, activate LTDC, DMA2D, QSPI and  whatever you need, then use HAL drivers to do your magic.
I tried and failed, maybe now I might have a better chance at it as back then I had very little understanding about what I just wrote in this post.
Is there another way?
Yes, as I said, software libraris.
We played with 3 of them during the training, all 3 looked pretty good to me even if you may want to use them for sligthly different purposes.
We tried Embedded Wizard , touchGFX and STemWin (Segger).
Those libraris smoothly and seamlessly integrate with the STM32 hardware, you don't even need to care about LTDC, DMA2D etc... they take care for you, and they do way more.
They have a PC based design where you build graphically your gui, including interaction with the touch screen, and then c code is generated for your device.
They work in slightly different ways, EW and tGFX have a well integrated environment, they deal with most of the tasks for you, while emWin requires a bit more coding.
I personally prefer the emWin approach because I feel it gives me better control over the code, but you pay that with more effort in most cases.
Also the PC tools are a bit less polished, but again, that's not a main concern for me.
One good thing about STemWin is that it comes for free with STM32 devices since ST made an agreement with Segger, customized/optimized the code for its devices and provided licenses for free to its customers.
If you have a medium to big sized project, you probably are not going to decide based on library license costs anyways.
My impression (but I still need to play more with those libraries) is that EW and tGFX may provide a faster time to market option and ensure good performances.
With STemWin I think you can achieve good performances, but it is up to you to optimize the process.

To wrap it up, ST seems to be quite committed in supporting graphics capabilities by :
- Providing fast MCUs (the STM32H7 will run at 400MHz!)
- Providing peripherals to remove load from the MCU and to ease the integration with memory and displays
- Working with partners to boost the software ecosystem
- Supporting customers with good documentation, examples and training
- PIZZA!

I may write more on this topic once I played a bit more with the libraries, maybe with some code examples.