Paper Byte Logs

Fix ESP32 firmware size in PlatformIO

2025-12-11T00:00:00+00:00

Error </div> </div>

Error: The program size (1564865 bytes) is greater than maximum allowed (1310720 bytes) </div> </div>

Most ESP32 boards ship with a 1.3 MB (1310720 bytes) application partition, unless a custom partition table is used. So if your firmware size is bigger than that, it won't compile.

Solution:

Change platformio.ini</code> to use a partition layout with a 2 MB app slot.

board_build</span></span>.</span>partitions =</span> partitions.</span>csv
</span></code></pre>
Create partitions.csv</code> in your project root:</p>
Option A - "huge app"</strong></p>
#</span></span> Name</span></span>,</span></span>   Type</span></span>,</span></span> SubType</span></span>,</span></span> Offset</span></span>,</span></span>  Size</span></span>,</span></span>    Flags
</span></span></span>nvs</span></span>,</span></span>      data</span></span>,</span></span> nvs</span></span>,</span></span>     0x9000</span></span>,</span></span>  0x5000</span></span>,</span></span>
</span></span></span>otadata</span></span>,</span></span>  data</span></span>,</span></span> ota</span></span>,</span></span>     0xe000</span></span>,</span></span>  0x2000</span></span>,</span></span>
</span></span></span>app0</span></span>,</span></span>     app</span></span>,</span></span>  ota_0</span></span>,</span></span>   0x10000</span></span>,</span></span> 2M</span></span>,</span></span>
</span></span></span>app1</span></span>,</span></span>     app</span></span>,</span></span>  ota_1</span></span>,</span></span>   0x210000</span></span>,</span></span> 2M</span></span>,</span></span>
</span></span></span>spiffs</span></span>,</span></span>   data</span></span>,</span></span> spiffs</span></span>,</span></span>  0x410000</span></span>,</span></span> 0x3F0000</span></span>,</span></span>
</span></span></span></code></pre>
Option B - "no OTA"</strong></p>
#</span></span> Name</span></span>,</span></span>   Type</span></span>,</span></span> SubType</span></span>,</span></span> Offset</span></span>,</span></span>  Size</span></span>,</span></span>     Flags
</span></span></span>nvs</span></span>,</span></span>      data</span></span>,</span></span> nvs</span></span>,</span></span>     0x9000</span></span>,</span></span>  0x5000</span></span>,</span></span>
</span></span></span>phy_init</span></span>,</span></span> data</span></span>,</span></span> phy</span></span>,</span></span>     0xe000</span></span>,</span></span>  0x1000</span></span>,</span></span>
</span></span></span>factory</span></span>,</span></span>  app</span></span>,</span></span>  factory</span></span>,</span></span> 0x10000</span></span>,</span></span> 0x2F0000</span></span>,</span></span>
</span></span></span>spiffs</span></span>,</span></span>   data</span></span>,</span></span> spiffs</span></span>,</span></span>  0x300000</span></span>,</span></span> 0x100000</span></span>,</span></span>
</span></span></span></code></pre>
See [[Understanding ESP32 partition tables]] for more information.</p>
Or have a look at those examples</a>.</p>

Understanding ESP32 partition tables 2025-12-11T00:00:00+00:00 I recently had the problem that my program size was greater than the default code partition. To fix this we can change the partition table of the ESP32. For a quick fix see this note</a>. ESP32 partitions</a></h1> The ESP32’s external flash (usually 4 MB, 8 MB, etc.) is divided into sections ("partitions"). Each partition has a name, type, subtype, offset, and size — just like a hard drive. Let's have a quick look at an example of a partitions.csv</code> file. # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x9000, 0x5000, otadata, data, ota, 0xe000, 0x2000, app0, app, ota_0, 0x10000, 2M, app1, app, ota_1, 0x210000, 2M, spiffs, data, spiffs, 0x410000, 0x3F0000, </code></pre> But before we dive into the partition table we need to have a look at the bigger picture. ESP32 Flash Layout</a></h2> ESP32 flash layout has mandatory system regions at fixed addresses, then a flexible area for your partition table and data. Here you can see an overview of the layout. Address</th> Description</th></tr></thead> 0x0000</code></td> Bootloader (fixed size, ~20-30 KB)</td></tr> 0x8000</code></td> Partition Table (default location, 4KB)</td></tr> 0x9000</code></td> First data partition (NVS by default)</td></tr> 0xXXXX</code></td> Application firmware partitions</td></tr> 0xYYYY</code></td> Filesystem partitions</td></tr> </tbody></table> So Bootloader and Partition Table use up the the first regions so the partition table starts at address 0x9000</code>. The partition table structure</a></h2> The partitions.csv</code> contains the following fields which represent the partition table. Field</th> Size</th> Description</th></tr></thead> Name</td> 16 bytes</td> Partition name (null-terminated string)</td></tr> Type</td> 1 byte</td> app=0x00, data=0x01</td></tr> Subtype</td> 1 byte</td> e.g. OTA_0=0x10, NVS=0x02, SPIFFS=0x82</td></tr> Offset</td> 4 bytes</td> Flash address (must be aligned to 0x1000 pages)</td></tr> Size</td> 4 bytes</td> Partition size</td></tr> Flags</td> 4 bytes</td> Rarely used</td></tr> </tbody></table> Types and Subtypes</a></h3> Only two major types exist: Type</th> Meaning</th></tr></thead> app</code></td> Contains executable firmware</td></tr> data</code></td> Contains NVS, SPIFFS, keys, system data</td></tr> </tbody></table> App subtypes</a></h4> For the app</code> type, you can specify the following subtypes. Subtype</th> Description</th></tr></thead> factory</code></td> Default firmware if no OTA used</td></tr> ota_0</code></td> OTA firmware slot 0</td></tr> ota_1</code></td> OTA firmware slot 1</td></tr> ota_x</code></td> Additional OTA slots (up to 15)</td></tr> </tbody></table> Data Subtypes</a></h4> For data</code>, those subtypes are available Subtype</th> Purpose</th></tr></thead> nvs</code></td> Key-value storage (Wi-Fi, preferences, calibration)</td></tr> otadata</code></td> OTA bootloader state (which slot is active)</td></tr> spiffs</code></td> SPIFFS filesystem</td></tr> fat</code></td> FAT filesystem</td></tr> nvs_keys</code></td> Secure NVS encryption keys</td></tr> phy</code></td> Radio calibration</td></tr> coredump</code></td> Crash dump storage</td></tr> efuse_em</code></td> Emulated eFuse storage (rare)</td></tr> </tbody></table> Tip </div> </div> You should include at least 0x3000</code> bytes for the NVS partition. </div> </div> Alignment rules</a></h2> ESP32 flash partitions must align to 4 KB. Note </div> </div> Partitions always start at addresses divisible by 0x1000 (4096 Bytes) Example: 0x9000</code>, 0x10000</code>, 0x210000</code> </div> </div> So to get back to our example from the start we have: # Name, Type, SubType, Offset, Size, Flags nvs, data, nvs, 0x9000, 0x5000, otadata, data, ota, 0xe000, 0x2000, app0, app, ota_0, 0x10000, 2M, app1, app, ota_1, 0x210000, 2M, spiffs, data, spiffs, 0x410000, 0x3F0000, </code></pre> A 20 KiB</code> Byte sized NVS partition starting at the address 0x9000</code>.</li> A 8 KiB</code> Byte sized otadata partition starting at 0x9000</code> + 0x5000</code> = 0xe000</code>.</li> Two 2 MB</code> Byte sized app partitions for our code.</li> A 4 MiB</code> Byte sized SPIFFS partition filling the rest of the available memory.</li> </ul> Ressources</a></h2> Espressif Docs - Partition Tables</a></li> Espressif Docs - Partition Tables (Arduino)</a></li> Medium - How to use custom partition tables</a></li> </ul> ESP32: Why GPIO 34-39 need external pullups 2025-06-01T00:00:00+00:00 While working on a small example project with an ESP32, I encountered an issue with button detection on pin 34. Despite multiple attempts to troubleshoot, the button simply wouldn't work as expected. Finally, I added an external pullup resistor—and voilà, it worked perfectly! Curious about the cause, I did some research and discovered an important detail: only GPIOs that can function as both input and output support software-configurable pullups. GPIO pins 34 to 39, however, are input-only and therefore require external pullups for proper operation. According to some Github issues, I'm not the first person this has happened to, so I’m leaving this note here as a reminder for future projects. Remember </div> </div> If you’re using GPIO 34-39, don’t forget to add external pullups! </div> </div> For more details, check out the official ESP-IDF documentation</a>. Writing an E-Paper Display Driver 2025-05-18T00:00:00+00:00 Situation and Goals</a></h1> I ordered some epaper display modules from waveshare</a> to let my students tinker with them. It is a three color display with 4.2inch and is based on the SSD1683</a> controller. Waveshare provides some examples on their wiki</a> to get started, but I found the instructions a bit tricky to follow. Especially if you are still getting started with embedded programming. For my students, I wanted to provide a simple example with a library that they can easily include in a platformio project. I also had a quick look at the GxEPD2</a> library which seemed to be more what I was looking for. But for educational reasons and for the purpose of learning something myself, I decided to write my own driver library. Goal </div> </div> Provide a simple library to use in a PlatformIO project.</li> Make the library compatible with an ESP32 and the Arduino Framework.</li> Keep the driver implementation as simple as possible.</li> </ul></div> </div> We will be using C++ as the primary programming language for this project, leveraging its object-oriented features to create a modular and reusable library for the e-paper display. While this guide aims to provide a starting point and easy to understand implementation, certain aspects have been intentionally simplified for clarity and ease of understanding. So let's get started. How to structure the code?</a></h1> To keep the design simple and modular, I decided to go with the following architecture: HAL: contains the microcontroller specific pin and spi initialization.</li> Driver: handles the display initialization and sending image data to the display.</li> Graphics: simple graphics functions to draw on a buffer.</li> User API: a wrapper to use the driver and graphics functions in a arduino typical way.</li> </ul> The Hardware Abstraction Layer</a></h2> Before we can decide how our HAL should look like, we need to understand how the display will be connected to the microcontroller. The display uses SPI to communicate with the microcontroller, but it also has some additional pins to control the display. The following table shows the pinout of the display: Pin Name</th> Description</th></tr></thead> VCC</td> Power supply (3.3V)</td></tr> GND</td> Ground</td></tr> DIN</td> SPI Data In (MOSI)</td></tr> CLK</td> SPI Clock (SCK)</td></tr> CS</td> Chip Select (active low)</td></tr> DC</td> Data (high) / Command (low)</td></tr> RST</td> Reset (active low)</td></tr> BUSY</td> Busy (active high)</td></tr> </tbody></table> Our HAL should handle everything which needs knowledge about the underlying hardware. This includes the SPI bus and some GPIO pins. So we create an abstract interface that can be implemented for different microcontrollers. The following code shows how the HAL interface looks like: class EPD_HAL { public: virtual void init() = 0; virtual void send_command(uint8_t cmd) = 0; virtual void send_data(uint8_t data) = 0; virtual void wait_busy() = 0; virtual void set_reset_pin(bool state) = 0; }; </code></pre> Initialization: Referring to the Datasheet</a> (Section 6.3) we need to set the following SPI settings. Clock Speed: 20MHz</li> Send MSBFIRST</code></li> SPI_MODE_0</code> (CPOL = 0 and CPHA = 0).</li> </ul> Data or Command: The display can be operated with either 3-wire SPI or 4-wire SPI depending if the DC Pin is used or not. The module provides the option to change the setting by soldering a resistor on the backside. The 4-wire configuration is the default option, so we will use it. This means: if the DC pin is LOW</code> we send a command.</li> if the DC pin is HIGH</code> we send some data.</li> </ul> The send_command</code> and send_data</code> methods just wrap this functionality by setting the DC Pin accordingly. Reset: To awake the panel from deep sleep mode we need to use the hardware reset pin. In the Datasheet is states that it needs to be pulled LOW</code> for 200μs and then back HIGH</code> for another 200μs. This behavior can vary depending on the specific version of the display panel being used. To accommodate these differences and keep the implementation flexible, I decided to encapsulate the reset logic within the driver itself and just provide a set_reset_pin</code> function in the HAL. Wait Busy: The busy</code> pin is used to signal that the panel is still processing and you should not send new commands or data to the display. The Driver</a></h2> Ok now that we have our EPD_HAL</code> as a foundation we can continue with the driver itself. The Driver should handle all the initialization of the display and provide functions to show an image on the screen and set it back into sleep mode afterwards. For convenience we also want a method to clear the screen. After some consideration this interface is what I came up with: class EPD_Driver { protected: EPD_HAL* hal; // Uses hardware abstraction layer const uint16_t width = 400; // Display width in pixels const uint16_t height = 300; // Display height in pixels public: EPD_Driver(EPD_HAL* hal) : hal(hal) {} void reset(); void init(); void clear(); void write_framebuffer(const uint8_t *image, bool use_red_ram); void update(); void sleep(); void display(const uint8_t *image, bool use_red_ram); }; </code></pre> Initialization and sleep mode</a></h3> Let's first look into the init</code> process. The description in the datasheet and the provided examples diverged somewhat, but both give us a good starting point. Basically we need to do the following: Do a hardware reset to awake the display from sleep mode.</li> Do a software reset to clear previous settings.</li> Send a bunch of commands to the display to config the options we want. These might change depending on your needs, but this should work as a minimal example.</li> </ol> Set the panels border waveform.</li> Select temperature sensor.</li> Set the displays RAMs Data Entry Mode.</li> Specify initial window (full) and cursor for the RAM.</li> </ul> Wait until the display is ready.</li> </ol> The Datasheet</a> (Chapter 7) provides a comprehensive table with all available commands and their options. I let a LLM generate a complete list of all commands from this table, which worked surprisingly well. namespace DisplayCmd { // System Control ... constexpr uint8_t DEEP_SLEEP_MODE = 0x10; // Deep Sleep Mode Control constexpr uint8_t DATA_ENTRY_MODE = 0x11; // Data Entry Mode Setting constexpr uint8_t SOFTWARE_RESET = 0x12; // SW RESET ... } </code></pre> There you also can find the command for the sleep mode 0x10</code>. There is a Deep Sleep Mode 1 and 2. However, I could not find any difference between the two. So i went with Mode 2 as this was used in the example code provided by waveshare even if the datasheet suggest to use Mode 1. Deep sleep mode </div> </div> As you should not keep the display in a high voltage state for too long it is important to send it back to deep sleep mode after updating the screen. </div> </div> We can use our HALs send_</code> methods, to send a command and it's options to the display. void EPD_Driver::sleep() { hal->send_command(DisplayCmd::DEEP_SLEEP_MODE); hal->send_data(0x01); // Deep Sleep Mode 2 } </code></pre> Show an Image</a></h3> To show something on the epaper display we need to load the image data into the RAM of the epaper module and then tell it to update the screen. But as the display supports RED</code>, BLACK</code> and WHITE</code> there are two memory locations. One for the BLACK</code> and WHITE</code> image and the other for the RED</code> and WHITE</code> image. Depending on your initial settings WHITE</code> is representate by a 1</code> and the others are 0</code>. Each byte which we will send to the panel represents 8 pixels. You can find a visual representation on the waveshare wiki page</a>. With this knowledge we can implement the write_framebuffer</code> and update</code> method: void EPD_Driver::write_framebuffer(const uint8_t *data, bool use_red_ram); { // select RAM hal->send_command(use_red_ram ? DisplayCmd::WRITE_RAM_RED: DisplayCmd::WRITE_RAM_BW); // Send all pixels uint32_t w = this->width / 8; for (uint32_t j = 0; j < this->height; j++) { for (uint32_t i = 0; i < w; i++) { hal->send_data(image[i + j * w]); } } } </code></pre> Depending on the selected color we tell the display which memory location to choose and send the data. As our display has a width of 400 pixel, which is a multiple of 8, we can savely just divide by 8 to calculate the bytes in one row. To make the changes visible to the user we need to call the update</code> function. To update the display we need to send two command and wait until the display is ready. void EPD_Driver::update() { hal->send_command(DISP_UPDATE_CTRL_2); hal->send_data(0xF7); // EN ANALOG, LOAD TEMP, DISP MODE 1, DIS ANALOG, DIS OSC hal->send_command(MASTER_ACTIVATION); hal->wait_busy(); } </code></pre> The display supports also an option for fast updates. But we will not look into this today. If we only use black and white images we could include the required commands in the write_framebuffer</code> method. But if we have an image with RED</code> and BLACK</code> we only want to update the screen after sending both images. So we keep both functions seperated. The display</code> method is basically just a wrapper around those two functions for the mentioned usecase. Also clear()</code> is the same as writing a complete white framebuffer. We are now able to show an image on the screen by sending an array of bytes to the module. As the flash memory of the esp32 is big enough for storing the complete image data we don't worry about partial updates for now. We can use a service like image2cpp</a> to convert an existing image into an array of bytes. Make sure your image has the correct dimensions (400x300) before uploading it. Then we copy the resulting bytes into the source code. const unsigned char TEST_IMAGE[] PROGMEM = { /* 0X00,0X01,0X90,0X01,0X2C,0X01, */ 0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF,0XFF, ... } </code></pre> For a quick demo we can test our newly written driver with the following program: // includes and defines ... auto hal = EPD_HAL_ESP32(EPD_CS_PIN, EPD_DC_PIN, EPD_RST_PIN, EPD_BUSY_PIN); auto epd = EPD_Driver(hal); void setup() { epd.init(); // Init the display epd.clear(); epd.display(TEST_IMAGE,false); epd.sleep(); } void loop(){} </code></pre> This works as expected and we should see our image on the screen. The Graphics Library</a></h2> We could call it a day and just always convert images and copy them into our program or write a simple server where to fetch new images from. But sometimes it is useful to be able to create a new image programmatically. This is where the graphics library comes in. Instead of writing our own graphics library from scratch we will use Adafruit_GFX</a> for this purpose. This makes it fairly simple. We only need to subclass the Adafruit_GFX</code> base class and implement the drawPixel</code> method. We could implement this in a way to draw directly to the epapers RAM. But this would need some more work on our EPD_Driver</code> class to make it a smooth experience. To keep it simple i decided to go with a different approach. In fact i decided to decouple it from the driver itself and only provide a Canvas</code> class which is basically a framebuffer where you can draw on. After that you can send the imge to the display using the driver. auto canvas = Canvas(400,300); // draw on the canvas with all functions which are provided by Adafruit_GFX auto img = canvas.getImage(); epd.display(img, BLACK_IMAGE); </code></pre> The User API</a></h2> To simplify usage and make the library feel more like a typical Arduino library, I designed a streamlined Epaper class. It abstracts away lower-level tasks like creating the HAL and driver instances, as well as automatically handling actions like calling sleep after displaying an image. This allows users to get started quickly with minimal setup. This is what is looks like: typedef enum{ BLACK = 0, RED = 1, } Color_t; class EPaper { private: EPD_HAL *hal; EPD_Driver *driver; public: EPaper(unsigned int cs_pin, unsigned int dc_pin, unsigned int reset_pin, unsigned int busy_pin); void begin(); void showImage(const byte *image, Color_t color); }; </code></pre> Conclusion</a></h1> Before starting this little project i thought it would be much harder and more work to get some minimal example up and running. But i made quick progress and the datasheet and the existing examples provided enough information to hack a first working solution together in a couple of hours. As it is often not more productive to write something from scratch it is very educative. This is what i love and my main motivation. There are probably a lot of things which can be further improved, like partial and fast updates or using dma to transfer the framebuffer. But i think i managed to keep it as simple as possible for a minimal starting point and still be able to improve it later. Source code </div> </div> If you are interested you can find the full library here</a>. But as this project evolves over time it will contain other functions to support more complex workflows as well. If you have suggestions for improvements feel free to contribute. </div> </div> Hello 👋 2025-04-01T00:00:00+00:00 Info </div> </div> I write this blog mainly for myself, but feel free to look around. </div> </div> Intro</a></h1> Hello, I'm a teacher and I like to tinker with electronics and programming. This blog is a collection of my projects, ideas, and experiments. I hope you find something useful or interesting here. Let's go ... and do something fun. PlatformIO 2025-01-02T00:00:00+00:00 By default, PlatformIO creates new projects inside your Documents directory, which might not be ideal for everyone. If you prefer to keep your projects elsewhere, like me, you can easily change the default location in the PlatformIO terminal with the following command: pio settings set projects_dir ~/Development/platformIO </code></pre> Just replace ~/Development/platformIO</code> with your preferred folder. This way, all new projects will be created in the directory you choose, keeping your workspace organized the way you like it.